Understanding and Utilizing Python Wheel to Speed Up Docker Builds

When deploying Python applications in a Docker environment, it's common to encounter issues where the image build times are excessively long or the build fails due to compiler dependencies like gcc. One of the most effective solutions to these problems is leveraging Python Wheel.

This article explores the concept and advantages of Python Wheel, as well as how to utilize it in your Dockerfile to optimize the build process.

1. What is Python Wheel?

Wheel (.whl) is Python's binary distribution format (Built Distribution).

Python packages are generally distributed in two forms:

1. Source Distribution (sdist, .tar.gz): This is the source code form. Installation requires a compilation (Build) process on the user's computer.

2. Built Distribution (bdist_wheel, .whl): This is a pre-compiled binary form. The installation completes with just copying the files without additional compilation.

To put it simply, the source distribution is like 'furniture that needs to be assembled', while Wheel is like 'finished furniture'. By using Wheel, there's no need for assembly time or tools (compilers).

2. Differences from Standard pip Install

When we execute the command pip install <package>, it operates internally with the following priority:

1. If a Wheel file exists: It downloads the binary and installs it immediately. (Very fast)

2. If no Wheel file exists: It downloads the source file (tar.gz), then compiles (Build) it locally and installs it. (Slow, requires compilation tools)

Summary Table of Differences

3. Why Use Wheel in Dockerfile (Advantages)

Using Wheel in a Docker environment, especially in a Multi-stage Build, provides several clear benefits:

1. Dramatically Improved Build Speed: You don't have to compile heavy libraries written in C/C++ like pandas and numpy each time.

2. Reduced Image Size: There's no need to include heavy compilation tools like gcc or build-essential in the final runtime image.

3. Increased Stability: Prevents build failures due to network issues or temporary outages in external repositories, and allows the use of pre-built files locally.

4. Writing and Using Dockerfile

A Docker build utilizing Wheel primarily involves creating Wheel files in the Builder Stage and then copying and installing them in the Final Stage.

Example: Dockerfile Using Multi-stage Build

Below is a pattern for efficiently installing packages with C library dependencies.

# [Stage 1] Builder: Create Wheel file
FROM python:3.9-slim as builder

# Install essential packages for compilation (not included in final image)
RUN apt-get update && apt-get install -y \
    build-essential \
    gcc \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app

# Copy requirements.txt
COPY requirements.txt .

# Generate .whl files in the /wheels directory using pip wheel command
# -w: Specify the path to save the wheel files
RUN pip wheel --no-cache-dir --no-deps --wheel-dir /app/wheels -r requirements.txt


# [Stage 2] Final: Runtime image
FROM python:3.9-slim

WORKDIR /app

# Copy only the Wheel files generated in the Builder stage
COPY --from=builder /app/wheels /wheels
COPY --from=builder /app/requirements.txt .

# Install packages using the Wheel files
# --no-index: Ignore PyPI indexes and only look for local files
# --find-links: Specify the path to look for packages to install
RUN pip install --no-cache --no-index --find-links=/wheels -r requirements.txt \
    && rm -rf /wheels

# Copy application code
COPY . .

CMD ["python", "main.py"]

Key Command Descriptions

• pip wheel: Generates the corresponding .whl file for the environment without installing the package, saving it to the specified directory.

• --wheel-dir: The location where the generated Wheel files will be saved.

• --find-links=/wheels: Instructs to look for packages in the specified local path instead of downloading from the PyPI server (internet).

Conclusion

Utilizing Python Wheel during Docker builds is not just a choice, but a necessary optimization technique. Especially in environments where repetitive builds occur in CI/CD pipelines, reducing build times and image sizes can lead to significant cost savings.

If your current Dockerfile includes the installation of gcc, consider transitioning to a multi-stage build utilizing Wheel.