To make a small enough container image is a complicated job, especially Python.

We can use multi-stage build process and virtualenv to reduce the overhead from the build and pip install. However, some Python libraries use the runtime C library, we have to overcome this issue; otherwise the package won’t work.

I will take TA-Lib as an example to show how to get everything done correctly.

FROM python:3.7-slim AS compile-image
RUN apt-get update
RUN apt-get install -y --no-install-recommends build-essential gcc wget

# Make sure we use the virtualenv:
RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"

RUN pip install numpy

# TA-Lib
RUN wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz && \
  tar -xvzf ta-lib-0.4.0-src.tar.gz && \
  cd ta-lib/ && \
  ./configure --prefix=/opt/venv && \
  make && \
  make install

RUN pip install --global-option=build_ext --global-option="-L/opt/venv/lib" TA-Lib==0.4.16
RUN rm -R ta-lib ta-lib-0.4.0-src.tar.gz

COPY requirements.txt .
RUN pip install -r requirements.txt

FROM python:3.7-slim AS build-image
COPY --from=compile-image /opt/venv /opt/venv

# Make sure we use the virtualenv:
ENV PATH="/opt/venv/bin:$PATH"
ENV LD_LIBRARY_PATH="/opt/venv/lib"

Here is an example, and there are 5 points should be aware:

1. Choose a small enough base image; nevertheless, avoid using alpine. The reason is explained by this article: https://pythonspeed.com/articles/alpine-docker-python/
2. Multi-stage is a good idea to reduce the overhead of building processes. In addition, applying virtualenv makes things much more easily. Therefore, install all Python packages into /opt/venv.
3. Now, we can try to build TA-Lib from the source. You have to specify the installation path to /opt/venv as well while configure.
4. Because we moved the default library path to /opt/venv, we need assign the global option or the build option to pip to let it know the library path.
5. Finally, remember to provide the LD_LIRARY_PATH to let python know where are the shared libraries.

Following the above procedure, the image size will be around 250MB. This is pretty small.