lxml is a very good Python library for parsing XML. Fast, powerful, and with C dependencies that are sometimes a pain to install.
That last part is the problem with Docker. lxml needs its compiled binaries in the final image, so keeping that image small is harder than usual, and with a multi-stage build it gets a bit trickier still.
It has a fix, though. This is the Dockerfile I use:
FROM python:3.8-alpine as base
RUN apk update && \
apk upgrade
FROM base as build_lxml
RUN apk add --no-cache build-base gcc musl-dev python3-dev libffi-dev libxml2-dev libxslt-dev
RUN python -OO -m pip install --no-cache-dir -U pip && \
python -OO -m pip wheel --no-cache-dir --wheel-dir=/root/lxml_wheel lxml
FROM base
COPY --from=build_lxml /root/lxml_wheel /root/lxml_wheel
# lxml binary dependencies
COPY --from=build_lxml /usr/lib/libxslt.so.1 /usr/lib/libxslt.so.1
COPY --from=build_lxml /usr/lib/libexslt.so.0 /usr/lib/libexslt.so.0
COPY --from=build_lxml /usr/lib/libxml2.so.2 /usr/lib/libxml2.so.2
COPY --from=build_lxml /usr/lib/libgcrypt.so.20 /usr/lib/libgcrypt.so.20
COPY --from=build_lxml /usr/lib/libgpg-error.so.0 /usr/lib/libgpg-error.so.0
RUN python -OO -m pip install --no-cache --no-index --find-links=/root/lxml_wheel/*
Step by step, this is what it does.
Create the build stage
There is a build_lxml stage. It holds the development packages that compiling lxml needs, and nothing from it reaches the final image unless we copy it over:
RUN apk add --no-cache build-base gcc musl-dev python3-dev libffi-dev libxml2-dev libxslt-dev
Compile lxml and optimise the wheel
Instead of installing lxml, we build a wheel from it, the same trick as in the post on Python images under 50 MB. Running pip with python -OO compiles the bytecode with optimisations on, and the wheel ends up in /root/lxml_wheel:
RUN python -OO -m pip install --no-cache-dir -U pip && \
python -OO -m pip wheel --no-cache-dir --wheel-dir=/root/lxml_wheel lxml
Copy the binary dependencies
This is the step that matters. The wheel alone is not enough: lxml links against a handful of .so files that only exist in the build stage, so the final stage has to copy them across by hand.
# lxml binary dependencies
COPY --from=build_lxml /usr/lib/libxslt.so.1 /usr/lib/libxslt.so.1
COPY --from=build_lxml /usr/lib/libexslt.so.0 /usr/lib/libexslt.so.0
COPY --from=build_lxml /usr/lib/libxml2.so.2 /usr/lib/libxml2.so.2
COPY --from=build_lxml /usr/lib/libgcrypt.so.20 /usr/lib/libgcrypt.so.20
COPY --from=build_lxml /usr/lib/libgpg-error.so.0 /usr/lib/libgpg-error.so.0
Install the wheel
The last line installs the wheel offline, from the folder we copied, with no index lookup:
RUN python -OO -m pip install --no-cache --no-index --find-links=/root/lxml_wheel/*
And that is all. A final image that knows nothing about compilers and still runs lxml.
