Arch Linux

fabiin

Member

Registered: 2021-07-08

Posts: 11

Why is debian's python much faster than arch's python ?

I did a little test comparing Python builds across distributions using Docker.

Here python in a Dockerfile based on debian :

FROM debian:sid
RUN apt update && apt install -y python3.12

docker build -f ./Dockerfile-debian -t python_debian .

The python code executed to benchmark :

import sys
import timeit
def test_function():
    numbers = list(range(1000))
    [num ** 2 for num in numbers]
benchmark_result = timeit.timeit(test_function, number=int(sys.argv[1]) )
print(benchmark_result)

Run tests:

$ docker run -v .:/app python_debian python3.12 /app/test.py
# 0.7174095689988462

$ python3.12 test.py
# 0.956413246000011

I don't understand why there is so much difference between the debian's python and the arch's one.

In the same fashion I've tried with many other distros, here the result (percents compare to official python docker image):

debian-on-docker    	0.7174095689988462	-33.074471847330045%
ubuntu-on-docker    	0.7872083639995253	-26.563098955608975%
fedora-on-docker    	0.9239705759991921	-13.804859220699843%
local-python               	0.956413246000011	-10.778355368071304%
arch-on-docker      	0.9577604120004253	-10.652681276195356%
official-python-docker     	1.0719520470011048	0.0%

Here are the compile arguments used by Debian and Arch : retrieved using "sysconfig.get_config_vars('CONFIG_ARGS')"

# Debian
# ["'--enable-shared'
# '--prefix=/usr'
# '--libdir=/usr/lib/x86_64-linux-gnu'
# '--enable-ipv6'
# '--enable-loadable-sqlite-extensions'
# '--with-dbmliborder=bdb:gdbm'
# '--with-computed-gotos'
# '--without-ensurepip'
# '--with-system-expat'
# '--with-dtrace'
# '--with-ssl-default-suites=openssl'
# '--with-wheel-pkg-dir=/usr/share/python-wheels/'
# 'MKDIR_P=/bin/mkdir -p'
# 'CC=x86_64-linux-gnu-gcc'"]

# Arch
# ["'--prefix=/usr'
# '--enable-shared'
# '--with-computed-gotos'
# '--enable-optimizations'
# '--with-lto'
# '--enable-ipv6'
# '--with-system-expat'
# '--with-dbmliborder=gdbm:ndbm'
# '--with-system-libmpdec'
# '--enable-loadable-sqlite-extensions'
# '--without-ensurepip'
# '--with-tzpath=/usr/share/zoneinfo'
# 'CFLAGS=-march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects'
# 'LDFLAGS=-Wl,-O1 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now          -Wl,-z,pack-relative-relocs -flto=auto'"]

Note that debian didn't use `--enable-optimizations`.

Git repository to have full test code : https://github.com/fabien-michel/compare-python

Any idea ?

Last edited by fabiin (2024-05-22 14:01:49)

fabiin

Member

Registered: 2021-07-08

Posts: 11

Re: Why is debian's python much faster than arch's python ?

I have also tried to build my own python using the same build args as debian, but the result is worse than anything! I really miss something. What black magic debian team use to build their python ?

Dockerfile to build my own python:

FROM debian:sid

RUN apt update && apt install -y git \
                                build-essential \
                                gdb \
                                lcov \
                                libbz2-dev \
                                libffi-dev \
                                libgdbm-dev \
                                liblzma-dev \
                                libncurses5-dev \
                                libreadline6-dev \
                                libsqlite3-dev \
                                libssl-dev \
                                lzma \
                                lzma-dev \
                                tk-dev \
                                uuid-dev \
                                xvfb \
                                zlib1g-dev \
                                systemtap-sdt-dev

RUN git clone --depth 1 -b 3.12 https://github.com/python/cpython.git
WORKDIR /cpython
RUN ./configure --enable-shared --enable-ipv6 --enable-loadable-sqlite-extensions --with-dbmliborder=bdb:gdbm --with-computed-gotos --without-ensurepip --with-system-expat --with-dtrace && \
    CC=x86_64-linux-gnu-gcc make && \
    make install
ENV LD_LIBRARY_PATH="/usr/local/lib/"

Last edited by fabiin (2024-05-22 13:59:54)

Maniaxx

Member

Registered: 2014-05-14

Posts: 757

Re: Why is debian's python much faster than arch's python ?

What are the results on bare metal python without docker?

---

sys2064

fabiin

Member

Registered: 2021-07-08

Posts: 11

Re: Why is debian's python much faster than arch's python ?

The comparison is about python from debian (in docker) against bare metal python 3.12 from arch repo (without docker)

Last edited by fabiin (2024-05-22 16:13:42)