Byron PeeblesBlog

Compiling Python using pyenv with optimizations

I noticed awhile ago that you can pass build arguments to pyenv when installing a new version of Python. I had also recently seen a mention of building Python targeted toward the native architecture, as well as Link Time Optimization (LTO). By setting PYTHON_CFLAGS and/or CONFIGURE_OPTS by doing something like:

PYTHON_CFLAGS=-march=native CONFIGURE_OPTS='--enable-optimizations --with-lto' pyenv install 3.10.2

So I decided to try it and benchmark using my sudoku solver.

Unoptimized
$ pyenv install 3.10.2
$ pyenv shell 3.10.2
$ time python sudoku.py 850002400720000009004000000000107002305000900040000000000080070017000000000036040
859612437723854169164379528986147352375268914241593786432981675617425893598736241

real	1m24.791s
user	1m24.770s
sys	0m0.021s
$ time python sudoku.py 850002400720000009004000000000107002305000900040000000000080070017000000000036040
859612437723854169164379528986147352375268914241593786432981675617425893598736241

real	1m25.261s
user	1m25.252s
sys	0m0.009s

So, an average real time of 85 seconds across two runs.

Optimized
$ PYTHON_CFLAGS=-march=native CONFIGURE_OPTS='--enable-optimizations --with-lto' pyenv install 3.10.2
$ pyenv shell 3.10.2
$ time python sudoku.py 850002400720000009004000000000107002305000900040000000000080070017000000000036040
859612437723854169164379528986147352375268914241593786432981675617425893598736241

real	1m6.683s
user	1m6.679s
sys	0m0.004s
$ time python sudoku.py 850002400720000009004000000000107002305000900040000000000080070017000000000036040
859612437723854169164379528986147352375268914241593786432981675617425893598736241

real	1m5.037s
user	1m5.032s
sys	0m0.005s

An average real time of 66 seconds, so only taking about 79% of the time of unoptimized Python. I’ve also noticed a large unit test suite that significantly uses Postgres as well and still saw it complete 5–10% faster. Building the Python took significantly more time (didn’t time it) but I only build a version of Python once and then run it many many times.