Arch Linux

wleivam

Member

Registered: 2024-09-16

Posts: 6

[REQUEST] python311

Hi everyone!

Please, I need help with an update to the package python311. Particularly, in the package version (pkgver), because there is a new version in the official repository Python 3.11.10.

Also, I found a wrong package name in the "makedepends" list. Actually the package "bluez-libs" is used, and should be used the "boost-libs" pkg.

I don't know if is it necessary tag the maintainers, but I'll really appreciate the help with these changes.

wleivam

Member

Registered: 2024-09-16

Posts: 6

Re: [REQUEST] python311

Sorry, I forget something, is it possible add the flag "--enable-optimizations" to the configure process?. This is for clean temp files of the previous compilations.

This have the "Readme.rst" file:

Profile Guided Optimization

PGO takes advantage of recent versions of the GCC or Clang compilers.  If used,
either via ``configure --enable-optimizations`` or by manually running
``make profile-opt`` regardless of configure flags, the optimized build
process will perform the following steps:

The entire Python directory is cleaned of temporary files that may have
resulted from a previous compilation.

An instrumented version of the interpreter is built, using suitable compiler
flags for each flavor. Note that this is just an intermediary step.  The
binary resulting from this step is not good for real-life workloads as it has
profiling instructions embedded inside.

After the instrumented interpreter is built, the Makefile will run a training
workload.  This is necessary in order to profile the interpreter's execution.
Note also that any output, both stdout and stderr, that may appear at this step
is suppressed.

The final step is to build the actual interpreter, using the information
collected from the instrumented one.  The end result will be a Python binary
that is optimized; suitable for distribution or production installation.

Scimmia

Fellow

Registered: 2012-09-01

Posts: 12,084

Re: [REQUEST] python311

Why in the world are you using that package? It shouldn't even exist as it does.

Just use the 'python' package in Extra. 3.12 didn't break that much, unless you've got a very specific use-case.

Last edited by Scimmia (2024-11-19 20:10:15)

loqs

Member

Registered: 2014-03-06

Posts: 18,042

Re: [REQUEST] python311

Also, I found a wrong package name in the "makedepends" list. Actually the package "bluez-libs" is used, and should be used the "boost-libs" pkg.

Do you mean bluez-libs should be replaced by boost-libs in makedepends?
https://github.com/python/cpython/blob/ … e.ac#L2237

I don't know if is it necessary tag the maintainers, but I'll really appreciate the help with these changes.

Leave a comment on https://aur.archlinux.org/packages/python311

This is for clean temp files of the previous compilations.

Cleanup is already provided by makepkg see the -c and -C options.

wleivam

Member

Registered: 2024-09-16

Posts: 6

Re: [REQUEST] python311

Why in the world are you using that package? It shouldn't even exist as it does.

Just use the 'python' package in Extra. 3.12 didn't break that much, unless you've got a very specific use-case.

I need this version to run llm models using the mlc engine. I use the vulkan driver and miniconda envs give me errors in the execution.

Do you mean bluez-libs should be replaced by boost-libs in makedepends?
https://github.com/python/cpython/blob/ … e.ac#L2237

In a server mode environment this can open unexplored backdoors, but maybe I'm wrong.

Leave a comment on https://aur.archlinux.org/packages/python311

Done!, Thanks!

Cleanup is already provided by makepkg see the -c and -C options.

The 3.10 version has the flag "--enable-optimizations" and this not, in the "configure" process.

Last edited by wleivam (2024-11-19 22:48:23)

Lone_Wolf

Administrator

From: Netherlands, Europe

Registered: 2005-10-04

Posts: 12,926

Re: [REQUEST] python311

Besides, we would recommend using Python 3.11;

Have you tried running it with python 3.12 or python 3.13 (python 3.13 rebuild has started in staging repo, so it will enter extra in the next few months) ?

---

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

clean chroot building not flexible enough ?
Try clean chroot manager by graysky

loqs

Member

Registered: 2014-03-06

Posts: 18,042

Re: [REQUEST] python311

Do you mean bluez-libs should be replaced by boost-libs in makedepends?
https://github.com/python/cpython/blob/ … e.ac#L2237

In a server mode environment this can open unexplored backdoors, but maybe I'm wrong.

Do you mean having bluez-libs on the system during package build opens unexplored backdoors or building python's socket module with bluetooth support opens unexplored backdoors?

Cleanup is already provided by makepkg see the -c and -C options.

The 3.10 version has the flag "--enable-optimizations" and this not, in the "configure" process.

Why must cleanup be performed by configure instead of makepkg?

wleivam

Member

Registered: 2024-09-16

Posts: 6

Re: [REQUEST] python311

Have you tried running it with python 3.12 or python 3.13 (python 3.13 rebuild has started in staging repo, so it will enter extra in the next few months) ?

I try with the 3.12, but I've gotten an error compiling the Apache TVM fork. The version 3.13 was released the last week, so no.

Do you mean having bluez-libs on the system during package build opens unexplored backdoors or building python's socket module with bluetooth support opens unexplored backdoors?

Both, I don't need the bluetooth working in my server, but again, maybe I'm wrong because is a personal use case.

Why must cleanup be performed by configure instead of makepkg?

The cleanup is the first step of the conf, the real target of this flag is:

The end result will be a Python binary
that is optimized; suitable for distribution or production installation.

loqs

Member

Registered: 2014-03-06

Posts: 18,042

Re: [REQUEST] python311

Have you tried running it with python 3.12 or python 3.13 (python 3.13 rebuild has started in staging repo, so it will enter extra in the next few months) ?

I try with the 3.12, but I've gotten an error compiling the Apache TVM fork. The version 3.13 was released the last week, so no.

Building relax worked for me. Possibly a difference in the cmake options you used or what you have installed on your system?

Both, I don't need the bluetooth working in my server, but again, maybe I'm wrong because is a personal use case.

That makes sense for your use case but why make that the default if you do not know of any actual issues it introduces when it removes functionality? What functionality does adding boost-libs add to the build?

wleivam

Member

Registered: 2024-09-16

Posts: 6

Re: [REQUEST] python311

That makes sense for your use case but why make that the default if you do not know of any actual issues it introduces when it removes functionality?

I compiled the 3.11.10 version without that lib, and I had no problems, but you're right.

What functionality does adding boost-libs add to the build?

It's to get a better PyTorch's performance, but again, It's for a personal use case.

loqs

Member

Registered: 2014-03-06

Posts: 18,042

Re: [REQUEST] python311

What functionality does adding boost-libs add to the build?

It's to get a better PyTorch's performance, but again, It's for a personal use case.

I searched through the 3.11.10 I could not find any use of boost. Also adding boost-libs which do not contain headers to the build would most likely not have any effect even if boost could be used:

grep -ir boost Python-3.11.10
Python-3.11.10/Doc/c-api/slice.rst:   ``PY_SSIZE_T_MAX``, silently boost the start and stop values less than
Python-3.11.10/Doc/c-api/slice.rst:   ``PY_SSIZE_T_MIN`` to ``PY_SSIZE_T_MIN``, and silently boost the step
Python-3.11.10/Doc/conf.py:    r'https://www.boost.org/libs/.+': r'https://www.boost.org/doc/libs/\d_\d+_\d/.+',
Python-3.11.10/Doc/faq/extending.rst:<https://cxx.sourceforge.net/>`_ `Boost
Python-3.11.10/Doc/faq/extending.rst:<https://www.boost.org/libs/python/doc/index.html>`_, or `Weave
Python-3.11.10/Doc/faq/extending.rst:The Boost Python Library (BPL, https://www.boost.org/libs/python/doc/index.html)
Python-3.11.10/Doc/library/functools.rst:      likely to provide an easy speed boost.
Python-3.11.10/Doc/whatsnew/2.4.rst:  resulting in performance boosts for :meth:`keys`, :meth:`values`, :meth:`items`,
Python-3.11.10/Include/internal/pycore_genobject.h:    /* Freelists boost performance 6-10%; they also reduce memory
Python-3.11.10/Lib/test/test_threadedtempfile.py:quickly. Guido reports needing to boost FILES_PER_THREAD to 500 before
Python-3.11.10/Lib/test/test_timeout.py:    # and Win98SE.  Boosting it to 2.0 helped a lot, but isn't a real
Python-3.11.10/Lib/test/test_urllibnet.py:        # domain name requests in an attempt to boost traffic
Python-3.11.10/Misc/HISTORY:- Issue #1087418: Boost performance of bitwise operations for longs.
Python-3.11.10/Misc/HISTORY:- Boosted the stack reservation for python.exe and pythonw.exe from
Python-3.11.10/Misc/HISTORY:  In addition, the default check interval was boosted from 10 to 100
Python-3.11.10/Misc/HISTORY:  don't.  It so happens that this gives a nice additional speed boost
Python-3.11.10/Misc/HISTORY:  examples also work again.  It is hoped that Gtk and Boost also work
Python-3.11.10/Misc/HISTORY:  can specify no more than 64 sockets.  Python now boosts
Python-3.11.10/Misc/NEWS:  This also boosts performance on Windows as it now uses the zlib crc32
Python-3.11.10/Misc/NEWS:  page size. Boosting the pool and arena size allows obmalloc to handle a
Python-3.11.10/Misc/NEWS:- bpo-32351: Use fastpath in asyncio.sleep if delay<0 (2x boost)
Python-3.11.10/Modules/_collectionsmodule.c:   boost (by moving each pointer only once instead of twice).
Python-3.11.10/Modules/cmathmodule.c:       this is essentially the "weak" test from the Boost library
Python-3.11.10/Modules/mathmodule.c:   used by the Boost library.  Following Boost (again), we re-express the
Python-3.11.10/Modules/mathmodule.c:   double-checked against the coefficients in the Boost source code.
Python-3.11.10/Modules/mathmodule.c:       this is essentially the "weak" test from the Boost library
Python-3.11.10/Modules/selectmodule.c:   Here we boost it.
Python-3.11.10/Modules/selectmodule.c:   Users who want even more than the boosted limit should #define
Python-3.11.10/Objects/listobject.c: * than this are boosted artificially via binary insertion.
Python-3.11.10/Objects/listsort.txt:artificially boosts it to minrun elements, via a stable binary insertion sort
Python-3.11.10/Python/ceval.c:   and silently boost values less than PY_SSIZE_T_MIN to PY_SSIZE_T_MIN.
Python-3.11.10/Tools/peg_generator/data/top-pypi-packages-365-days.json:            "project": "xgboost"
Python-3.11.10/Tools/peg_generator/data/top-pypi-packages-365-days.json:            "project": "catboost"

mithrial

Member

Registered: 2017-03-05

Posts: 51

Re: [REQUEST] python311

The bluez-libs dependency was introduced 11 years ago: https://gitlab.archlinux.org/archlinux/ … 81456089d6 and reintroduced again with FS#70438 after deemed necessary.

Last edited by mithrial (2024-11-21 14:47:44)