You are not logged in.

#1 2024-11-19 19:44:09

wleivam
Member
Registered: 2024-09-16
Posts: 6

[REQUEST] python311

Hi everyone!

Please, I need help with an update to the package python311. Particularly, in the package version (pkgver), because there is a new version in the official repository Python 3.11.10.

Also, I found a wrong package name in the "makedepends" list. Actually the package "bluez-libs" is used, and should be used the "boost-libs" pkg.

I don't know if is it necessary tag the maintainers, but I'll really appreciate the help with these changes.

Offline

#2 2024-11-19 19:55:59

wleivam
Member
Registered: 2024-09-16
Posts: 6

Re: [REQUEST] python311

Sorry, I forget something, is it possible add the flag "--enable-optimizations" to the configure process?. This is for clean temp files of the previous compilations.

This have the "Readme.rst" file:

Profile Guided Optimization

PGO takes advantage of recent versions of the GCC or Clang compilers.  If used,
either via ``configure --enable-optimizations`` or by manually running
``make profile-opt`` regardless of configure flags, the optimized build
process will perform the following steps:

The entire Python directory is cleaned of temporary files that may have
resulted from a previous compilation.

An instrumented version of the interpreter is built, using suitable compiler
flags for each flavor. Note that this is just an intermediary step.  The
binary resulting from this step is not good for real-life workloads as it has
profiling instructions embedded inside.

After the instrumented interpreter is built, the Makefile will run a training
workload.  This is necessary in order to profile the interpreter's execution.
Note also that any output, both stdout and stderr, that may appear at this step
is suppressed.

The final step is to build the actual interpreter, using the information
collected from the instrumented one.  The end result will be a Python binary
that is optimized; suitable for distribution or production installation.

Offline

#3 2024-11-19 20:09:27

Scimmia
Fellow
Registered: 2012-09-01
Posts: 12,130

Re: [REQUEST] python311

Why in the world are you using that package? It shouldn't even exist as it does.

Just use the 'python' package in Extra. 3.12 didn't break that much, unless you've got a very specific use-case.

Last edited by Scimmia (2024-11-19 20:10:15)

Offline

#4 2024-11-19 20:12:57

loqs
Member
Registered: 2014-03-06
Posts: 18,091

Re: [REQUEST] python311

wleivam wrote:

Also, I found a wrong package name in the "makedepends" list. Actually the package "bluez-libs" is used, and should be used the "boost-libs" pkg.

Do you mean bluez-libs should be replaced by boost-libs in makedepends?
https://github.com/python/cpython/blob/ … e.ac#L2237

wleivam wrote:

I don't know if is it necessary tag the maintainers, but I'll really appreciate the help with these changes.

Leave a comment on https://aur.archlinux.org/packages/python311

wleivam wrote:

This is for clean temp files of the previous compilations.

Cleanup is already provided by makepkg see the -c and -C options.

Offline

#5 2024-11-19 22:47:40

wleivam
Member
Registered: 2024-09-16
Posts: 6

Re: [REQUEST] python311

Scimmia wrote:

Why in the world are you using that package? It shouldn't even exist as it does.

Just use the 'python' package in Extra. 3.12 didn't break that much, unless you've got a very specific use-case.

I need this version to run llm models using the mlc engine. I use the vulkan driver and miniconda envs give me errors in the execution.

loqs wrote:

Do you mean bluez-libs should be replaced by boost-libs in makedepends?
https://github.com/python/cpython/blob/ … e.ac#L2237

In a server mode environment this can open unexplored backdoors, but maybe I'm wrong.

loqs wrote:

Done!, Thanks!

loqs wrote:

Cleanup is already provided by makepkg see the -c and -C options.

The 3.10 version has the flag "--enable-optimizations" and this not, in the "configure" process.

Last edited by wleivam (2024-11-19 22:48:23)

Offline

#6 2024-11-20 10:15:57

Lone_Wolf
Administrator
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 12,995

Re: [REQUEST] python311

https://llm.mlc.ai/docs/install/mlc_llm wrote:

Besides, we would recommend using Python 3.11;

Have you tried running it with python 3.12 or python 3.13 (python 3.13 rebuild has started in staging repo, so it will enter extra in the next few months) ?


Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

clean chroot building not flexible enough ?
Try clean chroot manager by graysky

Offline

#7 2024-11-20 12:31:19

loqs
Member
Registered: 2014-03-06
Posts: 18,091

Re: [REQUEST] python311

wleivam wrote:
loqs wrote:

Do you mean bluez-libs should be replaced by boost-libs in makedepends?
https://github.com/python/cpython/blob/ … e.ac#L2237

In a server mode environment this can open unexplored backdoors, but maybe I'm wrong.

Do you mean having bluez-libs on the system during package build opens unexplored backdoors or building python's socket module with bluetooth support opens unexplored backdoors?

wleivam wrote:
loqs wrote:

Cleanup is already provided by makepkg see the -c and -C options.

The 3.10 version has the flag "--enable-optimizations" and this not, in the "configure" process.

Why must cleanup be performed by configure instead of makepkg?

Offline

#8 2024-11-20 18:39:35

wleivam
Member
Registered: 2024-09-16
Posts: 6

Re: [REQUEST] python311

Lone_Wolf wrote:

Have you tried running it with python 3.12 or python 3.13 (python 3.13 rebuild has started in staging repo, so it will enter extra in the next few months) ?

I try with the 3.12, but I've gotten an error compiling the Apache TVM fork. The version 3.13 was released the last week, so no.

loqs wrote:

Do you mean having bluez-libs on the system during package build opens unexplored backdoors or building python's socket module with bluetooth support opens unexplored backdoors?

Both, I don't need the bluetooth working in my server, but again, maybe I'm wrong because is a personal use case.

loqs wrote:

Why must cleanup be performed by configure instead of makepkg?

The cleanup is the first step of the conf, the real target of this flag is:

The end result will be a Python binary
that is optimized; suitable for distribution or production installation.

Offline

#9 2024-11-20 19:07:30

loqs
Member
Registered: 2014-03-06
Posts: 18,091

Re: [REQUEST] python311

wleivam wrote:
Lone_Wolf wrote:

Have you tried running it with python 3.12 or python 3.13 (python 3.13 rebuild has started in staging repo, so it will enter extra in the next few months) ?

I try with the 3.12, but I've gotten an error compiling the Apache TVM fork. The version 3.13 was released the last week, so no.

Building relax worked for me. Possibly a difference in the cmake options you used or what you have installed on your system?

wleivam wrote:

Both, I don't need the bluetooth working in my server, but again, maybe I'm wrong because is a personal use case.

That makes sense for your use case but why make that the default if you do not know of any actual issues it introduces when it removes functionality? What functionality does adding boost-libs add to the build?

Offline

#10 2024-11-21 05:38:14

wleivam
Member
Registered: 2024-09-16
Posts: 6

Re: [REQUEST] python311

loqs wrote:

That makes sense for your use case but why make that the default if you do not know of any actual issues it introduces when it removes functionality?

I compiled the 3.11.10 version without that lib, and I had no problems, but you're right.

loqs wrote:

What functionality does adding boost-libs add to the build?

It's to get a better PyTorch's performance, but again, It's for a personal use case.

Offline

#11 2024-11-21 13:35:24

loqs
Member
Registered: 2014-03-06
Posts: 18,091

Re: [REQUEST] python311

wleivam wrote:
loqs wrote:

What functionality does adding boost-libs add to the build?

It's to get a better PyTorch's performance, but again, It's for a personal use case.

I searched through the 3.11.10 I could not find any use of boost. Also adding boost-libs which do not contain headers to the build would most likely not have any effect even if boost could be used:

grep -ir boost Python-3.11.10
Python-3.11.10/Doc/c-api/slice.rst:   ``PY_SSIZE_T_MAX``, silently boost the start and stop values less than
Python-3.11.10/Doc/c-api/slice.rst:   ``PY_SSIZE_T_MIN`` to ``PY_SSIZE_T_MIN``, and silently boost the step
Python-3.11.10/Doc/conf.py:    r'https://www.boost.org/libs/.+': r'https://www.boost.org/doc/libs/\d_\d+_\d/.+',
Python-3.11.10/Doc/faq/extending.rst:<https://cxx.sourceforge.net/>`_ `Boost
Python-3.11.10/Doc/faq/extending.rst:<https://www.boost.org/libs/python/doc/index.html>`_, or `Weave
Python-3.11.10/Doc/faq/extending.rst:The Boost Python Library (BPL, https://www.boost.org/libs/python/doc/index.html)
Python-3.11.10/Doc/library/functools.rst:      likely to provide an easy speed boost.
Python-3.11.10/Doc/whatsnew/2.4.rst:  resulting in performance boosts for :meth:`keys`, :meth:`values`, :meth:`items`,
Python-3.11.10/Include/internal/pycore_genobject.h:    /* Freelists boost performance 6-10%; they also reduce memory
Python-3.11.10/Lib/test/test_threadedtempfile.py:quickly. Guido reports needing to boost FILES_PER_THREAD to 500 before
Python-3.11.10/Lib/test/test_timeout.py:    # and Win98SE.  Boosting it to 2.0 helped a lot, but isn't a real
Python-3.11.10/Lib/test/test_urllibnet.py:        # domain name requests in an attempt to boost traffic
Python-3.11.10/Misc/HISTORY:- Issue #1087418: Boost performance of bitwise operations for longs.
Python-3.11.10/Misc/HISTORY:- Boosted the stack reservation for python.exe and pythonw.exe from
Python-3.11.10/Misc/HISTORY:  In addition, the default check interval was boosted from 10 to 100
Python-3.11.10/Misc/HISTORY:  don't.  It so happens that this gives a nice additional speed boost
Python-3.11.10/Misc/HISTORY:  examples also work again.  It is hoped that Gtk and Boost also work
Python-3.11.10/Misc/HISTORY:  can specify no more than 64 sockets.  Python now boosts
Python-3.11.10/Misc/NEWS:  This also boosts performance on Windows as it now uses the zlib crc32
Python-3.11.10/Misc/NEWS:  page size. Boosting the pool and arena size allows obmalloc to handle a
Python-3.11.10/Misc/NEWS:- bpo-32351: Use fastpath in asyncio.sleep if delay<0 (2x boost)
Python-3.11.10/Modules/_collectionsmodule.c:   boost (by moving each pointer only once instead of twice).
Python-3.11.10/Modules/cmathmodule.c:       this is essentially the "weak" test from the Boost library
Python-3.11.10/Modules/mathmodule.c:   used by the Boost library.  Following Boost (again), we re-express the
Python-3.11.10/Modules/mathmodule.c:   double-checked against the coefficients in the Boost source code.
Python-3.11.10/Modules/mathmodule.c:       this is essentially the "weak" test from the Boost library
Python-3.11.10/Modules/selectmodule.c:   Here we boost it.
Python-3.11.10/Modules/selectmodule.c:   Users who want even more than the boosted limit should #define
Python-3.11.10/Objects/listobject.c: * than this are boosted artificially via binary insertion.
Python-3.11.10/Objects/listsort.txt:artificially boosts it to minrun elements, via a stable binary insertion sort
Python-3.11.10/Python/ceval.c:   and silently boost values less than PY_SSIZE_T_MIN to PY_SSIZE_T_MIN.
Python-3.11.10/Tools/peg_generator/data/top-pypi-packages-365-days.json:            "project": "xgboost"
Python-3.11.10/Tools/peg_generator/data/top-pypi-packages-365-days.json:            "project": "catboost"

Offline

#12 2024-11-21 14:46:46

mithrial
Member
Registered: 2017-03-05
Posts: 51

Re: [REQUEST] python311

The bluez-libs dependency was introduced 11 years ago: https://gitlab.archlinux.org/archlinux/ … 81456089d6 and reintroduced again with FS#70438 after deemed necessary.

Last edited by mithrial (2024-11-21 14:47:44)

Offline

Board footer

Powered by FluxBB