You are not logged in.

Pages: 1

#1 2025-05-27 12:41:20

rocky
Member
Registered: 2025-01-13
Posts: 7

[Solved] Protobuf problems

I installed protobuf-31.0-2 and when I install AUR python libraries, this happens:
ImportError: libprotobuf.so.30.2.0: cannot open shared object file: No such file or directory
Maybe protobuf is too new?

Last edited by rocky (2025-06-15 02:15:44)

Offline

#2 2025-05-27 12:47:16

Lone_Wolf
Administrator
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 13,792

Re: [Solved] Protobuf problems

You probably need to rebuild those aur python librairies so they'll use the new protobuf .



Moderator Note: moving to aur board.

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

clean chroot building not flexible enough ?
Try clean chroot manager by graysky

Offline

#3 2025-05-27 13:41:41

rocky
Member
Registered: 2025-01-13
Posts: 7

Re: [Solved] Protobuf problems

# Maintainer: Claudia Pellegrino <aur ät cpellegrino.de>

pkgname=python-diffq
_gitpkgname=diffq
pkgver=0.2.4
pkgrel=2
pkgdesc='Differentiable quantization using pseudo quantization noise'
arch=('x86_64')
url='https://github.com/facebookresearch/diffq'
license=('CC-BY-NC-4.0')
depends=(
  'glibc'
  'python'
  'python-numpy'
  'python-pytorch'
)
makedepends=(
  'cython'
  'python-build'
  'python-installer'
  'python-pdoc'
  'python-setuptools'
  'python-wheel'
)
checkdepends=(
  'python-torchvision'
)
optdepends=(
  'python-einops: to run the included cifar example'
  'python-hydra-colorlog: to run the included cifar example'
  'python-hydra-core: to run the included cifar example'
  'python-submitit: to run the included cifar example'
  'python-timm: to run the included cifar example'
  'python-torchvision: to run the included cifar example'
)

source=(
  "${_gitpkgname}-${pkgver}.tar.gz::https://github.com/facebookresearch/diffq/archive/v${pkgver}.tar.gz"
)

sha512sums=(
  '0b47058c19e0ce841889c1f47b266cc292044d5483e7aa3f757bb5d050f48cdde10547b5ce7df654e54ee47271d566f24b667e228bbc20035e2345290eb06512'
)

prepare() {
  cd "${_gitpkgname}-${pkgver}"

  echo >&2 'Fixing circular imports in unit tests'
  mkdir tmp_test
  mv -v diffq/tests tmp_test/tests
  find tmp_test/tests -name '*.py' -exec sed -i -e 's/diffq\.\(tests\)/\1/' '{}' +

  echo >&2 'Fixing type mismatch in the bitpack.pack() call'
  sed -i -e 's/\(pack_fn(.*, \)\(self\.bits\)/\1int(\2)/' \
    diffq/uniform.py
}

_site_packages() {
  python -c 'import site; print(site.getsitepackages()[0])'
}

build() {
  cd "${_gitpkgname}-${pkgver}"

  echo >&2 'Building wheel'
  python -m build --wheel --no-isolation

  echo >&2 'Generating documentation'
  python -m installer --destdir=tmp_install dist/*.whl
  PYTHONPATH="${PWD}/tmp_install/$(_site_packages)" \
    pdoc -o docs diffq
}

check() {
  cd "${_gitpkgname}-${pkgver}"

  echo >&2 'Running unit tests'
  env -C tmp_test PYTHONPATH="${PWD}/tmp_install/$(_site_packages)" \
    python -m unittest discover
}

package() {
  cd "${_gitpkgname}-${pkgver}"

  echo >&2 'Packaging the wheel'
  python -I -m installer --destdir="${pkgdir}" dist/*.whl

  echo >&2 'Packaging the documentation'
  install -D -m 644 -t "${pkgdir}/usr/share/doc/${pkgname}" \
    README.md

  echo >&2 'Packaging examples'
  mkdir -p "${pkgdir}/usr/share/${pkgname}"
  cp -R --preserve=mode -t "${pkgdir}/usr/share/${pkgname}" \
    examples

  echo >&2 'Packaging the license'
  install -D -m 644 -t "${pkgdir}/usr/share/licenses/${pkgname}" \
    LICENSE
}
how?

Offline

#4 2025-05-27 13:46:18

Scimmia
Fellow
Registered: 2012-09-01
Posts: 12,773

Re: [Solved] Protobuf problems

Look at the FULL error, not just the part you posted. See what it is that's trying to import the old lib.

Offline

#5 2025-05-27 13:52:24

rocky
Member
Registered: 2025-01-13
Posts: 7

Re: [Solved] Protobuf problems

(the log is too long)
...(no problem)
==> Starting build()...
Building wheel
* Getting build dependencies for wheel...
Compiling bitpack.pyx because it changed.
[1/1] Cythonizing bitpack.pyx
running egg_info
creating diffq.egg-info
writing diffq.egg-info/PKG-INFO
writing dependency_links to diffq.egg-info/dependency_links.txt
writing requirements to diffq.egg-info/requires.txt
writing top-level names to diffq.egg-info/top_level.txt
writing manifest file 'diffq.egg-info/SOURCES.txt'
reading manifest file 'diffq.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files found matching 'examples/cifar/outputs/**'
adding license file 'LICENSE'
writing manifest file 'diffq.egg-info/SOURCES.txt'
* Building wheel...
running bdist_wheel
running build
running build_py
creating build/lib.linux-x86_64-cpython-313/diffq
copying diffq/utils.py -> build/lib.linux-x86_64-cpython-313/diffq
copying diffq/uniform.py -> build/lib.linux-x86_64-cpython-313/diffq
copying diffq/torch_pack.py -> build/lib.linux-x86_64-cpython-313/diffq
copying diffq/lsq.py -> build/lib.linux-x86_64-cpython-313/diffq
copying diffq/ts_export.py -> build/lib.linux-x86_64-cpython-313/diffq
copying diffq/diffq.py -> build/lib.linux-x86_64-cpython-313/diffq
copying diffq/base.py -> build/lib.linux-x86_64-cpython-313/diffq
copying diffq/__init__.py -> build/lib.linux-x86_64-cpython-313/diffq
running egg_info
writing diffq.egg-info/PKG-INFO
writing dependency_links to diffq.egg-info/dependency_links.txt
writing requirements to diffq.egg-info/requires.txt
writing top-level names to diffq.egg-info/top_level.txt
reading manifest file 'diffq.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files found matching 'examples/cifar/outputs/**'
adding license file 'LICENSE'
writing manifest file 'diffq.egg-info/SOURCES.txt'
running build_ext
building 'diffq.bitpack' extension
creating build/temp.linux-x86_64-cpython-313
gcc -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -g -ffile-prefix-map=/home/Rocky/aur/python-diffq/src=/usr/src/debug/python-diffq -flto=auto -fPIC -I/usr/include/python3.13 -c bitpack.c -o build/temp.linux-x86_64-cpython-313/bitpack.o
gcc -shared -Wl,-O1 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,-z,pack-relative-relocs -flto=auto -Wl,-O1 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,-z,pack-relative-relocs -flto=auto -Wl,-O1 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,-z,pack-relative-relocs -flto=auto -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -g -ffile-prefix-map=/home/Rocky/aur/python-diffq/src=/usr/src/debug/python-diffq -flto=auto build/temp.linux-x86_64-cpython-313/bitpack.o -L/usr/lib -o build/lib.linux-x86_64-cpython-313/diffq/bitpack.cpython-313-x86_64-linux-gnu.so
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/diffq
copying build/lib.linux-x86_64-cpython-313/diffq/utils.py -> build/bdist.linux-x86_64/wheel/./diffq
copying build/lib.linux-x86_64-cpython-313/diffq/uniform.py -> build/bdist.linux-x86_64/wheel/./diffq
copying build/lib.linux-x86_64-cpython-313/diffq/torch_pack.py -> build/bdist.linux-x86_64/wheel/./diffq
copying build/lib.linux-x86_64-cpython-313/diffq/lsq.py -> build/bdist.linux-x86_64/wheel/./diffq
copying build/lib.linux-x86_64-cpython-313/diffq/ts_export.py -> build/bdist.linux-x86_64/wheel/./diffq
copying build/lib.linux-x86_64-cpython-313/diffq/diffq.py -> build/bdist.linux-x86_64/wheel/./diffq
copying build/lib.linux-x86_64-cpython-313/diffq/bitpack.cpython-313-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/wheel/./diffq
copying build/lib.linux-x86_64-cpython-313/diffq/base.py -> build/bdist.linux-x86_64/wheel/./diffq
copying build/lib.linux-x86_64-cpython-313/diffq/__init__.py -> build/bdist.linux-x86_64/wheel/./diffq
running install_egg_info
Copying diffq.egg-info to build/bdist.linux-x86_64/wheel/./diffq-0.2.4-py3.13.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/diffq-0.2.4.dist-info/WHEEL
creating '/home/Rocky/aur/python-diffq/src/diffq-0.2.4/dist/.tmp-mfxhz599/diffq-0.2.4-cp313-cp313-linux_x86_64.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'diffq/__init__.py'
adding 'diffq/base.py'
adding 'diffq/bitpack.cpython-313-x86_64-linux-gnu.so'
adding 'diffq/diffq.py'
adding 'diffq/lsq.py'
adding 'diffq/torch_pack.py'
adding 'diffq/ts_export.py'
adding 'diffq/uniform.py'
adding 'diffq/utils.py'
adding 'diffq-0.2.4.dist-info/licenses/LICENSE'
adding 'diffq-0.2.4.dist-info/METADATA'
adding 'diffq-0.2.4.dist-info/WHEEL'
adding 'diffq-0.2.4.dist-info/top_level.txt'
adding 'diffq-0.2.4.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Successfully built diffq-0.2.4-cp313-cp313-linux_x86_64.whl
Generating documentation
Warn: 'diffq' may refer to either the installed Python module or the local file/directory with the same name. pdoc will document the installed module, prepend './' to force documentation of the local file/directory.
- Module location: /home/Rocky/aur/python-diffq/src/diffq-0.2.4/tmp_install/usr/lib/python3.13/site-packages/diffq/__init__.py
- Local file/directory: /home/Rocky/aur/python-diffq/src/diffq-0.2.4/diffq
Warn: Error loading diffq:
Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/pdoc/extract.py", line 220, in load_module
    return importlib.import_module(module)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/usr/lib/python3.13/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 1026, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/home/Rocky/aur/python-diffq/src/diffq-0.2.4/tmp_install/usr/lib/python3.13/site-packages/diffq/__init__.py", line 22, in <module>
    from .uniform import UniformQuantizer
  File "/home/Rocky/aur/python-diffq/src/diffq-0.2.4/tmp_install/usr/lib/python3.13/site-packages/diffq/uniform.py", line 11, in <module>
    import torch
  File "/usr/lib/python3.13/site-packages/torch/__init__.py", line 409, in <module>
    from torch._C import *  # noqa: F403
    ^^^^^^^^^^^^^^^^^^^^^^
ImportError: libprotobuf.so.30.2.0: cannot open shared object file: No such file or directory

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/pdoc/extract.py", line 303, in walk_packages2
    module = load_module(mod.name)
  File "/usr/lib/python3.13/contextlib.py", line 85, in inner
    return func(*args, **kwds)
  File "/usr/lib/python3.13/site-packages/pdoc/extract.py", line 222, in load_module
    raise RuntimeError(f"Error importing {module}") from e
RuntimeError: Error importing diffq
(/usr/lib/python3.13/site-packages/pdoc/extract.py:305)
Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/pdoc/extract.py", line 220, in load_module
    return importlib.import_module(module)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/usr/lib/python3.13/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 1026, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/home/Rocky/aur/python-diffq/src/diffq-0.2.4/tmp_install/usr/lib/python3.13/site-packages/diffq/__init__.py", line 22, in <module>
    from .uniform import UniformQuantizer
  File "/home/Rocky/aur/python-diffq/src/diffq-0.2.4/tmp_install/usr/lib/python3.13/site-packages/diffq/uniform.py", line 11, in <module>
    import torch
  File "/usr/lib/python3.13/site-packages/torch/__init__.py", line 409, in <module>
    from torch._C import *  # noqa: F403
    ^^^^^^^^^^^^^^^^^^^^^^
ImportError: libprotobuf.so.30.2.0: cannot open shared object file: No such file or directory

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/bin/pdoc", line 8, in <module>
    sys.exit(cli())
             ~~~^^
  File "/usr/lib/python3.13/site-packages/pdoc/__main__.py", line 198, in cli
    pdoc.pdoc(
    ~~~~~~~~~^
        *opts.modules,
        ^^^^^^^^^^^^^^
        output_directory=opts.output_directory,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/lib/python3.13/site-packages/pdoc/__init__.py", line 526, in pdoc
    all_modules[module_name] = doc.Module.from_name(module_name)
                               ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/pdoc/doc.py", line 406, in from_name
    return cls(extract.load_module(name))
               ~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 85, in inner
    return func(*args, **kwds)
  File "/usr/lib/python3.13/site-packages/pdoc/extract.py", line 222, in load_module
    raise RuntimeError(f"Error importing {module}") from e
RuntimeError: Error importing diffq
==> ERROR: A failure occurred in build().
    Aborting...

Last edited by rocky (2025-05-27 13:53:33)

Offline

#6 2025-05-27 14:22:19

Scimmia
Fellow
Registered: 2012-09-01
Posts: 12,773

Re: [Solved] Protobuf problems

rocky wrote:
Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/pdoc/extract.py", line 220, in load_module
    return importlib.import_module(module)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/usr/lib/python3.13/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 1026, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/home/Rocky/aur/python-diffq/src/diffq-0.2.4/tmp_install/usr/lib/python3.13/site-packages/diffq/__init__.py", line 22, in <module>
    from .uniform import UniformQuantizer
  File "/home/Rocky/aur/python-diffq/src/diffq-0.2.4/tmp_install/usr/lib/python3.13/site-packages/diffq/uniform.py", line 11, in <module>
    import torch
  File "/usr/lib/python3.13/site-packages/torch/__init__.py", line 409, in <module>
    from torch._C import *  # noqa: F403
    ^^^^^^^^^^^^^^^^^^^^^^
ImportError: libprotobuf.so.30.2.0: cannot open shared object file: No such file or directory

So this is happening when importing torch. Sounds like a partial update, did you ignore updates to whatever torch package you're using?

And please, use code tags when pasting these things!

Offline

#7 2025-05-27 14:49:56

rocky
Member
Registered: 2025-01-13
Posts: 7

Re: [Solved] Protobuf problems

Scimmia wrote:
rocky wrote:
Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/pdoc/extract.py", line 220, in load_module
    return importlib.import_module(module)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/usr/lib/python3.13/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 1026, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/home/Rocky/aur/python-diffq/src/diffq-0.2.4/tmp_install/usr/lib/python3.13/site-packages/diffq/__init__.py", line 22, in <module>
    from .uniform import UniformQuantizer
  File "/home/Rocky/aur/python-diffq/src/diffq-0.2.4/tmp_install/usr/lib/python3.13/site-packages/diffq/uniform.py", line 11, in <module>
    import torch
  File "/usr/lib/python3.13/site-packages/torch/__init__.py", line 409, in <module>
    from torch._C import *  # noqa: F403
    ^^^^^^^^^^^^^^^^^^^^^^
ImportError: libprotobuf.so.30.2.0: cannot open shared object file: No such file or directory

So this is happening when importing torch. Sounds like a partial update, did you ignore updates to whatever torch package you're using?

And please, use code tags when pasting these things!

what code tags(im newbie)
i have pytorch updates lol

Offline

#8 2025-05-27 14:54:46

Scimmia
Fellow
Registered: 2012-09-01
Posts: 12,773

Re: [Solved] Protobuf problems

https://en.wikipedia.org/wiki/BBCode

What does `pacman -Qi pytorch` show?

Offline

#9 2025-05-27 14:56:16

rocky
Member
Registered: 2025-01-13
Posts: 7

Re: [Solved] Protobuf problems

Scimmia wrote:

https://en.wikipedia.org/wiki/BBCode

What does `pacman -Qi pytorch` show?

I updated the system and it works now. Thanks

Offline

#10 2025-05-28 11:03:43

Lone_Wolf
Administrator
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 13,792

Re: [Solved] Protobuf problems

Please prepend [Solved] to the thread title (edit first post)

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

clean chroot building not flexible enough ?
Try clean chroot manager by graysky

Offline

Pages: 1

Board footer

Powered by FluxBB