[SOLVED] Python pip version inside and outside of virtual environments

Ferdinand · 2023-08-04 15:44:30

I am a little confused regarding python pip versions.

These are my installed versions of python and pip (the whole system is updated with pacman -Syu a couple of hours ago):

$ pacman -F python | grep installed
core/python 3.11.3-2 [installed]
$ pacman -F pip | grep installed
extra/python-pip 23.2.1-1 [installed]
$ pacman -Qo python
/usr/bin/python is owned by python 3.11.3-2
$ pacman -Qo pip
/usr/bin/pip is owned by python-pip 23.2.1-1
$ python --version
Python 3.11.3
$ pip --version
pip 23.2.1 from /usr/lib/python3.11/site-packages/pip (python 3.11)

However, inside a virtual environment, this is the case:

$ python -m venv Test
$ cd Test
$ . bin/activate
(Test) $ pacman -F python | grep -i installed
core/python 3.11.3-2 [installed]
(Test) $ pacman -F pip | grep -i installed
extra/python-pip 23.2.1-1 [installed]
(Test) $ pacman -Qo python
error: No package owns /tmp/Test/bin/python
(Test) $ pacman -Qo pip
error: No package owns /tmp/Test/bin/pip
(Test) $ python --version
Python 3.11.3
(Test) $ pip --version
pip 22.3.1 from /tmp/Test/lib/python3.11/site-packages/pip (python 3.11)

Now, /usr/bin/pip is a short script that runs pip._internal.cli.main, and /tmp/Test/lib/python3.11/site-packages/pip (which I really thought was a copy of /usr/lib/python3.11/site-packages/pip), is also a script that runs pip._internal.cli.main. I copy them below, for reference:

$ cat /usr/lib/python3.11/site-packages/pip/__main__.py 
import os
import sys

# Remove '' and current working directory from the first entry
# of sys.path, if present to avoid using current directory
# in pip commands check, freeze, install, list and show,
# when invoked as python -m pip <command>
if sys.path[0] in ("", os.getcwd()):
    sys.path.pop(0)

# If we are running from a wheel, add the wheel to sys.path
# This allows the usage python pip-*.whl/pip install pip-*.whl
if __package__ == "":
    # __file__ is pip-*.whl/pip/__main__.py
    # first dirname call strips of '/__main__.py', second strips off '/pip'
    # Resulting path is the name of the wheel itself
    # Add that to sys.path so we can import pip
    path = os.path.dirname(os.path.dirname(__file__))
    sys.path.insert(0, path)

if __name__ == "__main__":
    from pip._internal.cli.main import main as _main

    sys.exit(_main())
$ 
$ cat usr/bin/pip
#!/usr/bin/python
# -*- coding: utf-8 -*-
import re
import sys
from pip._internal.cli.main import main
if __name__ == "__main__":
    sys.argv[0] = re.sub(r"(-script\.pyw|\.exe)?$", "", sys.argv[0])
    sys.exit(main())
$

Now the two scripts are different, and somewhere in there is probably a (n evident, I bet) solution - but I'll be dashed if I can see it

I'd really appreciate it if i kind soul can shed some light on this one for me

Edit:
Aw, just realised I copied /usr/lib/python3.11/site-packages/pip/__main__.py, above.
Here's /tmp/Test/lib/python3.11/site-packages/pip/__main__.py
They differ, but in the end they both still run pip._internal.cli.main?

(Test) $ cat lib/python3.11/site-packages/pip/__main__.py
import os
import sys
import warnings

# Remove '' and current working directory from the first entry
# of sys.path, if present to avoid using current directory
# in pip commands check, freeze, install, list and show,
# when invoked as python -m pip <command>
if sys.path[0] in ("", os.getcwd()):
    sys.path.pop(0)

# If we are running from a wheel, add the wheel to sys.path
# This allows the usage python pip-*.whl/pip install pip-*.whl
if __package__ == "":
    # __file__ is pip-*.whl/pip/__main__.py
    # first dirname call strips of '/__main__.py', second strips off '/pip'
    # Resulting path is the name of the wheel itself
    # Add that to sys.path so we can import pip
    path = os.path.dirname(os.path.dirname(__file__))
    sys.path.insert(0, path)

if __name__ == "__main__":
    # Work around the error reported in #9540, pending a proper fix.
    # Note: It is essential the warning filter is set *before* importing
    #       pip, as the deprecation happens at import time, not runtime.
    warnings.filterwarnings(
        "ignore", category=DeprecationWarning, module=".*packaging\\.version"
    )
    from pip._internal.cli.main import main as _main

    sys.exit(_main())

Last edited by Ferdinand (2023-08-05 15:20:06)

Trilby · 2023-08-04 16:27:55

You say that somewhere in those scripts there is probably a "solution" - but to what? What is the problem?

You led into that simply by showing that a virtual env works just as it's supposed to. So what are you trying to "solve"?

Ferdinand · 2023-08-05 05:53:30

Sorry for being vague, Trilby.

My problem is simply that I don't understand why the pip inside the virtual environment is a different version than the pip from /usr/bin, considering that they both comes from python-pip and both basically calls main in the pip._internal.cli.main module?

It's not important, I just got hung up on it when I discovered it, and couldn't figure it out

If you have an answer, I'm still curious, but I don't have an actual problem with anything outside my head

Edit: Spelling

Last edited by Ferdinand (2023-08-05 05:55:40)

Trilby · 2023-08-05 12:00:30

Ferdinand wrote:

... they both comes from python-pip

No, they don't. The one inside the virtual environment comes from the python package. You can remove python-pip and repeat your commands from post #1 and you'll see you'll have pip inside the virtual environment, but not outside of it.

Ferdinand wrote:

I don't understand why the pip inside the virtual environment is a different version

Being able to use different versions that the system packages is one of the primary reasons for virtual environments to exist in the first place. In this case, the difference is not specifically deliberate, but more just coincidental: there is no direct connection between the virtual-environment pip provided by the python package, and the system pip provided by the python-pip package. So there is no expectation that they'd be the same version (sometimes they will be by coincidence, and the fact that they both are updated fairly frequently should result in them being pretty close in version number, but there's no reason to expect them to be identical).

Ferdinand · 2023-08-05 14:04:38

Thank you, Trilby!

Your answer explains it well - although I'll have to study this more, because it's not at all evident to me that the python package contains pip anywhere (file list) - and it's contrary to my understanding of how the virtual environments work; I thought you'd start out with a copy of the system installed packages

Anyway - I know you know this stuff, so I'll consider this answered and mark it as solved, but keep my book open; it's clear that I haven't quite understood this yet Thank's again!

Last edited by Ferdinand (2023-08-05 14:06:16)

Arch Linux

#1 2023-08-04 15:44:30

[SOLVED] Python pip version inside and outside of virtual environments

#2 2023-08-04 16:27:55

Re: [SOLVED] Python pip version inside and outside of virtual environments

#3 2023-08-05 05:53:30

Re: [SOLVED] Python pip version inside and outside of virtual environments

#4 2023-08-05 12:00:30

Re: [SOLVED] Python pip version inside and outside of virtual environments

#5 2023-08-05 14:04:38

Re: [SOLVED] Python pip version inside and outside of virtual environments

Board footer