[SOLVED] Python and related packages take long for web requests

gary8588 · 2026-01-13 19:44:24

Hello,

facing a very strange issue since today.

I noticed that yt-dlp took a long time (1-2 minutes) to display all available formats for a video. Then I tried to watch a stream on twitch via mpv (using the yt-dlp plugin) and that also took 1-2 minutes to open the stream. I then tried to get to the bottom of it and it looks like all web requests made by python programs or scripts hang those initial 1-2 minutes before responding. It is very easily reproducible by me:

$ python
Python 3.14.2 (main, Jan  2 2026, 14:27:39) [GCC 15.2.1 20251112] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.get("https://archlinux.org").text

This will hang for about 2 minutes and then respond with whatever the page is.

If I do the same web requests just via curl, this responds fast, almost instantly:

$ curl https://archlinux.org
<!DOCTYPE html>

<html lang="en">
<head>
    <meta charset="utf-8" />
    <meta name="theme-color" content="#08C" />
    <title>Arch Linux</title>
    <link rel="stylesheet" type="text/css" href="/static/archweb.css" media="screen" />
    <link rel="icon" type="image/png" href="/static/favicon.png" />
    <link rel="shortcut icon" type="image/png" href="/static/favicon.png" />
    <link rel="apple-touch-icon" href="/static/logos/apple-touch-icon-57x57.png" />
    <link rel="apple-touch-icon" sizes="72x72" href="/static/logos/apple-touch-icon-72x72.png" />
    <link rel="apple-touch-icon" sizes="114x114" href="/static/logos/apple-touch-icon-114x114.png" />
    ...
    ...
    ...

Anyone else running into this issue? Can reproduce on 2 different Computers. Updated Arch Linux right before making this post. I noticed that python packages were rebuilt due to python 3.14 but issue was also present on python 3.13

Last edited by gary8588 (2026-01-14 18:27:41)

astralc · 2026-01-13 20:09:31

what about

import urllib.request
urllib.request.urlopen('https://archlinux.org').read()

you can use "python -v" to have more verbose

gary8588 · 2026-01-13 20:20:19

Also slow (about 2 minutes) until the response appears unfortunately.

verbose output looks normal to me:

>>> urllib.request.urlopen("https://archlinux.org").read()
# /usr/lib/python3.14/encodings/__pycache__/idna.cpython-314.pyc matches /usr/lib/python3.14/encodings/idna.py
# code object from '/usr/lib/python3.14/encodings/__pycache__/idna.cpython-314.pyc'
# /usr/lib/python3.14/__pycache__/stringprep.cpython-314.pyc matches /usr/lib/python3.14/stringprep.py
# code object from '/usr/lib/python3.14/__pycache__/stringprep.cpython-314.pyc'

~2 minute hang here and then the response from archlinux.org:

b'<!DOCTYPE html>\n\n<html lang="en">\n<head>\n    <meta charset="utf-8" />\n    <meta name="theme-color" content="#08C" />\n    <title>Arch Linux</title>\n    <link rel="stylesheet" type="text/css" href="/static/archweb.css" media="screen" />\n    <link rel="icon" type="image/png" href="/static/favicon.png" />\n    <link rel="shortcut icon" type="image/png" href="/static/favicon.png" />\n    <l
...
...
...

seth · 2026-01-13 20:32:06

2 minutes would be a kernel timeout - do you get anything in the system journal or dmesg from those requests?
Does "requests.get("https://8.8.8.8").text" also timeout?

gary8588 · 2026-01-13 20:45:07

Nothing in journalctl or dmesg, had it open side by side while testing.

requests.get("https://8.8.8.8").text same behaviour. I do get the response eventually but only after about 2 minutes.

Found out that opening a socket manually works instantly:

$ python
Python 3.14.2 (main, Jan  2 2026, 14:27:39) [GCC 15.2.1 20251112] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> sock = socket.socket()
>>> sock.connect(("archlinux.org", 443))
>>> sock.send(b'test')
4
>>> sock.recv(200)
b'HTTP/1.1 400 Bad Request\r\nServer: nginx\r\nDate: Tue, 13 Jan 2026 20:42:54 GMT\r\nContent-Type: text/html\r\nContent-Length: 150\r\nConnection: close\r\n\r\n<html>\r\n<head><title>400 Bad Request</title></head>\r\n<b'
>>>

This returns the response from archlinux.org instantly. hmmm

seth · 2026-01-13 21:04:34

https://stackoverflow.com/questions/760 … s-on-linux

gary8588 · 2026-01-13 21:29:29

Looks like I am well within any limits

$ ss -s
Total: 657
TCP:   22 (estab 13, closed 3, orphaned 0, timewait 2)

Transport Total     IP        IPv6
RAW	  1         0         1        
UDP	  16        7         9        
TCP	  19        17        2        
INET	  36        24        12       
FRAG	  0         0         0

haven't needed to touch ulimits since installing so it is all default:

$ ulimit -a
real-time non-blocking time  (microseconds, -R) unlimited
core file size              (blocks, -c) unlimited
data seg size               (kbytes, -d) unlimited
scheduling priority                 (-e) 0
file size                   (blocks, -f) unlimited
pending signals                     (-i) 63210
max locked memory           (kbytes, -l) 8192
max memory size             (kbytes, -m) unlimited
open files                          (-n) 1024
pipe size                (512 bytes, -p) 8
POSIX message queues         (bytes, -q) 819200
real-time priority                  (-r) 0
stack size                  (kbytes, -s) 8192
cpu time                   (seconds, -t) unlimited
max user processes                  (-u) 63210
virtual memory              (kbytes, -v) unlimited
file locks                          (-x) unlimited

since curl or firefox or other non python programs return web requests fast I can only imagine something to be wrong on the python side

Lone_Wolf · 2026-01-14 12:25:47

Several deprecated options have been dropped from urllib with python 3.14 , maybe that's the cause ?
see https://docs.python.org/3/whatsnew/3.14.html#id12

seth · 2026-01-14 12:34:07

OP wrote:

issue was also present on python 3.13

Broadsword time:

strace -tt -f -o /tmp/python.strace python -c 'import requests; requests.get("https://archlinux.org").text'

gary8588 · 2026-01-14 17:57:49

Here you go: https://0x0.st/P8mO.strace

expires in 24h. Tried to redact some private information in the trace, please let me know if I shared private information and I will delete.

The relevant part being:

5494  18:02:23.735276 setsockopt(3, SOL_TCP, TCP_NODELAY, [1], 4) = 0
5494  18:02:23.735301 ioctl(3, FIONBIO, [0]) = 0
5494  18:02:23.735324 connect(3, {sa_family=AF_INET6, sin6_port=htons(443), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "2604:cac0:a104:d::3", &sin6_addr), sin6_scope_id=0}, 28) = -1 ETIMEDOUT (Connection timed out)
5494  18:04:36.879818 close(3)          = 0
5494  18:04:36.879941 socket(AF_INET, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_TCP) = 3
5494  18:04:36.879993 setsockopt(3, SOL_TCP, TCP_NODELAY, [1], 4) = 0
5494  18:04:36.880032 ioctl(3, FIONBIO, [0]) = 0
5494  18:04:36.880065 connect(3, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("209.126.35.79")}, 16) = 0

Looks definitely like a timeout. but why?
2604:cac0:a104:d::3 seems to be the HAproxy serving archlinux.org. So the IPv6 connection fails? Do I read this correctly?
After the timeout there is another connect attempt using the IPv4 address (209.126.35.79) and this succeeds?

EDIT: Yep its IPv6 connections that hang:

$ python
Python 3.14.2 (main, Jan  2 2026, 14:27:39) [GCC 15.2.1 20251112] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> sock = socket.socket(socket.AF_INET6, socket.SOCK_STREAM, 0)
>>> sock.connect(("2604:cac0:a104:d::3", 443))
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    sock.connect(("2604:cac0:a104:d::3", 443))
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TimeoutError: [Errno 110] Connection timed out
>>>

Provider issue then?

Last edited by gary8588 (2026-01-14 18:09:37)

seth · 2026-01-14 18:11:37

Yup, it's IPv6
Try to https://wiki.archlinux.org/title/IPv6#Disable_IPv6 and see whether the problem remains

Before that check

curl -svL https://archlinux.org > /dev/null

Edit: can you "ping -6 2001:4860:4860::8888" ?

Last edited by seth (2026-01-14 18:12:58)

gary8588 · 2026-01-14 18:27:15

curl seems to take the IPv4 route:

$ curl -svL https://archlinux.org > /dev/null
* Host archlinux.org:443 was resolved.
* IPv6: 2604:cac0:a104:d::3
* IPv4: 209.126.35.79
*   Trying [2604:cac0:a104:d::3]:443...
*   Trying 209.126.35.79:443...
* ALPN: curl offers h2,http/1.1
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [1557 bytes data]
* SSL Trust Anchors:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS change cipher, Change cipher spec (1):
{ [1 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [19 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [2040 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [80 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [52 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [52 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / x25519 / id-ecPublicKey
* ALPN: server accepted h2
* Server certificate:
*   subject: CN=archlinux.org
*   start date: Dec 14 12:56:31 2025 GMT
*   expire date: Mar 14 12:56:30 2026 GMT
*   issuer: C=US; O=Let's Encrypt; CN=E7
*   Certificate level 0: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using ecdsa-with-SHA384
*   Certificate level 1: Public key type EC/secp384r1 (384/192 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha256WithRSAEncryption
*   subjectAltName: "archlinux.org" matches cert's "archlinux.org"
* SSL certificate verified via OpenSSL.
* Established connection to archlinux.org (209.126.35.79 port 443) from 192.168.178.20 port 48272 
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://archlinux.org/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: archlinux.org]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.18.0]
* [HTTP/2] [1] [accept: */*]
} [5 bytes data]
> GET / HTTP/2
> Host: archlinux.org
> User-Agent: curl/8.18.0
> Accept: */*
> 
* Request completely sent off
} [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [265 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [265 bytes data]
< HTTP/2 200 
< server: nginx
< date: Wed, 14 Jan 2026 18:20:46 GMT
< content-type: text/html; charset=utf-8
< content-length: 25228
< cache-control: max-age=307
< etag: "3cdac82154b8a2e5daa219fd6f730862"
< x-content-type-options: nosniff
< referrer-policy: strict-origin
< cross-origin-opener-policy: same-origin
< x-frame-options: DENY
< vary: Cookie
< strict-transport-security: max-age=31536000; includeSubdomains; preload
< x-cache-status: HIT
< 
{ [5 bytes data]
* Connection #0 to host archlinux.org:443 left intact

seth wrote:

can you "ping -6 2001:4860:4860::8888" ?

Nope, just hangs:

$ ping -6 2001:4860:4860::8888
PING 2001:4860:4860::8888 (2001:4860:4860::8888) 56 data bytes

added "ipv6.disable=1" to kernel parameters, rebooted and things work fast again! Thanks for debugging this.

seth · 2026-01-14 18:29:57

It's messy but the broadsword never fails the job

gary8588 · 2026-01-14 18:33:28

Hehe yes thanks a lot, broadsword now in my inventory as well

Arch Linux

#1 2026-01-13 19:44:24

[SOLVED] Python and related packages take long for web requests

#2 2026-01-13 20:09:31

Re: [SOLVED] Python and related packages take long for web requests

#3 2026-01-13 20:20:19

Re: [SOLVED] Python and related packages take long for web requests

#4 2026-01-13 20:32:06

Re: [SOLVED] Python and related packages take long for web requests

#5 2026-01-13 20:45:07

Re: [SOLVED] Python and related packages take long for web requests

#6 2026-01-13 21:04:34

Re: [SOLVED] Python and related packages take long for web requests

#7 2026-01-13 21:29:29

Re: [SOLVED] Python and related packages take long for web requests

#8 2026-01-14 12:25:47

Re: [SOLVED] Python and related packages take long for web requests

#9 2026-01-14 12:34:07

Re: [SOLVED] Python and related packages take long for web requests

#10 2026-01-14 17:57:49

Re: [SOLVED] Python and related packages take long for web requests

#11 2026-01-14 18:11:37

Re: [SOLVED] Python and related packages take long for web requests

#12 2026-01-14 18:27:15

Re: [SOLVED] Python and related packages take long for web requests

#13 2026-01-14 18:29:57

Re: [SOLVED] Python and related packages take long for web requests

#14 2026-01-14 18:33:28

Re: [SOLVED] Python and related packages take long for web requests

Board footer