[SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

GOKOP · 2019-04-08 15:54:28

[ Full previous title: youtube-dl - "Can't download anything, stuck on invoking downloader" ]

I can't download anything from Youtube with youtube-dl.
When I run it with -v flag, this is what it shows:

gokop@demunarch ~$ youtube-dl -v https://www.youtube.com/watch\?v\=N6hVmn9FM7o
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.youtube.com/watch?v=N6hVmn9FM7o']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.03.18
[debug] Python version 3.7.3 (CPython) - Linux-5.0.5-arch1-1-ARCH-x86_64-with-arch
[debug] exe versions: ffmpeg 4.1.2, ffprobe 4.1.2
[debug] Proxy map: {}
[youtube] N6hVmn9FM7o: Downloading webpage
[youtube] N6hVmn9FM7o: Downloading video info webpage
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on 'https://r5---sn-u2oxu-ajwz.googlevideo.com/videoplayback?key=yt6&dur=25.592&source=youtube&lmt=1550709814240786&pl=38&mime=video%2Fmp4&txp=5535432&expire=1554760042&initcwndbps=682500&clen=5953150&mn=sn-u2oxu-ajwz%2Csn-i5heen7s&mm=31%2C29&keepalive=yes&sparams=aitags%2Cclen%2Cdur%2Cei%2Cgir%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Ckeepalive%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Crequiressl%2Csource%2Cexpire&ip=2a01%3A117f%3A511%3Ad600%3A872%3A6163%3A3a52%3Ae1c1&mv=m&mt=1554738313&ms=au%2Crdu&itag=137&ei=Cm2rXJLaD8mZ1gLI95bQAw&requiressl=yes&ipbits=0&signature=34DBFD0D1F89FDB406ACD7497AC7656C0BE38510.D22D664C593284641A350030F08A262674F28089&gir=yes&fvip=5&c=WEB&id=o-AKgezRKkQgQt1PP8SuP-k3XnapXLfIrdcuQKftDpJPD9&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C278&ratebypass=yes'

It never gets past that point. Showed url also doesn't respond in browser. (gets stuck on TLS negotation)
I'm able to ping "r5---sn-u2oxu-ajwz.googlevideo.com".
Any ideas?

Last edited by GOKOP (2019-04-09 15:29:01)

archimboldo · 2019-04-08 17:30:44

rtmpdump is installed?

GOKOP · 2019-04-08 17:46:18

It wasn't, but after I installed it the same thing is happening

archimboldo · 2019-04-08 18:04:04

Hmm, user config seems empty. What's the content of .config/youtube-dl/config?

GOKOP · 2019-04-08 18:09:53

There's no such file. So as I understand I need to specify something in the config for youtube-dl to work?

archimboldo · 2019-04-08 18:14:46

You can try to create a config file with this content

-o ~/Downloads/%(title)s-%(id)s.%(ext)s

GOKOP · 2019-04-08 18:31:43

Didn't help unfortunately.

I have to add one thing though - it appears that the behavior changes with different videos - most ones I've tried behave like in the OP, but for example, this video: https://www.youtube.com/watch?v=CH3hdCP3yL4
It starts the download, and network traffic indicates that it's actually downloading something (in a very spiky manner) but then stops, and the process never appears to be finished. Also the .part file doesn't get bigger than 0B and .ytdl file doesn't get bigger than 50B.

gokop@demunarch ~$ youtube-dl -v https://www.youtube.com/watch\?v\=CH3hdCP3yL4        
[debug] System config: []
[debug] User config: ['-o', '~/Downloads/%(title)s-%(id)s.%(ext)s']
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.youtube.com/watch?v=CH3hdCP3yL4']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.03.18
[debug] Python version 3.7.3 (CPython) - Linux-5.0.5-arch1-1-ARCH-x86_64-with-arch
[debug] exe versions: ffmpeg 4.1.2, ffprobe 4.1.2, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] CH3hdCP3yL4: Downloading webpage
[youtube] CH3hdCP3yL4: Downloading video info webpage
[youtube] CH3hdCP3yL4: Downloading MPD manifest
[debug] Default format spec: bestvideo+bestaudio/best
WARNING: Requested formats are incompatible for merge and will be merged into mkv.
[debug] Invoking downloader on 'https://manifest.googlevideo.com/api/manifest/dash/mm/31%2C29/mn/sn-u2oxu-ajwz%2Csn-i5heen7s/mv/m/id/087de17423f7c8be/pl/38/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/itag/0/ip/2a01%3A117f%3A511%3Ad600%3A872%3A6163%3A3a52%3Ae1c1/requiressl/yes/mt/1554747687/playback_host/r5---sn-u2oxu-ajwz.googlevideo.com/sparams/as%2Cei%2Chfr%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cplayback_host%2Crequiressl%2Csource%2Cexpire/signature/2E7AFD6B02FE3F6C26827B80B4F706FBB064A685.A7F3CA108F671E42942B7E2F841549509ED82DFC/key/yt6/ipbits/0/hfr/all/expire/1554769368/ei/eJGrXP-gIM2P-gaQn72IAw/fvip/5/initcwndbps/608750/ms/au%2Crdu/source/youtube'
[dashsegments] Total fragments: 118
[download] Destination: /home/gokop/Downloads/Song Loop (Cats on Mars)-CH3hdCP3yL4.f247.webm

Same was happening before adding that output file pattern to the config so I don't think that was the problem

archimboldo · 2019-04-08 18:35:01

youtube-dl -f 22 https://www.youtube.com/watch?v=CH3hdCP3yL4?

GOKOP · 2019-04-08 18:39:57

Hmm, interesting. Now it doesn't even start the download just like everything else but there's additional step of downloading MPD manifest.

seth · 2019-04-08 21:09:42

youtube-dl  --external-downloader curl --external-downloader-args -v 'https://www.youtube.com/watch?v=N6hVmn9FM7o'

Post the output.

@archimboldo, why would he need rtmpdump? It's http.

GOKOP · 2019-04-08 21:13:32

gokop@demunarch ~$ youtube-dl  --external-downloader curl --external-downloader-args -v 'https://www.youtube.com/watch?v=N6hVmn9FM7o'
[youtube] N6hVmn9FM7o: Downloading webpage
[youtube] N6hVmn9FM7o: Downloading video info webpage
[download] Destination: /home/gokop/Downloads/Farewell.-N6hVmn9FM7o.f137.mp4
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 2a00:f46:0:a301::10...
* TCP_NODELAY set
* Connected to r5---sn-u2oxu-ajwz.googlevideo.com (2a00:f46:0:a301::10) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
  0     0    0     0    0     0      0      0 --:--:--  0:01:05 --:--:--     0

Counts time but nothing is being downloaded

seth · 2019-04-08 21:19:25

youtube-dl  --external-downloader curl --external-downloader-args "-v --tls-max 1.2" 'https://www.youtube.com/watch?v=N6hVmn9FM7o'

Edit: parameter requires quotes.

Last edited by seth (2019-04-08 21:19:51)

GOKOP · 2019-04-08 21:21:43

Same thing happens

seth · 2019-04-08 21:23:01

Did the TLS version drop to 1.2?

GOKOP · 2019-04-08 21:38:54

Oh yeah I didn't notice that, it did.

seth · 2019-04-08 21:47:24

No problem, just to ensure it's not due to TLS 1.3 issues.

Are you behind some firewall or anything else that might tamper w/ the TLS hello?
Can you watch youtube in a browser?

GOKOP · 2019-04-08 21:52:23

Yes, youtube in browser works fine so I don't think firewall is making problems

Lone_Wolf · 2019-04-09 10:01:31

Linux-5.0.5-arch1-1-ARCH-x86_64-with-arch

That url works fine here with linux 5..0.7 / fully updated archlinux.
Do things improve after you run pacman -Syu ?

GOKOP · 2019-04-09 11:15:04

Nope, still the same thing

Ropid · 2019-04-09 11:38:26

Can you go watch a video on youtube.com in your web browser, and then do right-click -> "stats for nerds" and look for this line here:

Codecs  vp09.00.51.08.01.01.01.01 (244) / opus (251)

Then go and run "youtube-dl -F ..." with the URL and see what that output is? Maybe your browser is not using the thing that would be seen as "best" video+audio stream by youtube-dl.

In that example video where I got that "Codecs ..." line from, downloading the same video and audio streams that the browser is using would be done like this with youtube-dl:

youtube -f 244+251 https://www.youtube.com/...

See if youtube-dl manages to download stuff if you use the same streams that the browser uses.

GOKOP · 2019-04-09 12:04:14

No luck here either. Browser uses 247+251 and that gives me the same result.

seth · 2019-04-09 13:27:50

There's some issue w/ the TLS negotiation and it's apparently not limited to TLSv1.3 (feel free to go down to 1.1)
It's not a problem here (or apparently for anybody else) so my best guess remains that some firewall or ISP filter messes around.

a) Are you behind a (corporate) firewall?
b) W/o going into details: is your geolocation on the more assholery side when it comes to "what our subjects may know"?

GOKOP · 2019-04-09 13:40:29

I'm behind firewall of my home router and I might be behind ISP's NAT, but I'm not sure. As for geolocation, I'm not sure what you mean

seth · 2019-04-09 14:21:32

I'm not sure what you mean

Are you in restrictive country, known to notoriously filter your traffic?

Is your home routers FW actually a firewall or just NAT?

Here're test results for the google CDN server:
https://www.ssllabs.com/ssltest/analyze … 34.214.208

Random guess (hunch, actually makes no sense at all):

cat /proc/sys/kernel/random/entropy_avail

https://wiki.archlinux.org/index.php/Haveged

You could also wireshark the connection to inspect the actual client_hello package.

GOKOP · 2019-04-09 15:27:25

Hmm, ok, that's interesting. I don't live in a country that blocks traffic in any way (except for illegal hazard websites but on DNS level) however I know my ISP blocks russia today (rt.com) for whatever reason so I tried to use youtube-dl through VPN in US and guess what, it worked.
I'm not sure if ISP is somehow blocking me here or is it some other region-related connection problem but I don't mind using VPN to download stuff from YouTube so I guess my problem is solved.

Arch Linux

#1 2019-04-08 15:54:28

[SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#2 2019-04-08 17:30:44

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#3 2019-04-08 17:46:18

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#4 2019-04-08 18:04:04

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#5 2019-04-08 18:09:53

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#6 2019-04-08 18:14:46

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#7 2019-04-08 18:31:43

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#8 2019-04-08 18:35:01

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#9 2019-04-08 18:39:57

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#10 2019-04-08 21:09:42

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#11 2019-04-08 21:13:32

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#12 2019-04-08 21:19:25

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#13 2019-04-08 21:21:43

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#14 2019-04-08 21:23:01

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#15 2019-04-08 21:38:54

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#16 2019-04-08 21:47:24

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#17 2019-04-08 21:52:23

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#18 2019-04-09 10:01:31

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#19 2019-04-09 11:15:04

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#20 2019-04-09 11:38:26

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#21 2019-04-09 12:04:14

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#22 2019-04-09 13:27:50

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#23 2019-04-09 13:40:29

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#24 2019-04-09 14:21:32

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

#25 2019-04-09 15:27:25

Re: [SOLVED] youtube-dl - Can't download anything, stuck on invoking do...

Board footer