You are not logged in.

#1 2013-06-06 08:08:37

Darksoul71
Member
Registered: 2010-04-12
Posts: 319

youtube-dl called from python script fails...why ? [solved]

Hi there,

since downloading youtube stuff with Browser plugins has been pretty unstable for me lately (read: mostly incomplete videos) I switched over to youtube-dl again. Unfortunately youtube-dl often stops downloading as well...

I wrote a small Python script which looks for incomplete downloads in my Youtube folder and restarts the download again. The code is quite simple:

#!/usr/bin/python
# -*- coding: utf8 -*-

# Modules used:
import sys, os

part_files = os.popen ("ls *.part")

for part_file in part_files:
  target_name = part_file.strip()[:-5]
  target_name = target_name.split(".")[0]
  yt_id = target_name [len(target_name)-11:]
  cli = 'youtube-dl -t http://www.youtube.com/watch?v=' + yt_id
  print cli
  os.popen (cli)

Unfortunately youtube-dl bombs out with the following stack trace and I have got no clue why...

youtube-dl -t http://www.youtube.com/watch?v=XNVaEoQMarg
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/bin/youtube-dl/__main__.py", line 17, in <module>
  File "/usr/bin/youtube-dl/youtube_dl/__init__.py", line 516, in main
    
  File "/usr/bin/youtube-dl/youtube_dl/__init__.py", line 500, in _real_main
    
  File "/usr/bin/youtube-dl/youtube_dl/FileDownloader.py", line 507, in download
  File "/usr/bin/youtube-dl/youtube_dl/InfoExtractors.py", line 92, in extract
  File "/usr/bin/youtube-dl/youtube_dl/InfoExtractors.py", line 87, in initialize
  File "/usr/bin/youtube-dl/youtube_dl/InfoExtractors.py", line 313, in _real_initialize
  File "/usr/bin/youtube-dl/youtube_dl/InfoExtractors.py", line 197, in report_lang
  File "/usr/bin/youtube-dl/youtube_dl/FileDownloader.py", line 192, in to_screen
IOError: [Errno 32] Broken pipe

Does anybody have an idea why this happens ? Using youtube-dl without the "resume download" script works perfectly though...

TIA,
D$

Last edited by Darksoul71 (2013-06-06 11:00:51)


My archlinux x86_64 host:
AMD E350 (2x1.6GHz) / 8GB DDR3 RAM / GeForce 9500GT (passive) / Arch running from 16GB USB Stick

Offline

#2 2013-06-06 10:51:28

mychris
Member
From: Munich
Registered: 2012-09-15
Posts: 67

Re: youtube-dl called from python script fails...why ? [solved]

I think the problem is that you are using os.popen.
The function returns a file object to which the pipe of the new spawned process is connected. If you don't catch that file object, python might garbage collect it. (In addition, your script terminates, so the file object will be garbage collected).
Check out the subprocess module from the standard lib. I think there you can redirect the pipe of the new spawned processs to /dev/null or something.
http://docs.python.org/2/library/subprocess.html

Offline

#3 2013-06-06 10:57:58

Darksoul71
Member
Registered: 2010-04-12
Posts: 319

Re: youtube-dl called from python script fails...why ? [solved]

@mychris:
Thanks a lot ! Using the subprocess module fixed my issue.

The working code looks like this:

#!/usr/bin/python
# -*- coding: utf8 -*-

# Modules used:
import sys, os, subprocess

part_files = os.popen ("ls *.part")

for part_file in part_files:
  target_name = part_file.strip()[:-5]
  target_name = target_name.split(".")[0]
  yt_id = target_name [len(target_name)-11:]
  cli = 'youtube-dl -t http://www.youtube.com/watch?v=' + yt_id
  subprocess.call(cli, shell=True)

Perfect !


My archlinux x86_64 host:
AMD E350 (2x1.6GHz) / 8GB DDR3 RAM / GeForce 9500GT (passive) / Arch running from 16GB USB Stick

Offline

#4 2013-06-06 11:00:15

mychris
Member
From: Munich
Registered: 2012-09-15
Posts: 67

Re: youtube-dl called from python script fails...why ? [solved]

No problem.
Please mark the thread as [SOLVED].

Offline

#5 2013-06-06 11:11:59

Darksoul71
Member
Registered: 2010-04-12
Posts: 319

Re: youtube-dl called from python script fails...why ? [solved]

Done ! smile


My archlinux x86_64 host:
AMD E350 (2x1.6GHz) / 8GB DDR3 RAM / GeForce 9500GT (passive) / Arch running from 16GB USB Stick

Offline

#6 2013-06-06 11:16:50

WorMzy
Forum Moderator
From: Scotland
Registered: 2010-06-16
Posts: 5,155

Re: youtube-dl called from python script fails...why ? [solved]

Just a note: you may want to exchange the ambiguous shebang with the more portable '#!/usr/bin/env python3', or at least '#!/usr/bin/python3', if you ever want to share the script across multiple distros.


Sakura:-
Mobo: ASUS P8Z77-V PRO // Processor: Intel Core i7-3770K 3.4GHz // GFX: nVidia GeForce GTX 970 Ti // RAM: 32GB (4x 8GB) Corsair DDR3 (@ 2133MHz) // Storage: 1x 3TB Seagate SATAII 5x 1TB Samsung SATAII, 2x 120GB Corsair SSD

Offline

#7 2013-06-06 11:40:26

Darksoul71
Member
Registered: 2010-04-12
Posts: 319

Re: youtube-dl called from python script fails...why ? [solved]

Thanks for the hint !
I will consider this as change for my scripts. I am always happy to use Python because it often runs under other distros / platforms (e.g. Raspberry Pi).


My archlinux x86_64 host:
AMD E350 (2x1.6GHz) / 8GB DDR3 RAM / GeForce 9500GT (passive) / Arch running from 16GB USB Stick

Offline

#8 2013-06-07 10:01:27

whitie
Member
Registered: 2011-03-13
Posts: 16

Re: youtube-dl called from python script fails...why ? [solved]

Hello,
maybe a little bit more pythonic:

#!/usr/bin/env python3

import glob
import os
import subprocess


def main():
    part_files = glob.glob('./*.part')
    for part_file in part_files:
        target_name = os.path.splitext(part_file)[0]
        yt_id = target_name[len(target_name)-11:]
        cmd = ['youtube-dl', '-t',
               'http://www.youtube.com/watch?v={}'.format(yt_id)]
        subprocess.call(cmd)


if __name__ == '__main__':
    main()

Whitie

Offline

#9 2013-06-07 10:05:15

Darksoul71
Member
Registered: 2010-04-12
Posts: 319

Re: youtube-dl called from python script fails...why ? [solved]

Whitie,

thanks for posting ! I know that my scripts often look like a bad reinterpretation of shell script in Python smile
Honestly what counts for me is that things work. They do not have to be pretty or well implemented, but just work. big_smile

Regards,
D$


My archlinux x86_64 host:
AMD E350 (2x1.6GHz) / 8GB DDR3 RAM / GeForce 9500GT (passive) / Arch running from 16GB USB Stick

Offline

#10 2013-06-08 09:33:33

Mr.Elendig
#archlinux@freenode channel op
From: The intertubes
Registered: 2004-11-07
Posts: 3,721

Re: youtube-dl called from python script fails...why ? [solved]


Evil #archlinux@freenode channel op and general support dude.
. files on github, Screenshots, Random pics and the rest

Offline

#11 2013-06-09 09:31:11

Darksoul71
Member
Registered: 2010-04-12
Posts: 319

Re: youtube-dl called from python script fails...why ? [solved]

Thanks...one never stops to learn smile

I wasn't even aware of quvi.


My archlinux x86_64 host:
AMD E350 (2x1.6GHz) / 8GB DDR3 RAM / GeForce 9500GT (passive) / Arch running from 16GB USB Stick

Offline

Board footer

Powered by FluxBB