[SOLVED] FFMPEG appears to be trashing bash script variables

jwm-art · 2014-05-26 19:04:03

Hi,

I have a directory containg MP4 files which were downloaded via youtube-dl upon which I want to extract audio using FFMPEG.
I am using the usual find/while/read/do/done loop to process each file individually. The files were downloaded using Youtube-dl (invoked with --restrict-filenames) so the youtube video ID is recorded within the filename which can then be used to obtain the video title (ie to tag the audio with once extracted, ie ID3).

Without FFMPEG the loop works as expected as can be seen when using echo, but as soon as I introduce FFMPEG into the loop, no such file or directory errors are reported because the filenames passed to FFMPEG are nonsense and appear to have been trashed, for instance, characters are stripped from the start or end of the filename.

find . -name '*.mp4' | while read FILE
do
    FNAME=${FILE%.mp4}
    FID=${FNAME: -11}
    FTITLE=${FNAME%-$FID}
    TITLE=$(youtube-dl --get-title $FID)
#    MP3="${FNAME}.mp3"
    AAC="${FNAME}.aac"
    ffmpeg -i $FILE -vn -acodec copy $AAC

done

Frequently green text like so reams down the terminal:

stream #1:
  keyframe=1
  duration=0.023
  dts=355.033  pts=355.033
  size=252

Swiftly followed by:

00000000  21 0a 4f ff ff e0 3f fe a2 c6 b1 41 31 a0 88 41 !.O...?....A1..A
00000010  28 3e 00 03 d2 15 25 4a 92 17 40 0b e4 f5 4c e5 (>....%J..@...L.
00000020  af 56 45 aa 31 0c eb 87 45 ac 26 54 8a ed c9 b4 .VE.1...E.&T....
00000030  94 98 ca a8 03 34 aa bf 04 51 38 12 c9 43 80 62 .....4...Q8..C.b
00000040  5b 16 94 99 20 18 9e cd 55 d0 6c 15 38 13 ce cc [... ...U.l.8...
00000050  d4 2e 80 3b 10 e9 07 70 bc 23 8c 8c 84 96 8a 38 ...;...p.#.....8
00000060  29 63 8d 87 75 5e 66 dc fa de 7f 4e 70 b2 44 09 )c..u^f....Np.D.
00000070  39 c7 30 50 78 2b 6a b0 48 90 ba c4 14 c4 41 5c 9.0Px+j.H.....A\
00000080  b6 3c 64 38 4d 92 3a b6 d1 01 d3 99 86 d5 54 23 .<d8M.:.......T#
00000090  2a c4 13 49 61 50 b7 d0 3c 1e 2b 4b 90 c4 32 ae *..IaP..<.+K..2.
000000a0  bc 6b 57 a6 de df 19 53 8e 38 66 d1 c4 fc a4 d5 .kW....S.8f.....
000000b0  f9 6f a9 2d 20 6c eb 29 1e 1a ef 96 a6 84 82 63 .o.- l.).......c
000000c0  0a cc 75 82 14 b4 08 09 fa 11 b8 0d e7 11 80 84 ..u.............
000000d0  20 31 18 00 05 2a 54 af 46 ac 08 79 b9 a8 11 70  1...*T.F..y...p
000000e0  f5 c0 2f 88 4b 27 7c a4 fd f9 f1 e4 77 53 c5 4a ../.K'|.....wS.J
000000f0  d7 74 d1 02 80 46 40 08 16 a1 e7                .t...F@....

It doesn't matter if the filenames originate from find or redirected from the contents of a text file. Anyone have any suggestions for workarounds?

Last edited by jwm-art (2014-05-27 21:39:39)

Trilby · 2014-05-26 19:13:48

Have you tried quoting the variables in the ffmpeg command (this is a good idea regardless of whether it is the solution).

jwm-art · 2014-05-26 21:10:29

Trilby wrote:

Have you tried quoting the variables in the ffmpeg command (this is a good idea regardless of whether it is the solution).

Yes tried, no difference. Often requires `pkill -9 ffmpeg` to break out of the hexdump-like output pouring forth.

jwm-art · 2014-05-27 19:49:03

Can someone try and reproduce this please? Here is a self contained bash script to do so:
(note: nothing offensive in the videos, though music may not be your taste.)

edit: sorry code had problems... please wait...

Last edited by jwm-art (2014-05-27 19:51:34)

Trilby · 2014-05-27 19:56:24

I'll see if I can try it when it gets posted.

But have you also just ran the loop echoing the full ffmpeg command that *would have* executed. You can then take those individual ffmpeg commands to see how many of them are causing the problem - this might reveal a pattern among the problematic commands.

jwm-art · 2014-05-27 20:06:26

Self contained example:

#!/bin/bash

while read URL
do
    echo -n "Downloading video from ${URL}... "

    youtube-dl --restrict-filenames "${URL}" --quiet

    if test $? -eq 0; then
        echo "Ok"
    else
        echo "FAIL"
        echo "Aborting..."
        exit
    fi
done <<YT_URLS
    https://www.youtube.com/watch?v=XqdYnxv01yM
    https://www.youtube.com/watch?v=lX_o5t2YoUE
    https://www.youtube.com/watch?v=V9sI6VEDE5M
YT_URLS

find . -name '*.mp4' | while read -r FILE
do
    echo "-----------------------------------------"
    echo -n "Converting ${FILE} to MP3... "
    MP3="${FILE%.mp4}.mp3"
    echo "${MP3}"
    ffmpeg -loglevel warning -y -i "${FILE}" "${MP3}"

    if test $? -eq 0; then
        echo "Ok"
    else
        echo "FAIL"
        echo "Abortingm..."
        exit
    fi
    echo

done

Here's the output on my system:

$ ../ffmpeg_test 
Downloading video from https://www.youtube.com/watch?v=XqdYnxv01yM... Ok
Downloading video from https://www.youtube.com/watch?v=lX_o5t2YoUE... Ok
Downloading video from https://www.youtube.com/watch?v=V9sI6VEDE5M... Ok
-----------------------------------------
Converting ./Rabbit_City_White_Lable_-_Beyond_Control-lX_o5t2YoUE.mp4 to MP3... ./Rabbit_City_White_Lable_-_Beyond_Control-lX_o5t2YoUE.mp3

Enter command: <target>|all <time>|-1 <command>[ <argument>]
Parse error, at least 3 arguments were expected, only 1 given in string 'osmo_Dibs_-_Xultation-V9sI6VEDE5M.mp4'
debug=1
debug=2  1984kB time=00:02:06.95 bitrate= 128.0kbits/s    
error parsing debug value
debug=0
[output stream 0:0 @ 0xd90b60] EOF on sink link output stream 0:0:default.
No more output streams to write to, finishing.
[libmp3lame @ 0xc71de0] Trying to remove 815 more samples than there are in the queue
size=    6971kB time=00:07:26.12 bitrate= 128.0kbits/s    
video:0kB audio:6971kB subtitle:0 data:0 global headers:0kB muxing overhead 0.004483%
19212 frames successfully decoded, 0 decoding errors
[AVIOContext @ 0xd9ff60] Statistics: 2 seeks, 17081 writeouts
[AVIOContext @ 0xc77200] Statistics: 7082450 bytes read, 176 seeks
Ok

The first MP4 file found is converted, but alongside strange unexplained output, and then it just stops. In the first iteration of the second loop how does FFMPEG get hold of the second filename (incomplete, missing first character)? What is the "Enter command" all about? Its all very weird.

Last edited by jwm-art (2014-05-27 20:17:57)

jwm-art · 2014-05-27 20:22:21

Hi Trilby,
I've run the script to echo the ffmpeg commands and then ran them outside of the script and there is no problem whatsoever, no strange output, it works as expected. The problem seems to arise as soon as ffmpeg is put within a bash script.
Cheers,
James.

Trilby · 2014-05-27 20:26:22

That is odd. I can confirm the issue. When I put just an "echo" before the ffmpeg command I can copy the command that is echoed and execute it without issue. But in the script I get the same result you do.

EDIT: I've done some tinkering with this, and I'm baffled. Sorry.

Last edited by Trilby (2014-05-27 21:06:36)

jwm-art · 2014-05-27 21:24:22

Apparently, ffmpeg reads from stdin, this is the problem.

Fixed by appending the command with `< /dev/null` or `-nostdin`

See:
https://lists.ffmpeg.org/pipermail/ffmp … 21179.html

Arch Linux

#1 2014-05-26 19:04:03

[SOLVED] FFMPEG appears to be trashing bash script variables

#2 2014-05-26 19:13:48

Re: [SOLVED] FFMPEG appears to be trashing bash script variables

#3 2014-05-26 21:10:29

Re: [SOLVED] FFMPEG appears to be trashing bash script variables

#4 2014-05-27 19:49:03

Re: [SOLVED] FFMPEG appears to be trashing bash script variables

#5 2014-05-27 19:56:24

Re: [SOLVED] FFMPEG appears to be trashing bash script variables

#6 2014-05-27 20:06:26

Re: [SOLVED] FFMPEG appears to be trashing bash script variables

#7 2014-05-27 20:22:21

Re: [SOLVED] FFMPEG appears to be trashing bash script variables

#8 2014-05-27 20:26:22

Re: [SOLVED] FFMPEG appears to be trashing bash script variables

#9 2014-05-27 21:24:22

Re: [SOLVED] FFMPEG appears to be trashing bash script variables

Board footer