Game recording issues with ffmpeg

SuperV1234 · 2013-07-22 08:33:23

Hello, I'm trying to create a script to record windowed games with ffmpeg.

This is what I have so far:

#!/bin/sh 
 
INFO=$(xwininfo -frame)
 
WIN_GEO=$(echo $INFO | grep -oEe 'geometry [0-9]+x[0-9]+' | grep -oEe '[0-9]+x[0-9]+')
WIN_XY=$(echo $INFO | grep -oEe 'Corners:\s+\+[0-9]+\+[0-9]+' | grep -oEe '[0-9]+\+[0-9]+' | sed -e 's/+/,/' )
 
ffmpeg -f x11grab -y -r 60 -s $WIN_GEO -i :0.0+$WIN_XY -f alsa -i pulse -crf 18 -preset ultrafast -threads 8 -async 5000 $1.mkv

It allows me to click on a window and record it - it works, but has some issues:

> I manually have to redirect the game audio with pavucontrol to be able to record it
> Audio gets out of sync, I tried many options with no luck
> I can't find a way to record both game audio and microphone audio

So,

> Could the script be improved somehow for better quality/performance?
> How can I prevent the audio going out of sync?
> How can I record both game video, game audio and microphone audio at the same time?

Thanks.

skottish · 2013-07-22 15:59:52

The synchronization issue may because of the CRF value. On one hand, you're using ultrafast as a setting. On the other, your CRF value is low enough where it requires significant processing. Maybe recording losslessly on the first run and re-encoding afterwards can fix this?

Max-P · 2013-07-22 19:14:11

SuperV1234 wrote:

> I manually have to redirect the game audio with pavucontrol to be able to record it

Usually, PulseAudio will remember the last used device. Once you run ffmpeg once, PA should remember that it listened on the monitor input and use that.

SuperV1234 wrote:

> Audio gets out of sync, I tried many options with no luck

I got this problem too a while back. Try removing the -async option, it's deprecated and caused more trouble than I had. Also, make sure you always have enough processing power to process everything in time, otherwise it will become out of sync quickly. ffmpeg isn't smart enough yet to fill the gaps, so if you can't grab the audio fast enough (pulseaudio glitch), can't encode the video fast enough 100% of the time, or anything else, it will lose frames and result in a delay. Even funnier, also make sure the drive you output to is fast enough to save the file so ffmpeg never block on writing the file to disk (I used to send my encode directly to a streaming server, and the network latency messed it up).

SuperV1234 wrote:

> I can't find a way to record both game audio and microphone audio

You can set this up easily with either PA or Jack. Jack is a bit easier to use, but here's the way I do it myself:

- Create a dummy output device (module-null-sink)
- Use a loopback to play your microphone in the dummy output (module-loopback)
- Use a loopback to play your sound card monitor into the dummy output (module-loopback again)
- Make ffmpeg record from the monitor of the dummy output.

You can also juste encode 2 audio streams using ffmpeg (-f alsa -i pulse twice, or -f pulse -i "your pulseaudio input") and edit it after.

I think jack would use less CPU to do the same thing (audio streams are piped and mixed directly in the right place instead of using loopbacks and dummy outputs)

SuperV1234 wrote:

> Could the script be improved somehow for better quality/performance?

- Unless you really want 60 FPS output, I'd suggest reducing the FPS to 30. Keep the 60 FPS input, but output into 30 FPS. It will use half the processing power and there will be almost no noticable difference.
- If you really want 60 FPS output and know how it will be displayed, you can actually encode in interlaced video (60i), which looks like 60 FPS but weights about the same as 30 FPS.
- You can increase the quality, using something like -preset fast or even -preset medium given you have enough spare processing. Remember, encoding speed should be constant. If you record 30 FPS, you have to output 30 FPS all the time!
- Like said by skottish, try not using CFR. It will cause the encoder to have to spend more time one some frames and this extra processing might be enough to cause FPS drop.
- Tune your x264 options (-x264args). Options reference
- Try reducing the number of threads: make room for your game and give a ffmpeg as few cores as possible. using 8 cores will make it have full processing power, but there are some cases where your game will have CPU spikes and ffmpeg will wait for a single thread to finish a frame and get out of sync. They will be rescheduled to another core, but this has some overhead, and x264 encoding uses the CPU cache a lot with inline assembler code. Having as few context switch as possible for ffmpeg is preferable over having two ffmpeg threads encoding on the same CPU.

I hope this helps you a bit. Unfortunately, ffmpeg is still pretty bad at recording because of audio/video sync issues In the end, there will always be sync issues after a while, so I suggest you restart ffmpeg once in a while. I opened a bug on their tracker a while ago about similar issues. Never got any attention, no comment, not even moved from new to open. They need to fix the input buffer size in a way they always grab exactly N frames of video and audio. They seem to use some timecodes but it looks like the muxer often fails to resync them back. I'd fix it myself, but ffmpeg is way too big for me to find what I need, especially for a bug that seem to have existed forever and still not being fixed or even addressed.

Last edited by Max-P (2013-07-22 19:16:12)

AleXoundOS · 2013-10-04 08:46:56

I fully confirm the sync issues. Always had problems with drifting audio delay. Tried a ton of options variations, nothing helped except setting itsoffset, that anyways gets away after some time.

DrZaius · 2013-10-06 01:07:13

AleXoundOS wrote:

I fully confirm the sync issues. Always had problems with drifting audio delay. Tried a ton of options variations, nothing helped except setting itsoffset, that anyways gets away after some time.

Instead of encoding you can try to use stream copy mode:

-codec:a copy

This may make a difference. Note that your output container will need to support PCM audio. Should be fine with Matroska (mkv).

Last edited by DrZaius (2013-10-06 01:07:47)

AleXoundOS · 2013-10-06 10:12:01

DrZaius wrote:

AleXoundOS wrote:
I fully confirm the sync issues. Always had problems with drifting audio delay. Tried a ton of options variations, nothing helped except setting itsoffset, that anyways gets away after some time.
Instead of encoding you can try to use stream copy mode:
-codec:a copy
This may make a difference. Note that your output container will need to support PCM audio. Should be fine with Matroska (mkv).

Assuming I'm getting audio on-the-fly (from pulseaudio or alsa) it won't work.
And it's not a pulseaudio problem or alsa. I tested same recording scenario with VLC and it worked in perfect sync for 12 hours. If anybody interested here are vlc parameters I used:

cvlc \
 screen:// --screen-fps=24.000000 --screen-left 0 --screen-top 0 --screen-width 640 --screen-height 480 --live-caching=3000 \
 --input-slave=pulse:// \
 --sout "#transcode{\
            venc=x264{preset=ultrafast,keyint=2},\
            vcodec=h264,fps=24,vb=1600,\
            acodec=mp3,ab=160,channels=2,samplerate=44100,\
            audio-sync, threads=1}:\
         rtp{dst=127.0.0.1,port=1234,sdp=file:///tmp/vlc.sdp}"

After that I used jtvlc to stream it further.

DrZaius · 2013-10-06 18:42:19

AleXoundOS wrote:

Assuming I'm getting audio on-the-fly (from pulseaudio or alsa) it won't work.

By "won't work" do you mean it is still out of sync, that the command results in an error, or something else? The command at least works for me (I wasn't recording any sound however).

$ ffmpeg -f alsa -i hw:0 -t 5 -c:a copy out.wav
ffmpeg version N-56333-g7129935 Copyright (c) 2000-2013 the FFmpeg developers
  built on Sep 16 2013 19:08:37 with gcc 4.8.1 (GCC) 20130725 (prerelease)
  configuration: --prefix=/usr --extra-libs=-ldl --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libvorbis --enable-libvpx --enable-libx264 --enable-nonfree --enable-x11grab
  libavutil      52. 43.100 / 52. 43.100
  libavcodec     55. 31.101 / 55. 31.101
  libavformat    55. 16.103 / 55. 16.103
  libavdevice    55.  3.100 / 55.  3.100
  libavfilter     3. 85.100 /  3. 85.100
  libswscale      2.  5.100 /  2.  5.100
  libswresample   0. 17.103 /  0. 17.103
  libpostproc    52.  3.100 / 52.  3.100
Guessed Channel Layout for  Input Stream #0.0 : stereo
Input #0, alsa, from 'hw:0':
  Duration: N/A, start: 1381084359.501811, bitrate: 1536 kb/s
    Stream #0:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
Output #0, wav, to 'out.wav':
  Metadata:
    ISFT            : Lavf55.16.103
    Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, 1536 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
size=     867kB time=00:00:05.00 bitrate=1418.1kbits/s    
video:0kB audio:867kB subtitle:0 global headers:0kB muxing overhead 0.009011%

Arch Linux

#1 2013-07-22 08:33:23

Game recording issues with ffmpeg

#2 2013-07-22 15:59:52

Re: Game recording issues with ffmpeg

#3 2013-07-22 19:14:11

Re: Game recording issues with ffmpeg

#4 2013-10-04 08:46:56

Re: Game recording issues with ffmpeg

#5 2013-10-06 01:07:13

Re: Game recording issues with ffmpeg

#6 2013-10-06 10:12:01

Re: Game recording issues with ffmpeg

#7 2013-10-06 18:42:19

Re: Game recording issues with ffmpeg

Board footer