Arch Linux

Simargol

Member

Registered: 2015-06-11

Posts: 10

FFmpeg + PulseAudio. Audio delay if FFmpeg is launched before audio.

Screen capture has been giving me "occasional" ~2 seconds (as it felt) audio delay. But it seems that now I can constantly reproduce it if FFmpeg is started before any other audio application (if state of PA Sinks/Sources is SUSPENDED).

Test #1. Launch FFmpeg, wait 8 seconds, launch vlc with audio sync test:

ffmpeg -f x11grab -video_size 1600x900 -r 30 -i :0.0 -f pulse -i default -c:v libx264 -preset ultrafast -c:a libvorbis -t 30 out1.mkv -y -nostdin &
sleep 8
cvlc synctest.mp4 --zoom 0.4 --stop-time 20 vlc://quit &>/dev/null

RESULT: Audio delay is there.
VIDEO: https://uploadfiles.io/mqthm
LOG: https://pastebin.com/raw/g9SQMcZD

Test #2. Reversed order (vlc, wait, ffmpeg):

cvlc synctest.mp4 --zoom 0.4 --stop-time 20 vlc://quit &>/dev/null &
sleep 8
ffmpeg -f x11grab -video_size 1600x900 -r 30 -i :0.0 -f pulse -i default -c:v libx264 -preset ultrafast -c:a libvorbis -t 20 out2.mkv -y -nostdin

RESULT: No audio delay.
VIDEO: https://uploadfiles.io/ky8gn
LOG: https://pastebin.com/raw/UDuryGK1

FFmpeg prints multiple warnings in both cases while vlc is not running yet (or already).

...
[matroska @ 0x55fe123f7680] Non-monotonous DTS in output stream 0:1; previous: 10631, current: 9746; changing to 10631. This may result in incorrect timestamps in the output file.
[libvorbis @ 0x55fe123fb500] Queue input is backward in time
...

At the same time Latency of PulseAudio Sink is huge (as you can see in video at top left corner).

System info:

$ pacman -Qs 'alsa|pulse|ffmpeg'
local/alsa-lib 1.1.6-1
    An alternative implementation of Linux sound support
local/alsa-plugins 1.1.6-2
    Extra alsa plugins
local/alsa-utils 1.1.6-1
    An alternative implementation of Linux sound support
local/ffmpeg 1:4.0.2-2
    Complete solution to record, convert and stream audio and video
local/ffmpegthumbnailer 2.2.0-2
    Lightweight video thumbnailer that can be used by file managers.
local/lash 0.6.0~rc2-14
    Session management system for JACK and ALSA
local/lib32-alsa-lib 1.1.6-1
    An alternative implementation of Linux sound support (32 bit)
local/lib32-alsa-plugins 1.1.6-1
    Extra alsa plugins (32-bit)
local/lib32-libpulse 12.2-1
    A featureful, general-purpose sound server (32-bit client libraries)
local/libcanberra-pulse 0.30+2+gc0620e4-1
    PulseAudio plugin for libcanberra
local/libpulse 12.2-2
    A featureful, general-purpose sound server (client library)
local/pavucontrol 1:3.0+23+g335c26c-1
    PulseAudio Volume Control
local/pulseaudio 12.2-2
    A featureful, general-purpose sound server
local/pulseaudio-alsa 2-4
    ALSA Configuration for PulseAudio
local/pulseaudio-bluetooth 12.2-2
    Bluetooth support for PulseAudio
local/zita-alsa-pcmi 0.2.0-4
    The successor of clalsadrv, provides easy access to ALSA PCM devices.

$ pactl list sources
Source #0
	State: SUSPENDED
	Name: alsa_output.pci-0000_00_1b.0.analog-stereo.monitor
	Description: Monitor of Built-in Audio Analog Stereo
	Driver: module-alsa-card.c
	Sample Specification: s16le 2ch 44100Hz
	Channel Map: front-left,front-right
	Owner Module: 7
	Mute: no
	Volume: front-left: 65536 / 100% / 0.00 dB,   front-right: 65536 / 100% / 0.00 dB
	        balance 0.00
	Base Volume: 65536 / 100% / 0.00 dB
	Monitor of Sink: alsa_output.pci-0000_00_1b.0.analog-stereo
	Latency: 0 usec, configured 0 usec
	Flags: DECIBEL_VOLUME LATENCY 
	Properties:
		device.description = "Monitor of Built-in Audio Analog Stereo"
		device.class = "monitor"
		alsa.card = "1"
		alsa.card_name = "HDA Intel PCH"
		alsa.long_card_name = "HDA Intel PCH at 0xd1614000 irq 30"
		alsa.driver_name = "snd_hda_intel"
		device.bus_path = "pci-0000:00:1b.0"
		sysfs.path = "/devices/pci0000:00/0000:00:1b.0/sound/card1"
		device.bus = "pci"
		device.vendor.id = "8086"
		device.vendor.name = "Intel Corporation"
		device.product.id = "8c20"
		device.product.name = "8 Series/C220 Series Chipset High Definition Audio Controller"
		device.form_factor = "internal"
		device.string = "1"
		module-udev-detect.discovered = "1"
		device.icon_name = "audio-card-pci"
	Formats:
		pcm

Source #1
	State: SUSPENDED
	Name: alsa_input.pci-0000_00_1b.0.analog-stereo
	Description: Built-in Audio Analog Stereo
	Driver: module-alsa-card.c
	Sample Specification: s16le 2ch 44100Hz
	Channel Map: front-left,front-right
	Owner Module: 7
	Mute: no
	Volume: front-left: 3547 /   5% / -76.00 dB,   front-right: 10387 /  16% / -48.00 dB
	        balance 0.66
	Base Volume: 6554 /  10% / -60.00 dB
	Monitor of Sink: n/a
	Latency: 0 usec, configured 0 usec
	Flags: HARDWARE HW_MUTE_CTRL HW_VOLUME_CTRL DECIBEL_VOLUME LATENCY 
	Properties:
		alsa.resolution_bits = "16"
		device.api = "alsa"
		device.class = "sound"
		alsa.class = "generic"
		alsa.subclass = "generic-mix"
		alsa.name = "92HD95 Analog"
		alsa.id = "92HD95 Analog"
		alsa.subdevice = "0"
		alsa.subdevice_name = "subdevice #0"
		alsa.device = "0"
		alsa.card = "1"
		alsa.card_name = "HDA Intel PCH"
		alsa.long_card_name = "HDA Intel PCH at 0xd1614000 irq 30"
		alsa.driver_name = "snd_hda_intel"
		device.bus_path = "pci-0000:00:1b.0"
		sysfs.path = "/devices/pci0000:00/0000:00:1b.0/sound/card1"
		device.bus = "pci"
		device.vendor.id = "8086"
		device.vendor.name = "Intel Corporation"
		device.product.id = "8c20"
		device.product.name = "8 Series/C220 Series Chipset High Definition Audio Controller"
		device.form_factor = "internal"
		device.string = "front:1"
		device.buffering.buffer_size = "352800"
		device.buffering.fragment_size = "176400"
		device.access_mode = "mmap+timer"
		device.profile.name = "analog-stereo"
		device.profile.description = "Analog Stereo"
		device.description = "Built-in Audio Analog Stereo"
		alsa.mixer_name = "IDT 92HD95"
		alsa.components = "HDA:111d7695,17aa3800,00100101"
		module-udev-detect.discovered = "1"
		device.icon_name = "audio-card-pci"
	Ports:
		analog-input-internal-mic: Internal Microphone (priority: 8900, not available)
		analog-input-mic: Microphone (priority: 8700, available)
	Active Port: analog-input-mic
	Formats:
		pcm

$ pactl list sinks
Sink #0
	State: SUSPENDED
	Name: alsa_output.pci-0000_00_1b.0.analog-stereo
	Description: Built-in Audio Analog Stereo
	Driver: module-alsa-card.c
	Sample Specification: s16le 2ch 44100Hz
	Channel Map: front-left,front-right
	Owner Module: 7
	Mute: no
	Volume: front-left: 28443 /  43% / -21.75 dB,   front-right: 28443 /  43% / -21.75 dB
	        balance 0.00
	Base Volume: 65536 / 100% / 0.00 dB
	Monitor Source: alsa_output.pci-0000_00_1b.0.analog-stereo.monitor
	Latency: 0 usec, configured 0 usec
	Flags: HARDWARE HW_MUTE_CTRL HW_VOLUME_CTRL DECIBEL_VOLUME LATENCY 
	Properties:
		alsa.resolution_bits = "16"
		device.api = "alsa"
		device.class = "sound"
		alsa.class = "generic"
		alsa.subclass = "generic-mix"
		alsa.name = "92HD95 Analog"
		alsa.id = "92HD95 Analog"
		alsa.subdevice = "0"
		alsa.subdevice_name = "subdevice #0"
		alsa.device = "0"
		alsa.card = "1"
		alsa.card_name = "HDA Intel PCH"
		alsa.long_card_name = "HDA Intel PCH at 0xd1614000 irq 30"
		alsa.driver_name = "snd_hda_intel"
		device.bus_path = "pci-0000:00:1b.0"
		sysfs.path = "/devices/pci0000:00/0000:00:1b.0/sound/card1"
		device.bus = "pci"
		device.vendor.id = "8086"
		device.vendor.name = "Intel Corporation"
		device.product.id = "8c20"
		device.product.name = "8 Series/C220 Series Chipset High Definition Audio Controller"
		device.form_factor = "internal"
		device.string = "front:1"
		device.buffering.buffer_size = "352800"
		device.buffering.fragment_size = "176400"
		device.access_mode = "mmap+timer"
		device.profile.name = "analog-stereo"
		device.profile.description = "Analog Stereo"
		device.description = "Built-in Audio Analog Stereo"
		alsa.mixer_name = "IDT 92HD95"
		alsa.components = "HDA:111d7695,17aa3800,00100101"
		module-udev-detect.discovered = "1"
		device.icon_name = "audio-card-pci"
	Ports:
		analog-output-speaker: Speakers (priority: 10000, not available)
		analog-output-headphones: Headphones (priority: 9000, available)
	Active Port: analog-output-headphones
	Formats:
		pcm

And I haven't been changing configuration of alsa or pulse:
/usr/bin/alsa-info.sh - https://pastebin.com/raw/PHka2PsL
tail -n +1 /etc/pulse/* - https://pastebin.com/raw/Q9EB7ZAC

Why does this delay happen and is it possible to fix it?

Last edited by Simargol (2018-08-26 12:52:33)

dbermond

Package Maintainer (PM)

Registered: 2017-05-06

Posts: 28

Website

Re: FFmpeg + PulseAudio. Audio delay if FFmpeg is launched before audio.

ffmpeg -f x11grab -video_size 1600x900 -r 30 -i :0.0 -f pulse -i default -c:v libx264 -preset ultrafast -c:a libvorbis -t 30 out1.mkv -y -nostdin &

1) Have you tried with '-f alsa' instead of pulse? In my experince, using alsa causes less audio problems. I use '-f pulse' only as a fallback option.

2) When capturing the desktop with x11grab, use '-framerate' instead of '-r'.

3) Avoid using non-lossless software-based video encoders in the recording process. This also causes audio problems in my experience (unless if using a hardware-based video encoder). When using a software-based video encoder (like libx264), try to make a two step process like this:

a) record with lossless encoders (for example, use ffv1 for video and pcm_s16le for audio). This will produce a intermediate/temporary video file.

b) use a second ffmpeg call to encode the lossless video file to the desired audio/video encoders.

Simargol

Member

Registered: 2015-06-11

Posts: 10

Re: FFmpeg + PulseAudio. Audio delay if FFmpeg is launched before audio.

1) Yes, I forgot to mention that pure Alsa works fine.
2) -framerate worked fine for me for tests with no audio delay. But for the test with audio delay it resulted in video stuttering. Changing to -r somewhy fixed it.
3) With codecs you mentioned I have no this stuttering issue when using -framerate. But audio delay issue is still there.

Last edited by Simargol (2018-08-28 08:31:11)

dbermond

Package Maintainer (PM)

Registered: 2017-05-06

Posts: 28

Website

Re: FFmpeg + PulseAudio. Audio delay if FFmpeg is launched before audio.

1) Yes, I forgot to mention that pure Alsa works fine.

I understand that you may want to use pulseaudio for some reason, but you should try to stick with alsa since it works fine for you.

You could try to use my screencast interface to record the desktop and see if it helps in some way. It defaults to use alsa though. AUR packages: screncast and screencast-git.

V1del

Forum Moderator

Registered: 2012-10-16

Posts: 23,430

Re: FFmpeg + PulseAudio. Audio delay if FFmpeg is launched before audio.

Can you post a

pacmd list-source-outputs

during the working set up and the broken one?

Just for giggles, what happens if you define some initial latency for the sink?

PULSE_LATENCY_MSEC=30 $ffmpegcommand

another potential contributor might be a wakeup out of the suspension state that leads to this issue. Does e.g. setting exit-idle-time to a negative value in /etc/pulse/daemon.conf and/or commenting load-module module-suspend-on-idle in /etc/pulse/default.pa help with the issue?

Simargol

Member

Registered: 2015-06-11

Posts: 10

Re: FFmpeg + PulseAudio. Audio delay if FFmpeg is launched before audio.

Output of pacmd list-source-outputs doesn't differ in working and not working tests:
test1 (delay, ffmpeg is started first): https://www.diffchecker.com/82cfEFAr
test2 (no delay, vlc is started first): https://www.diffchecker.com/vMxvWCHy
Left column is when only ffmpeg is running. Right column is when vlc is running as well.

With PULSE_LATENCY_MSEC=30, pactl shows requested/configured latency as 30msec, FFmpeg produces much less warnings, but delay is still there.

Forbidding falling into SUSPENDED state also doesn't seem to change anything.

Later I'll try to find better tests maybe as these don't seem to explain much.