File buffers and fifos/pipes

kokoko3k · 2011-03-28 11:52:45

Hi, i'm searching for an utility that buffers data to a real file.
What i need would allows me to do something like that:

dd if=/my/movie.avi | filebuffer /tmp/mybuffer -s 100M | mplayer -

/tmp/mybuffer will be a fixed size 100M file containing data from /my/movie.avi
filebuffer will also pipe data to mplayer.

Does a similar tool exists?
I tried mbuffer from aur, but even if the man page states that -T argument should make it buffer to a file, no file is created at all and ram is used instead

Thanks.

Last edited by kokoko3k (2011-03-28 11:55:15)

Ramses de Norre · 2011-03-28 12:05:49

RAM will probably be used anyway as the kernel caches the data without writing it to disk. Why exactly do you need to do it this way?

kokoko3k · 2011-03-28 12:31:04

Several times i tought of it, but lately i just needed it to dump multiple dvbt channels using mplayer (if you tune to a special pid 8192 it will dump several stations at once) and playing them back using another mplayer instance while allowing large time-shifting.
Doing that in ram would kill my system in a matter of seconds, while a file buffer would do the trick.
Also pagecache-managment from googlecode would allow the kernel to "forget" about that data as soon as possible, reducing pagecache footprints.

scorpyn · 2011-03-28 16:17:09

I don't get it.

Why don't you just start 2 instances of mplayer and have them load the same file?

I doubt that mplayer will actually be able to read that 100MB file properly. And even if it did, how should it behave when it gets full? And how would mplayer know when it's full?

You'd probably have to create a new file in /dev to make this work...

kokoko3k · 2011-03-29 07:07:33

Because that file would become extremly big over time, it is about 6MB/sec and i cannot start 2 mplayer instances for dvb:// streams because it just won't work.

mplayer will be able to read that file because dvbt streams are basically mpegTS, you can cut them as you like and start playback where you like.
I don't care mplayer to know when the buffer is full, 'filebuffer' should manage that by its own overwriting the file with new data when 100MB limit is reached.

scorpyn · 2011-03-29 08:18:40

So...

You want it to write from scratch every time it reaches 100MB?

Or do you want it to create a new file every 100MB?

Or do you want it to remove stuff at the beginning and keep filling it up at the end?

The problem is this : What will happen when the 2nd instance of mplayer reaches the end of the buffer file? The file is being rewritten, but mplayer is at the end of the file...

...or perhaps mplayer can be set to restart from the beginning of the file once it reaches the end?

The next question is whether the file should be truncated before filling it the 2nd time. If not, mplayer may simply keep reading old data after a while. If truncated, it'll be empty for a while, which would cause issues for mplayer.

Since it's a stream it'll probably work to split it, but not all formats support that. Sometimes you need to be able to read stuff at the end/beginning to be able to decode.

My suggestion is this :

1. Start writing to file 1
2. When file 1 is full, close it and start writing to file 2
3. When file 2 is full, close it and start writing to file 3
...and so on...

Perhaps an option could be added to only keep n files, so you don't fill the partition with 100MB files.

kokoko3k · 2011-03-29 10:08:44

"I want it to remove stuff at the beginning and keep filling it up at the end", like a circular buffer.
When mplayer reaches the end of the file, it would just start from the beginning (-loop 0 option).
If i understood properly, your suggestion may lead to glitches/interruptions when mplayer change files, since it doesn't support gapless playback (maybe mplayer2 fork can do the trick.)

--
Edit:
Or maybe new data could just be appended to the buffer file and old data removed from the beginning (i don't know if it is possible)

Last edited by kokoko3k (2011-03-29 10:20:00)

Arch Linux

#1 2011-03-28 11:52:45

File buffers and fifos/pipes

#2 2011-03-28 12:05:49

Re: File buffers and fifos/pipes

#3 2011-03-28 12:31:04

Re: File buffers and fifos/pipes

#4 2011-03-28 16:17:09

Re: File buffers and fifos/pipes

#5 2011-03-29 07:07:33

Re: File buffers and fifos/pipes

#6 2011-03-29 08:18:40

Re: File buffers and fifos/pipes

#7 2011-03-29 10:08:44

Re: File buffers and fifos/pipes

Board footer