You are not logged in.
Hi all, some times ago flash video were stored in /tmp and I was able to copy them out. Now this desn't happen any more. I have searched out for a while and I have written a command to copy them from the process npviewer.bin to the home.
Here's the script:
#!/bin/bash
cp `lsof | grep Flash | \
awk '
{
if (NF != 10) {
proc="pnf";
inode="inf";
}
else {
proc=$2;
inode=substr($4, 1, length($4)-1);
print "/proc/"proc"/fd/"inode;
}
} '` ~
This does the trick but I'd like to improve/correct it, there are some things about cp and awk which I haven't understood.
For example: if I moove the '~' inside print, like this:
#!/bin/bash
cp `lsof | grep Flash | \
awk '
{
if (NF != 10) {
proc="pnf";
inode="inf";
}
else {
proc=$2;
inode=substr($4, 1, length($4)-1);
print "/proc/"proc"/fd/"inode, "~";
}
} '`
the code between `...` produce the correct (I think) output like this (I had 3 Youtube tabs opened on Chrome):
/proc/5149/fd/11 ~
/proc/5149/fd/14 ~
/proc/5149/fd/15 ~
but cp complies that the target "~" is not a directory.
Any comment, suggestion, improvement? Thanks!
Last edited by DarioP (2011-02-20 14:46:20)
Offline
I deleted your duplicate thread. For future reference, you can use the 'report' button to get a mod's attention (someone else did that for you in this case).
Allan-Volunteer on the (topic being discussed) mailn lists. You never get the people who matters attention on the forums.
jasonwryan-Installing Arch is a measure of your literacy. Maintaining Arch is a measure of your diligence. Contributing to Arch is a measure of your competence.
Griemak-Bleeding edge, not bleeding flat. Edge denotes falls will occur from time to time. Bring your own parachute.
Offline
but cp complies that the target "~" is not a directory.
read this:
http://mywiki.wooledge.org/Arguments
and this:
http://mywiki.wooledge.org/BashParser
It will help explain a lot. But in this case, the issue is that tilde expansion is performed before command substitution. So that, combined with the tilde being within single quotes, means that it never gets expanded, and cp(1) sees the literal "~". Changing it to "$HOME" wouldn't work either, as parameter expansions also occur before command substitution (but after tilde expansion). The other issue is that you would be trying to copy your home dir to... your home dir. As it is now, after awk does its thing, cp(1) sees this as its arguments:
cp /proc/5149/fd/11 /proc/5149/fd/14 /proc/5149/fd/15 /home/<your username>/
cp(1) isn't performed once per line, it doesn't care. It just sees each output from awk as another argument. Your version would look like this:
cp /proc/5149/fd/11 ~ /proc/5149/fd/14 ~ /proc/5149/fd/15 ~
Even if the tildes were expanded, you should see the issue here.
Even more correct would be using cp -t...
cp -t "$HOME" $(lsof | grep Flash | awk '
{
if (NF != 10) {
proc="pnf";
inode="inf";
} else {
proc=$2;
inode=substr($4, 1, length($4)-1);
print "/proc/"proc"/fd/"inode;
}
} ')
This would prevent cp from possibly doing something strange, and is technically the correct way to copy multiple files with one invocation. "$HOME" should always be used over "~" in scripts, as well.
There may also be a better way to do this whole thing, as this method seems prone to breaking, but I'd have to play with it more. The only reason this works is because none of the filenames should ever have spaces in them.
Last edited by freak (2011-02-21 18:24:58)
Offline
Thank you very much for your reply. Now things are much clearer!!
I won't fool around with many words file name, as the inode is always a number followed by a char (which must be dropped).
I have another question instead: the function system allows awk to execute bash command. So a script like this:
lsof | grep Flash | awk '
{
if (NF != 10) {
proc="pnf";
inode="inf";
} else {
proc=$2;
inode=substr($4, 1, length($4)-1);
system(cp " /proc/"proc"/fd/"inode " $HOME");
}
} '
should produce the correct results.. instead it returns:
sh: /proc/3661/fd/15: Permission denied
Where does this fail? Is there an option that should be pass to awk?
Offline
Why don't you use a simple find in /proc/fd?
find /proc/*/fd/ -lname "/tmp/Flash*" -exec cp {} $HOME \;
Help me to improve ssh-rdp !
Retroarch User? Try my koko-aio shader !
Offline
Why don't you use a simple find in /proc/fd?
find /proc/*/fd/ -lname "/tmp/Flash*" -exec cp {} $HOME \;
My command looks for files that are opened by a process and that contains "Flash" in their instance. Those kind of files are generated for example when you buffer a flash video and are deleted when you close the browser tab.
Find command doesn't make this selection and gets lost! On my pc your command generates this term output (sorry for the Italian):
dario:/~ > find /proc/*/fd/ -lname "/tmp/Flash*" -exec cp {} $HOME \;
find: "/proc/1087/fd/": Permesso negato
find: "/proc/10/fd/": Permesso negato
find: "/proc/1167/fd/": Permesso negato
find: "/proc/1180/fd/": Permesso negato
find: "/proc/1181/fd/": Permesso negato
find: "/proc/1182/fd/": Permesso negato
find: "/proc/1297/fd/": Permesso negato
find: "/proc/1298/fd/": Permesso negato
find: "/proc/12/fd/": Permesso negato
find: "/proc/1307/fd/": Permesso negato
find: "/proc/1326/fd/": Permesso negato
find: "/proc/1327/fd/": Permesso negato
find: "/proc/1328/fd/": Permesso negato
find: "/proc/1329/fd/": Permesso negato
find: "/proc/1330/fd/": Permesso negato
find: "/proc/1331/fd/": Permesso negato
find: "/proc/1332/fd/": Permesso negato
find: "/proc/1358/fd/": Permesso negato
find: "/proc/1361/fd/": Permesso negato
find: "/proc/1370/fd/": Permesso negato
find: "/proc/1376/fd/": Permesso negato
find: "/proc/1378/fd/": Permesso negato
find: "/proc/1381/fd/": Permesso negato
find: "/proc/1384/fd/": Permesso negato
find: "/proc/1387/fd/": Permesso negato
find: "/proc/1388/fd/": Permesso negato
find: "/proc/1398/fd/": Permesso negato
find: "/proc/13/fd/": Permesso negato
find: "/proc/1465/fd/": Permesso negato
find: "/proc/14/fd/": Permesso negato
find: "/proc/1504/fd/": Permesso negato
find: "/proc/1510/fd/": Permesso negato
find: "/proc/1521/fd/": Permesso negato
find: "/proc/1580/fd/": Permesso negato
find: "/proc/15/fd/": Permesso negato
find: "/proc/16/fd/": Permesso negato
find: "/proc/1733/fd/": Permesso negato
find: "/proc/1770/fd/": Permesso negato
find: "/proc/1771/fd/": Permesso negato
find: "/proc/17/fd/": Permesso negato
find: "/proc/1889/fd/": Permesso negato
find: "/proc/18/fd/": Permesso negato
find: "/proc/1934/fd/": Permesso negato
find: "/proc/1971/fd/": Permesso negato
find: "/proc/19/fd/": Permesso negato
find: "/proc/1/fd/": Permesso negato
find: "/proc/2080/fd/": Permesso negato
find: "/proc/20/fd/": Permesso negato
find: "/proc/2175/fd/": Permesso negato
find: "/proc/21/fd/": Permesso negato
find: "/proc/22/fd/": Permesso negato
find: "/proc/23/fd/": Permesso negato
find: "/proc/24/fd/": Permesso negato
find: "/proc/25/fd/": Permesso negato
find: "/proc/26/fd/": Permesso negato
find: "/proc/27/fd/": Permesso negato
find: "/proc/28/fd/": Permesso negato
find: "/proc/29/fd/": Permesso negato
find: "/proc/2/fd/": Permesso negato
find: "/proc/3/fd/": Permesso negato
find: "/proc/436/fd/": Permesso negato
find: "/proc/437/fd/": Permesso negato
find: "/proc/438/fd/": Permesso negato
find: "/proc/447/fd/": Permesso negato
find: "/proc/448/fd/": Permesso negato
find: "/proc/449/fd/": Permesso negato
find: "/proc/485/fd/": Permesso negato
find: "/proc/486/fd/": Permesso negato
find: "/proc/518/fd/": Permesso negato
find: "/proc/6849/fd/": Permesso negato
find: "/proc/6889/fd/": Permesso negato
find: "/proc/6/fd/": Permesso negato
find: "/proc/7008/fd/": Permesso negato
find: "/proc/7652/fd/": Permesso negato
find: "/proc/785/fd/": Permesso negato
find: "/proc/792/fd/": Permesso negato
find: "/proc/7/fd/": Permesso negato
find: "/proc/802/fd/": Permesso negato
find: "/proc/803/fd/": Permesso negato
find: "/proc/8162/fd/": Permesso negato
find: "/proc/816/fd/": Permesso negato
find: "/proc/828/fd/": Permesso negato
find: "/proc/8390/fd/": Permesso negato
find: "/proc/8412/fd/": Permesso negato
find: "/proc/8422/fd/": Permesso negato
find: "/proc/8440/fd/": Permesso negato
find: "/proc/8516/fd/": Permesso negato
find: "/proc/851/fd/": Permesso negato
find: "/proc/8540/fd/": Permesso negato
find: "/proc/864/fd/": Permesso negato
find: "/proc/867/fd/": Permesso negato
find: "/proc/871/fd/": Permesso negato
find: "/proc/8/fd/": Permesso negato
find: "/proc/self/fd/5": File o directory non esistente
and no file is copied. Did you test your command before posting?
Last edited by DarioP (2011-02-21 11:10:20)
Offline
Of course, it actually works for me,
redirect stderr messages to dev null for a less distracting output and add a -v to cp for a more verbose message:
# chromium-browser http://www.youtube.com/watch?v=clIUE4SRDd8&feature=topvideos &
# (wait for the video to load...)
# find /proc/*/fd/ -lname "/tmp/Flash*" -exec cp -v {} $HOME 2>/dev/null \;
"/proc/23907/fd/24" -> "/home/koko/24"
# mplayer /home/koko/24
MPlayer git-cba6d60-4.5.2 (C) 2000-2011 MPlayer Team
161 audio & 349 video codecs
[..cut]
Playing /home/p2p/24.
Detected file format: FLV format (libavformat)
[..cut]
[AO OSS] audio_setup: Can't open audio device /dev/dsp: Device or resource busy
AO: [alsa] 48000Hz 2ch s16le (2 bytes per sample)
Starting playback...
A: -0.0 V: 0.0 A-V: 0.000 ct: 0.000 0/ 0 ??% ??% ??,?% 0 0
Movie-Aspect is 1.70:1 - prescaling to correct movie aspect.
VO: [xv] 524x308 => 524x308 Planar YV12
A: 1.9 V: 1.9 A-V: 0.001 ct: 0.000 0/ 0 4% 1% 1.8% 0 0
Exiting... (Quit)
Last edited by kokoko3k (2011-02-21 13:19:33)
Help me to improve ssh-rdp !
Retroarch User? Try my koko-aio shader !
Offline
This works better:
#!/bin/bash
for s in $(find /proc/*/fd/ -lname "/tmp/Flash*" 2>/dev/null); do
d=$HOME/$(basename $s)
p=$(echo $s|cut -d "/" -f 3)
echo tailing $s to $d from pid $p
tail -n +0 --pid=$p --follow $s > $d &
done
...because using tail instead of cp allows you to start copying the videos while they are still buffering.
Tail will exit when you close the browser or when the plugin ends.
There's the drawback that if you launch it multiple times you'll 'tail the tail'
Help me to improve ssh-rdp !
Retroarch User? Try my koko-aio shader !
Offline
redirect stderr messages to dev null for a less distracting output
That's ok, now is working for me too!
I don't know which approach is better, yours is probably faster but I like to use awk...
Offline
using tail instead of cp allows you to start copying the videos while they are still buffering.
That's great!! You can also use cp when still buffering and produce readable files, but you have to relaunch the command from time to time...
There's the drawback that if you launch it multiple times you'll 'tail the tail'
That's bad, because if you start to buffer a second video you'll corrupt the first... uhm... I have to think a solution for this... maybe exporting the file names of the files that are already been tailing...
Offline
This works better:
#!/bin/bash for s in $(find /proc/*/fd/ -lname "/tmp/Flash*" 2>/dev/null); do d=$HOME/$(basename $s) p=$(echo $s|cut -d "/" -f 3) echo tailing $s to $d from pid $p tail -n +0 --pid=$p --follow $s > $d & done
Not to nit-pick, but I'm going to. Once again, in this case it doesn't really matter, but you should never use a for loop to iterate through the output of a command. This, as well, will break for any filename containing spaces.
I also have issues with using basename(1), but that's a different story.
#!/bin/bash
current=/tmp/flashdownload
download() {
if ! grep -q "$1" "$current"; then
printf "%s\n" "$1" >> "$current"
tail -n +0 --pid="$3" --follow "$1" > "$2"
sed -i "\|$1|d" "$current"
else
printf "%s\n" "$1 is alread downloading" >&2
fi
}
while IFS= read -rd '' s; do
d=$HOME/${s##*/}
p=$(cut -d '/' -f 3 <<<"$s")
printf "%s\n" "tailing $s to $d from PID $p"
download "$s" "$d" "$p" &
done < <(find /proc/*/fd -lname "/tmp/Flash*" -print0 2>/dev/null)
wait
Never hurts to write safe(r) code This isn't the best thing in the world, but it should make sure you're only tailing once per file (untested).
Good idea, though... definitely a better solution than parsing lsof(1)
Last edited by freak (2011-02-21 21:56:05)
Offline
Since i dont understand that syntax very well, here's another, safer, try.
This will simply not overwrite a file which already exists,
and since the filename is taken using readlink instead of the descriptor id,
we'll avoid to 'tail the tail' or download a file twice in a shot:
#!/bin/bash
export IFS=$'\n' #Treat spaces as... spaces!
ddir=$HOME/Flashes #Destination directory
mkdir -p "$ddir" 2>/dev/null #...create it,eventually
pref="FlashDownload" #Prefix name
dp="$ddir"/"$pref" #destdir/prefixname
for s in $(find /proc/*/fd/ -lname "/tmp/Flash*" 2>/dev/null); do
pt=$(readlink "$s"|cut -f 1 -d " ") #Get real flash name
d=$dp$(basename "$pt") #FullPath/Prefix+RealFlashName
if [ ! -f "$d" ] #Download if the file does not exists
then
p=$(echo "$s"|cut -d "/" -f 3) #get the pid
echo tailing "$s" to "$d" from pid $p
tail -n +0 --pid=$p --follow "$s" > "$d" & #copy using tail
else
echo File "$pt" is downloaded/downloading #...skip if the file exists
fi
done
try:
Gozer scripts # getflash.sh
tailing /proc/2272/fd/3 to /tmp/My flash files/Download_FlashXX9Jbc7T from pid 2272
tailing /proc/2280/fd/3 to /tmp/My flash files/Download_FlashXX187Sux from pid 2280
tailing /proc/2289/fd/3 to /tmp/My flash files/Download_FlashXXm9ej4O from pid 2289
tailing /proc/2297/fd/3 to /tmp/My flash files/Download_FlashXXnKcXO7 from pid 2297
File /tmp/FlashXX9Jbc7T is downloaded/downloading
File /tmp/FlashXX187Sux is downloaded/downloading
File /tmp/FlashXXm9ej4O is downloaded/downloading
File /tmp/FlashXXnKcXO7 is downloaded/downloading
File /tmp/FlashXX9Jbc7T is downloaded/downloading
File /tmp/FlashXX187Sux is downloaded/downloading
File /tmp/FlashXXm9ej4O is downloaded/downloading
File /tmp/FlashXXnKcXO7 is downloaded/downloading
File /tmp/FlashXX9Jbc7T is downloaded/downloading
File /tmp/FlashXX187Sux is downloaded/downloading
File /tmp/FlashXXm9ej4O is downloaded/downloading
File /tmp/FlashXXnKcXO7 is downloaded/downloading
File /tmp/FlashXX9Jbc7T is downloaded/downloading
File /tmp/FlashXX187Sux is downloaded/downloading
File /tmp/FlashXXm9ej4O is downloaded/downloading
File /tmp/FlashXXnKcXO7 is downloaded/downloading
Gozer scripts # getflash.sh
File /tmp/FlashXX9Jbc7T is downloaded/downloading
File /tmp/FlashXX187Sux is downloaded/downloading
File /tmp/FlashXXm9ej4O is downloaded/downloading
File /tmp/FlashXXnKcXO7 is downloaded/downloading
File /tmp/FlashXX9Jbc7T is downloaded/downloading
File /tmp/FlashXX187Sux is downloaded/downloading
File /tmp/FlashXXm9ej4O is downloaded/downloading
File /tmp/FlashXXnKcXO7 is downloaded/downloading
File /tmp/FlashXX9Jbc7T is downloaded/downloading
File /tmp/FlashXX187Sux is downloaded/downloading
File /tmp/FlashXXm9ej4O is downloaded/downloading
File /tmp/FlashXXnKcXO7 is downloaded/downloading
File /tmp/FlashXX9Jbc7T is downloaded/downloading
File /tmp/FlashXX187Sux is downloaded/downloading
File /tmp/FlashXXm9ej4O is downloaded/downloading
File /tmp/FlashXXnKcXO7 is downloaded/downloading
File /tmp/FlashXX9Jbc7T is downloaded/downloading
File /tmp/FlashXX187Sux is downloaded/downloading
File /tmp/FlashXXm9ej4O is downloaded/downloading
File /tmp/FlashXXnKcXO7 is downloaded/downloading
File /tmp/FlashXX9Jbc7T is downloaded/downloading
File /tmp/FlashXX187Sux is downloaded/downloading
File /tmp/FlashXXm9ej4O is downloaded/downloading
File /tmp/FlashXXnKcXO7 is downloaded/downloading
Last edited by kokoko3k (2011-02-22 13:28:43)
Help me to improve ssh-rdp !
Retroarch User? Try my koko-aio shader !
Offline
Very nice work! A great improvement respect my first work. Still not perfect as the terminal outputs refers many time to the same file. I wonder if it would be better to come back to lsof(1) and awk...
Too less time to think about this in these days
Offline
Yep, same output is repeated because of tail.
One way to suppress it is to append everything to a file and then "sort -u" the results.
-edit-
i don't think lsof would give better results, because it would catch multiple tails too.
about awk, i don't know.
Last edited by kokoko3k (2011-02-22 16:27:35)
Help me to improve ssh-rdp !
Retroarch User? Try my koko-aio shader !
Offline
A (imo)better way to conserve the files works like this
Flash calls unlink() directly after opening a file, i.e. it deletes the reference to it. Using LD_PRELOAD you can override any dynamically linked functions, so in this case you can just change unlink(). Of course your browser might create files that it should delete, so our unlink() replacement, should provide that functionality and only filter calls that include the word "Flash" in the pathname, otherwise redirecting it to the original unlink (using dlsym to get the address that the original symbol of unlink() pointed to).
My attempt looks like this
unlink_mod.c
#define _GNU_SOURCE
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <dlfcn.h>
static int (*functionSurrogate_unlink)(const char* s) = NULL;
int unlink(const char *pathname) {
if (functionSurrogate_unlink == NULL) functionSurrogate_unlink = dlsym(RTLD_NEXT, "unlink");
return strstr(pathname , "Flash") != NULL ? 0 : functionSurrogate_unlink(pathname);
}
Makefile
CC=gcc
CFLAGS=-fPIC -Wall -shared -ldl
unlink_mod.so: unlink_mod.c
$(CC) $(CFLAGS) $^ -o $@
Now you can start browser with
LD_PRELOAD=/path/to/your/unlink_mod.so chromium
(Of course if your browser isn't chromium, you should replace that word. Also be shure not to have other instances of it running, because it needs to be a fresh session, otherwise it might just send a signal to the running instance to open up new window)
This approach allows many modifications. For example you could hard link the files in your home directory, or open a save dialog. Because the hack relies on a binary file virtually everything is possible.
I prefer this method, as it much more bug proof. Only if the flash binary has the unlink() call statically compiled this method gets tricky (In that case it might be possible to locate the unlink syscall in the binary and modify it).
Offline
I don't understand c syntax, but that seems a really nice and cleaner (under a certain point of view) solution, it force new flash versions to behave just like old one, right?
Help me to improve ssh-rdp !
Retroarch User? Try my koko-aio shader !
Offline
I don't understand c syntax, but that seems a really nice and cleaner (under a certain point of view) solution, it force new flash versions to behave just like old one, right?
If the old version just left the files in the /tmp folder, then yes, it behaves like the old version.
I originally got the idea, when there was (or maybe still is) a bug in flash causing a sound glitch (if i remember correctly). The bug was introduced, when glibc changed the way memcpy worked. The C standard doesn't define the behavior of memcpy when used on overlapping arrays, in that case one should call memmove. At first memcpy behaved identically (at least regarding its results an overlapping arrays) to memmove, but this changed. But the developers didn't use memmove in a case they should have done so. Linus Torwalds commented on it (also he mentions that this change was pretty unnecessary (I don't see the point of this 'optimization' too)), promoting this kind of solution.
https://bugzilla.redhat.com/show_bug.cgi?id=638477#c38 (I don't use a Redhead based distribution, but posts by Linus are pretty interesting somehow)
I also had another idea yesterday, which might be even more robust; just block the unlink call with selinux or apparmor. I haven't really worked with them, but I'm pretty sure, that it is possible to create rules, that do the trick.
Offline
lsof /usr/lib/mozilla/plugins/libflas <tab>
Then /proc/<process name>/fd
Then
for x in *; do file $x; done;
Check for "---> deleted" (Normally file 11 or 10)
That is how I used to do it before I bumped.
Awesome thread.
Offline
... At first memcpy behaved identically (at least regarding its results an overlapping arrays) to memmove, but this changed...
Not quite true, it only behaved identically when the destination buffer was before the source buffer (which it happened to be in flash I guess).
I also had another idea yesterday, which might be even more robust; just block the unlink call with selinux or apparmor. I haven't really worked with them, but I'm pretty sure, that it is possible to create rules, that do the trick.
You could use ptrace to block the unlink system call, which wouldn't require anything as heavy as selinux. However, does flash actually delete the cached files? Every time I try to grab something from its cache, it's still there.
And while we're at it, what's the need for this script in the first place? I can find all the files I need in ~/.cache/chromium/Default/Cache.
Offline
DerRoteBaron wrote:... At first memcpy behaved identically (at least regarding its results an overlapping arrays) to memmove, but this changed...
Not quite true, it only behaved identically when the destination buffer was before the source buffer (which it happened to be in flash I guess).
I also had another idea yesterday, which might be even more robust; just block the unlink call with selinux or apparmor. I haven't really worked with them, but I'm pretty sure, that it is possible to create rules, that do the trick.
You could use ptrace to block the unlink system call, which wouldn't require anything as heavy as selinux. However, does flash actually delete the cached files? Every time I try to grab something from its cache, it's still there.
And while we're at it, what's the need for this script in the first place? I can find all the files I need in ~/.cache/chromium/Default/Cache.
under firefox, flash videos are stored in /tmp and unlinked right after preloading starts. this makes it harder to copy videos directly.
all methods presented here try to copy the video in order to use an alternate player since flash is such a joke in linux.
Offline
Perhaps you will find the flash files back in /tmp again as I do with the latest firefox.
Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit! X-ray confirms Iam spineless!
Offline
And while we're at it, what's the need for this script in the first place? I can find all the files I need in ~/.cache/chromium/Default/Cache.
For me this dosen't work (on two PCs with different distros), neither firefox nor chromium places videos in cache folders. It seems to have done that in previous versions.
You could use ptrace to block the unlink system call, which wouldn't require anything as heavy as selinux.
Thanks for that tip. This seems to be one of the most stable way, to prevent the unlink call. There is a tool using ptrace to enforce certain rules called systrace. It seems pretty simple. If I have got some free time on my hand I will try it out (I have a lot of exams in the next few weeks).
After skimming through the documentation of selinux and apparmor, I'm not so sure if it can be accomplished using either of them.
Offline
I'm running flashplugin-prerelease 10.3.162.29-4 and the files are stored in the cache as always. Maybe they get deleted with the stable version?
Offline
with the 10.3 version it seems this method doesnt work anymore. i went through all my fd, and the Flash files dont show up anymore
Offline
Just tried with firefox + youtube and it works as usual
flashplugin version 10.3.181.34-1
Help me to improve ssh-rdp !
Retroarch User? Try my koko-aio shader !
Offline