[SOLVED] How do I mass rename files: change "_20" to single space

RobF · 2009-08-10 14:40:19

I have a large set of files in a nested directory tree in which all spaces in the original filenames and directory names had been expanded into strings of "_20" (and "(" into "_28" and ")" into "_29", etc.). How can I with a single command or script or program replace these strings of "_20" etc. with single spaces (or the other characters) in all of these file and directory names?

Edit: This problem was solved to my satisfaction. A summary of the solutions offered is posted at the end of this thread (post #22).

Last edited by RobF (2009-08-12 20:29:15)

alessandro_ufms · 2009-08-10 14:47:22

Hmm... I only know graphical programs for this: GPRename (GTK) and KRename (KDE).

tadzik · 2009-08-10 14:53:51

Why graphical? Use rename, console thing.
`rename _20 ' ' *` will replace _20 with space in every file in a dir.
Regards

jdiez · 2009-08-10 15:00:19

Try this shell script (UNTESTED)

for i in *;
do
mv $i "`echo $i | sed 's/_20/ /g'`"
done

EDIT:
Just tested it, it works.

Last edited by jdiez (2009-08-10 15:03:18)

jdiez · 2009-08-10 15:04:20

Whoops! Sorry tadzik, I didn't read your answer. Looks more versatile than my quick and ugly shell script

RobF · 2009-08-10 15:48:21

tadzik wrote:

Why graphical? Use rename, console thing.
`rename _20 ' ' *` will replace _20 with space in every file in a dir.
Regards

Thanks, tadzik.

rename _20 ' ' *

does some of what's needed, i.e. it does replace a single occurrence of _20 with a single space. However, every filename has multiple occurrences of _20, hence I have to run this command as many times. Also, it only works within the pwd and it doesn't rename directory names. I'd like to find a way of doing the mass renaming on every occurrence of _20 etc. in every file and directory name throughout a nested directory structure up to 4 levels deep.

I'll try jdiez's script next.

jdiez · 2009-08-10 15:50:28

My script renames every occurence of _20, but only on the script's directory.

tadzik · 2009-08-10 15:56:27

RobF - just read the rename's manpage, You'll know everything, I'm quite sure rename can handle directory tree as well. And it can handle regexps afair, so It'll solve your multiple _20's problem

RobF · 2009-08-10 17:17:01

Thanks, tadzik and jdiez. I managed to get the job done partially manually, using your ideas, i.e. running the following from the command line through all directories and all of their levels:

for i in *; do mv $i "`echo $i | sed 's/_20/ /g'`"; done
rename _28 '(' *
... and so on

brisbin33 · 2009-08-10 18:20:43

this has come up so many times. i wrote this script which i find works best, recursively renaming files and directories from the bottom up:

find "$1" -depth | while read file do
  dir="$(dirname "$file")"
  old="$(basename "$file")"
  new="$(echo $old | sed 's/foo/bar/g;s/baz/bat/g')"
  [ "$old" != "$new" ] && mv -iv "$dir/$old" "$dir/$new"
done

the above switches foo with bar, then baz with bat... adjust these to 's/_20/\ /g;s/_28/$/g;s/_29/$/g' and so on.

funny, i use it to remove spaces...

Last edited by brisbin33 (2009-08-10 18:22:31)

pizmooz · 2009-08-10 18:31:04

Working with spaces in files can be tricky, since bash uses them as a param delimeters. Using perl allows you to not have to deal with all the space escaping BS. Since find acts recursively, I would combine find and perl and do it in one shot:

find . | perl -n -e 'chomp;$old=$new=$_; $new=~s/(_20|_28|_29)/ /g; rename($old,$new)'

of course you need to cd to the directory first. This has been tested BTW

RobF · 2009-08-11 13:50:21

pizmooz wrote:

Working with spaces in files can be tricky, since bash uses them as a param delimeters. Using perl allows you to not have to deal with all the space escaping BS. Since find acts recursively, I would combine find and perl and do it in one shot:
find . | perl -n -e 'chomp;$old=$new=$_; $new=~s/(_20|_28|_29)/ /g; rename($old,$new)'
of course you need to cd to the directory first. This has been tested BTW

Thanks, pizmooz, for your solution. It works well; I wished I'd had it yesterday, it would have saved me 45 min of doing many of these swaps manually.

However, as it stands, it doesn't seem to work recursively, down the directory tree. Can that be added to it? Also, is it possible to do multiple swaps in one pass, i.e. changing _20 to <space>, _28 to (, _29 to ), _2C to &, _26 to , etc, all at the same time?

brisbin33 · 2009-08-11 13:53:40

RobF wrote:

However, as it stands, it doesn't seem to work recursively, down the directory tree. Can that be added to it? Also, is it possible to do multiple swaps in one pass, i.e. changing _20 to <space>, _28 to (, _29 to ), _2C to &, _26 to , etc, all at the same time?

if i knew perl i'd try to adjust his one-liner; but as it stands, the short script i posted above does both of these things already.

Last edited by brisbin33 (2009-08-11 13:54:06)

Trent · 2009-08-11 14:26:36

This uses the hex and chr functions, along with the /e (evaluate) switch to s///, to replace all the codes at once. It will choke if you feed it _2F sequences, though, because you can't create files with slashes in their names.

find . | perl -n -e 'chomp; $new=$_; $new=~s/_(\d{2})/chr hex $1/eg; rename($_, $new)'

Edit: copied my working version instead of my finished version
Edit 2: ok, maybe not... it will fail if it tries to change a file inside a folder that has been renamed already. If it were my script, I'd just run it seven times or so and make sure it penetrates all the way to the bottom of the hierarchy, but it may be that you need to do a variation on brisbin33's idea instead.

Last edited by Trent (2009-08-11 14:46:09)

RobF · 2009-08-11 20:04:09

brisbin33 wrote:

RobF wrote:
However, as it stands, it doesn't seem to work recursively, down the directory tree. Can that be added to it? Also, is it possible to do multiple swaps in one pass, i.e. changing _20 to <space>, _28 to (, _29 to ), _2C to &, _26 to , etc, all at the same time?
if i knew perl i'd try to adjust his one-liner; but as it stands, the short script i posted above does both of these things already.

Thanks, brisbin33, for your script. I tried to run it in the following version ("swapstrings"):

#!bin/bash
find "$1" -depth | while read file do
  dir="$(dirname "$file")"
  old="$(basename "$file")"
  new="$(echo $old | sed 's/_20/\ /g;s/_28/\(/g;s/_29/\)/g;s/_2C/\,/g;s/_26/\&/g')"
  [ "$old" != "$new" ] && mv -iv "$dir/$old" "$dir/$new"
done

but I got the error

swapstrings: line 7: syntax error near unexpected token `done'                                  
swapstrings: line 7: `done'

The most efficient way that I've found to get this job done is by running a perl program "RecursiveRegexpRename", written by Andrew Hardwick (http://duramecho.com/ComputerPrograms/R … expRename/). I simply ran it from the top directory for each of the cases of swapping the "???" strings to <space>, (, ), &, and , i.e.

perl RecursiveRegexpRename.pl -m g '_20' ' '
perl RecursiveRegexpRename.pl -m g '_28' '('
... and so on

That was all. It also has the virtue of giving you the option of doing a dry run, with no changes written, to show you what will be changed.

brisbin33 · 2009-08-11 21:08:28

RobF wrote:

but I got the error...

bah, i fat fingered.

change it like this:

#!bin/bash
find "$1" -depth | while read file; do
  dir="$(dirname "$file")"
  old="$(basename "$file")"
  new="$(echo $old | sed 's/_20/\ /g;s/_28/\(/g;s/_29/\)/g;s/_2C/\,/g;s/_26/\&/g')"
  [ "$old" != "$new" ] && mv -iv "$dir/$old" "$dir/$new"
done

and it should work just fine.

*while read file ; do is the mistake if you didn't catch it.

RobF · 2009-08-11 23:06:57

brisbin33, I'll try your corrected script tomorrow.

RobF · 2009-08-12 08:50:37

Ran

#!bin/bash
find "$1" -depth | while read file; do
  dir="$(dirname "$file")"
  old="$(basename "$file")"
  new="$(echo $old | sed 's/_20/\ /g;s/_28/\(/g;s/_29/\)/g;s/_2C/\,/g;s/_26/\&/g')"
  [ "$old" != "$new" ] && mv -iv "$dir/$old" "$dir/$new"
done

Got error:

find: cannot search `': No such file or directory

RobF · 2009-08-12 11:41:41

The Perl script "RecursiveRegexpRename" actually is able to do the entire job in one pass:

Dry run:
perl RecursiveRegexpRename.pl -t -m ge '_([0-9A-F]{2})' 'chr(hex($1))' > dryrun.txt

Actual run:
perl RecursiveRegexpRename.pl -m ge '_([0-9A-F]{2})' 'chr(hex($1))'

A dry run and checking the output is advisable as it's possible that strings may be renamed that don't actually represent hex characters. If you also want to change lowercase hex character strings, change the flag ge to gei.

Many thanks to script author Andrew Hardwick for these tips. Good program!

brisbin33 · 2009-08-12 13:32:30

lolwut?

┌─[ 09:30 ][ blue:~/Temp ]
└─> lt test
test
|-- dum_20dir
|   |-- dum_20dir2
|   |   |-- dum_20file
|   |   |-- dum_26file
|   |   |-- dum_28file
|   |   `-- dum_2Cfile
|   |-- dum_20file
|   |-- dum_26dir2
|   |   |-- dum_20file
|   |   |-- dum_26file
|   |   |-- dum_28file
|   |   `-- dum_2Cfile
|   |-- dum_26file
|   |-- dum_28file
|   `-- dum_2Cfile
`-- dum_28dir
    |-- dum_20dir2
    |   |-- dum_20file
    |   |-- dum_26file
    |   |-- dum_28file
    |   `-- dum_2Cfile
    |-- dum_20file
    |-- dum_26dir2
    |   |-- dum_20file
    |   |-- dum_26file
    |   |-- dum_28file
    |   `-- dum_2Cfile
    |-- dum_26file
    |-- dum_28file
    `-- dum_2Cfile

6 directories, 24 files
┌─[ 09:24 ][ blue:~/Temp ]
└─> find ./test -depth | while read file; do
  >   dir="$(dirname "$file")"
  >   old="$(basename "$file")"
  >   new="$(echo $old | sed 's/_20/\ /g;s/_28/\(/g;s/_29/\)/g;s/_2C/\,/g;s/_26/\&/g')"
  >   [ "$old" != "$new" ] && mv -iv "$dir/$old" "$dir/$new"
  > done
`./test/dum_28dir/dum_26dir2/dum_2Cfile' -> `./test/dum_28dir/dum_26dir2/dum,file'
`./test/dum_28dir/dum_26dir2/dum_28file' -> `./test/dum_28dir/dum_26dir2/dum(file'
`./test/dum_28dir/dum_26dir2/dum_20file' -> `./test/dum_28dir/dum_26dir2/dum file'
`./test/dum_28dir/dum_26dir2/dum_26file' -> `./test/dum_28dir/dum_26dir2/dum&file'
`./test/dum_28dir/dum_26dir2' -> `./test/dum_28dir/dum&dir2'
`./test/dum_28dir/dum_2Cfile' -> `./test/dum_28dir/dum,file'
`./test/dum_28dir/dum_28file' -> `./test/dum_28dir/dum(file'
`./test/dum_28dir/dum_20file' -> `./test/dum_28dir/dum file'
`./test/dum_28dir/dum_20dir2/dum_2Cfile' -> `./test/dum_28dir/dum_20dir2/dum,file'
`./test/dum_28dir/dum_20dir2/dum_28file' -> `./test/dum_28dir/dum_20dir2/dum(file'
`./test/dum_28dir/dum_20dir2/dum_20file' -> `./test/dum_28dir/dum_20dir2/dum file'
`./test/dum_28dir/dum_20dir2/dum_26file' -> `./test/dum_28dir/dum_20dir2/dum&file'
`./test/dum_28dir/dum_20dir2' -> `./test/dum_28dir/dum dir2'
`./test/dum_28dir/dum_26file' -> `./test/dum_28dir/dum&file'
`./test/dum_28dir' -> `./test/dum(dir'
`./test/dum_20dir/dum_26dir2/dum_2Cfile' -> `./test/dum_20dir/dum_26dir2/dum,file'
`./test/dum_20dir/dum_26dir2/dum_28file' -> `./test/dum_20dir/dum_26dir2/dum(file'
`./test/dum_20dir/dum_26dir2/dum_20file' -> `./test/dum_20dir/dum_26dir2/dum file'
`./test/dum_20dir/dum_26dir2/dum_26file' -> `./test/dum_20dir/dum_26dir2/dum&file'
`./test/dum_20dir/dum_26dir2' -> `./test/dum_20dir/dum&dir2'
`./test/dum_20dir/dum_2Cfile' -> `./test/dum_20dir/dum,file'
`./test/dum_20dir/dum_28file' -> `./test/dum_20dir/dum(file'
`./test/dum_20dir/dum_20file' -> `./test/dum_20dir/dum file'
`./test/dum_20dir/dum_20dir2/dum_2Cfile' -> `./test/dum_20dir/dum_20dir2/dum,file'
`./test/dum_20dir/dum_20dir2/dum_28file' -> `./test/dum_20dir/dum_20dir2/dum(file'
`./test/dum_20dir/dum_20dir2/dum_20file' -> `./test/dum_20dir/dum_20dir2/dum file'
`./test/dum_20dir/dum_20dir2/dum_26file' -> `./test/dum_20dir/dum_20dir2/dum&file'
`./test/dum_20dir/dum_20dir2' -> `./test/dum_20dir/dum dir2'
`./test/dum_20dir/dum_26file' -> `./test/dum_20dir/dum&file'
`./test/dum_20dir' -> `./test/dum dir'
┌─[ 09:31 ][ blue:~/Temp ]
└─> lt test
test
|-- dum dir
|   |-- dum dir2
|   |   |-- dum file
|   |   |-- dum&file
|   |   |-- dum(file
|   |   `-- dum,file
|   |-- dum file
|   |-- dum&dir2
|   |   |-- dum file
|   |   |-- dum&file
|   |   |-- dum(file
|   |   `-- dum,file
|   |-- dum&file
|   |-- dum(file
|   `-- dum,file
`-- dum(dir
    |-- dum dir2
    |   |-- dum file
    |   |-- dum&file
    |   |-- dum(file
    |   `-- dum,file
    |-- dum file
    |-- dum&dir2
    |   |-- dum file
    |   |-- dum&file
    |   |-- dum(file
    |   `-- dum,file
    |-- dum&file
    |-- dum(file
    `-- dum,file

6 directories, 24 files

i can't explain your error. worked like a charm for me.

/edit: i fear you didn't realize that `find "$1"...` means you need to pass the directory to act on

script.sh /path/to/directory

Last edited by brisbin33 (2009-08-12 13:37:13)

RobF · 2009-08-12 15:13:27

brisbin33 wrote:

/edit: i fear you didn't realize that `find "$1"...` means you need to pass the directory to act on
script.sh /path/to/directory

Yes, I did fail to pass the directory to act on. I apologize if I caused you unnecessary work - I know practically nothing about bash shell scripting, although I want to learn at least the rudiments of it. I ran your (corrected) script on my set of 352 files in 74 subdirectories, nested four levels deep, and indeed it worked like a charm, making all the correct changes. Thanks for your help.

RobF · 2009-08-12 20:23:45

=== Summary of methods of bulk renaming files/dirs discussed in this thread ===

Problem: I have a large set of files (352) in a nested directory tree (total of 74 subdirectories, up to 4 levels deep) in which all spaces and several other characters in the original file and directory names had been expanded into strings of "_20" for spaces (and into "_28" for "(" and "_29" for ")", etc.). How can I, with a single command or script or program, replace these strings of "_20" etc. with single spaces or the other characters, resp., for every occurrence of these strings in every single one of these file and directory names?

To be specific, I want to change:

_20 to <space>
_28 to (
_29 to )
_2C to ,
_26 to &

Solutions offered:

1.

$ rename _20 ' ' *     (... and so on for all 5 cases)

This command replaces a single occurrence of _20 in every filename with a single space. Repeat for every other occurrence of this string in the same filename. It only works within the pwd, and it doesn't rename directory names.

2. Write the following shell script (courtesy of jdiez) and run it in the top directory

#!bin/bash
for i in *;
do
mv $i "`echo $i | sed 's/_20/ /g'`"
done

or simply run it on the command line:

$ for i in *; do mv $i "`echo $i | sed 's/_20/ /g'`"; done

Repeat for all 5 cases.

3. Run the following Perl script (courtesy of pizmooz)

$ find . | perl -n -e 'chomp;$old=$new=$_; $new=~s/(_20|_28|_29)/ /g; rename($old,$new)'

However, this script doesn't seem to work recursively, all the way down the directory tree. The above example, as written, creates spaces in lieu of _20, _28, _29. For other replacements adjust accordingly.

4. Write the following shell script "swapstrings.sh" (courtesy of brisbin33)

#!bin/bash
find "$1" -depth | while read file; do
  dir="$(dirname "$file")"
  old="$(basename "$file")"
  new="$(echo $old | sed 's/_20/\ /g;s/_28/\(/g;s/_29/\)/g;s/_2C/\,/g;s/_26/\&/g')"
  [ "$old" != "$new" ] && mv -iv "$dir/$old" "$dir/$new"
done

and run it in the top directory:

$ sh swapstrings.sh directory_name

5. Run the perl program "RecursiveRegexpRename", written by Andrew Hardwick (http://duramecho.com/ComputerPrograms/R … expRename/). Run it from the top directory for each of the five cases, i.e.

perl RecursiveRegexpRename.pl -m g '_20' ' '
perl RecursiveRegexpRename.pl -m g '_28' '('
... and so on

OR run a single pass with a syntax specific to replacing hex chars, as follows:

Dry run (check dryrun.txt to see whether everything came out all right):

perl RecursiveRegexpRename.pl -t -m ge '_([0-9A-F]{2})' 'chr(hex($1))' > dryrun.txt

Actual run, writes all changes:

perl RecursiveRegexpRename.pl -m ge '_([0-9A-F]{2})' 'chr(hex($1))'

Methods #4 and #5 are the best.

portix · 2009-08-12 21:47:07

If globstar is enabled you can also use

for i in **/*; do 
    mv $i `sed 's/_20/\ /g;s/_28/\(/g;s/_29/\)/g;s/_2C/\,/g;s/_26/\&/g' <<<$i`
done

Procyon · 2009-08-12 22:54:15

And an extra echo -e will interpret the hex codes if you convert it to \x20

$ file "$(echo -e $(echo a_20_28b_29 | sed 's/_\([0-9A-F]\{2\}\)/\\x\1/g'))"
a (b): directory

Arch Linux

#1 2009-08-10 14:40:19

[SOLVED] How do I mass rename files: change "_20" to single space

#2 2009-08-10 14:47:22

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#3 2009-08-10 14:53:51

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#4 2009-08-10 15:00:19

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#5 2009-08-10 15:04:20

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#6 2009-08-10 15:48:21

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#7 2009-08-10 15:50:28

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#8 2009-08-10 15:56:27

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#9 2009-08-10 17:17:01

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#10 2009-08-10 18:20:43

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#11 2009-08-10 18:31:04

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#12 2009-08-11 13:50:21

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#13 2009-08-11 13:53:40

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#14 2009-08-11 14:26:36

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#15 2009-08-11 20:04:09

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#16 2009-08-11 21:08:28

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#17 2009-08-11 23:06:57

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#18 2009-08-12 08:50:37

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#19 2009-08-12 11:41:41

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#20 2009-08-12 13:32:30

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#21 2009-08-12 15:13:27

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#22 2009-08-12 20:23:45

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#23 2009-08-12 21:47:07

Re: [SOLVED] How do I mass rename files: change "_20" to single space

#24 2009-08-12 22:54:15

Re: [SOLVED] How do I mass rename files: change "_20" to single space

Board footer