Mass renaming files (space to underscore)

justin · 2007-08-15 21:44:39

Is there an easy way to replace spaces in filenames with underscores?

A command that renames all files in a folder would be great, or maybe a recommendation for a program that is decent at renaming files.

paramthegreat · 2007-08-15 21:54:16

You can create a bash script that will rename all the files and directories under your current directory. Just create a file called rename or whatever with the following code and make it executable.

#!/bin/bash

ls | while read -r FILE
do
    mv -v "$FILE" `echo $FILE | tr ' ' '_' | tr -d '[{}(),\!]' | tr -d "\'" | tr '[A-Z]' '[a-z]' | sed 's/_-_/_/g'`
done

Then just execute that file. The above code will convert all your spaces to underscores, remove characters like {}(),\! and convert the filename to lowercase. If you just want to convert spaces to underscrore, use this.

#!/bin/bash

ls | while read -r FILE
do
    mv -v "$FILE" `echo $FILE | tr ' ' '_' `
done

justin · 2007-08-15 22:13:03

Is it possible to incorporate that into .bashrc?

shining · 2007-08-15 22:19:01

justin wrote:

Is it possible to incorporate that into .bashrc?

What do you mean ?

Otherwise, an alternative that you can try :

rename ' ' '_' *

justin · 2007-08-15 22:23:00

so instead having to run a script named rmspaces, I just have a command 'rmspaces'

paramthegreat · 2007-08-15 22:51:04

You know running a script is the same as running a command. Lets say you put your script called rmspaces in /home/justin/scripts. You can always run it as /home/justin/scripts/rmspaces. Otherwise, just append /home/justin/scripts to your PATH variable in your .bashrc and you should be able to run 'rmspaces'.

shining · 2007-08-15 22:52:53

justin wrote:

so instead having to run a script named rmspaces, I just have a command 'rmspaces'

Ok, I was not sure that's what you meant.
For simple commands, you can just define an alias :

alias rmspaces="rename ' ' '_' *"

otherwise you can make a function :

rmspaces () {
rename ' ' '_' *
}

I just noticed rename only replaces the first occurence, so that's not ideal, it would need to be run several times.
Rather use the first proposed one.

Gilneas · 2007-08-15 23:13:48

imho, your "ls | while read -r FILE" might be nicer as a "for FILE in *"

paramthegreat · 2007-08-16 00:06:35

Gilneas wrote:

imho, your "ls | while read -r FILE" might be nicer as a "for FILE in *"

I thought so too, but its turns out that 'ls | while read -r' is better for working with filenames that have funky characters in them, especially when you are working with filenames from a windows system. I had to process a large library of songs, and the for loop failed a bunch of times but WHILE read worked quite well.

Off course, I do not claim to be an expert on bash scripts, I might be wrong, but personal experience leads me to trust WHILE better (in terms of accuracy, don't know about speed)

drakosha · 2007-08-16 07:12:47

Nice discussion. '*' should not be used with filenames with spaces, it doesn't work well. I once wrote about my experience with such renaming here

George_K · 2007-08-16 07:26:52

drakosha wrote:

'*' should not be used with filenames with spaces, it doesn't work well. I once wrote about my experience with such renaming here

In fact `ls` shouldn`t be used for filenames with spaces but '*' should. Of course "for f in `ls`" will not work because bash treats whitespaces as delimiters. Your script with "cut -b 5-" will look like this:

for f in *; do mv $f "`echo $f | cut -b 5-`"; done

Purch · 2007-08-16 08:53:22

I use this script to do the job recursively. Just add or remove sed or tr commands to modify the result.

Note this fixes the directory names too recursively. First it converts the file names of the directory and after that fixes the directory name.

#!/bin/bash
# Convert filenames to lowercase
# and replace characters recursively
#####################################

if [ -z $1 ];then echo Give target directory; exit 0;fi

find "$1" -depth -name '*' | while read file ; do
        directory=$(dirname "$file")
        oldfilename=$(basename "$file")
        newfilename=$(echo "$oldfilename" | tr 'A-Z' 'a-z' | tr ' ' '_' | sed 's/_-_/-/g')
        if [ "$oldfilename" != "$newfilename" ]; then
                mv -i "$directory/$oldfilename" "$directory/$newfilename"
                echo ""$directory/$oldfilename" ---> "$directory/$newfilename""
                #echo "$directory"
                #echo "$oldfilename"
                #echo "$newfilename"
                #echo
        fi
        done
exit 0

MrWeatherbee · 2007-08-16 09:07:45

George_K wrote:

Of course "for f in `ls`" will not work because bash treats whitespaces as delimiters. Your script with "cut -b 5-" will look like this:
for f in *; do mv $f "`echo $f | cut -b 5-`"; done

You can set the Input Field Separator (IFS variable) ... so `ls` could be used like this:

> orig_ifs="$IFS";IFS=$'\n';for f in `ls`; do echo mv $f `echo $f | cut -b 5-`; done; IFS="$orig_ifs"

mv this is file 1.txt  is file 1.txt
mv this is file 2.txt  is file 2.txt

Though I'm not recommending this method when other methods are more suitable, it does illustrate how things usually can be done one way or the other. At any rate, it is usually best to reset the IFS variable explicitly as shown above, but if you don't, it will be reset automatically upon opening a new shell so you really can't mess things up too badly. .

Last edited by MrWeatherbee (2007-08-16 09:10:51)

drakosha · 2007-08-16 09:10:34

George_K wrote:

drakosha wrote:
'*' should not be used with filenames with spaces, it doesn't work well. I once wrote about my experience with such renaming here
In fact `ls` shouldn`t be used for filenames with spaces but '*' should. Of course "for f in `ls`" will not work because bash treats whitespaces as delimiters. Your script with "cut -b 5-" will look like this:
for f in *; do mv $f "`echo $f | cut -b 5-`"; done

There're problems with file names which have leading/trailing spaces. Here's an example: (note first filename has leading space)

bash-3.00$ ls -l
total 0
-rw-r--r--  1 uu uu 0 Aug 16 12:08  1 2 3
-rw-r--r--  1 uu uu 0 Aug 16 12:08 1 2 3
bash-3.00$ for f in *; do echo $f; done
1 2 3
1 2 3

Last edited by drakosha (2007-08-16 09:11:04)

justin · 2007-08-16 17:00:55

Someone at the Ubuntu forums suggested this awhile back:

find . -depth|rename 's/\ /_/g'

but it didn't seem to work under Arch.

quantax · 2007-08-16 18:40:17

justin wrote:

but it didn't seem to work under Arch.

I have the same problem. But according to http://wiki.linuxquestions.org/wiki/Rename there are two different programs called "rename". The default one and the Perl-thingy which understands regular expressions.
On the page it also says:

It comes with the perl package in the Debian distribution.

But I can't find it here on Arch. Is it hidden somewhere, or isn't it in the Arch Perl package?

Edit: Maybe this is the Perl-script: http://search.cpan.org/~pederst/rename-1.4/

Last edited by quantax (2007-08-16 19:24:40)

George_K · 2007-08-16 21:42:00

drakosha wrote:

There're problems with file names which have leading/trailing spaces. Here's an example: (note first filename has leading space)
bash-3.00$ ls -l
total 0
-rw-r--r--  1 uu uu 0 Aug 16 12:08  1 2 3
-rw-r--r--  1 uu uu 0 Aug 16 12:08 1 2 3
bash-3.00$ for f in *; do echo $f; done
1 2 3
1 2 3

Interesting that this command works fine in zsh, but not in bash. In bash you should use

for f in *; do echo "$f"; done

My code also works only in zsh. In bash it will look like this:

for f in *; do mv "$f" "`echo $f | cut -b 5-`"; done

Sorry for not testing this in bash, but after I switched to zsh I became a little more lazier.

MrWeatherbee wrote:

orig_ifs="$IFS";IFS=$'\n';for f in `ls`; do echo mv $f `echo $f | cut -b 5-`; done; IFS="$orig_ifs"
At any rate, it is usually best to reset the IFS variable explicitly as shown above, but if you don't, it will be reset automatically upon opening a new shell so you really can't mess things up too badly. :)

Thank you. Also, I would use this method to back up IFS ;)

(IFS=$'\n';for f in `ls`; do echo mv $f `echo $f | cut -b 5-`; done)

Last edited by George_K (2007-08-16 21:42:47)

chilebiker · 2007-08-16 22:56:13

In case you use KDE, there's a nice tool called KRename. It's perfect for that kind of jobs!

pelle.k · 2007-08-17 01:44:54

Another way i guess...

 for file in *; do [ -f "$file" ] && ( mv "$file" "$(echo $file | sed -e 's/ /_/g')" ); done

codemac · 2007-08-17 15:49:57

If you want to use zsh, check out zmv.

cdhotfire · 2007-08-21 16:27:49

You could also use mmv which is in AUR.

Gilneas · 2007-11-15 20:31:46

I recently ran into a problem with one of the scripts here that makes all the entries in a directory normal.
The first problem is files that start with a -
And also, if a normal version of a filename already exists it will be overwritten.
Here's a script I've been trying out. Using "mv --", and it appends a + in case of the second problem, to preserve (double) extension either before .foo.bar, before .bar, or at the end of the filename (i.e. it has no extension).

#! /bin/bash

for FILE in *; do
NEWFILE="$(echo $FILE | \
        sed -e 'y#QWERTYUIOPASDFGHJKLZXCVBNM#qwertyuiopasdfghjklzxcvbnm#
                s#[^_.a-z0-9]#_#g
                s#__*#_#g
                s#^_##')"
if [ x"$FILE" = x"$NEWFILE" ]; then
        continue
fi
if [ -z $NEWFILE ]; then NEWFILE=+; fi
while [ -e $NEWFILE ]; do
        NEWFILE=$(echo $NEWFILE | sed -e 's#^\(.*\)\.\(.*\)\.\(.*\)$#\1+.\2.\3#
                                          t
                                          s#^\(.*\)\.\(.*\)$#\1+.\2#
                                          t
                                          s#$#+#')
done
mv -v -- "$FILE" "$NEWFILE"
done

Edit: I changed the first line in the first sed command. s can't do translations that way, you need to use y, and y can only work with two alphabets, not a range. (11/21/07)
Edit2: I added a check to see if the new filename is not empty. This happens when the file has only special characters, so it's not uncommon at all.(12/29/07)

Last edited by Gilneas (2007-12-29 00:28:15)

tam1138 · 2007-11-15 20:37:45

I found myself needing to rename files according to a random regex, so I wrote this script, which I now use all the time:

#!/bin/sh
#
# mv-regex.sh [-n] regex < filenames
#
# TODO: make "orig -> new" output optional with -v flag

[ "$1" == "-n" ] && export PRETEND="true" && shift

if [ -z "$1" ]; then
    echo "usage: $0 [-n] regex... < filenames" >&2
fi

SED_CMD="sed"
while [ -n "$*" ]; do
    SED_CMD="$SED_CMD -e $1"
    shift
done

echo "sed command is: $SED_CMD"
#exit 0

while read file; do
    newfile=`echo $file | $SED_CMD`

    if [ "$newfile" != "$file" ]; then
        echo "$file -> $newfile"
        [ "$PRETEND" == "true" ] || mv "$file" "$newfile"
    else
        echo "$file: no change"
    fi
done

I think the only caveat is that it doesn't like more than one regex on the command-line.

Last edited by tam1138 (2007-11-15 20:38:20)

Gilneas · 2007-11-15 20:45:15

tam1138: I can see how that's more useful in general purpose, and note that you _can_ put more regexes on there, just like you can with sed (i.e. in a big single quoted multi-line script).
And also note the two problems I states and their solution. Although they are rare, you may want to add them to your script.

bluetech · 2007-11-15 22:21:57

I needed a script to recursively rename everything to lowercase. Naturally, I started out with bash:

#!/bin/sh
# recursively rename all files and directories in a given folder to lowercase

if [[ $# == 0 ]]; then
    echo "usage: lower PATH"
    exit 2
elif [[ ! -d "$1" ]]; then
    echo "lower: $1 not a directory"
    exit 1
fi

find "$1" -nowarn -depth | head -n-1 | while read -r path; do
    newpath="$(dirname "$path")/$(basename "$path" | tr 'A-Z' 'a-z')"
    if [[ ! -e "$newpath" ]]; then
        mv "$path" "$newpath"
    fi
done

but quickly ended up with a python script:

#!/usr/bin/python
# recursively rename all files and directories in a given folder to lowercase

import sys, os, itertools

if len(sys.argv) < 2:
    print >> sys.stderr, "usage: lower PATH"
    sys.exit(2)
if not os.path.isdir(sys.argv[1]):
    print >> sys.stderr, sys.argv[1], "not a directory"
    sys.exit(1)

for root, dirs, files in os.walk(sys.argv[1]):
    for path in itertools.chain(files, dirs):
        old, new = os.path.join(root, path), os.path.join(root, path.lower())
        if not os.path.exists(new):
            os.rename(old, new)

which works faster and better and spares me the entire shell headache. Admittedly, as my bash script would suggest, I am not a shell guru, but I just couldn't figure out why it fails on certain filenames, and also couldn't find a way to make it as quick as the python version.

Arch Linux

#1 2007-08-15 21:44:39

Mass renaming files (space to underscore)

#2 2007-08-15 21:54:16

Re: Mass renaming files (space to underscore)

#3 2007-08-15 22:13:03

Re: Mass renaming files (space to underscore)

#4 2007-08-15 22:19:01

Re: Mass renaming files (space to underscore)

#5 2007-08-15 22:23:00

Re: Mass renaming files (space to underscore)

#6 2007-08-15 22:51:04

Re: Mass renaming files (space to underscore)

#7 2007-08-15 22:52:53

Re: Mass renaming files (space to underscore)

#8 2007-08-15 23:13:48

Re: Mass renaming files (space to underscore)

#9 2007-08-16 00:06:35

Re: Mass renaming files (space to underscore)

#10 2007-08-16 07:12:47

Re: Mass renaming files (space to underscore)

#11 2007-08-16 07:26:52

Re: Mass renaming files (space to underscore)

#12 2007-08-16 08:53:22

Re: Mass renaming files (space to underscore)

#13 2007-08-16 09:07:45

Re: Mass renaming files (space to underscore)

#14 2007-08-16 09:10:34

Re: Mass renaming files (space to underscore)

#15 2007-08-16 17:00:55

Re: Mass renaming files (space to underscore)

#16 2007-08-16 18:40:17

Re: Mass renaming files (space to underscore)

#17 2007-08-16 21:42:00

Re: Mass renaming files (space to underscore)

#18 2007-08-16 22:56:13

Re: Mass renaming files (space to underscore)

#19 2007-08-17 01:44:54

Re: Mass renaming files (space to underscore)

#20 2007-08-17 15:49:57

Re: Mass renaming files (space to underscore)

#21 2007-08-21 16:27:49

Re: Mass renaming files (space to underscore)

#22 2007-11-15 20:31:46

Re: Mass renaming files (space to underscore)

#23 2007-11-15 20:37:45

Re: Mass renaming files (space to underscore)

#24 2007-11-15 20:45:15

Re: Mass renaming files (space to underscore)

#25 2007-11-15 22:21:57

Re: Mass renaming files (space to underscore)

Board footer