You are not logged in.

#1 2012-05-18 08:04:44

whoops
Member
Registered: 2009-03-19
Posts: 891

Merge directory content w. different capitalization ("windows style")

Hi!

Is there already a Program or script or simple way to do this?

I have a folder with a huge mess in it resulting from people copying files forth and back from a windows system and randomly changing capitalizations of directory and file names on their windows machines. Files have versioning in their names, so there's no problem with overwriting - anything that would have the same name (if it weren't for the capitalization) can be overwritten. The Problem is just, that all the files & directories are spread over many many different caps versions of the same directories and there's lots of duplicates too (several times the same file with different caps in one directory or in multiple directories and subdirectories... *sigh*).

Doesn't have to be a "clean" way: my first Idea would be to just move the whole thing into a container of some sort that does not support capitalization with an "overwrite all" option and then extract it again from that container... any other quick solution is fine too, though...


thx!

Offline

#2 2012-05-18 09:27:58

tomk
Forum Fellow
From: Ireland
Registered: 2004-07-21
Posts: 9,839

Re: Merge directory content w. different capitalization ("windows style")

Change all file names to lower case, then merge all identically-named dirs.

Offline

#3 2012-05-18 09:46:48

whoops
Member
Registered: 2009-03-19
Posts: 891

Re: Merge directory content w. different capitalization ("windows style")

Yes, that's what I'm trying to do. How?

Offline

#4 2012-05-18 10:09:17

tomk
Forum Fellow
From: Ireland
Registered: 2004-07-21
Posts: 9,839

Re: Merge directory content w. different capitalization ("windows style")

You've heard of Google right? There's a whole bunch of hits for "linux change filename to lowercase".

Offline

#5 2012-05-18 11:08:22

whoops
Member
Registered: 2009-03-19
Posts: 891

Re: Merge directory content w. different capitalization ("windows style")

Yes, there's 745.000 to look trough, yay. So if you know of a method that can handle my problem, please just tell me. I found nothing that can handle a deep directory structure where making everything lowercase would result in many directories & files having the same name on many different levels... without making even more of a mess at least.

Last edited by whoops (2012-05-18 11:09:50)

Offline

#6 2012-05-18 14:50:57

tomk
Forum Fellow
From: Ireland
Registered: 2004-07-21
Posts: 9,839

Re: Merge directory content w. different capitalization ("windows style")

sigh... why don't people experiment anymore?

I'm not going to tell you how to handle your problem, because I would have to work that out for you. I don't have the data in question to hand, so it would be impossible to work it out for you anyway, even if I wanted to.

That said, if you want to provide a more detailed description, and a representative sample of what you're dealing with, I may be able to provide some pointers - if I'm in the right mood, the moon is in the fourth house, and 10,000 llamas are sacrificed.

Offline

#7 2012-05-18 16:15:25

Inxsible
Forum Fellow
From: Chicago
Registered: 2008-06-09
Posts: 9,183

Re: Merge directory content w. different capitalization ("windows style")

tomk wrote:

....if I'm in the right mood, the moon is in the fourth house, and 10,000 llamas are sacrificed.

haha !! I'd rather the llamas be spared !

@op, find is a very powerful tool.


Forum Rules

There's no such thing as a stupid question, but there sure are a lot of inquisitive idiots !

Offline

#8 2012-05-18 16:23:59

Stebalien
Member
Registered: 2010-04-27
Posts: 1,239
Website

Re: Merge directory content w. different capitalization ("windows style")

@op, to add to what Inxsible said, take a look at it's -iname and -exec flags.


Steven [ web : git ]
GPG:  327B 20CE 21EA 68CF A7748675 7C92 3221 5899 410C

Offline

#9 2012-05-18 17:23:45

whoops
Member
Registered: 2009-03-19
Posts: 891

Re: Merge directory content w. different capitalization ("windows style")

So, I guess that means the answer to my question is:
1) There's no (existing) application out there especially designed to "convert" such directories to a "windows friendly structure"?
( Thought of find and exec first, too... but if it can be done with a "simple" one-liner that's beyond me. I need loops / recursion to do something like that.)
2) There's no simple "workaround" method of mounting a "non caps filesystem" and just move everything there so the problem "overwrites" itself?
( Tried copying it to a mounted windows file system, but that only made windows crash later, the wrong capitalizations worked just fine xD )

The problem is simply, that the capitalization is random on multiple levels with many duplicates - I thought that was clear? Just imagine multiple windows users that all don't care what happens when they use caps, they just push everything into the same drive and it ends up looking like this:

About like that...:

Blue_Folder
-- Green_Folder
----- banana_file
----- BANANA_FILE
----- strawberry_file
BLUE_FOLDER
-- Green_Folder
----- BANANA_FILE
-- GREEN_Folder
----- CHERRY_FILE
----- strawberry_file
Red_Folder
-- YELLOW_FOLDER
----- apple_file
----- APPLE_file
-- yellow_folder
----- apple_file
----- tomato_file
-- YELLOW_folder
----- TOMATO_FILE

(and it's about 1-2k unique folders and files...  which means, it's about 12k total. Because the deeper the hierarchy  gets, the more duplicates. Or something. Not sure if I get it tongue - not really sure how many "real" files there are. It's a mess. Anyway, I can't test on the whole tree either, that takes too long. And if I only test on a part... well.. there could ANYTHING be hidden somewhere in there. Any sort of strange / invalid / script-unfriendly characters and probably a whole lot of stuff that I didn't even think off. )

One of the problems is, that I can't use (simple) mv's in a find/one-liner like I usually do because there might already be an existing folder with the new name and then I'll end up with something like:
-- green_folder
-- banana_file
---- green_folder
---- banana_file
------ green_folder
------ banana_file

When I tried to have the mini-script search directories only and do files later / check if the directory already exist first... something else went horribly wrong (no idea what... probably because I suck at writing scripts. Then I got confused and started searching for an existing tool that does the trick (which I didn't find) wink

Well, guess if there's no simple solution / existing program for that, I'll just have to postpone this and start working on a "real" script when I have more time...

Last edited by whoops (2012-05-18 17:30:00)

Offline

#10 2012-05-18 17:50:32

drcouzelis
Member
From: Connecticut, USA
Registered: 2009-11-09
Posts: 4,092
Website

Re: Merge directory content w. different capitalization ("windows style")

Ok, I'll take a shot at it...

mkdir ../merged # You can copy the files into this new directory
find . -exec install -d '{}' ../merged/"`to-lowercase '{}'`" ';'

So:

  • For every file and directory inside the current directory... ("find .")

  • ..."install" the file to the "merged" directory... ("-d" will create the full path if necessary)

  • ...after converting the destination path and filename to lowercase (you're responsible for finding a "to-lowercase" script from somewhere)

How's that look? I haven't tried it. Do I get an Internet cookie?

Whatever, I always wanted to learn how the "find -exec" command worked, and this gave me an excuse to.

Edit: I alost forgot. Get those people to start using a version control system, for goodness sake!

Last edited by drcouzelis (2012-05-18 17:53:32)

Offline

#11 2012-05-18 18:16:43

ewaller
Administrator
From: Pasadena, CA
Registered: 2009-07-13
Posts: 20,727

Re: Merge directory content w. different capitalization ("windows style")

You may want to look at awk.  One can put the recipe on the command line.  Even better you can create a 3 or 4 line awk program, put a magic cookie on the first line to identify it as an awk script, make it executable, and you have created a nice command you can post to the 'handy command line utilities thread' smile

I am still learning awk, so I will not embarrass myself here by even attempting an example.


Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
The shortest way to ruin a country is to give power to demagogues.— Dionysius of Halicarnassus
---
How to Ask Questions the Smart Way

Offline

#12 2012-05-18 18:21:28

drcouzelis
Member
From: Connecticut, USA
Registered: 2009-11-09
Posts: 4,092
Website

Re: Merge directory content w. different capitalization ("windows style")

ewaller wrote:

I am still learning awk, so I will not embarrass myself here by even attempting an example.

Oh, man, I "learned" awk once for a class. I was able to use it, and it's certainly powerful, but I've never felt so beaten up by a programming language. sad

Offline

#13 2012-05-18 18:49:42

Cloudef
Member
Registered: 2010-10-12
Posts: 636

Re: Merge directory content w. different capitalization ("windows style")

Another reason to use zsh

zmv '*' '${(L)f}'

http://strcat.de/zsh/#zmv

Offline

#14 2012-05-18 18:59:32

drcouzelis
Member
From: Connecticut, USA
Registered: 2009-11-09
Posts: 4,092
Website

Re: Merge directory content w. different capitalization ("windows style")

Cloudef wrote:

[zsh awesomeness]

O_O

I think you get the cookie instead.

Offline

#15 2012-05-19 08:42:11

whoops
Member
Registered: 2009-03-19
Posts: 891

Re: Merge directory content w. different capitalization ("windows style")

Thanks!

While those examples don't work without adjustments, both "install" and "zsh" look as if they're going to make writing a script a lot easier than my original fiddling around with cp / mv / etc... maybe I only have to clean up the data a little and then it'll already work.
What I figured out so far:
- Need to use fdupes first when using "install" or I'll run out of disk space.
- Next time, I'm going to delete all symlinks in there first. No idea what those are doing in there anyway and I somehow managed to overwrite some files with symlinks to those files.
- install -D looks more suited than -d, not sure yet though.
- Towards the end I somehow get a "frozen prompt" or something (behaves like an unfinished while loop without "do / done" in a script)... so I guess I really need to search for strange characters in file names first.
- Also, I'm scared of + more importantly: way too clumsy to use zsh and it's probably best for everyone if I never ever even look at it again lol

Well, guess now its only a matter of letting the machine try to do the merging a couple of times with small adjustments / additions in between... which takes up to 2 hours T_T ... but as I don't have to sit in front of it staring at the screen that shouldn't be too much of a problem - hope I won't find too many more unexpected things in there :3

Last edited by whoops (2012-05-19 08:43:00)

Offline

Board footer

Powered by FluxBB