[Bug?] wildcards

iasmb · 2009-09-18 23:43:15

This was apparently because I had a file beginning with a hyphen, removing that file fixed it. Strange it works with some versions of ls and not others.

example:

touch ./-E
ls *
rm ./-E
ls *

Is this a bug? More tests below.

-----

On x86-64, every other upgrade of coreutils appears to have issues with passing certain wilcard arguments to ls.
For example,

ls *.pdf

is fine, while

ls *.*

or

ls *.flv

results in.

ls: invalid option -- 'E'
Try `ls --help' for more information.

Can anyone else confirm before I investigate further and file a bug?

Last edited by iasmb (2009-09-19 00:53:04)

ahcaliskan · 2009-09-18 23:48:38

ls *.* works fine here on x86_64

iasmb · 2009-09-19 00:02:08

Interesting, I'm using the stock coreutils 7.6-1 package.

[iasmb@a1 ~]$ /bin/ls -l *
/bin/ls: invalid option -- 'E'
Try `/bin/ls --help' for more information.
[iasmb@a1 ~]$ md5sum /bin/ls
e7239bd2536ad4c47cabdf73506239f3  /bin/ls

Alias is the standard "--color=auto" and clearly the error occurs at option parsing before the util attempts to stat any files. The exact same thing happened on my system with the update prior to the last one, the last one worked fine again and now with this latest one it's regressed. Does anybody have any ideas about what could possibly cause this?

Last edited by iasmb (2009-09-19 00:04:55)

ahcaliskan · 2009-09-19 00:31:55

well, I thought I recognized the error message, since it happened to me a few times with files starting with hypen.

iasmb · 2009-09-19 00:46:36

ahcaliskan wrote:

well, I thought I recognized the error message, since it happened to me a few times with files starting with hypen.

In this case it was a stupid video I grabbed with youtube-dl.

What's curious is that the behaviour appears to change with different versions of ls. For cautions sake, in case this were a bug with GNU getopts, I tested rm by creating a file called "-rf *".

rm *
rm: invalid option -- ' '
Try `rm ./'-rf *'' to remove the file `-rf *'.
Try `rm --help' for more information.

Hmm. Is it normal for the result of a shell glob to be passed as a command line argument?

[iasmb@a1 ~]$ mkdir test && cd test
[iasmb@a1 test]$ touch ./-n
[iasmb@a1 test]$ echo "test" > ./test
[iasmb@a1 test]$ cat *
     1    test
[iasmb@a1 test]$

Why would ls be inconsistent between versions?

Last edited by iasmb (2009-09-19 01:13:51)

ahcaliskan · 2009-09-19 13:27:51

obviously coreutils is parsing an argument option if hypen is present. If you run "cat --help", you'll see that "-n" argument is valid, and stands for "--number", so that's why it works this time. It might work with various arguments but not all, depending on the options available for coreutils.

Last edited by ahcaliskan (2009-09-19 13:31:51)

iasmb · 2009-09-20 13:40:12

The cat example was one I created to illustrate the problem. When shell globbing is enabled and a raw wildcard used ("*" instead of "./*"), a file beginning with a hyphen should expand to "./-filename" otherwise it will be passed as an argument. IOW, don't run commands using raw wildcards on a directory where untrusted users can create files with arbitrary names. Using path prefixes "./? ./*" avoids the problem.

I still don't understand why the globbing behaviour is inconsistent between versions. I don't recall updating glibc (assuming the shell uses the glob() function from there), so it must be something else.

Last edited by iasmb (2009-09-20 13:59:01)

Daenyth · 2009-09-20 14:12:49

This is normal and unavoidable behavior. Your shell expands wildcards BEFORE the command is run, meaning that the command has no way of telling apart options you pass it from wildcard-expanded ones.

iasmb · 2009-09-21 09:17:42

Sure, but why has the behavior been inconsistent between updates to ls? Wouldn't it be safer if raw wildcards expanding to a file beginning with a hyphen were consistently escaped to "./*" or "./?"?

Something like this must have been happening because running "ls *" on a directory containing a file named like "-E" worked fine until a previous update to coreutils. The last but one update apparently escaped and the current update does not. I noticed the updates to coreutils along with the change in behaviour and I'm merely assuming that something shipping in that package is responsible. If someone can give me a start, I'll dive in and look at the code. Do both coreutils and bash use the glob() function from glibc?

ahcaliskan · 2009-09-21 13:50:21

I didn't find glob() in ls.c but ignore pattern function was found tough. I would personally follow the development at coreutils git and try to compare the revisions: http://git.savannah.gnu.org/gitweb/?p=coreutils.git

Arch Linux

#1 2009-09-18 23:43:15

[Bug?] wildcards

#2 2009-09-18 23:48:38

Re: [Bug?] wildcards

#3 2009-09-19 00:02:08

Re: [Bug?] wildcards

#4 2009-09-19 00:31:55

Re: [Bug?] wildcards

#5 2009-09-19 00:46:36

Re: [Bug?] wildcards

#6 2009-09-19 13:27:51

Re: [Bug?] wildcards

#7 2009-09-20 13:40:12

Re: [Bug?] wildcards

#8 2009-09-20 14:12:49

Re: [Bug?] wildcards

#9 2009-09-21 09:17:42

Re: [Bug?] wildcards

#10 2009-09-21 13:50:21

Re: [Bug?] wildcards

Board footer