You are not logged in.

#1 2020-04-03 08:32:52

mixxoo
Member
Registered: 2017-05-04
Posts: 10

[SOLVED] BASH - parsing options via command substitution

Hello,

I was trying to build (actually preprocess only, but here it does not matter) the first simple GTK application with following command :

gcc -v -E $(pkgconf --cflags --libs gtk+-3.0) -o gtkhello gtkhello.c

Output of the pkgconf command alone is following :

pkgconf --cflags --libs gtk+-3.0
-I/usr/include/gtk-3.0 -I/usr/include/pango-1.0 ...etc... -lglib-2.0

seems to be correct.

However GCC interprets it like this :

gcc -v -E $(pkgconf --cflags --libs gtk+-3.0) -o gtkhello gtkhello.c
Using built-in specs.
COLLECT_GCC=gcc
Target: x86_64-pc-linux-gnu
Configured with: /build/gcc/src/gcc/configure --prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-pkgversion='Arch Linux 9.3.0-1' --with-bugurl=https://bugs.archlinux.org/ --enable-languages=c,c++,ada,fortran,go,lto,objc,obj-c++,d --enable-shared --enable-threads=posix --with-system-zlib --with-isl --enable-__cxa_atexit --disable-libunwind-exceptions --enable-clocale=gnu --disable-libstdcxx-pch --disable-libssp --enable-gnu-unique-object --enable-linker-build-id --enable-lto --enable-plugin --enable-install-libiberty --with-linker-hash-style=gnu --enable-gnu-indirect-function --enable-multilib --disable-werror --enable-checking=release --enable-default-pie --enable-default-ssp --enable-cet=auto gdc_include_dir=/usr/include/dlang/gdc
Thread model: posix
gcc version 9.3.0 (Arch Linux 9.3.0-1) 
COLLECT_GCC_OPTIONS='-v' '-E' '-I' '/usr/include/gtk-3.0 -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/lib/libffi-3.2.1/include -I/usr/include/harfbuzz -I/usr/include/fribidi -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/gio-unix-2.0 -I/usr/include/atk-1.0 -I/usr/include/at-spi2-atk/2.0 -I/usr/include/dbus-1.0 -I/usr/lib/dbus-1.0/include -I/usr/include/at-spi-2.0 -pthread -lgtk-3 -lgdk-3 -lz -lpangocairo-1.0 -lpango-1.0 -lharfbuzz -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 ' '-o' 'gtkhello' '-mtune=generic' '-march=x86-64'
 /usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/cc1 -E -quiet -v -I /usr/include/gtk-3.0 -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/lib/libffi-3.2.1/include -I/usr/include/harfbuzz -I/usr/include/fribidi -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/gio-unix-2.0 -I/usr/include/atk-1.0 -I/usr/include/at-spi2-atk/2.0 -I/usr/include/dbus-1.0 -I/usr/lib/dbus-1.0/include -I/usr/include/at-spi-2.0 -pthread -lgtk-3 -lgdk-3 -lz -lpangocairo-1.0 -lpango-1.0 -lharfbuzz -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0  gtkhello.c -o gtkhello -mtune=generic -march=x86-64
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/../../../../x86_64-pc-linux-gnu/include"
ignoring nonexistent directory "/usr/include/gtk-3.0 -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/lib/libffi-3.2.1/include -I/usr/include/harfbuzz -I/usr/include/fribidi -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/gio-unix-2.0 -I/usr/include/atk-1.0 -I/usr/include/at-spi2-atk/2.0 -I/usr/include/dbus-1.0 -I/usr/lib/dbus-1.0/include -I/usr/include/at-spi-2.0 -pthread -lgtk-3 -lgdk-3 -lz -lpangocairo-1.0 -lpango-1.0 -lharfbuzz -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 "
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/include
 /usr/local/include
 /usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/include-fixed
 /usr/include
End of search list.
gtkhello.c:1:10: fatal error: gtk/gtk.h: No such file or directory
    1 | #include <gtk/gtk.h>
      |          ^~~~~~~~~~~
compilation terminated.

Executing the build command without bash command substitution (writing all required include dirs and libs) works without problems.

Any ideas what might be wrong? I have no other means to investigate what's going on there.

Thanks for any clarification.

Last edited by mixxoo (2020-04-05 08:34:18)

Offline

#2 2020-04-03 15:31:11

mpan
Member
Registered: 2012-08-01
Posts: 1,181
Website

Re: [SOLVED] BASH - parsing options via command substitution

gcc does not perform substitution. It’s your shell that does it even before it knows gcc has to be executed. Are you sure this is bash and not, by any chance, fish?

If this is bash, what does this command print:

bash -c 'while (( $# )); do echo "$1"; shift; done' ignore $(pkgconf --cflags --libs gtk+-3.0)

Last edited by mpan (2020-04-03 15:31:32)


Sometimes I seem a bit harsh — don’t get offended too easily!

Offline

#3 2020-04-03 16:14:13

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,422
Website

Re: [SOLVED] BASH - parsing options via command substitution

I find it very hard to believe that the command you put in the code tags is what you actually used.  In contrast to mpan's comment, using a different shell like fish would not explain this.  Fish does not recognize command substitution with $() at all.  It'd error out immediately.  Any shell that does do $() command substitution would do this correctly.  From GCC's output, however, it's clear that the result of the command substitution was quoted.  No POSIX shell would add quotes where you didn't include them.  The simplest explanation would be that your actual command was the following, which would produce the exact results you provided:

gcc -v -E "$(pkgconf --cflags --libs gtk+-3.0)" -o gtkhello gtkhello.c

This could also be likely if that command itself was part of a script or makefile where you added quotes around a variable when you should not have.

EDIT: or I suppose if you had an alternative pkgconf script/function/alias that added quotes ... but this is very highly unlikely.

Last edited by Trilby (2020-04-03 16:15:38)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#4 2020-04-03 17:56:53

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: [SOLVED] BASH - parsing options via command substitution

Disregarding the almost certain chance that we were lied to...

As a general rule, incidentally, you should -- instead of using gcc -v to debug your failing command -- use 'set -x':

$ set -x
$ gcc -E $(pkgconf --cflags --libs gtk+-3.0) -o gtkhello gtkhello.c
++ pkgconf --cflags --libs gtk+-3.0
+ gcc -E -I/usr/include/gtk-3.0 -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/lib/libffi-3.2.1/include -I/usr/include/harfbuzz -I/usr/include/fribidi -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/gio-unix-2.0 -I/usr/include/atk-1.0 -I/usr/include/at-spi2-atk/2.0 -I/usr/include/dbus-1.0 -I/usr/lib/dbus-1.0/include -I/usr/include/at-spi-2.0 -pthread -lgtk-3 -lgdk-3 -lz -lpangocairo-1.0 -lpango-1.0 -lharfbuzz -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 -o gtkhello gtkhello.c


$ gcc -E "$(pkgconf --cflags --libs gtk+-3.0)" -o gtkhello gtkhello.c
++ pkgconf --cflags --libs gtk+-3.0
+ gcc -E '-I/usr/include/gtk-3.0 -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/lib/libffi-3.2.1/include -I/usr/include/harfbuzz -I/usr/include/fribidi -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/gio-unix-2.0 -I/usr/include/atk-1.0 -I/usr/include/at-spi2-atk/2.0 -I/usr/include/dbus-1.0 -I/usr/lib/dbus-1.0/include -I/usr/include/at-spi-2.0 -pthread -lgtk-3 -lgdk-3 -lz -lpangocairo-1.0 -lpango-1.0 -lharfbuzz -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 ' -o gtkhello gtkhello.c

It becomes very obvious in the second case that the final command which bash is executing is... not what you expected.

Or somewhat fancier, but not built in:

$ args() { printf '<%s>\n' "$@"; }

$ args gcc -E $(pkgconf --cflags --libs gtk+-3.0) -o gtkhello gtkhello.c
<gcc>
<-E>
<-I/usr/include/gtk-3.0>
<-I/usr/include/pango-1.0>
<-I/usr/include/glib-2.0>
<-I/usr/lib/glib-2.0/include>
<-I/usr/lib/libffi-3.2.1/include>
<-I/usr/include/harfbuzz>
<-I/usr/include/fribidi>
<-I/usr/include/freetype2>
<-I/usr/include/libpng16>
<-I/usr/include/cairo>
<-I/usr/include/pixman-1>
<-I/usr/include/gdk-pixbuf-2.0>
<-I/usr/include/libmount>
<-I/usr/include/blkid>
<-I/usr/include/gio-unix-2.0>
<-I/usr/include/atk-1.0>
<-I/usr/include/at-spi2-atk/2.0>
<-I/usr/include/dbus-1.0>
<-I/usr/lib/dbus-1.0/include>
<-I/usr/include/at-spi-2.0>
<-pthread>
<-lgtk-3>
<-lgdk-3>
<-lz>
<-lpangocairo-1.0>
<-lpango-1.0>
<-lharfbuzz>
<-latk-1.0>
<-lcairo-gobject>
<-lcairo>
<-lgdk_pixbuf-2.0>
<-lgio-2.0>
<-lgobject-2.0>
<-lglib-2.0>
<-o>
<gtkhello>
<gtkhello.c>

$ args gcc -E "$(pkgconf --cflags --libs gtk+-3.0)" -o gtkhello gtkhello.c
<gcc>
<-E>
<-I/usr/include/gtk-3.0 -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/lib/libffi-3.2.1/include -I/usr/include/harfbuzz -I/usr/include/fribidi -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/gio-unix-2.0 -I/usr/include/atk-1.0 -I/usr/include/at-spi2-atk/2.0 -I/usr/include/dbus-1.0 -I/usr/lib/dbus-1.0/include -I/usr/include/at-spi-2.0 -pthread -lgtk-3 -lgdk-3 -lz -lpangocairo-1.0 -lpango-1.0 -lharfbuzz -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 >
<-o>
<gtkhello>
<gtkhello.c>

Last edited by eschwartz (2020-04-03 17:58:33)


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#5 2020-04-04 09:19:28

mixxoo
Member
Registered: 2017-05-04
Posts: 10

Re: [SOLVED] BASH - parsing options via command substitution

Thank you all for valuable comments. I agree the name of the topic was not chosen very correctly. Actually the problem is related to bash, not gcc nor pkgconf.
Yes, you are both right, the issue is related to unnecessary quoting. @eschwartz : thank you for suggesting 'set -x'. I did not know about such option.

However, believe it or not, I confirm that I entered the command WITHOUT any quotes. But after bash command expansion they (single quotes) mysteriously appeared. I can show this with following simple example I tried this morning :

$ set -x
++ printf '\033]0;%s@%s:%s\007' mirek NUC '~'
$ pacman $(echo -Qs meson)
++ echo -Qs meson
+ pacman '-Qs meson'
pacman: invalid option -- ' '

It is copy-paste from my terminal.
@eschwartz: I am sorry you feel I am joking. It is not my nature wasting people's time with such jokes. I was seriously looking for advice or explanation. However, I agree it looks like a primitive issue at first sight.
Nevertheless, thank you for pointing the direction. I have learnt something new again.
Trilby said :

No POSIX shell would add quotes where you didn't include them.

Here it appears that sometimes the opposite is true. I do not know, maybe I am still missing something (or have somewhere deeply buried bash setting, I do not know of, producing this behavior).

Offline

#6 2020-04-04 12:58:52

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,422
Website

Re: [SOLVED] BASH - parsing options via command substitution

mixxoo wrote:

... or have somewhere deeply buried bash setting, I do not know of, producing this behavior.

Then test for that.  Start a clean bash shell and try again:

env -i bash --noprofile -l

(edit: in that clean bash shell, retry the same commands with set -x as in the previous post).

Last edited by Trilby (2020-04-04 13:04:36)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#7 2020-04-04 20:00:39

mixxoo
Member
Registered: 2017-05-04
Posts: 10

Re: [SOLVED] BASH - parsing options via command substitution

It works in the clean shell.

env -i bash --noprofile -l
bash-5.0$ set -x
bash-5.0$ pacman $(echo -Qi meson)
++ echo -Qi meson
+ pacman -Qi meson

Now I have to find settings which make the difference.

Offline

#8 2020-04-04 20:24:08

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,422
Website

Re: [SOLVED] BASH - parsing options via command substitution

Using a similar procedure could likely narrow it down pretty fast:

env -i bash --noprofile -l
. ~/.bashrc
set -x
pacman $(echo -Qi meson)

This would test whether your bashrc is sufficient to cause the problem, which I'd bet is a very safe bet, but worth quickly confirming before bothering to dig into it.  If that does replicate the problem, post your bashrc.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#9 2020-04-05 05:06:26

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: [SOLVED] BASH - parsing options via command substitution

mixxoo wrote:

It is copy-paste from my terminal.
@eschwartz: I am sorry you feel I am joking. It is not my nature wasting people's time with such jokes. I was seriously looking for advice or explanation. However, I agree it looks like a primitive issue at first sight.
Nevertheless, thank you for pointing the direction. I have learnt something new again.

It is indeed a bizarre behavior, which is why I leaned towards the belief that we weren't getting the full picture here... but I did say "almost" because I can't completely rule out something completely unexpected.

I'm happy to have helped you figure out where your args went, and given you can narrow down the problem to not appearing when you use bash without a profile, it seems my suspicion was unfounded. Thank you for bearing with us!

And I'll second Trilby's request to test your bashrc and post it if it proves to be causing the issue.


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#10 2020-04-05 08:18:17

mixxoo
Member
Registered: 2017-05-04
Posts: 10

Re: [SOLVED] BASH - parsing options via command substitution

So it is finally solved. Some time ago I needed to modify IFS variable (because of some other batch processing) which has been set in .bashrc to IFS=$'\n'. It is not the default value. After removing it from .bashrc everything works as expected. However I do not really understand how it is related to quoted command substitution. As far as I know it is mainly input field separator which defines the characters used by word splitting. It was just try and see what it does smile

Offline

#11 2020-04-05 13:15:03

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,422
Website

Re: [SOLVED] BASH - parsing options via command substitution

Ha!  IFS hadn't even occurred to me, but yes, that makes comlete sense.

I was actually wrong.  I suspected the output of the command was quoted, which it wasn't.  The reason I suspected that was actually not directly because GCC presented the whole string quoted.  Rather, GCC presenting the whole string in quotes meant the whole thing was passed to gcc as a single argument.  Aside: every program is passed a list of arguments, in C code these are generally represented as the "argv" array.  What is in each one is determined by the shell.  The shell parses / tokenizes the string on the command line and seperates "words".  Quoting is one of the common ways to allow spaces to be within a single word / token.  Escaping the space with a backslash is another.  I had not been thinking that setting IFS was yet another way to do this.  The end result is that the result of the command substitution in $() is treated read and split only on newlines to create tokens on the command line.

This actually is quite interesting as it depends on the setting of the IFS and the order or events in processing the command line.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

Board footer

Powered by FluxBB