xdg-open is slow opening files

setzer22 · 2014-07-27 15:31:54

Whenever I use xdg-open to open an application it takes a while until the application is shown, but if I try to open it with the application itself it works almost instantaneously. While using xdg-open I can notice a high increase of the cpu usage.

What could it be that's causing this issue? Is there maybe a log file I can check to see what's going on under the hood? This issue is happening with gpicview (for images), nemo, and thunar; I haven't tried with other applications but I assume it's going to be the same.

As relevant info I might add that I'm using openbox and no desktop environment. I'll upload any config files if requested, but I don't really know what to include as a start besides my mimeapps.list:

#~/.local/share/applications/mimeapps.list
[Default Applications]
inode/directory=Thunar.desktop
image/png=gpicview.desktop

[Added Associations]
image/png=gpicview.desktop;

I've checked the wiki for xdg-open and all the forums (arch and outside of arch) and have just found people with the same issue, but no answers whatsoever.

Any help on this would be much appreciated.

emeres · 2014-07-27 15:48:24

Have you run 'xdg-open file' in a terminal? Any relevant output, logs from dmesg maybe? Do you have XDG_* variables set? What if you disable mimeapps.list, local applications directory or the whole ~/.local?

setzer22 · 2014-07-27 16:05:39

Yes, I've tried using xdg-open in the terminal and the behaviour is exactly the same. I can't see any relevant output, after a long delay xdg-open prints a success string i.e: "START /usr/bin/thunar "Pictures/". No logs that I know of, and dmesg isn't showing any new messages, also dmesg | grep xdg doesn't output anything.

The output of printenv | grep XDG is:

XDG_VTNR=7
XDG_SESSION_ID=c3
XDG_SEAT=seat0
XDG_RUNTIME_DIR=/run/user/1000

I manually checked and there aren't any other XDG relevant variables I can see i there, I should say that I haven't set any of those variables myself at any point.

Disabling mimeapps.list or any of the folders you mentioned (renaming them) doesn't change a thing.

emeres · 2014-07-27 16:31:34

Maybe if you set XDG_DATA_DIRS to "$HOME/.local/share:$XDG_DATA_DIRS" and/or XDG_DATA_HOME to "$HOME/.local/share" in ~/.profile or whatever you use, it will work after relogging in, probably not though, since disabling ~/.local does nothing. I assume xdg searches for something or is calculating extensively. Do you have/did you change any configuration that could affect xdg? You could try to see if this is user specific or global with another account. Could this be shell related? There are approximately 900 lines of code in the xdg-open, you might want to take a look.

Last resort would be strace.

Edit: typos.

Last edited by emeres (2014-07-27 17:39:55)

setzer22 · 2014-07-27 17:33:19

Ok after checking the code (and putting a lot of echoes in it) I've narrowed down the issue. The part of the code that's taking most of the time is:

DEBUG 3 "$xdg_user_dir:$xdg_system_dirs"
for x in `echo "$xdg_user_dir:$xdg_system_dirs" | sed 's/:/ /g'`; do
	    echo "searching file $default $x/applications/ $1"                   #I added this
            search_desktop_file "$default" "$x/applications/" "$1"
done

Also I managed to enable debug messages by modifying the code and the DEBUG function is now working, this is the output I get:

#The DEBUG line:
/home/josep/.local/share:/usr/local/share/:/usr/share/

#My own echo
searching file Thunar.desktop /home/josep/.local/share/applications/ /home/josep/Pictures/
searching file Thunar.desktop /usr/local/share//applications/ /home/josep/Pictures/
searching file Thunar.desktop /usr/share//applications/ /home/josep/Pictures/

#Final output
START /usr/bin/thunar "/home/josep/Pictures/"

I should also add that the program stops before finishing the loop, so I assume it's just doing an exhaustive search of my disk to find the Thunar.desktop binary and stopping when it finds it. Isn't there a way I could make this faster?

I'll take a closer look into the search_desktop_file function now, but I thought this was a useful update.

Thanks for your help, any more help on that will be much appreciated.

EDIT: Also I'm intrigued, why does that path /usr/share//applications contain a double slash. Is that a valid path? Does a double slash mean anything relevant in unix or is this something the program is doing perhaps?

Last edited by setzer22 (2014-07-27 17:36:29)

emeres · 2014-07-27 17:51:29

Remove the slashes at the end of $xdg_user_dir:$xdg_system_dirs:==:/usr/local/share/:/usr/share/, then this should go away. Analyze step by step how the x variable is read in that loop.

As for the issue at hand, look into mounting options. maybe your hdd is the bottleneck or something along the path to it. Putting the whole searched directories into ram might speed things up, but there are probably at least a couple aspects to consider, then again asd might help.

setzer22 · 2014-07-27 18:11:24

After analyzing the loop (and the inner search_desktop_file function), I can tell that function is what's taking all the CPU time, also I can tell that only 3 iterations of the loop are run, one for each X, which have the values:

$x_1 = Thunar.desktop /home/josep/.local/share/
$x_2 = Thunar.desktop /usr/local/share/
$x_3 = /usr/share/

I can tell the first two searches take close to no time, and actually, the binary /usr/bin/thunar, which is located in the third iteration, doesn't take much time either, so surprisingly what's eating most of CPU resources is this part of the script:

        command="$(get_key "${file}" "Exec" | first_word)"
        command_exec=`which $command 2>/dev/null`
        arguments="$(get_key "${file}" "Exec" | last_word)"
        arg_one="`echo "$arg" | sed 's/[&*\\]/\\\\&/g'`"
        icon="$(get_key "${file}" "Icon")"
        if [ "${icon}" != "" ]
        then
            icon="--icon '${icon}'"
        else
            icon="''"
        fi
        # FIXME: Actually LC_MESSAGES should be used as described in
        # http://standards.freedesktop.org/desktop-entry-spec/latest/ar01s04.html
        localised_name="'$(get_key "${file}" "Name")'"
        arguments_exec="$(echo "$arguments" | sed -e 's*%[fFuU]*"'"$arg_one"'"*g' \
                                                  -e 's*%i*'"$icon"'*g' \
                                                  -e 's*%c*'"$localised_name"'*g')"

The first two variables command and command_exec are set to "thunar" and "/usr/bin/thunar", and this happens relatively fast (although it takes some time to find it, it's not even a 10% of the total time).

EDIT: I think all signs lead to the disk read taking too long (for example the get_key function which reads the .desktop file looking for the asked key, takes too much), but this is weird, as I had never encountered this issue in other linux distributions, everything has ran always smooth and fine, despite not having a good machine.

EDIT2: After finally tracking down the issue, I've found this loop: (inside the get_key function)

while read line
    do
	echo "readline" 1>&2
        case "$line" in
            "[Desktop Entry]")
                desktop_entry="y"
            ;;
            # Reset match flag for other groups
            [*)
                desktop_entry=""
            ;;
            *)
                # Only match Desktop Entry group
                if [ -n "${desktop_entry}" ]
                then
                    echo "${line}" | grep -E "^${key}=" | cut -d= -f 2-
                fi
        esac
    done < "${file}"

More than 90% of CPU time is spent in this function, but I can't see why. Parsing a .desktop file is not a difficult task by any means and even if this was reading the file every time, everything would be cached in the ram anyway, as modern OSs use paging for that matter (and by modern I mean not modern at all) and I'm well aware that linux does).

So TL;DR I don't understand how can the simple task of parsing a .desktop file take more time than opening google chrome or gimp. Any help on that?

Last edited by setzer22 (2014-07-27 19:07:43)

setzer22 · 2014-07-27 19:52:48

And finally I solved it!

It appears that the scrpit I had was terribly broken, more precisely that get_key function. I tried looking at the source of xdg-open at linux mint and crunchbang and neither of them had that function, and instead had one that parsed the contents to binary only once to read them afterwards from the RAM.

I just copied my xdg-open from crunchbang's /usr/bin to my /usr/bin folder in my arch machine and now it works pefectly fine.

Still, I wonder: I've just installed arch, so my packages should be up-to-date, why is there such a broken package in the official repositories? Should I report this as a bug?

I'll mark this as solved. Thank you very much emeres to guide me towards the problem.

emeres · 2014-07-27 19:59:18

Should it really be corrupted in any way, then yes. If a bug for xdg-open has not been filled, do that. Force updating repositories (pacman -Syy) and check the package xdg-utils first however. There is also xdg-utils-git in aur.

You are welcome.

Edit: It might be a feature request however depending on the code. My xdg-open seems also affected.

Last edited by emeres (2014-07-27 20:09:27)

setzer22 · 2014-07-27 20:17:30

The package I downloaded from the official repos seems to be the one I had in the first time. Not that it doesn't work, but I'd call 12 seconds to parse a .desktop file broken.

The xdg-open I copied from crunchbang is clearly different from the one I just downloaded from the arch repositories, so I guess I'll send a bug report.

ninian · 2014-07-28 20:42:21

@setzer22:
xdg-open is a pretty yucky bit of scripting, especially if you don't use a DE. Also. it doesn't really follow the XDG standards anyway.
It might amuse to try opening your desktop files with my ayr launcher which was coded from scratch to try and do a better job for my own DE-free systems.

4z01235 · 2014-08-07 14:02:34

@setzer: do you have a copy of the script you took from Crunchbang? Or a patch that can be applied to the one in the Arch repos? I'm having the same difficulty with xdg-open being aggravatingly slow.

Arch Linux

#1 2014-07-27 15:31:54

xdg-open is slow opening files

#2 2014-07-27 15:48:24

Re: xdg-open is slow opening files

#3 2014-07-27 16:05:39

Re: xdg-open is slow opening files

#4 2014-07-27 16:31:34

Re: xdg-open is slow opening files

#5 2014-07-27 17:33:19

Re: xdg-open is slow opening files

#6 2014-07-27 17:51:29

Re: xdg-open is slow opening files

#7 2014-07-27 18:11:24

Re: xdg-open is slow opening files

#8 2014-07-27 19:52:48

Re: xdg-open is slow opening files

#9 2014-07-27 19:59:18

Re: xdg-open is slow opening files

#10 2014-07-27 20:17:30

Re: xdg-open is slow opening files

#11 2014-07-28 20:42:21

Re: xdg-open is slow opening files

#12 2014-08-07 14:02:34

Re: xdg-open is slow opening files

Board footer