You are not logged in.

#126 2016-01-22 02:35:46

bashnev
Member
Registered: 2012-06-11
Posts: 24

Re: mimeo: open files by mimetype or file name

Apologies for the poorly expressed bug report. I was being rushed at the time. Thanks for the fix and to Eschwartz for adding clarity.

Offline

#127 2016-01-22 03:25:57

eschwartz
Trusted User/Bug Wrangler
Registered: 2014-08-08
Posts: 3,309

Re: mimeo: open files by mimetype or file name

Happy to clarify the issue. smile

As I noted in the other thread, I've seen the problem manifest before, so I knew what to look for.


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#128 2016-01-25 13:21:50

Treferwynd
Member
Registered: 2014-07-03
Posts: 6

Re: mimeo: open files by mimetype or file name

Using mimeo 2016.1.23-1, I have the same issue as xduugu, it starts two programs if there's a match in associations.txt:

$ cat ~/.config/mimeo/associations.txt
/usr/bin/firefox %U
  ^https?://example

$ mimeo -c http://example
/usr/bin/firefox http://example
firefox http://example

$ mimeo -c http://notexample
firefox http://notexample

And is terribly slow...

$ time mimeo -c http://example
/usr/bin/firefox http://example
firefox http://example

real	0m2.777s
user	0m0.130s
sys	0m0.020s

EDIT: Actually not quite the same as xduugu, the mimetype detection seems fine

$ mimeo -m http://example
http://example
  x-scheme-handler/http

Last edited by Treferwynd (2016-01-25 13:41:33)

Offline

#129 2016-01-25 16:29:13

xduugu
Member
Registered: 2008-10-16
Posts: 292

Re: mimeo: open files by mimetype or file name

I found an issue with URLs. For some reason, the path of an URL is first treated as local path. I guess it's because of the latest file://-URI changes.

Everything fine:

$ mimeo -m 'http://127.0.0.1'
http://127.0.0.1
  x-scheme-handler/http

mimeo opens "/" in a file manager, because the local path "/" exists:

$ mimeo -m 'http://127.0.0.1/'
http://127.0.0.1/
  inode/directory

"/root" also exists

$ mimeo -m 'http://127.0.0.1/root'
http://127.0.0.1/root
  inode/directory

"/foo" does not exist and the URL is correctly recognized:

$ mimeo -m 'http://127.0.0.1/foo'
http://127.0.0.1/foo
  x-scheme-handler/http

Offline

#130 2016-01-25 18:41:28

Treferwynd
Member
Registered: 2014-07-03
Posts: 6

Re: mimeo: open files by mimetype or file name

Interestingly enough, I don't have that issue

$ mimeo -m 'http://127.0.0.1/'
http://127.0.0.1/
  x-scheme-handler/http

Offline

#131 2016-01-25 22:45:58

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 6,513
Website

Re: mimeo: open files by mimetype or file name

  • The double-command with custom associations has been fixed.

  • Only "file" URLs that point to localhost will be treated as files.

@Treferwynd
Is it slow on repeated runs? If so it's probably just the delay from reading the initially uncached (on disk) files. I have added more debugging commands to track file reading and writing. Use the --debug flag to see what's taking so long and I'll see what I can do to speed it up.

It's also strange that the behavior for 'http://127.0.0.1/' was different on your system. I may have made some erroneous assumptions in the code that checks for the localhost in the URL. Please save the following script as "localhost_test" and post the output of

./localhost_test 'http://localhost/' 'http://127.0.0.1/'

localhost_test

#!/usr/bin/env python3

import socket
import sys
import urllib.parse

def test(arg):
  parsed_url = urllib.parse.urlparse(arg)
  localhost = socket.getfqdn(socket.gethostname())
  hostname = parsed_url.hostname if parsed_url.hostname else 'localhost'
  remotehost = socket.getfqdn(hostname)
  print('{}\n  remote host: {}\n  local host:  {}\n  equal:       {}'.format(
    arg,
    remotehost,
    localhost,
    remotehost == localhost
  ))

if __name__ == '__main__':
  try:
    for arg in sys.argv[1:]:
      test(arg)
  except (KeyboardInterrupt, BrokenPipeError):
    pass

If those URLs are not detected as localhost, do you have localhost lines in /etc/hosts?


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#132 2016-01-25 23:52:36

xduugu
Member
Registered: 2008-10-16
Posts: 292

Re: mimeo: open files by mimetype or file name

Everything fine now. Thanks again.

Offline

#133 2016-01-26 19:19:55

Treferwynd
Member
Registered: 2014-07-03
Posts: 6

Re: mimeo: open files by mimetype or file name

I've added my hostname to /etc/hosts (never had any use for that) and now the script returns true.
But I don't get why is there a problem with 'http://127.0.0.1' being detected as 'x-scheme-handler/http'

$ ./localhost_test 'http://localhost/' 'http://127.0.0.1/'
http://localhost/
  remote host: localhost.localdomain
  local host:  localhost.localdomain
  equal:       True
http://127.0.0.1/
  remote host: localhost.localdomain
  local host:  localhost.localdomain
  equal:       True

$ mimeo -m 'http://127.0.0.1/'
http://127.0.0.1/
  x-scheme-handler/http

For the speed issue, I'm gonna check tomorrow, right now I'm pretty fried right now, but yes, it is kinda slow every time I invoke it

$ mimeo -c "http://example.wot" --debug
DEBUG: loading arguments from /home/tref/.config/mimeo/default_arguments.txt
DEBUG: opening /home/tref/.config/mimeo/associations.txt
/usr/bin/firefox http://example.wot

$ mimeo -c "http://notexample.wot" --debug
DEBUG: loading arguments from /home/tref/.config/mimeo/default_arguments.txt
DEBUG: opening /home/tref/.config/mimeo/associations.txt
DEBUG: loading /home/tref/.config/mimeapps.list
DEBUG: parsing /usr/share/applications/firefox.desktop
firefox http://notexample.wot

It hangs a bit after the "opening ..." and "parsing ..." lines. Sometimes it's fast, sometimes it's slow. Tomorrow I'm gonna look into the source

Offline

#134 2016-01-26 20:19:02

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 6,513
Website

Re: mimeo: open files by mimetype or file name

Treferwynd wrote:

But I don't get why is there a problem with 'http://127.0.0.1' being detected as 'x-scheme-handler/http'

The localhost detection and the MIME-type for URLs were independent issues. I realized that Mimeo was failing to recognize 127.0.0.1 as localhost on your system because of your previous post:

Treferwynd wrote:

Interestingly enough, I don't have that issue

$ mimeo -m 'http://127.0.0.1/'
http://127.0.0.1/
  x-scheme-handler/http

Before the update, you should have seen the same output as xduugu. After the last update that issue would only appear for 'file://127.0.0.1/path/to/local/file'. I'll see if I can find a way to reliably detect the localhost regardless of the system hosts file.

Let me know if you figure what's slowing down mimeo.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#135 2016-01-27 11:08:00

Treferwynd
Member
Registered: 2014-07-03
Posts: 6

Re: mimeo: open files by mimetype or file name

So I thought 2016.1.25.2 also solved the speed issue, so I wrote a little (ugly) script to measure things, but 2016.1.25.2 is noticeably slower than 2016.1.18.2

Running the same mimeo command 100 times:

$ pacman -Q mimeo
mimeo 2016.1.18.2-1
$ runtime 100 mimeo -c "http://example.wot"
Average:   0.102 s
Max:       0.135 s
Min:       0.074 s
Total:    10.191 s
$ pacman -Q mimeo
mimeo 2016.1.25.2-1
$ runtime 100 mimeo -c "http://example.wot"
Average:   0.184 s
Max:       1.034 s
Min:       0.103 s
Total:    18.442 s

The new version, while also being a tad slower, sometimes just hangs for a second or so.

I'll see if I can find the cause.
But... am I the only one with this issue? Did I do something stupid? :S

Offline

#136 2016-01-27 16:18:57

Treferwynd
Member
Registered: 2014-07-03
Posts: 6

Re: mimeo: open files by mimetype or file name

Found the culprit, it's the socket.getfqdn() call, sometimes it takes an unreasonable time.
So mimeo is fine, but I have no idea why this happens neutral

E.g.

$ time python -c 'import socket; socket.getfqdn("google")'
real	0m0.954s

Offline

#137 2016-01-27 21:11:07

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 6,513
Website

Re: mimeo: open files by mimetype or file name

Thanks for tracking down the delay. I would not have been able to reproduce it on my system. I have no idea of how to speed up the call itself within Mimeo and I suspect that it depends on some system configuration. I have, however, restructured the code slightly to avoid that call if the URL is not a "file" URL.

Btw, where can I find "runtime" from your previous post? I wasn't able to find it in any official package with "pkgfile" and I see no obvious candidate in the AUR. It looks quite useful.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#138 2016-01-27 23:56:37

Treferwynd
Member
Registered: 2014-07-03
Posts: 6

Re: mimeo: open files by mimetype or file name

No problem! Actually I don't think you can do something, I think it's my system that it's borked, I mean, it works fine 95 times out of 100, I've really no idea what's happeninng neutral
The script is just a super ugly bash function that I wrote just to test the speed issue, I wouldn't recommend it, but here it is anyway: https://gist.github.com/Treferwynd/b32515f9d5ca6c564335

EDIT: I updated mimeo and behold, the issue is gone!

$ pacman -Q mimeo
mimeo 2016.1.27-1
$ runtime -q 100 mimeo -c "http://example.wot"
Average:      0.104 s
Max:          0.142 s
Min:          0.076 s
Total:       10.367 s

Thanks for the quick fixes, and well, you know, mimeo in general!

Last edited by Treferwynd (2016-01-28 00:08:35)

Offline

#139 2016-01-28 19:43:33

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 6,513
Website

Re: mimeo: open files by mimetype or file name

Thanks for posting the script. I started playing around with the idea in Python after discovering the "timeit" module and ended up with a little script that I have named timestats.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#140 2016-02-10 16:53:58

xduugu
Member
Registered: 2008-10-16
Posts: 292

Re: mimeo: open files by mimetype or file name

I found a bug big_smile

Apparently, the associations are only considered for the first file argument.

associations.txt:

pymol %F
  \.pdb$
$ mimeo -c a.pdb 
pymol a.pdb
$ mimeo -c b.pdb 
pymol b.pdb
$ mimeo -c a.pdb b.pdb 
WARNING: failed to determine command precursor for b.pdb
pymol a.pdb
WARNING: failed to determine command(s) for b.pdb

By the way, the behavior I try to archive is:

$ mimeo -c a.pdb b.pdb 
pymol a.pdb b.pdb

But I suppose that mimeo cannot combine arguments by the command when using the %F place holder.

Offline

#141 2016-02-10 20:30:35

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 6,513
Website

Re: mimeo: open files by mimetype or file name

It handles %F, %U and the other placeholders. The problem was a broken loop condition which has been fixed. It behaves as expected now. smile


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#142 2016-02-10 20:39:18

xduugu
Member
Registered: 2008-10-16
Posts: 292

Re: mimeo: open files by mimetype or file name

Works. Thanks again.

Offline

#143 2016-02-10 20:41:27

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 6,513
Website

Re: mimeo: open files by mimetype or file name

xduugu wrote:

Works. Thanks again.

Whoa , that was a fast upgrade. smile


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#144 2016-04-11 20:32:24

rtfupi
Member
Registered: 2016-04-11
Posts: 4

Re: mimeo: open files by mimetype or file name

Xyne wrote:

NB! To use files in the now-deprecated $XDG_DATA_HOME/applications directory (~/.local/share/applications by default), you must use the --deprecated flag. See the changelog for further information.

In which directory mimeo will look for user defined desktop files?

Offline

#145 2016-04-11 20:35:43

ewaller
Administrator
From: Pasadena, CA
Registered: 2009-07-13
Posts: 17,197

Re: mimeo: open files by mimetype or file name

rtfupi,  Welcome to Arch Linux.

In case you are wondering where your post went, you hit the 'report' link, not the 'reply' link.


Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
Sometimes it is the people no one can imagine anything of who do the things no one can imagine. -- Alan Turing
---
How to Ask Questions the Smart Way

Offline

#146 2016-04-14 23:31:03

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 6,513
Website

Re: mimeo: open files by mimetype or file name

rtfupi wrote:
Xyne wrote:

NB! To use files in the now-deprecated $XDG_DATA_HOME/applications directory (~/.local/share/applications by default), you must use the --deprecated flag. See the changelog for further information.

In which directory mimeo will look for user defined desktop files?

$ mimeo --help
...
  --create <filename> <Name> <Exec> <MIME-type matcher or ""> <"term" or "">
                        Create a minimal desktop file. Edit the created file
                        if necessary. If an empty string is passed instead of
                        a MIME-type matcher, the file will not specify any
                        MIME-type associations. The fifth argument indicates
                        if "Terminal" should be set to "true" in the created
                        file. See "--mimeman-help" for more information. The
                        created files are saved in $XDG_DATA_HOME/applications
                        (~/.local/share/applications by default). This
                        directory is only checked when the --deprecated flag
                        is used.

As the help message indicates, desktop files created directly with mimeo are saved in the deprecated applications directory, for legacy reasons and because the current XDG standard does not support the files in any user-controlled directory. If you add your own files elsewhere, you must add them to one of the locations specified by the standard. Just check where desktop files are stored on your computer or read the latest XDG specification.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#147 2016-04-15 10:01:09

rtfupi
Member
Registered: 2016-04-11
Posts: 4

Re: mimeo: open files by mimetype or file name

Xyne wrote:

As the help message indicates, desktop files created directly with mimeo are saved in the deprecated applications directory, for legacy reasons and because the current XDG standard does not support the files in any user-controlled directory. If you add your own files elsewhere, you must add them to one of the locations specified by the standard. Just check where desktop files are stored on your computer or read the latest XDG specification.

Hm... I understood from the standard that files ($XDG_DATA_HOME/applications/$desktop-mimeapps.list and $XDG_DATA_HOME/applications/mimeapps.list) are deprecated but not  the directory ($XDG_DATA_HOME/applications/).

Last edited by rtfupi (2016-04-15 10:05:32)

Offline

#148 2016-04-16 22:52:42

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 6,513
Website

Re: mimeo: open files by mimetype or file name

rtfupi wrote:

Hm... I understood from the standard that files ($XDG_DATA_HOME/applications/$desktop-mimeapps.list and $XDG_DATA_HOME/applications/mimeapps.list) are deprecated but not  the directory ($XDG_DATA_HOME/applications/).

In the "Adding/removing associations" section of the specification, the author wrote:

(in the context of reading mimeapps.list files)
The adding and removal of associations only applies to desktop files in the current directory, or a later one (in precedence order).
...
The suggested algorithm for listing (in preference order) the applications associated to a given mimetype is:

  • create an empty list for the results, and a temporary empty "blacklist"

  • visit each "mimeapps.list" file, in turn; a missing file is equivalent to an empty file

  • add to the results list any "Added Associations" in the mimeapps.list, excluding items on the blacklist

  • add to the blacklist any "Removed Associations" in mimeapps.list

  • add to the results list any .desktop file found in the same directory as the mimeapps.list which lists the given type in its MimeType= line, excluding any desktop files already in the blacklist. For directories based on XDG_CONFIG_HOME and XDG_CONFIG_DIRS, there are (by definition) no desktop files in the same directory.

  • add to the blacklist the names of any desktop files found in the same directory as the mimeapps.list file (which for directories based on XDG_CONFIG_HOME and XDG_CONFIG_DIRS, is none)

  • repeat the last four steps for each subsequent directory

See the step in bold in particular. The specification determines the directory order for reading mimeapps.list files. Only desktop files in those directories are considered. Desktop files in the user configuration directories are explicitly prohibited as is the mimeapps.list file in the user applications directory, which thus prevents desktop files in that directory from being considered.

Basically, the current specification has deprecated user desktop files. I have no idea why they would remove such a useful functionality, but they did.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#149 2016-04-17 19:05:50

rtfupi
Member
Registered: 2016-04-11
Posts: 4

Re: mimeo: open files by mimetype or file name

Xyne wrote:

See the step in bold in particular. The specification determines the directory order for reading mimeapps.list files. Only desktop files in those directories are considered. Desktop files in the user configuration directories are explicitly prohibited as is the mimeapps.list file in the user applications directory, which thus prevents desktop files in that directory from being considered.

Basically, the current specification has deprecated user desktop files. I have no idea why they would remove such a useful functionality, but they did.

Thank you for the clarification. I'll try to ask about this in freedesktop@lists.freedesktop.org

Offline

#150 2016-04-18 00:54:32

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 6,513
Website

Re: mimeo: open files by mimetype or file name

rtfupi wrote:

Thank you for the clarification. I'll try to ask about this in freedesktop@lists.freedesktop.org

If it turns out that I have misunderstood the specification or if they decide to change it, let me know and I will update mimeo.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

Board footer

Powered by FluxBB