qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

skanky · 2015-10-02 08:33:33

I'm getting an issue trying to run qutebrowser, since the recent python update. It silently fails on one machine and on the other says

Traceback (most recent call last):
  File "/usr/bin/qutebrowser", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 3084, in <module>
    @_call_aside
  File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 3070, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 3097, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 651, in _build_master
    ws.require(__requires__)
  File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 952, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 839, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'pypeg2' distribution was not found and is required by qutebrowser

I have python-pypeg2 installed and up to date. I'm running the latest qutebrowser-git. It's not the only python application that can't find a required distribution - the other is reflect.
So something's awry with my python update, and it may be that the mirror is out of date with one package or other. Does that sound plausible, or is it likely that there's some other issue crept into my python set-up?

byte · 2015-10-02 08:54:29

python-pypeg2 is an AUR package, so it's your duty to rebuild it against Python 3.5.

skanky · 2015-10-02 09:00:07

byte wrote:

python-pypeg2 is an AUR package, so it's your duty to rebuild it against Python 3.5.

Mea culpa. I knew I was missing/forgetting something (been a long week). Many thanks.

Last edited by skanky (2015-10-02 09:00:26)

navi_se · 2015-10-02 14:10:02

byte wrote:

python-pypeg2 is an AUR package, so it's your duty to rebuild it against Python 3.5.

I have the same issue, but I am not sure what is meant by rebuild against Python 3.5. Could you explain it to me?

I have tried redownloading the package, run makepkg and install again, but that's not working. I tried to look into the PKGBUILD file to see if there was a reference to Python x.x hat I could tweak, but couldn't spot one.

EDIT: Fixed. Download, rebuilt and reinstalled python-pypeg2 first and qutebrowser to version 0.4.1, in this order.

Last edited by navi_se (2015-10-02 14:20:25)

navi_se · 2015-10-03 09:19:20

karlch wrote:

I find the script to import mappings from the vimrc an interesting topic although I have no use for it personally. So I came up with this little script which basically greps for the mappings and then tries to substitute them in the keys.conf using vim. It will ask the user for confirmation as not all mappings are surely useful in qutebrowser (or the plain cut might not deliver the excpected result...).
#! /bin/bash

# Safety first
cp ~/.config/qutebrowser/keys.conf ~/.config/qutebrowser/keys.conf.bak
# Catch mappings from vimrc
cat ~/.vimrc | grep "map\|noremap\|nmap\|nnoremap" > ~/tmp.txt

# Run a substitution in vim for the mappings
IFS=$'\n'
for i in $(cat ~/tmp.txt)
do
    cur=$(echo $i | cut -d " " -f3)
    new=$(echo $i | cut -d " " -f2)
    vim +"%s /    ${cur}$/    ${new}TMP/ceI" +"wq" ~/.config/qutebrowser/keys.conf
done

# Remove the TMP which stops remapping recursively
vim +"%s /TMP$//" +"wq" ~/dummyqute
# Remove the file used for catching the mappings
rm ~/tmp.txt
I only tested it with a dummyvimrc including the four mappings equivalent to the ones mentioned by @nevi_se:
nnoremap j h
nnoremap k j
nnoremap l k
nnoremap ö l
The script almost definitely has flaws (I am rather new to shell scripting) and will have lots of empty runs (e.g. leader mappings, function mappings in the vimrc) but it could be a basis to work on. Have fun

Tested this and it works, only change was to replace the ~/dummyqute above with the actual ~/.config/qutebrowser/keys.conf path to make the substitution work. I will have a look at what other config could be taken from .vimrc to the qutebrowser config.

karlch · 2015-10-03 09:28:37

Oops, I thought I had substituted them all, my bad, I shall edit it...

Glad to hear it works though.

If you come up with something useful please share it.

navi_se · 2015-10-03 09:39:05

I will do. Plan is to see if there is anything else that has an immediate port from vimrc to qutebrowser config. I am fairly new to vim, though, so I have a very short config file for now. I was also thinking of translating your script in python, as that seems to be the preferred language of the compiler and the one the browser is running on. I figure it could become a true/false option in the settings config. In the meanwhile, thanks for your work, it was very useful in many ways

Mercader1902 · 2015-10-06 13:51:10

I have random xorg crashes while using qutebrowser after the python update. I don't know how to debug it.

File "/usr/lib/python3.5/site-packages/qutebrowser/app.py", line 119 in qt_mainloop and line 109 in run
File "/usr/lib/python3.5/site/packages/qutebrowser/qutebrowser.py", line 149 in main
File "/usr/bin/qutebrowser", line 9 in <module>

Xorg.0.log:
https://ptpb.pw/WeKZ

Last edited by Mercader1902 (2015-10-06 13:53:47)

The Compiler · 2015-10-07 06:21:04

Mercader1902 wrote:

I have random xorg crashes while using qutebrowser after the python update. I don't know how to debug it.
File "/usr/lib/python3.5/site-packages/qutebrowser/app.py", line 119 in qt_mainloop and line 109 in run
File "/usr/lib/python3.5/site/packages/qutebrowser/qutebrowser.py", line 149 in main
File "/usr/bin/qutebrowser", line 9 in <module>
Xorg.0.log:
https://ptpb.pw/WeKZ

I've not heard about any such problems so far....

What do you mean with xorg crashes exactly? Your whole X crashes, or just qutebrowser? I don't really see anything interesting in Xorg.0.log.

Could you share the full output you get from qutebrowser? What exactly is the context of the stacktrace you posted above?

Mercader1902 · 2015-10-07 09:36:53

The whole xorg crashes and that's what I have from qutebrowser in the tty.

It only happens when I'm using qutebrowser. Before the python update it worked flawlessly...

The Compiler · 2015-10-07 09:47:43

That's the full output you get?

Can you run with --no-err-windows (and | tee the output to some file ideally) and see if that helps (and if not, show the output)?

Mercader1902 · 2015-10-07 10:54:01

I executed the browser with "... | tee output" and it continuous to crash. I have nothing in the file.

The Compiler · 2015-10-07 10:59:09

Can you run it via qutebrowser --debug 2>&1 | tee output, make it crash, and then sanitize and pastebin that file? Can you check if it also happens when launched with --temp-basedir?

Mercader1902 · 2015-10-07 13:41:13

What do you mean with sanitize?

https://ptpb.pw/Rfsi

The last 100 lines:
https://ptpb.pw/a82Y

Do you see something useful? I'm really lost...

The Compiler · 2015-10-07 13:58:05

The log could contain informations you consider sensitive, like what URLs you visited - in case you didn't want me to know that, you'd need to remove them before pasting

Is there a way you can reliably reproduce the issue?

Other than that I'm lost as well, no useful information there... Could you run qutebrowser inside gdb and see if it says something about what happened?

Can you also show the version numbers in qutebrowser --version please?

Mercader1902 · 2015-10-07 14:09:54

qutebrowser v0.4.0
Git commit: v0.4.0-194-g739d2cf (2015-10-02 22:56:18 +0200)

CPython: 3.5.0
Qt: 5.5.0, runtime: 5.5.0
PyQt: 5.5

It just happens, when testing I tried to follow a pattern but it just crash randomly. Sometimes after 20 seconds, sometimes after 30 minutes.
I will test the browser with --temp-basedir and gdb this evening, I'm at the college library right now.

Thank you for all @The Compiler.

Awebb · 2015-10-07 14:55:40

How about supporting one of the existing plugin formats? I doubt too many people are eager to learn yet another method.

The Compiler · 2015-10-07 15:01:41

Awebb wrote:

How about supporting one of the existing plugin formats? I doubt too many people are eager to learn yet another method.

As mentioned in the issue I plan to look into the WebExtensions (Chromium, Opera, soon Firefox) and dwb's javascript API, and see how much it makes sense to stick to those.

As for the Python plugin API (which will definitely arrive quite some time before a javascript API), I'm not aware of any Python APIs for browser plugins - are you?

Last edited by The Compiler (2015-10-07 15:02:34)

Awebb · 2015-10-07 15:13:49

I was talking about WebExtensions, indeed, not about Python plugins. Since I've been a bit of a smartass here, I will now do some research and post here, should I find anything.

Mercader1902 · 2015-10-07 19:32:11

The Compiler wrote:

The log could contain informations you consider sensitive, like what URLs you visited - in case you didn't want me to know that, you'd need to remove them before pasting
Is there a way you can reliably reproduce the issue?
Other than that I'm lost as well, no useful information there... Could you run qutebrowser inside gdb and see if it says something about what happened?
Can you also show the version numbers in qutebrowser --version please?

gbm doesn't show any log.

I took a picture right after the xorg crash:
http://i.imgur.com/S00lXTn.jpg

The Compiler · 2015-10-08 07:49:33

Mercader1902 wrote:

The Compiler wrote:
The log could contain informations you consider sensitive, like what URLs you visited - in case you didn't want me to know that, you'd need to remove them before pasting
Is there a way you can reliably reproduce the issue?
Other than that I'm lost as well, no useful information there... Could you run qutebrowser inside gdb and see if it says something about what happened?
Can you also show the version numbers in qutebrowser --version please?
gbm doesn't show any log.
I took a picture right after the xorg crash:
http://i.imgur.com/S00lXTn.jpg

I'm out of ideas, sorry - this seems more like an X or graphic driver issue triggered by... something :-/

Maybe you can run X inside gdb, or check how to turn on debugging output for your graphic driver and check dmesg, but I doubt I can be of much help at this point

andmars · 2015-10-12 05:53:23

@The Compiler

Hi,

I just read on phoronix.com, that with Qt 5.6 qtwebkit will be dropped. Now I'm curious: What are your thoughts and plans on that?

http://phoronix.com/scan.php?page=news_ … ature-Look

The Compiler · 2015-10-12 06:31:28

andmars wrote:

@The Compiler
Hi,
I just read on phoronix.com, that with Qt 5.6 qtwebkit will be dropped. Now I'm curious: What are your thoughts and plans on that?
http://phoronix.com/scan.php?page=news_ … ature-Look

I just answered to the Phoronix comment, so I'm going to quote myself:

The Compiler wrote:

peppercats;n828679 wrote:
glad that webkit is being dropped, a lot of projects(otter, qupzilla, qutebrowser, etc) still use it as a default despite it having a ton of bugs/performance issues so hopefully they'll be forced to upgrade to webengine.
qutebrowser developer here, and I had to sign up just to answer to this.
It's not like we are too lazy or whatever to switch to QtWebEngine. Hell, it's not like you'd need to force us to switch. I personally would've liked to use QtWebEngine since the beginning - when I started 2 years ago, I even considered waiting until QtWebEngine is released and usable (I'm glad I didn't - I'd still be waiting).
Otter and QupZilla already have a work-in-progress QtWebEngine backend, I'm working hard on writing more tests so I can maintain both backends for the time being.
The problem is that QtWebEngine still is kind of a joke for anything beyond using HTML/CSS to do some presentation kiosk or whatever. This is slowly getting better (especially with 5.6), but it's still a pain. Some examples:
Until 5.5: Controlling storage paths/policies for cache, cookies etc. isn't possible.
Until 5.5: Downloading files isn't possible
Until 5.5: Setting any settings isn't possible
Until 5.6: Form input in Otter won't work with QtWebEngine because of a QtWebEngine bug
Until 5.6: Scrollbars are painted red in QtWebEngine with Otter because - you guessed it - a QtWebEngine bug
Until 5.6: Adblocking is impossible
Until 5.6: custom HTTP headers are impossible (think selective Referer blocking, Do-Not-Track, etc.)
Until 5.6: Flash doesn't work (arguably less important than it used to be, but some users still need it)
Until 5.6: Custom URL schemes for browser-internal pages are not possible
Until 5.6: Setting proxies isn't really possible
Until 5.7: Getting the scroll position isn't possible.
Even after 5.6 (and probably 5.7): Many things possible with QtWebKit still aren't possible (mainly QNetworkAccessManager/QWebElement/QWebFrame APIs missing). For example, anything other than a single, global proxy isn't possible. More sophisticated adblocking isn't possible. Setting the scroll position isn't.
QtWebEngine isn't getting packaged for Fedora/Debian/Ubuntu because it bundles a patched Chromium, so dropping QtWebKit support is infeasible.
And you're blaming us for not switching over yet? So you prefer a completely broken browser over a working but suboptimal one? Honestly, with 5.5 I'm quite happy with QtWebKit, even more so comparing to QtWebEngine... I think QtWebEngine will be the better solution at some point, but it's far from being ready yet, and switching to it will mean losing many features and adding many workarounds.
So yeah, I'm not happy about the removal of QtWebEngine before it can be considered stable/mature, and I certainly don't need to be "forced" to switch.

So I'm having QtWebEngine support on my radar since months (see #666 for a rough list of issues/TODO's), but I first want to work on having more tests so I can maintain both backends without going insane.

I'm not sure if I'll be ready for Qt 5.6, but since the source of QtWebKit is still distributed, I hope distributions (especially those not packaging QtWebEngine) will continue packaging QtWebKit until QtWebEngine is more usable (Qt 5.7? 5.8? Hell, if I knew...). For AUR, that's not so much of an issue - if it's really needed, I'll just package QtWebKit in the AUR.

Ambrevar · 2015-10-20 10:33:49

I have been using qutebrowser for about a day now, and I am very impressed so far!
Thanks for the good work, you've saved my power-web-browsing experience.

After Luakit and dwb, I am wondering if switching webbrowser every 2 years will become a new trend.
(Let's hope this QtWebEngine won't be too much trouble...)

I've noticed that Google Image uses the old display with the default user agent.
Changing it to this did the trick:

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/538.15 (KHTML, like Gecko) Version/8.0 Safari/538.15 qutebrowser/0.4.1

For now the one feature I miss from dwb is the lack of bookmark page.
By the way, why are the bookmarks stored in 'bookmarks/url' and not simply 'bookmarks'? Are you planning to add folder support? That would be great!

I'll post the few issues I've noticed on GitHub asap.

The Compiler · 2015-10-20 11:16:58

Ambrevar wrote:

I have been using qutebrowser for about a day now, and I am very impressed so far!
Thanks for the good work, you've saved my power-web-browsing experience.
After Luakit and dwb, I am wondering if switching webbrowser every 2 years will become a new trend.
(Let's hope this QtWebEngine won't be too much trouble...)

Glad to hear that! QtWebEngine definitely will be some trouble - it'll mostly be some work until everything (downloads, hints, etc.) will work in QtWebEngine-mode. But I'm working hard on the first step towards that, having lots and lots of automated tests

For now the one feature I miss from dwb is the lack of bookmark page.

So just a list of all bookmarks as a HTML page? That should be quite easy - feel free to open an issue!

By the way, why are the bookmarks stored in 'bookmarks/url' and not simply 'bookmarks'? Are you planning to add folder support? That would be great!

Yeah, or at least I wanted to keep that possibility open. I won't add folder support anytime soon though, busy enough with the big things (tests, QtWebEngine, config revolution, per-domain settings) for the next months probably.

See #882 for some discussions.

I'll post the few issues I've noticed on GitHub asap.

Cool! By the way, as you're a TU: Are you interested on packaging qutebrowser for [community] by any chance? That'd be most appreciated!

Florian

Arch Linux

#151 2015-10-02 08:33:33

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#152 2015-10-02 08:54:29

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#153 2015-10-02 09:00:07

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#154 2015-10-02 14:10:02

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#155 2015-10-03 09:19:20

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#156 2015-10-03 09:28:37

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#157 2015-10-03 09:39:05

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#158 2015-10-06 13:51:10

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#159 2015-10-07 06:21:04

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#160 2015-10-07 09:36:53

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#161 2015-10-07 09:47:43

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#162 2015-10-07 10:54:01

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#163 2015-10-07 10:59:09

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#164 2015-10-07 13:41:13

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#165 2015-10-07 13:58:05

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#166 2015-10-07 14:09:54

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#167 2015-10-07 14:55:40

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#168 2015-10-07 15:01:41

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#169 2015-10-07 15:13:49

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#170 2015-10-07 19:32:11

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#171 2015-10-08 07:49:33

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#172 2015-10-12 05:53:23

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#173 2015-10-12 06:31:28

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#174 2015-10-20 10:33:49

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#175 2015-10-20 11:16:58

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

Board footer