qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

Trilby · 2016-12-21 01:53:33

I *finally* just rid my system of chromium (which meant I could get rid of gtk and all of it's depdendencies) because we can now have netflix without it.

For some time I've been able to watch netflix in qutebrowser but only if both chromium and chromium-widevine were installed. As of qt5-webengine 5.7 webenegine should manage this itself.

I have to admit that most of this is over my head - so I've been doing a bit of flailing before finding success. But I checked the arch build of qt5-webengine and was pleased to see that the compile flag mentioned in that blog is enabled in the precompiled version in the repos. So the only hurdle left was the widevine lib/plugin.

I didn't have any luck with just chromium-widevine installed, but then I checked the contents of the chromium package and found the needed libwidevinecdmadapter.so file. As a first pass, I simply copied this file from the packae into /usr/lib/chromium/ and uninstalled chromium, gtk2, and all the dependencies. I figured it was a long shot - but it worked: I'm now watching netflix without any of that baggage.

I'll soon test whether it also works without the chromium-widevine package (no bloat there, but if it's not needed that's good to know). And assuming it is kosher license-wise to do so, I may soon make a qutebrowser-widevine AUR package that pulls this single needed lib file out of chromium to save the hassle for others. If anyone wants to beat me to the punch on this - please do.

For the time being I wanted to share this success so others might be able to do the same (if you also loathe widget toolkits, but want to watch netflix).

ewaller · 2016-12-21 04:27:13

Trilby wrote:

if you also loathe widget toolkits, but want to watch netflix.

Isn't that why $DEITY invented the Raspberry Pi?

The Compiler · 2016-12-21 06:28:08

Trilby wrote:

I'll soon test whether it also works without the chromium-widevine package (no bloat there, but if it's not needed that's good to know). And assuming it is kosher license-wise to do so, I may soon make a qutebrowser-widevine AUR package that pulls this single needed lib file out of chromium to save the hassle for others. If anyone wants to beat me to the punch on this - please do.

Yay! You might want to consider naming it qt5-webengine-widevine or so instead, as it should work fine with e.g. QupZilla too.

Trilby · 2016-12-21 12:29:42

I found that an aur package for this would be nearly identical to the chromium-widevine package with just a few changes: switch 'chromium' from a dependency to a conflict, add the cdmadapter.so to the bsdtar line and add an install line for the cdmadapter.so. I've posted a PKGBUILD as a github gist for now and have posted a comment on the chromium-widevine page for Scimmia to see if it'd be feasible to have a single package cover both use cases before I submit the new one to the AUR.

EDIT: in considering a way to prevent this from conflicting with chromium I was thinking that changing the path from /usr/lib/chromium/ to something qtwebengine related might be useful. From the above linked blog there is this excerpt:

Qt Blog wrote:

Qt WebEngine will try to find an already downloaded or installed version of the plugin, but while convinient this might not be enough if you want to ship an application that relies on it being present. In that case you need to extract from Google Chrome and place it in the application or ‘plugins/ppapi’ folder.

This sounded promising: I could put the so files in the qt5-webengine lib folder and not have to worry about conflicts with other browsers. Unfortunately I'm not sure where this path actually is. I grepped the package contents of qt5-webengine and didn't find any path with ppapi in it, but there is a /usr/lib/qt/plugins path - I tried placing the widevine .so files there, but they were not found when I tried to play a video.

~~The Compiler, do you happen to know if there is a qt5-webengine specific path in which these ppapi plugins would be detected?~~

EDIT 2: nevermind that. It turns out that was the right path, I just needed to create the ppapi subdirectory. Now I can make a qt5-webengine specific widevine package that neither depends on nor conflicts with any other browsers (AUR Package).

Trilby · 2017-01-05 04:19:19

Pasting text has not been working for me for some while now. I'd not reported it yet as I hadn't taken any time to investigate. But over the past few days I've been trying everything I can think of and have tried to search the issue tracker, but I'm making no progress: the shift-insert binding fails to insert the primary selection contents.

I've swapped the key binding for insert-text {primary} and opening an external editor to ensure the key symbols were right. When swapped, shift+insert opened an editor as expected, and my Ctrl-e binding did nothing. When switched back, vice-versa (ctrl-e opened the editor, and shift-insert did nothing). Notably, the key bound to "text-insert {primary}" did give a suitable error message when the primary selection was empty which confirms the binding is working. But when there was something in the primary selection there was no response at all.

I tried various command mode entries including "insert-text {primary}" which behaves just like the keybinding: with an empty selection it gives an error, with anything in the primary selection, it does nothing at all. I then went to just doing "insert-text testing" for a keybinding and directly from the command mode and similarly had no response. I tried "later 3000 insert-text testing" and ensured a text field was focused within 3 seconds, but there was still no response.

Inserting with the insert-text command and/key binding works fine with the webkit backend but not with webengine.

I've just now ran qutebrowser in debug mode (qutebrowser --backend=webengine --debug) and I see these lines when I try to insert "testing" into the search field of a duckduckgo page:

22:57:04 DEBUG    misc       app:on_focus_object_changed:768 Focus object changed: <PyQt5.QtWidgets.QOpenGLWidget object at 0x7f7fa855d678>
22:57:06 DEBUG    commands   command:run:509 command called: insert-text ['"testing"']
22:57:06 DEBUG    commands   command:run:524 Calling qutebrowser.browser.commands.CommandDispatcher.insert_text(<qutebrowser.browser.commands.CommandDispatcher>, '"testing"')
22:57:06 DEBUG    webview    webenginetab:_js_cb_single:418 Got element from JS: {'outer_xml': '<input type="text" name="q" tabindex="1" autocomplete="off" id="search_form_input" class="search__input--adv  js-search-input" value="ddg" autocapitalize="off" autocorrect="off">', 'class_name': 'search__input--adv  js-search-input', 'id': 16, 'value': '', 'rects': [{'left': 361.984375, 'top': 19, 'width': 494.765625, 'height': 38, 'bottom': 57, 'right': 856.75}], 'tag_name': 'INPUT', 'attributes': {'autocomplete': 'off', 'name': 'q', 'type': 'text', 'id': 'search_form_input', 'class': 'search__input--adv  js-search-input', 'autocapitalize': 'off', 'tabindex': '1', 'value': 'ddg', 'autocorrect': 'off'}}
22:57:06 DEBUG    webelem    webelem:is_editable:252 Checking if element is editable: <qutebrowser.browser.webengine.webengineelem.WebEngineElement html='<input type="text" name="q" tabindex="1" autocomplete="off" id="search_form_input" class="search__input--adv  js-search-input" value="ddg" autocapitalize="off" autocorrect="off">'>
22:57:06 DEBUG    webelem    webengineelem:insert_text:103 Inserting text into element <qutebrowser.browser.webengine.webengineelem.WebEngineElement html='<input type="text" name="q" tabindex="1" autocomplete="off" id="search_form_input" class="search__input--adv  js-search-input" value="ddg" autocapitalize="off" autocorrect="off">'>
22:57:06 DEBUG    misc       app:on_focus_object_changed:768 Focus object changed: None

The command with my input of "testing" is received and processed and the appropriate web element is referred to. The output suggests that text was inserted into that web element and there are no apparent errors here - but nothing was actually inserted into the text field.

I get equivalent results with the textboxes on these forums, and every other textarea/input field I've tried.

As a final test I just moved my entire ~/.config/qutebrowser directory and started qutebrowser again with a fully default configuration, but this had no effect on the symptoms: inserting with a keybinding or from command mode fails silently.

The above diagnostics were all conducted on my home comuputer running qutebrowser-git from the AUR, but I've been having the same problem on my work computer which has qutebrowser from [community] (it should by the stable version from [community], I can confirm this tomorrow if needed). I've just updated the git version on my home computer today and there has been no change in symptoms (updated r10933.59b378e29-1 -> r10971.48d4c9311-1).

Are there any other diagnostics I could provide? Are others able to use insert-text with webengine?

EDIT: xref to github issue. As noted there, the input fields on the suggested page below also fail for me.

EDIT 2: For other's struggling with this, middle-button paste does work. This will serve as a much better alternative for the time being than syncing selections and using Ctrl-V.

Last edited by Trilby (2017-01-06 19:50:16)

The Compiler · 2017-01-05 06:41:23

Hmm, this is odd - do you remember ever seeing it working on DuckDuckGo with QtWebEngine?

From what I've seen from a few quick tests:

It works fine on e.g. http://www.tizag.com/htmlT/htmlinput.php
It doesn't work when frames/iframes are involved (which is not implemented yet)
It also doesn't work on DuckDuckGo (which is the part which should work...)

I'm a bit busy with university exam preparations for the next weeks, but please open an issue!

F34R · 2017-01-12 14:10:16

I always failed with webengine backened, why not working ? Crashed immediately.

Last edited by F34R (2017-01-12 14:10:34)

Trilby · 2017-01-12 14:21:05

F34R, there's not much to go on there. Do you have qt5-webengine installed? Has this been a persistent problem, or is it new (if new, are you trying to use infinality)? What is the output from running `qutebrowser --backend=webengine` from a terminal? If nothing obvious comes up there (post it here anyways, it might not be obvious to you but others could see it) then use `qutebrowser --backend=webengine --debug` to get more verbose output.

F34R · 2017-01-15 09:25:14

Trilby wrote:

F34R, there's not much to go on there. Do you have qt5-webengine installed? Has this been a persistent problem, or is it new (if new, are you trying to use infinality)? What is the output from running `qutebrowser --backend=webengine` from a terminal? If nothing obvious comes up there (post it here anyways, it might not be obvious to you but others could see it) then use `qutebrowser --backend=webengine --debug` to get more verbose output.

Yes I installed, but I also have Qtwebkit packages.

normal terminal output.
Pastebin

debugged terminal output.
debug

qutebrowser was restarted after a fatal crash!
Unfortunately, this crash occurred in Qt (the library qutebrowser uses), and QtWebKit (the current backend) is not maintained anymore.

Since I can't do much about those crashes I disabled the crash reporter for this case, but this will likely be resolved in the future with the new QtWebEngine backend.

I don't have any idea why not working.

Trilby · 2017-01-15 12:54:34

There are some odd nouveau errors there. But the debug output doesn't seem to match your description. The debug output shows that duckduckgo.com was successfully opened (presumably as a home page) then you opened a new url 10fastfingers.com and that at least attempted to open.

What are the actual symptoms? Does it just crash on that webpage?

EDIT: this looks like a previously reported issue between nouveau and qtWebEngine and there is open bug report here and another with an alleged workaround.

Does the following work:

export LIBGL_ALWAYS_SOFTWARE=1
qutebrowser --backend=webegine

I can't quite tell if that should have an effect at runtime as it is suggested as a workaround for packaging - but it's worth a shot.

EDIT 2: Also, do you have mesa-libgl installed?

F34R · 2017-01-15 13:18:31

Trilby wrote:

There are some odd nouveau errors there. But the debug output doesn't seem to match your description. The debug output shows that duckduckgo.com was successfully opened (presumably as a home page) then you opened a new url 10fastfingers.com and that at least attempted to open.
What are the actual symptoms? Does it just crash on that webpage?
EDIT: this looks like a previously reported issue between nouveau and qtWebEngine and there is open bug report here and another with an alleged workaround.
Does the following work:
export LIBGL_ALWAYS_SOFTWARE=1
qutebrowser --backend=webegine
I can't quite tell if that should have an effect at runtime as it is suggested as a workaround for packaging - but it's worth a shot.
EDIT 2: Also, do you have mesa-libgl installed?

No, not just this page, this is the example. All my visited webpage crashed...

Yepp the mesa-libgl also installed.

the

export LIBGL_ALWAYS_SOFTWARE=1

just working, thanks Trilby.

Last edited by F34R (2017-01-15 13:18:57)

Trilby · 2017-01-15 13:45:09

F34R wrote:

No, not just this page, this is the example. All my visited webpage crashed...

I'm glad it worked, but my point was you were able to open qutebrowser just fine. It only crashed when you navigated to some pages: even if almost any page crashes, this is different from crashing on startup which is what you first described. Once you gave accurate symptoms and actual error messages, it was pretty easy to get to the bottom of this. Please keep this in mind in the future on these forums, and start by giving accuarate descriptions of the symptoms and exact error messages.

WorMzy · 2017-01-16 12:55:29

Is there a hidden setting or something that stops the browser entering input mode when textboxes/areas are focused, and only enters input mode when explicitly told to (by pressing 'i')?

Something in my .cache/qutebrowser folder seems to be causing this behaviour, since starting the browser with a fresh cache restores the normal behaviour (lets me enter input mode on textbox/area focus again without having to press 'i'). I have a suspicion that my cat accidentally (or purposely, who knows with cats?) hit a key combination when walking accross my keyboard that toggled *something*, but nothing under :set input seems relevant to the problem (and starting with a fresh config did not resolve the problem, although a fress config did start with my old session, so perhaps there's a session-specific option for this behaviour?).

EDIT: the old cache folder is now working, so I can't check if this is a session-specific behaviour at the moment.

Last edited by WorMzy (2017-01-16 12:58:47)

lahwaacz · 2017-01-16 13:41:23

There is `input -> auto-insert-mode`, which is `false` by default, but it does not always work as expected:
https://github.com/The-Compiler/qutebrowser/issues/89
https://github.com/The-Compiler/qutebrowser/issues/673
https://github.com/The-Compiler/qutebrowser/issues/793

Trilby · 2017-01-16 14:03:03

I was thinking of auto-insert-mode too, but that only applies immediately after a page load (if an input field is focused, should qutebrowser go into insert mode). Whenever you explicitly select an input field via mouse click or following a hint, input mode is entered. I don't know of a setting that changes that behavior. I'm not sure why one would want to explicitly select an input field without entering insert mode. WorMzy, what would be a use-case for this behavior? Or did I misunderstand and you are only referring to auto-focused fields when a page loads?

WorMzy · 2017-01-16 14:13:53

Oh, I can't think of a use case for it, I was just experiencing it.

I wasn't sure if it was a bug or expected behaviour, so I was hoping to get a reply along the lines of "Oh you've enabled XYZ mode, press Ctrl+Shift+? to disable it again". Restarting qutebrowser with a fresh cache isn't too cumbersome of a workaround if the behaviour reoccurs, but I'd be happier knowing what happened and how to enable/disable it properly.

I tried switching auto-insert-mode on while exploring the :set input options, but it didn't change the behaviour I was experiencing. I also toggled auto-leave-insert-mode which didn't help either.

EDIT: I should mention that I upgraded qutebrowser last night (along with zlib libinput gnutls xorg-fonts-misc akonadi lib32-libwebp lib32-zlib libvpx libwps minizip mpd nginx), but downgrading my packages to the pre-update state did not affect the behaviour (the qutebrowser update was only a pkgrel bump to add an opt depend, so I didn't think that would've been the cause of the problem anyway). I am now running a fully up-to-date system and don't have the problem.

I should also mention that I am using the webengine backend.

Last edited by WorMzy (2017-01-16 14:24:07)

lahwaacz · 2017-01-19 10:32:17

Heh, now I've noticed this behaviour too. But I don't think it's bound to the session as WorMzy described, because while clicking an input field in internet banking system does not trigger insert mode, in another tab in the same instance clicking DuckDuckGo's search box triggers insert mode as expected. I must say though that the banking system is full of Javascript and other crap, there are dozens of buttons for which qutebrowser does not create a hint, so I'm not surprised that something does not work there.

WorMzy · 2017-01-19 13:07:26

Unfortunately DDG was affected for me too, so I don't think this was the same problem, although it sounds like you had the same symptoms on the banking website. I've not been able to reproduce it so far though.

Docbroke · 2017-01-19 13:38:26

Though I am not using qutebrowser anymore, this issue with insertmode was present in earlier versions too ( before this migration towards qtwebengine started). At that time unpredictable behavious of insertmode along with few issues of webkit made me quit.
In my view main issue is with this dual mode vim like nature of qutebrowser. I would have been more happier with singlemode behaviour in this otherwise great browser.

The Compiler · 2017-01-20 06:51:00

There are a few related issues here:

Either backend and rich-text editors: #253
Frames/iframes with QtWebEngine won't work yet, tracked in #666 (the generic QtWebEngine issue)
QtWebEngine and some zoom levels: #2169

Docbroke wrote:

In my view main issue is with this dual mode vim like nature of qutebrowser. I would have been more happier with singlemode behaviour in this otherwise great browser.

I don't think this will ever be the primary focus of qutebrowser, but I wonder how close you could get once there are autocmds (or a plugin API) by simply entering insert mode on start and rebinding keys accordingly.

Docbroke · 2017-01-20 07:04:56

The Compiler wrote:

I wonder how close you could get once there are autocmds (or a plugin API) by simply entering insert mode on start and rebinding keys accordingly.

Is such autocmds available in current versions? How to apply that, I would like to test it.

The Compiler · 2017-01-20 07:10:33

No - there's a PR for it, but it would need an update for all the QtWebEngine refactorings, and I'd prefer to add something like that after there's a Python plugin API (so events in the plugin API and autocmds are in sync and not two completely separate APIs).

kvtb · 2017-01-21 13:57:43

I'm experimenting with Wayland and Sway (i3)
I've set QT_WAYLAND_DISABLE_WINDOWDECORATION=1
to hide client side window decorations

Other qt5 apps such as Trojita indeed hide the additional Qt window decoration when this env.var is set to 1, but qutebrowser isn't.
So, in Wayland with Sway, qutebrowser has two window decorations...

I have no idea if this is a Qutebrowser issue or perhaps a qt-py issue, but if someone else has seen the same issue and knows how to fix this, please share!
Thanks in advance!

Last edited by kvtb (2017-01-21 14:15:02)

The Compiler · 2017-01-21 14:24:06

kvtb wrote:

I'm experimenting with Wayland and Sway (i3)
I've set QT_WAYLAND_DISABLE_WINDOWDECORATION=1
to hide client side window decorations
Other qt5 apps such as Trojita indeed hide the additional Qt window decoration when this env.var is set to 1, but qutebrowser isn't.
So, in Wayland with Sway, qutebrowser has two window decorations...
I have no idea if this is a Qutebrowser issue or perhaps a qt-py issue, but if someone else has seen the same issue and knows how to fix this, please share!
Thanks in advance!

qutebrowser has a ui -> hide-wayland-decoration setting which overrides the environment variable, so just set that instead

kvtb · 2017-01-23 18:00:15

@The Compiler: thanks for your quick response and thanks for foreseeing this issue and implementing this option

Arch Linux

#376 2016-12-21 01:53:33

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#377 2016-12-21 04:27:13

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#378 2016-12-21 06:28:08

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#379 2016-12-21 12:29:42

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#380 2017-01-05 04:19:19

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#381 2017-01-05 06:41:23

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#382 2017-01-12 14:10:16

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#383 2017-01-12 14:21:05

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#384 2017-01-15 09:25:14

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#385 2017-01-15 12:54:34

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#386 2017-01-15 13:18:31

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#387 2017-01-15 13:45:09

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#388 2017-01-16 12:55:29

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#389 2017-01-16 13:41:23

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#390 2017-01-16 14:03:03

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#391 2017-01-16 14:13:53

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#392 2017-01-19 10:32:17

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#393 2017-01-19 13:07:26

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#394 2017-01-19 13:38:26

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#395 2017-01-20 06:51:00

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#396 2017-01-20 07:04:56

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#397 2017-01-20 07:10:33

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#398 2017-01-21 13:57:43

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#399 2017-01-21 14:24:06

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

#400 2017-01-23 18:00:15

Re: qutebrowser - A keyboard-driven, vim-like browser based on PyQt5

Board footer