You are not logged in.

#151 2021-04-29 18:12:22

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,378

Re: Weaver: a socket controlled web browser

I've really not had much time to work on this (though I'm using it constantly).  I see I did screw up the "list" command.  For some reason I was thinking of renaming the command and changed it in the main code but not the help output.  I fixed that.  List works again, but with the new approach it only gives window ids which can then be fed back to weaver, e.g.:

weaver :list | while read xid; do weaver -w $xid '=location.href+"\t"+document.title'; done

I also added output for any unknown command.

Right now the only commands that accept any other arguments are 'ua' and 'profile'.  I've been thinking about expanding download abilities, and including a stop command could be a good part of that.  I'd like to implement restarting downloads if weaver is exited in the middle of a long-running download.  It's happened a few times now that I've been downloading a large file, and also working on weaver's code which results in my restarting it to test changes.  This currently results in the download stopping, but a .download partial file remains.

Last edited by Trilby (2021-04-29 18:14:19)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#152 2021-06-30 04:46:43

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,260

Re: Weaver: a socket controlled web browser

Any idea how to receive push notification in weaver or somehow block the notification asking permission for push notification. With some sites I get notification asking to allow push notification everytime I visit.


Arch is home!
cwm rofi weaver vifm vis lizzy pass terminator

Online

#153 2021-06-30 13:49:53

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,378

Re: Weaver: a socket controlled web browser

It is pretty easy to "block" the notification by providing a default answer - there is a function that handles various such permission requests from websites, and for now I've only implemented the simplest (in code) approach of asking the user each time.  I've been considering the best way for a user to provide default answers for various types of permission.  I believe some browsers store this information after the first time you answer and reuse you previous answers - I'm not inclined to go this route, but a simple means of user-specified defaults is an eventual intended goal.

In the short term, if you'd like to hack on the code, all this is just in lines 232-237 of server.cpp.  It'd be trivially easy to provide a universal "yes"/"accept" answer to all such requests, though that might not be desirable.  It'd also be just as easy to provide a universal "no"/"reject" to all such requests, though this might break some things that user would expect to work (e.g., all video conferencing, mic input, etc).

The current system of prompting the user every time was partially for initial simplicity to get things up and running without taking any control away from the user (i.e., I was testing weaver on zoom calls, and needed this working).  But it was also a way for me to get a sense of how often, and in what cases websites really make these requests: so by prompting every time it's clear what's actually happening in the background.

Sidenote: there hasn't been any activity on the weaver code in a bit as I've been tinkering with a parallel / alternative project to work similarly with firefox / gecko.  The memory handling issues of qt5-webengine are a real PITA and likely unsolvable (as they are internal to that library, not my code) - when the background process lives on effectively indefinitely, the gradual leaks in webengine accumulate to be quite a mess.  Firefox is *much* better on ram use, but there are a few requirements that I'm not sure it'll be able to meet.  So I'm still tinkering with it (I also can't stand that it pulls in all the gtk3 stuff, but if I can get it otherwise functional, that'd be the lesser of two evils compared to webengine's ram-cancer).

Last edited by Trilby (2021-06-30 13:55:24)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#154 2021-07-04 03:47:55

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,378

Re: Weaver: a socket controlled web browser

I just implemented a user config file which - at the moment - only has settings for the above-mentioned feature permission requests that are generated by some websites.  For any feature you can set defaults of always rejecting, always accepting, or (the default) of asking each time.  You can also specify different permissions for given domains as exemplified in the config.conf provided in the examples directory.

Please test and feel free to report any issues.

Side note on the firefox/gecko avenue ... that was a dead end, so I'm back to focusing on the webengine project here.  As a nice bonus, I was worried my ISP was giving me issues, but it turns out webengine is really just a whole lot faster at loading pages than firefox.

Last edited by Trilby (2021-07-04 03:49:16)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#155 2021-07-04 15:04:48

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,260

Re: Weaver: a socket controlled web browser

Thanks for adding one more feature. I am glad that you are back to working on weaver, which has become my primary browser.
I have selected always rejecting notifications which seems to work at present.
Yes, indeed webengine is fast, but I thought that was because server client architecture keeping webengine loaded in memory.


Arch is home!
cwm rofi weaver vifm vis lizzy pass terminator

Online

#156 2021-07-04 16:45:50

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,378

Re: Weaver: a socket controlled web browser

Perhaps webengine remaining running in the background helps, but only relative to a cold start.  Opening additional tabs / windows in firefox was notably slower for me than even cold starting webengine - it could just be for me as I did no real benchmarking, but it was pretty annoying.  It also seems updates of webengine may have made my memory issues a bit less dramatic, but there still appears to be a very gradual leak.

In anycase, now that I implemented a config file to address the feature-permissions, it'll be pretty easy to start adding other user-configurable options for many bits that are currently hard coded.

Last edited by Trilby (2021-07-04 16:49:04)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#157 2021-07-05 08:14:29

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,260

Re: Weaver: a socket controlled web browser

After setting "Notifications=reject", I am still getting message asking for permission to enable push notifications. Atleast on one site https://pyra-handheld.com/boards/ . Unless you login there, you won't get that message. However I am not getting that message from protonmail now.


Arch is home!
cwm rofi weaver vifm vis lizzy pass terminator

Online

#158 2021-07-05 13:33:39

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,378

Re: Weaver: a socket controlled web browser

I just create an account there to test, and I believe I see what you are describing, but that is actually part of their website code, not an API request sent to the browser.  Some websites put up their own prompt / request for permission before they even attempt to request permission from the browser - essentially their site is asking for your permission to ask the browser's permission.  Weaver settings can only impact the outcome of the second of those two requests.

You can test / confirm this by toggling the Notifications setting for weaver, then going to your profile page which this should go to:

https://pyra-handheld.com/boards/account/preferences

There is an option for "Push notifications" with a red button labeled "Enable push notifications".  If you set weaver's setting to "ask", pressing this red button will present webengine's own prompt box asking for your permission.  If you set weaver's setting to "accept" pressing this red button will enable notifications with no prompt from webengine.  If you set weaver's setting to "reject" (or really anything other than "accept" or "ask") pressing that button will have no effect at all (as weaver rejects the request without asking you first).

If the message you see shows up at the bottom of the page at the full width of the page, matching the style of the page, then that is their "pre-request" request.  It seems I only saw that once, and once rejected I no longer see it.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#159 2021-07-06 08:14:02

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,260

Re: Weaver: a socket controlled web browser

Yes, indeed. Now I can see the difference between webengine request and website request. Thanks.


Arch is home!
cwm rofi weaver vifm vis lizzy pass terminator

Online

#160 2021-07-21 17:11:52

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,260

Re: Weaver: a socket controlled web browser

I just noticed few updates in the project and tried to build it, but I am getting error when building with makepkg and older PKGBUILD.
https://termbin.com/9lor


Arch is home!
cwm rofi weaver vifm vis lizzy pass terminator

Online

#161 2021-07-21 18:34:03

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,378

Re: Weaver: a socket controlled web browser

Sorry about that - I'm surprised the second one even got pushed (the QString -> char * issue), but that has been fixed.  I also pushed an update to use the proper %lld in the format string.  I overlooked this as on a 64-bit machine (long) == (long long), or at least long is 64 bit.  I gather you are building on a 32 bit system, right?

Last edited by Trilby (2021-07-21 18:34:17)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#162 2021-07-21 18:46:20

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,260

Re: Weaver: a socket controlled web browser

I tried again, but no luck. There are less errors now though.
https://termbin.com/vk7u

I am building using arch.

───▶ uname -a
Linux asusarch 5.12.15-arch1-1 #1 SMP PREEMPT Wed, 07 Jul 2021 23:35:29 +0000 x86_64 GNU/Linux

Last edited by Docbroke (2021-07-21 18:50:33)


Arch is home!
cwm rofi weaver vifm vis lizzy pass terminator

Online

#163 2021-07-21 19:33:00

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,378

Re: Weaver: a socket controlled web browser

Ah, and I see makepkg has much stricter flags (including -Wformat) as of a week ago.  I've just patched up all the format strings to avoid all these warnings.

Last edited by Trilby (2021-07-21 19:33:11)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#164 2021-07-22 02:11:52

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,260

Re: Weaver: a socket controlled web browser

thanks, build is successful now.

EDIT: hinter appears to have improved.

EDIT2: something changed with downloads, "weaver :downloads" stuck at 2% after download is finished.

Last edited by Docbroke (2021-07-22 02:43:32)


Arch is home!
cwm rofi weaver vifm vis lizzy pass terminator

Online

#165 2021-07-22 03:21:27

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,378

Re: Weaver: a socket controlled web browser

The format of the downloads display is now customizable - but not really documented yet.  I also recently noticed what you describe *if* you mean it says 10e2% as that is 100% just in exponential notation which I'm going to get rid off next time I tinker with that part of the code.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#166 2021-07-22 05:51:47

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,260

Re: Weaver: a socket controlled web browser

One more thing, when you  tinker with hinter.js, can you add possibility to open hinted link in new window? Like "f" to open link in current window and "F" to open in new window.


Arch is home!
cwm rofi weaver vifm vis lizzy pass terminator

Online

#167 2021-07-23 01:51:20

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,378

Re: Weaver: a socket controlled web browser

I've added the option for opening hints in a new window to hinter.js and added "F" to keys.js to activate this option.

On a different note, I just started tinkering with wayland (using sway) and I was pleasantly surprised to see that weaver runs natively on wayland as-is.

Last edited by Trilby (2021-07-23 01:52:28)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#168 2021-07-23 02:27:57

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,260

Re: Weaver: a socket controlled web browser

F still opens in same window. I have updated my hinter.js from example files and added keybinding for F.

I had too tried weaver with hikari(wayland wm based on cwm) and it was working last time I checked, I will look that again but aur doesn't appear reachable at present.


Arch is home!
cwm rofi weaver vifm vis lizzy pass terminator

Online

#169 2021-07-23 03:16:49

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,378

Re: Weaver: a socket controlled web browser

I just saw the inconsistent behavior and I believe it's fixed.  Apparently the "newWindow" variable was being elavuated prior to the event handler being called ... I think.  Anyways I made a "getter" style function for the value and it appears to be working better - but it still hasn't been tested much.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#170 2021-07-23 04:46:55

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,260

Re: Weaver: a socket controlled web browser

Behavious is still inconsistent. When I press "F" and select unique hint (without pressing enter), link opens in current window only, but if I press enter key immediately after selecting hint, link opens in new-window as well as in the current window.


Arch is home!
cwm rofi weaver vifm vis lizzy pass terminator

Online

#171 2021-07-23 11:31:58

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,378

Re: Weaver: a socket controlled web browser

Ah!  Thank you, that description makes it clear what I missed.  Standby ...

edit: I just pushed an update to hinter.js.  Given how many times it's been wrong, I don't want to be too confident, but this fix actually makes total sense (both why it seemed inconsistent and how it needed to be fixed).  The last attempt was very unsatisfactory as I was suspecting that there was some odd scope / time-of-evaluation issue in the JS which didn't make a whole lot of sense.

That's the value of good feedback / bug reports, thanks.

Last edited by Trilby (2021-07-23 11:40:36)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#172 2021-07-23 11:54:40

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,260

Re: Weaver: a socket controlled web browser

This appears to be working, thanks.


Arch is home!
cwm rofi weaver vifm vis lizzy pass terminator

Online

#173 2021-07-24 18:06:50

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,260

Re: Weaver: a socket controlled web browser

There are still issues with hinter
1) It doesn't work on all sites, e.g, when I try to use hinter with https://wiki.archlinux.org/title/List_of_applications
all userscript stops working (including all keybinds)
2) some links are not detected by hinter, see the below image comparing hinter with similar script from lariza https://www.uninformativ.de/git/lariza/ … ts.js.html

using-lariza-hints.png
using-hinter-js-weaver.png


Arch is home!
cwm rofi weaver vifm vis lizzy pass terminator

Online

#174 2021-07-24 20:40:16

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,378

Re: Weaver: a socket controlled web browser

Well, it's working fine, it's just not doing what you want.  Feel free to edit, it's just meant as an example.

In the first case, there are over 7000 links on that page, and the example hinter creates hints for all of them - so it just takes a *long* time for the script to finish.  As all user scripts run in the same script "world" other user scripts will not be able to run until it finishes.  You can cap the number of hints created by editting line 91 to something like the following:

for (var i = 0; i < links.length && i < 200; i++) {

This could also be editted within the loop to skip any anchor element not currently on screen.  But then you couldn't hit 'f' then scroll and still have followable links.  Having this later ability on a vast majority of web pages (with manageable number of total links) is more value to me than having hinting work just on the viewport on the rare page with thousands of links.

Theoretically one could check the total number of links on the page and conditionally do one or the other approach: e.g., on pages with less than 400 total links, create hints for all of them, on pages with more than that, create hints only for the links that are currently in the viewport.

And for the second case, it's due to their being no way to give a css style for something to be "really really actually on top" you can only give arbitrarily high z-index values.  The default for hinter.js is 1000, feel free to edit line 33 to increase this further.  I just changed it to 99000 and it works on the weaver project page (but they will still not show on any web page with absolute position elements with a z-index even higher than 99000).  I don't know what the maximum allowable value for z-index actualy is - and I suspect it will vary by browser (though I'm pretty sure it'd at least be 2.1 billion).


EDIT: it turns out the conditional approach for pages with huge numbers of links was pretty easy to implement, so I just pushed a revision that does that.  But really, feel free to edit it.  I'd suspect you'd at least edit the styling to your liking.

Last edited by Trilby (2021-07-24 21:06:26)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#175 Today 13:12:11

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,378

Re: Weaver: a socket controlled web browser

Weaver has just received major revisions which consolodated control mechanisms between js input and socket commands.  This drastically simplified the code and made the "client" program nearly irrelevant (it's now just a very short script).  As a result command format has changed; any key bindings that call weaver will need to be updated (not within-page keybindings in keys.js, these are the same, except the js function weaverCommand is no longer available).

There is now what I expect to be a much more useful README, help page here.  That page call also be viewed via `weaver //help` from an updated version of weaver.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

Board footer

Powered by FluxBB