You are not logged in.

#51 2021-02-01 03:40:09

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,282

Re: Weaver: a socket controlled web browser

Oops, thanks.  I just pushed the fix for that.  I had changed the data structure used to pass some information between functions and forgot to update one use of it.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#52 2021-02-01 06:05:58

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,235

Re: Weaver: a socket controlled web browser

Can I call some function from a script using something like "weaver load_function reading_mode".
Here is my usecase.

I have this script to provide readingmode , which I used to read articles without distractions (with vimb browser) by calling function simplyread. I am not able to access the original source of simplyread therefor I have uploaded that file.

One more thing, I would like is ability to get link_url, which may-be url under the mouse or (some hinting system if you are willing to add some more bloat), for example "weaver link-url" to return url under mouse.
I know one can alway copy that with right click and copy-link-address, but it makes it two step process to use that url somewhere else, with "weaver link-url", that url can be directly opened with link-handler script using single shortcut.

Last edited by Docbroke (2021-02-01 06:56:33)


Arch is home!
cwm rofi weaver vifm vim lizzy pass terminator

Offline

#53 2021-02-01 14:02:22

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,282

Re: Weaver: a socket controlled web browser

Yes both will be very easy to do and the mechanism to do so is on the TODO list: being able to send JS over the socket and have weaver run it.  For the first use case, I should be able to have that within a day or so and there might be two ways you could do it: 1) you put that JS code that defines the function in a userscript which loads it into all pages (that the userscript @ lines match), then you call the function via a soon-to-be-implemented function (whose name will probably change) like "weaver run-javascript function_name(parameters)".  The other way is to to just read the whole thing in every time it's called with something like "weaver run-js-from-file /path/to/the_script.js".  Again, these abilities will be coming very soon.

There are two ways I could implement the second goal as both JS and Qt can get the url under the mouse.  I'd prefer to keep this in JS to keep the weaver code simpler.  So the goal would be something like "weaver run-javascript-with-return <jscode to get href from element under mouse>".  This JS approach that provides a return value, however, will take some work.  Qt only allows asyncronous JS calls.  So there's no obvious way to wait for the return value of the JS code to send back over the socket to the client.  Lots of people have tried to do this and have come up with ugly and fragile hacks which are not good enough for me.  What I think will end up being most likely will be two commands "weaver run-javascript <whatever>" then a moment later and potentially in a loop until a result is returned "weaver check-javascript-result".

If asyncronous JS is not doable, I'll revist this and consider implementing the Qt based mechanism to retrieve the url under the mouse.  The reason I'd favor the JS mechanism is that implementing it will cover an enormous range of behavior and use-cases, while the Qt approach would be very specific: it'd do what you want, but not be useful for anything else.


EDIT: Actually that was easier than I though.  js-* functions are done.  I will eventually add a wiki page about some potential 'gotchas' with using them.  But as a very short overview:

"js-run [script]" to run the provided script.  Obviously some legal JS symbols/code would be challenging to escape on a command line, so I strongly recommend putting any complex code into a userscript file in a named function, then just use "js-run functionName()".  Also note that as a side-effect of how command line arguments are parsed and then sent over the socket, whitespace ends up collapsed, so the following would be both be treated like the first form is intended "js-run alert('hello world');" and "js-run alert('hello   world');".

"js-result" returns the result from the last JS code sent via "js-run".  A result in this context is the evaluation of the last command in the javascript.  This should not be used, however, until "js-result-ready" returns 'true' as all javascript is run asyncronously.

So when you do not need any return value from the javascript just use the single command.  This example will have the same result as "weaver scroll-top" (this is essentially how that command is implemented):

$ weaver js-run 'window.scrollTo(0,0)'

If you need the result use (at least) three commands:

$ weaver js-run 'prompt("What is your name?");'
$ weaver js-result-ready         # will return an empty string while script is still running (e.g., while prompt box is still open)

$ weaver js-result-ready         # when the script is done, this returns "true"
true
$ weaver js-result                  # this will always return a result - but if js-result-ready is not true, the data may be stale
Trilby

So in a script, you may want to loop on 'js-result-ready':

weaver js-run 'doSomething();'
while [ -z "$(weaver js-result-ready)" ]; do sleep 0.1; done
retval=$(weaver js-result)

CAUTION weaver does not do any checking of your JS code.  None.  At all.  If you do something silly, you could very easily hang a web worker thread, eat up ram, etc.  Weaver does not prevent you from doing stupid things, as doing so would also prevent you from doing creative things.  However, I do plan to have some sort of option to disable the js-* functions all together for (hyper) security purposes.  As is, if any process running as your user on your system simply executes a "weaver js-run BLAH BLAH" it could cause a fair bit of trouble for you (that is if it was deliberately designed to be malicious, garbage input will just fail to do anything).

Last edited by Trilby (2021-02-01 17:59:33)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#54 2021-02-01 18:09:59

qinohe
Member
From: Netherlands
Registered: 2012-06-20
Posts: 1,276

Re: Weaver: a socket controlled web browser

@Trilby, PKGBUILD 'weaver-dev' fails to build if there is a marginal difference between the time on your system and mine!
So, if remote is r202102011754 but mine is r202102011755 an error is caused, which makes sense;)
Tree of a build:

.
├── pkg
│   └── weaver-dev
├── PKGBUILD
├── src
│   ├── weaver-dev-r202102011754
│   │   ├── client.o
│   │   ├── examples
│   │   │   ├── shortcut
│   │   │   └── xbindkeysrc
│   │   ├── LICENSE
│   │   ├── Makefile
│   │   ├── packaging
│   │   │   └── PKGBUILD
│   │   ├── server.o
│   │   ├── src
│   │   │   ├── client.cpp
│   │   │   ├── client.h
│   │   │   ├── server.cpp
│   │   │   ├── server.h
│   │   │   ├── weaver.cpp
│   │   │   ├── weaver.h
│   │   │   ├── webengine.cpp
│   │   │   ├── webengine.h
│   │   │   ├── webkit.cpp
│   │   │   └── webkit.h
│   │   ├── weaver
│   │   ├── weaver.o
│   │   └── webengine.o
│   └── weaver-dev-r202102011754.tar.gz -> /home/mark/build/weaver-dev/weaver-dev-r202102011754.tar.gz
└── weaver-dev-r202102011754.tar.gz

7 directories, 23 files

The error is:

==> Entering fakeroot environment...
==> Starting package()...
/home/mark/build/weaver-dev/PKGBUILD: line 20: cd: weaver-dev-r202102011755: No such file or directory
==> ERROR: A failure occurred in package().
    Aborting...

weaver-nightly builds okay

Offline

#55 2021-02-01 18:13:24

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,282

Re: Weaver: a socket controlled web browser

I saw that once before but then could never replicate it.  I don't think it's about the difference in time between machines, but if the makepkg was running accross a "minut tick" on your system as the _ver variable expands to different values in different places in the PKGBUILD.  This would be the case if _ver=$(date -u ....) was "lazy evaluated" and the date command was run more than once while processing the PKGBUILD.  But I don't think this could be the case.

EDIT: I just confirmed Makepkg will re-evaluate such a variable multiple times - that's a problem, and seems like a very bad idea.  I confirmed it with this:

pkgname=test
_ver=$(date -u +%s)
pkgver=r${_ver}
pkgrel=1
pkgdesc='testing'
arch=('x86_64')

build() {
	echo $_ver
}

package() {
	echo $_ver
}

That will most likely echo different numbers in the build and package stages (if not, add a sleep to the functions and it definitely will).  I'll create a thread on this ...

Last edited by Trilby (2021-02-01 18:18:31)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#56 2021-02-01 18:17:43

qinohe
Member
From: Netherlands
Registered: 2012-06-20
Posts: 1,276

Re: Weaver: a socket controlled web browser

True, it succeeds if I build it exactly  on the minute tick;)
It's just, who does that... I'm probably not the only one it would happen to?
Not that I have a solution I just wanted you to know ;-)

edit: I saw your edit, makes sense yes!
edit: also tried the test, the result is clear.

Last edited by qinohe (2021-02-01 18:50:22)

Offline

#57 2021-02-01 19:15:44

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,282

Re: Weaver: a socket controlled web browser

I just pushed a working PKGBUILD that will avoid this issue and work around this bug in makepkg.

Last edited by Trilby (2021-02-01 19:16:28)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#58 2021-02-01 19:57:29

qinohe
Member
From: Netherlands
Registered: 2012-06-20
Posts: 1,276

Re: Weaver: a socket controlled web browser

Yes, very nice, 'pkgver=d20210201.1209'  is translated to 'LICENCE' date in PKGBUILD, cool.. problem gone, thanks;)

Offline

#59 2021-02-01 20:23:11

ugjka
Member
From: Latvia
Registered: 2014-04-01
Posts: 1,560
Website

Re: Weaver: a socket controlled web browser

Thanks Trilby, I use this to look up documentation on my second laptop via ssh


Fix yo shit: journalctl -b -p warning
Github

Offline

#60 2021-02-01 21:07:24

qinohe
Member
From: Netherlands
Registered: 2012-06-20
Posts: 1,276

Re: Weaver: a socket controlled web browser

Could be I'm misunderstanding something, It's possible to have multiple user scripts?
If I have 2 different script in the weaver dir. '.config/weaver/scripts'.
For testing I used the examples given and I don't get default zoom on every page only 'bbs.archlinux.org'
It which works on every page if I remove 'bbs.archlinux-css.js' and only use the 'defaultZoom.js'
The 2 scripts:
defaultZoom.js:

// ==UserScript==
// ==/UserScript==

document.body.style.zoom = "120%"

bbs.archlinux-css.js:

// ==UserScript==
// @include https://bbs.archlinux.org/*
// ==/UserScript==

css = document.createElement('style')
document.head.appendChild(css);
css.innerText = `
.pun {
        max-width: 95rem !important;
        font-size: 1rem !important;
}
`;

Last edited by qinohe (2021-02-01 21:14:08)

Offline

#61 2021-02-01 22:12:08

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,282

Re: Weaver: a socket controlled web browser

Thanks ugjka, I'm glad it's useful.

Qinohe, that's odd.  You can have countless scripts.  I just tested with the scripts you posted and even used the same filenames in case order of reading impacted them (which it shouldn't as they don't have conflicting code) - but the defaultZoom.js still applied to other pages for me.  Note that all the scripts are loaded into the profile when the weaver server is started - so changing the on-disk files under ~/.config/weaver/scripts/*.js will not have any impact until the server restarts.

Can you check the stdout/stderr from the server process and see if any js errors were reported?

Last edited by Trilby (2021-02-01 22:12:22)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#62 2021-02-01 22:28:45

qinohe
Member
From: Netherlands
Registered: 2012-06-20
Posts: 1,276

Re: Weaver: a socket controlled web browser

Trilby wrote:

Qinohe, that's odd.  You can have countless scripts.  I just tested with the scripts you posted and even used the same filenames in case order of reading impacted them (which it shouldn't as they don't have conflicting code) - but the defaultZoom.js still applied to other pages for me.  Note that all the scripts are loaded into the profile when the weaver server is started - so changing the on-disk files under ~/.config/weaver/scripts/*.js will not have any impact until the server restarts.

Yes, I already thought nothing is wrong with the scripts.
I also understand I need to restart 'socket server' so I did that on every change I made'

Can you check the stdout/stderr from the server process and see if any js errors were reported?

No errors it just runs with the following output(which is fine):

$weaver 2>&1
Listening on socket "/run/user/1000/weaver"

I do use firejail/apparmor but there seems to be nothing in the way of any of the tools weaver uses.

edit: changed weaver command(nothing changed to output..)

edit2: To rule out any possibility I changed to default kernel & turned off apparmor/firejail.
The result is the same I only get default zoom if 'bbs.archlinux-css.js' is disabled.

Last edited by qinohe (2021-02-01 23:23:33)

Offline

#63 2021-02-01 23:42:35

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,282

Re: Weaver: a socket controlled web browser

Can you add these two lines, recompile, then test again (running the newly compiled version):

Index: src/webengine.cpp
==================================================================
--- src/webengine.cpp
+++ src/webengine.cpp
@@ -224,11 +224,13 @@
 		QStringList("*.js"),
 		QDir::Readable | QDir::Files);
 	QWebEngineScript script;
+int i = 0;
 	while (scriptFiles.hasNext()) {
 		QFile scriptFile(scriptFiles.next());
 		if (!scriptFile.open(QIODevice::ReadOnly | QIODevice::Text))
 			continue;
 		script.setName(scriptFile.fileName());
+script.setWorldId(++i);
 		script.setSourceCode(QString(scriptFile.readAll()));
 		script.setInjectionPoint(QWebEngineScript::DocumentReady);
 		script.setRunsOnSubFrames(true);

Also, an even easier test - please put some other js in the defaultZoom.js file that will be obvious - even a "alert('hello');" would do so ensure that script is running on page load.

Last edited by Trilby (2021-02-01 23:55:50)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#64 2021-02-02 00:20:49

qinohe
Member
From: Netherlands
Registered: 2012-06-20
Posts: 1,276

Re: Weaver: a socket controlled web browser

Trilby wrote:

Also, an even easier test - please put some other js in the defaultZoom.js file that will be obvious - even a "alert('hello');" would do so ensure that script is running on page load.

Yes, I get a popup but only after I disabled 'bbs.archlinux-css.js', otherwise no popup.
There may be unwanted symbols in the script, I copied it from #1...?

Offline

#65 2021-02-02 00:27:35

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,282

Re: Weaver: a socket controlled web browser

Did you test the patch?


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#66 2021-02-02 00:40:17

qinohe
Member
From: Netherlands
Registered: 2012-06-20
Posts: 1,276

Re: Weaver: a socket controlled web browser

No, but I figured out why it happened 'bbs.archlinux-css.js' vs 'bbsarchlinuxcss.js'
The later works! You said you tried even with my 'naming' correct?
I wonder what the difference is on my system but the score '-' seems to be illegal.
Of course I wish to figure out why for I haven't got a clue...

I didn't try the patch, yet, I may try that later too, thanks;)

edit:I did patch 'webengine.cpp'.
It had no influence on the naming of the script, scores still seem to still be illegal in '*.js" in the weaver dir.
Though, any symbol '-_.' (score, underscore, dot) in the name seem to be illegal here!

Last edited by qinohe (2021-02-02 01:16:04)

Offline

#67 2021-02-02 01:33:54

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,282

Re: Weaver: a socket controlled web browser

Yup, I had the same file names as you and couldn't replicate the issue.  I'm not sure how the filename of the archlinux file could be relevant as that one always worked, right - it was the other one that was failing.

FYI, the patch was for diagnostic purposes and should not be used otherwise.  Using that patch would make js-run unable to call any functions in userscripts as the purpose of the patch was to put every chunk of JS in a seperate "world".

Last edited by Trilby (2021-02-02 01:34:45)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#68 2021-02-02 01:39:44

qinohe
Member
From: Netherlands
Registered: 2012-06-20
Posts: 1,276

Re: Weaver: a socket controlled web browser

Thanks, the issue must be on my side than, however, dmenu scripts which are in '.config/dmenu/d-script' seem to have no problems with underscores '_' though, they aren't '.js'...
I will revert that patch ;-)

Offline

#69 2021-02-02 04:47:01

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,235

Re: Weaver: a socket controlled web browser

When I run

weaver js-run 'simplyread()'

I am getting

js: Uncaught ReferenceError: simplyread is not defined

I have posted the script I am using HERE, which is placed as .config/weaver/scipts/ReadingMode.js

I am sure I am doing something wrong, but I don't know it yet.

EDIT: "weaver reload skip-cache" doesn't seem to work. "weaver reload" works as expected.

EDIT2: REM usage keeps increasing when weaver server is on, ultimately forcing me to kill and restart the server.

Last edited by Docbroke (2021-02-02 13:39:32)


Arch is home!
cwm rofi weaver vifm vim lizzy pass terminator

Offline

#70 2021-02-02 15:53:50

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,282

Re: Weaver: a socket controlled web browser

Thanks for the reports.  I just pushed a fix for the scripts - and I suspect this fix will patch up the previously-observed oddities.  It seems I misunderstood where the class objects for scripts were really supposed to be maintained - this is a recurring issue I'm hitting as a C programer using C++ where abstract objects that are sometimes pointers but sometimes actual data are passed around.  The documentation never clarifies when a function will maintain a reference to the passed object and when the caller needs to.

The memory leak is almost certainly due to similar circumstances - though I'm a bit surprised.  I err very far on the side of removing references and deleting objects.  It'll take a bit to track down where the memory leak is.

On "reload skip-cache", I'm not yet able to replicate a problem.  Can you describe specifically how / when you use it and how you determine that the cache was not skipped?


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#71 2021-02-02 15:59:25

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,235

Re: Weaver: a socket controlled web browser

When I run "weaver reload skip-cache", nothing happens, while after running "weaver reload" page is refreshed.

EDIT: Javascript function works after updating to the latest version. Thanks so much.

Last edited by Docbroke (2021-02-02 16:16:09)


Arch is home!
cwm rofi weaver vifm vim lizzy pass terminator

Offline

#72 2021-02-02 16:33:10

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,282

Re: Weaver: a socket controlled web browser

Ah, got it!  Skip-cache should work now.  I misunderstood a webengine function "pageAction(...)" which doesn't actualy run the page action, but just gives a pointer to it - I needed to use "pageAction(...)->trigger()" to actually run it.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#73 2021-02-02 16:37:39

qinohe
Member
From: Netherlands
Registered: 2012-06-20
Posts: 1,276

Re: Weaver: a socket controlled web browser

Trilby wrote:

Thanks for the reports.  I just pushed a fix for the scripts - and I suspect this fix will patch up the previously-observed oddities.  It seems I misunderstood where the class objects for scripts were really supposed to be maintained - this is a recurring issue I'm hitting as a C programer using C++ where abstract objects that are sometimes pointers but sometimes actual data are passed around.  The documentation never clarifies when a function will maintain a reference to the passed object and when the caller needs to.

Welcome, yes it fixes the '-_.' issue for me and forgiven ;-), I did start to doubt the validity of my own env. though, even tried a clean (new user) bash env - I'm a ZSH user - but the results were the same.
Glad you fixed it. It may be nice for you if a 'good' programmer can help you fix the oddities you're unable to find/see - I trust weaver though and I think you're a good programmer if you can even make this!!
BTW. you don't want to be the maintainer of 'your' PKGBUILD anymore?;)

Offline

#74 2021-02-02 16:40:24

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 25,282

Re: Weaver: a socket controlled web browser

qinohe wrote:

BTW. you don't want to be the maintainer of 'your' PKGBUILD anymore?;)

Nope.  The PKGBUILD works, but apparently is not appropriate for the AUR.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#75 2021-02-02 16:44:37

qinohe
Member
From: Netherlands
Registered: 2012-06-20
Posts: 1,276

Re: Weaver: a socket controlled web browser

Trilby wrote:
qinohe wrote:

BTW. you don't want to be the maintainer of 'your' PKGBUILD anymore?;)

Nope.  The PKGBUILD works, but apparently is not appropriate for the AUR.

??Why, what's wrong with it? namcap is fine with it...
edit: Or do you mean it is not ready to publish, yet?

Last edited by qinohe (2021-02-02 16:53:56)

Offline

Board footer

Powered by FluxBB