[SOLVED] XMLStarlet problems after update

7thSon · 2017-07-24 22:15:34

I've had xmlstarlet set up for use in some weather scripts for conky, running for quite a number of weeks without issue.
However the past week it has broken due to an update, as the exact same commands that worked previously now give me an error.

My xmlstarlet commands all look something like this, and used to work;

xmlstarlet sel -t -v "//time[@to='$NOW'][@period='$PERIOD']/symbol/@name" http://www.yr.no/place/Sweden/Norrbotten/Lule%C3%A5/forecast.xml

But now I get the output in terminal;

Attempt to load network entity http://www.yr.no/place/Sweden/Norrbotten/Lule%C3%A5/forecast.xml

I've searched for this error, and found some suggestions about defining namespace in the command, but it didn't work.
Hoping someone else here also uses xmlstarlet and might know what this is about and how to fix it?

Last edited by 7thSon (2017-08-18 05:56:43)

seth · 2017-07-24 23:13:33

Does

curl -s 'http://www.yr.no/place/Sweden/Norrbotten/Lule%C3%A5/forecast.xml' | xmlstarlet sel -t -v "//time[@to='$NOW'][@period='$PERIOD']/symbol/@name"

work?

progandy · 2017-07-24 23:13:57

You'll have to allow net access for xmlstarlet:

xmlstarlet sel --net ...

Edit: Or use an external command like curl

Last edited by progandy (2017-07-24 23:15:01)

7thSon · 2017-07-25 00:03:52

progandy wrote:

You'll have to allow net access for xmlstarlet:
xmlstarlet sel --net ...
Edit: Or use an external command like curl

Adding "--net" worked Thanks!
Was this something that was recently added to xmlstarlet, since it worked before without it?

seth · 2017-07-25 07:08:01

The option is old and xmlstarlet had its last update in 2015.
The message you get from libxml, so there'll have been a behavioral change there.

There've been some memory fixes, the redirection of a parser call and "Prevent unwanted external entity reference", see https://git.gnome.org/browse/libxml2/log/?ofs=50
This could account for a bug, because it's really only about the input file (tried curl'ing) and not about loading entities linked inside the file, ie. the input is now accidentally covered by this protection.

I can however not recall operating on remote files before (I tend to fetch them into tmp files to operate on them there multiple times - and because i've more faith in curl or wget accessing the interwebs than in less used, tested and updated tools)
So you'd downgrade your latest libxml2 update, re-check behavior and in doubt feel free to file a bug.

progandy · 2017-07-25 13:21:20

Xmlstarlet directly calls libxml with the commandline argument and lets it handle opening the file regardless of its location. Specifically, xmlReaderForFile is used. It is by default called with the parameter called "XML_PARSE_NONET" which is described as "Forbid network access". It feels like the new behaviour is the correct choice.

https://sourceforge.net/p/xmlstar/code/ … ect.c#l708
http://xmlsoft.org/html/libxml-xmlreade … derForFile
http://xmlsoft.org/html/libxml-parser.h … rserOption

Last edited by progandy (2017-07-25 13:27:23)

Arch Linux

#1 2017-07-24 22:15:34

[SOLVED] XMLStarlet problems after update

#2 2017-07-24 23:13:33

Re: [SOLVED] XMLStarlet problems after update

#3 2017-07-24 23:13:57

Re: [SOLVED] XMLStarlet problems after update

#4 2017-07-25 00:03:52

Re: [SOLVED] XMLStarlet problems after update

#5 2017-07-25 07:08:01

Re: [SOLVED] XMLStarlet problems after update

#6 2017-07-25 13:21:20

Re: [SOLVED] XMLStarlet problems after update

Board footer