[Solved] growing nodeset hit limit error during build

d_fajardo · 2021-07-01 15:16:53

I am trying to update dict-freedict-deu-eng package. I have a previous working version.
During this new build however, I get flagged this error:

mkdir -p build/tei/
/usr/lib/freedict-tools/teiaddphonetics -o build/tei/deu-eng-phonetics.tei deu-eng.tei
Platform dictd supports this dictionary module.
xsltproc --novalid --xinclude --stringparam dictname deu-eng --path /home/brightchip/.cache/yay/dict-freedict-deu-eng/src/deu-eng/ /usr/lib/freedict-tools/xsl/tei2c5.xsl build/tei/deu-eng-phonetics.tei >build/dictd/deu-eng.c5
XPath error : Memory allocation failed : growing nodeset hit limit

growing nodeset hit limit

The build continues but very, very slow.
I've never encountered this on previous build of the same package so I don't know what it is. Internet searches has not given me any satisfactory answer.
Does anyone have a fix?

Last edited by d_fajardo (2021-07-02 06:37:57)

seth · 2021-07-01 15:42:35

https://stackoverflow.com/questions/401 … 8#40176398

d_fajardo · 2021-07-01 15:50:15

Or you could change your XPath expressions so that they never encounter nodesets larger than 10M nodes.

How does one do that? And where?

seth · 2021-07-01 15:56:54

In the files you compile.
The problem is (probably) /usr/lib/freedict-tools/xsl/tei2c5.xsl - idk about its nature, but it could be corrupted (you may want to ask upstream whether this node depth/width is expected)

d_fajardo · 2021-07-01 16:28:56

but it could be corrupted (you may want to ask upstream whether this node depth/width is expected)

Good point
I'll have a look at the files.

d_fajardo · 2021-07-01 18:24:11

The setup is actually simple. There's a dictionary called deu-eng.tei in the source and then tei2c5.xsl converts tei to c5, the format for dictd or related.
The dictionary however is 400+ MB. The conversion takes ages. From PKGBUILD:

build()
{
	local _freedict_tools_dir=/usr/lib/freedict-tools
	local _ding2tei_dir="${_freedict_tools_dir}/importers/ding2tei"

	"$_ding2tei_dir"/preprocess/de-en/all.sh \
		< "ding-${_dingver}/de-en.txt" \
		> de-en.txt.preprocessed

	"${_ding2tei_dir}/ding2tei" de-en.txt.preprocessed "${_lang}/${_lang}.tei"

	cd "$_lang"
	make FREEDICT_TOOLS="$_freedict_tools_dir" build-dictd
}

It seems ding needs to be called first to convert to tei THEN to convert to c5.
Am I understanding this right?

seth · 2021-07-01 19:21:56

I have no idea, sorry.
Is the build() script actually the content of the xsl???
https://en.wikipedia.org/wiki/XSLT

d_fajardo · 2021-07-02 06:09:19

Is the build() script actually the content of the xsl???

No. It is in the PKGBUILD. The xsl file is like a formatting xml file.
I contacted the maintainer and he says it is normal behaviour and that it takes a day with 4GB of RAM to build the dictionary and suggested to use the bin version instead which I did.
I'll mark this as solved as there's nothing to do now.

Arch Linux

#1 2021-07-01 15:16:53

[Solved] growing nodeset hit limit error during build

#2 2021-07-01 15:42:35

Re: [Solved] growing nodeset hit limit error during build

#3 2021-07-01 15:50:15

Re: [Solved] growing nodeset hit limit error during build

#4 2021-07-01 15:56:54

Re: [Solved] growing nodeset hit limit error during build

#5 2021-07-01 16:28:56

Re: [Solved] growing nodeset hit limit error during build

#6 2021-07-01 18:24:11

Re: [Solved] growing nodeset hit limit error during build

#7 2021-07-01 19:21:56

Re: [Solved] growing nodeset hit limit error during build

#8 2021-07-02 06:09:19

Re: [Solved] growing nodeset hit limit error during build

Board footer