How to fix the growing nodset with xpath, memory allocation failed

Question

I am using xmllint to fetch some data from xml file, by using following command:

xmllint --xpath '//Module[TestCase/Test[@result="fail"]]/@name' test.xml

The same command worked for some time, but triggering it few time, I am getting an error as

XPath error : Memory allocation failed : growing nodeset hit limit

growing nodeset hit limit

^ XPath evaluation failure

Please note: The xmllint --xpath '//Module[TestCase/Test[@result="fail"]]/@name' test.xml command is working fine when I using the test.xml whose size is 1MB, but when the use the same command with test.xml whose size is 390MB then I am facing the issue

I tried troubleshooting by trying to increase the xpath nodeset length XPATH_MAX_NODESET_LENGTH 90000000 and export XPATH_MAX_NODESET_LENGTH 90000000 but it dint helped me.

Answer 1

A file size of 390Mb should be possible these days if you configure enough memory, but it's pushing it.

You could try downloading Saxon and using the XQuery command line interface:

java net.sf.saxon.Query -s:test.xml 
  -qs:"//Module[TestCase/Test[@result='fail']]/@name

If necessary, you could get a license for Saxon-EE and run this in streaming mode, which would definitely solve the memory problems.