Pig xmlloader error when loading tag with colon

465 views Asked by At

Ive been using Pig and XMLLOADER to load xml files. I've been practising on BOOK example. However, XML file I need to process has colons in tag. When I run a script it says that due to ':' it cannot be processed.(exact log at the end)

This is the file I have. Modified for the purpose of ":" case. BOOKT.xml

<CATALOG>
<BC:BOOK id="1">
<TITLE>Hadoop Defnitive Guide</TITLE>
<AUTHOR>Tom White</AUTHOR>
<COUNTRY>US</COUNTRY>
<COMPANY>CLOUDERA</COMPANY>
<PRICE>24.90</PRICE>
<YEAR>2012</YEAR>
</BC:BOOK>
<BOOK id="2">
<TITLE>Programming Pig</TITLE>
<AUTHOR>Alan Gates</AUTHOR>
<COUNTRY>USA</COUNTRY>
<COMPANY>Horton Works</COMPANY>
<PRICE>30.90</PRICE>
<YEAR>2013</YEAR>
</BOOK>
</CATALOG>

Now this is the BOOK.pig (note: tried this with regex and Xpath thats why both appear and error is still there)

REGISTER piggybank.jar
DEFINE XPath org.apache.pig.piggybank.evaluation.xml.XPath();

A =  LOAD 'BOOKT' using org.apache.pig.piggybank.storage.XMLLoader('BC:BOOK') as (x:chararray);
dump A; 
--B = foreach A GENERATE FLATTEN(REGEX_EXTRACT_ALL(x,'<BC:BOOK>\\s*<TITLE>(.*)</TITLE>\\s*<AUTHOR>(.*)</AUTHOR>\\s*<COUNTRY>(.*)</COUNTRY>\\s*<COMPANY>(.*)</COMPANY>\\s*<PRICE>(.*)</PRICE>\\s*<YEAR>(.*)</YEAR>\\s*</BC:BOOK>'));
B = FOREACH A GENERATE flatten XPath(x, 'BC:BOOK/AUTHOR'), XPath(x, 'BC:BOOK/PRICE');
describe B;

This is the error:

ERROR org.apache.pig.tools.pigstats.PigStats - ERROR 0:java.lang.RuntimeException: java.lang.RuntimeException: XML tag identifier 'BC:BOOK' does not match the regular expression /[a-zA-Z\_][0-9a-zA-Z\-_]+/

My question is what should i put in XMLLOADE(STRING identifier) so that I can have tags with ":" ( I cannot modify piggybank.jar, i tried putting : as a xml special code,and i tried using XMLLOADER('sth'+'sth')...

1

There are 1 answers

0
eof On

One , not so neat solution, is to load it to pig storage and then to replace ':' with '', and then to load it with XMLLOADER.