yacc fails to reduce (Python Lex-Yacc)

Question

I'm trying to write a fairly simple grammar with PLY (python implementation of yacc), and am having trouble getting yacc to reduce strings of tokens when I want it to.

I want to interpret a series of commands that take different kinds of arguments. Each different kind of argument has a different token. The string of tokens that comes out of lex might look like this:

COMMAND VARARG VARARG STRARG
COMMAND VARARG STRARG STRARG

I want yacc to reduce each of those lines into a rule called instruction. However, yacc refuses to stop reducing the first line after the last argument (STRARG) and generates a syntax error because of the unexpected COMMAND token.

That is, instead of reducing COMMAND VARARG VARARG STRARG to instruction, yacc shifts one more time to get COMMAND VARARG VARARG STRARG COMMAND (picking up the last COMMAND from the next line, which shouldn't have been read).

The yacc portion of my code looks like this:

def p_rule1(p):
    r'instruction : COMMAND VARARG VARARG STRARG'

    # do stuff

def p_rule2(p):
    r'instruction : COMMAND VARARG STRARG STRARG'

    # do other stuff

Am I making some obvious error in my rule specifications? This is the first time I've used lex/yacc, so I wouldn't be surprised.

Answer 1

You need to include additional rules for handling multiple instructions, so yacc will know what to do with that second COMMAND token. Something like below should work.

instructions : instructions '\n' instruction
             | instruction

instruction : COMMAND VARARG VARARG STRARG
            { do stuff }
            | COMMAND VARARG STRARG STRARG
            {do other stuff }

See http://luv.asn.au/overheads/lex_yacc/yacc.html#recusive