boost::spirit::lex, how does one generate a end of file token?

Question

The question is rather simple, I've written a lexer, using boost::spirit, however I can't seem to find a way, to generate an EOF token. - So how would one go about doing this?

Answer 1

What is an EOF token?

Historically, some platforms have associated special 'EOF' (ascii 26, e.g.) characters with text files. Like the use of 0x15 as newline character, such uses are now largely defunct. The end of a file is better defined as the absense of further input, in other words: it is a stream state, not a character.

The token iterators Spirit Lex signal 'EOF' by returning the end iterator.

Both the tokenizer API (lex::tokenize(...)) as well as Spirit Qi understand this behaviour (by exiting the tokenizing loop (lex) and/or by making the qi::eoi parser succeed match).

E.g. if you need to assert that parsing reached the end of the input, you'd just say

 myrule = subrule1 >> subrule2 > qi::eoi;

Or if you want to assert the presence of something (say, a closing ;) unless at end of input:

 myrule = subrule1 >> subrule2 >> (qi::eoi | ';');

Did I miss something about the question that isn't addressed like this?