How to specify unicode characters in patterns for fslex

424 views Asked by At

What is the correct way to specify Unicode characters in pattern for FSharp Lexer. Following code is not compiled with the FsLex.exe utility:

let lexeme lexbuf = LexBuffer<char>.LexemeString lexbuf
...
rule tokenize = parse   

| ['a'-'z' 'A'-'Z'] { TOKEN1 }  
| [\u0100\u0101]    { TOKEN2 } 
| [\u0102-\u01FF]   { TOKEN3 }  
...
| [eof]             { EOF }

What I'm doing wrong?

P.S: I'm using fslex.exe with --unicode option

Thanks, Vitaliy

1

There are 1 answers

0
wmeyer On BEST ANSWER

I think you have to put the unicode characters in single quotes, just like in normal F# code.

At least that seems to work for a small example that I tested.