Lex (software)

Open source and Versions
– Some versions of Lex are now open-source.
– Open-source versions of Lex are distributed with open-source operating systems like OpenSolaris and Plan 9 from Bell Labs.
– The popular open-source version of Lex is called flex.
– Flex is not derived from proprietary coding.
– Flex is known as the fast lexical analyzer.

Structure of a Lex file
– Lex files are divided into three sections: definitions, rules, and C code.
– The definitions section defines macros and imports header files written in C.
– C code can be written in the definitions section and will be copied into the generated source file.
– The rules section associates regular expression patterns with C statements.
– When the lexer sees text matching a pattern, it executes the associated C code.

Example of a Lex file
– An example Lex file for the flex version of Lex recognizes strings of numbers and prints them out.
– The file has a definitions section, a rules section, and a C code section.
– The rules section uses regular expressions to match strings of digits.
– The C code section contains the main function that calls the lexer.
– The input given to flex will be converted into a C file called lex.yy.c.

Using Lex with other programming tools
– Lex can be used with parser generators like Yacc or Bison.
– Lex limits rules to those describable by regular expressions.
– Parser generators use a formal grammar to parse an input stream.
– Lex is often used to produce a token-stream for parser generators.
– Scannerless parsing refers to parsing the input character-stream directly, without a distinct lexer.

Lex and make
– Make is a utility used to maintain programs involving Lex.
– Make assumes that a file with the extension ‘.l’ is a Lex source file.
– The make internal macro LFLAGS can be used to specify Lex options for make.
– Lex can be integrated into the make workflow for automatic compilation.
– Make simplifies the build process for Lex programs.