Download as a PDF

BUILDING LANGUAGE IMPLEMENTATIONS
913
see Figure 2 ). This may be true at least for the first pass of the language implementation, if it is composed of several successive passes.
The scanner
This takes the source text as input, and transforms it into a sequence of tokens,
while omitting some information, such as comments. The lexical part of programming
languages may be described using regular expressions. Practical scanner generators
2
exist; one of the most widely-known scanner generators is Lex. However, it is often
more expeditious to program the scanner by hand, since its work is not very
complicated, is very similar from language to language, and the interface between
scanning and parsing is more easily realized.
The parser
This checks the syntax of the source text, and builds or enumerates some tree
representation of its syntactic structure. Parser generators have been the subject of
intensive studies for a long time, and the result is currently satisfactory. The syntax
of the 3source language
is specified
using a context-free grammar, for example in
4
5
Yacc, LLGen or Syntax. On the other hand, it is very cumbersome, or even
impossible, to build ‘a parser manually, except when using the recursive-descent
6
method.
Figure 2. A syntax-directed compiler