1
Download A Python Book - Dave Kuhlman dot org
Transcript
A Python Book Node Type FuncCallNodeType Node Type TermNodeType Child: bbb Node Type FuncCallListNodeType Node Type FuncCallNodeType Node Type TermNodeType Child: ccc Node Type FuncCallListNodeType Node Type CommandNodeType Node Type FuncCallNodeType Node Type TermNodeType Child: ddd Node Type FuncCallListNodeType Node Type FuncCallNodeType Node Type TermNodeType Child: eee Node Type FuncCallListNodeType Node Type FuncCallNodeType Node Type TermNodeType Child: fff Node Type FuncCallListNodeType Node Type FuncCallNodeType Node Type TermNodeType Child: ggg Node Type FuncCallListNodeType Node Type FuncCallNodeType Node Type TermNodeType Child: hhh Node Type FuncCallListNodeType Node Type FuncCallNodeType Node Type TermNodeType Child: iii Node Type FuncCallListNodeType Comments: ● ● ● ● We can now put comments in our input, and they will be ignored. Comments begin with a "#" and continue to the end of line. See the definition of comment in function genTokens. This tokenizer does not require us to separate tokens with whitespace as did the simple tokenizer in the earlier version of our recursive descent parser. The changes we made over the earlier version were to: 1. Import Plex. 2. Replace the definition of the tokenizer function genTokens. 3. Change the call to genTokens so that the call passes in the file name, which is needed to create the scanner. Our new version of genTokens does the following: 1. Create patterns for scanning. 2. Create a lexicon (an instance of Plex.Lexicon), which uses the patterns. Page 140
