3.21 Scanner/Parser Interface

If the scanner terminates without any errors, control is passed to the parser. The parser parses the token list generated by the scanner. The token list consists of text scraps, freestanding typesetter directives, and special sequence tokens.

The user should bear in mind that the scanner finishes running before the parser starts running.  This means that the scanner cannot be influenced in any way by higher order structures such as the parser might parse. For example, it is impossible to write a FunnelWeb macro to include a file, or to write a macro that inserts a vskip pragma into the input text.