The ugliest part of this code is the DTD parsing, because you want
(say) SYSTEM returned in some places as a keyword, and in others as a
name. To achieve this, the yacc layer has to be constantly setting
the lexer mode ("lexical tie-ins"). Contrast this with C (surprise!)
where you can't have a variable with the same name as a keyword.
As Henry said, the performance is one reason why we switched to a plain
C parser. Another is the question of 16-bit characters, though this
could probably have been kludged since all the syntactically important
characters are < 128.
-- Richard