This entire chapter is about the get_next function (defined from 145 to 155, the end), which gets the next token of input.
This function produces the next token in almost the form that ultimately goes into the data structures (tok_mem etc – what we saw in chapters 5 to 8) except that instead of two-byte tokens for identifiers, module names and module numbers, it instead returns a certain special one-byte value and then other procedures (which?) turn them into the actual two-byte tokens.