How do TeX's four processors function in tandem with one another?

  • LaTeX
  • Thread starter Eclair_de_XII
  • Start date
  • Tags
    Function
In summary, the TeX processor first transforms the code into a token list, which is then processed by the execution and input processors. The input processor may run multiple times, affecting the catcodes of characters and potentially causing confusion. The concept of simultaneous processing may be difficult to understand, but it is important to grasp in order to fully understand TeX.
  • #1
Eclair_de_XII
1,083
91
TL;DR Summary
I copied a piece of TeX code from TeX by Topic, and I don't quite know how the following code is interpreted. I am informed that the eyes, mouth, gullet, and intestines of TeX do not run one after another, but rather, in tandem. But this is rather confusing to me, and it is too vague an explanation for me to understand the following code.
Code:
{\catcode`\^^M=12 \endlinechar=`\^^J \catcode`\^^J=5 This is an M:
This is a middle line.
This is a "J":
}

As I see it, the TeX processor would first need to feed this to the input processor to be transformed into a token list.
The input processor would see the code I posted above with their default cat-codes and transform it into a list of (char-code,cat-code) pairs.

\begin{speculation}
There's nothing in the token list that is going to be expanded, I think, so the expansion processor does nothing.
Then the token list is fed into the execution processor, and the assignments on line 1 are made.

I assume then, that the input processor is run again after the execution processor, but that just makes things even more confusing, if it were true. For example:

- If the assignments on the first line have been made, then why is the eol-character on this line still ^^M?
- Why is the eol-character on the second and third lines ^^J with cat-code 5 while the terminator character on the first is ^^M?
- Why does it seem that the eol-character is ^^J outside the group, and moreover, why does it not have cat-code 5? Isn't grouping supposed to isolate the assignment from the rest of the document outside the group?
- Consider the following:

Code:
{\catcode`\^^M=12 %
\endlinechar=`\^^J %
\catcode`\^^J=5 This is an M:
This is a middle line.
This is a "J":
}

If separated each assignment statement by a comment-character followed by a line break, the terminator character does not seem to be a catcode-12 ^^M after the phrase, ``This is an M:''. This is does not seem to be in-line with the fact that the three resulting lines are read as though they are all on one line.
- Overall, this piece of code is a total anomaly to me and makes me question all that I think I know about TeX.
\end{speculation}

I know this is far from the earlier remark that the input, expansion, execution, and visual processors work simultaneously, and actually affect one another when TeX is run. Yet, I cannot seem to grasp how they would work at the same exact time. Mind you, TeX by Topic claims to be an advanced book on TeX, to be read after an introductory text, and would thus expect its reader to know this sort of thing. It might be my incompetence as an inattentive reader, but all the same, the books I've skimmed do not seem to go too in-depth on the interdependence of these processors.
 

Attachments

  • Screenshot_2021-07-28_15-38-42.png
    Screenshot_2021-07-28_15-38-42.png
    8.1 KB · Views: 153
Physics news on Phys.org
  • #2
If someone could explain how the input, expansion, execution, and visual processors work together during a TeX run, it would be greatly appreciated.
 

FAQ: How do TeX's four processors function in tandem with one another?

What are the four processors in TeX and what is their function?

The four processors in TeX are the input processor, the expansion processor, the execution processor, and the output processor. The input processor reads and interprets the input file, the expansion processor expands macros and other commands, the execution processor executes the expanded commands, and the output processor formats the output document.

How do the four processors work together in TeX?

The four processors in TeX work together in a sequential manner. Once the input processor finishes reading the input file, it passes the processed data to the expansion processor. The expansion processor then passes the expanded data to the execution processor, which executes the commands. Finally, the output processor formats the output document based on the executed commands.

What is the role of the expansion processor in TeX?

The expansion processor in TeX is responsible for expanding macros and other commands. This allows for the creation of customized commands and simplifies the input file by replacing long sequences of code with shorter, more readable commands.

How does the execution processor in TeX handle errors?

The execution processor in TeX has a built-in error handling mechanism. If an error occurs during the execution of a command, the execution processor will stop and display an error message. The user can then make corrections and resume the execution process.

Can the four processors in TeX be modified or customized?

Yes, the four processors in TeX can be modified or customized through the use of packages or macros. Users can also define their own commands and macros to be used in the input file, which will then be processed by the expansion processor.

Back
Top