Those who write scientific papers, especially if they contain lots of formulas like papers in mathematics or physics, generally use a typesetting system called LaTeX. LaTeX is not a WYSIWYG editor like, for example, MS Word: it is in fact a programming language (or rather, a “markup” language), so the text is written in a source file according to the rules of the language, and the formatted file (usually a PDF) is generated at a later stage through a “compilation” step, much as one does for an executable computer program.
As in source files written in a programming language such as C++ or Python, a LaTeX “source” file can contain, besides text and various commands, comment lines, which the compiler ignores and which will not end up in the final PDF. In LaTeX, comment lines start with the percent sign % (a bit like // in C++ and # in Python), and they are useful for removing (ideally temporarily) a piece of text that one is not sure one wants to appear in the final document. The abuse of comments in LaTeX, however, means that they sometimes become the junk room where cut paragraphs end up, numbers that were not entirely convincing, notes to co-authors, or perhaps earlier versions of a table whose numbers have evolved over time.
Physicists first of all (but by now many other “hard” scientific disciplines as well), when they submit a paper to a peer-reviewed journal for consideration for publication, have got into the habit of uploading the submitted (and therefore preliminary) version to a pre-publication (pre-print) server called arXiv, launched in 1991 and by now the reference point for scientific publications. Long before the advent of Open Access publishing, it was in fact already possible to get hold of a version of a paper for free, without paying for a journal subscription, or even just waiting for the peer review process to conclude. It is such a standard practice, at least in the world of physics, that even once a paper has been published in a journal we tend to replace the preliminary version on arXiv with the final one, and arXiv remains the primary reference for quickly finding the publications we need.
What many forget is that, when a paper is uploaded to arXiv, one normally uploads, besides the PDF, the LaTeX source file as well, together with a separate file for every plot or figure in the paper. It is not mandatory, but it is in fact standard practice, at least in physics. And when the LaTeX source is uploaded, unless it has been cleaned up beforehand, the secret and hidden comments are uploaded too, which nobody sees when reading the PDF, but which are in principle there, available to anyone who cares to go and look for them.
Let’s be clear, almost nobody does, because reading the uncompiled LaTeX source of a paper, especially one written by others, is not exactly a fun pastime. But a large language model, which reads everything put in front of it without getting tired, has no such qualms. This is the second episode of the series of posts about my adventures with my LLM wikis. If in the first one I described how the problem was a model adding to the sources things that were not there, here the story begins, at least in appearance, the other way round.
From the PDF to the source
The second LLM wiki I have had fun with during 2026 (and also the first one I started working and experimenting on) is related to my work, and collects papers and notes on FCC, CERN’s Future Circular Collider. I started with the idea of building myself a “second brain” on Higgs boson physics at FCC-ee, together with the development of the future calorimeter of the ALLEGRO detector and with machine learning methods for optimising detectors, with a few forays into the documents produced for the Débat Public on FCC (the French public consultation) that took place this summer, and perhaps also doing a few experiments with RAG. Today my FCC LLM wiki has about 180 pages, built from about a hundred sources, and it has already been quite useful more than once for quickly finding a piece of information or making a connection that is not necessarily trivial.
I started building this LLM wiki by feeding the agent the papers on FCC physics or detectors in PDF format, the very same ones I was reading myself. It worked rather well, but on a couple of occasions I noticed that digesting formulas extracted directly from the PDFs was not always straightforward, and required the agent to first generate a graphical version to be interpreted afterwards, where wrong Greek letters, lost superscripts and mangled cross sections were more frequent than I liked. Why go through all this (costly, in terms of tokens) effort, I told myself, if the formulas are also written out in plain text in the LaTeX sources of the same papers? So I wrote a script that, given an arXiv identifier, also downloads the LaTeX source, and I asked the agent to reread all the papers starting from there, obtaining exact equations, tables transcribed word for word, and a text that costs three to five times less to read. The agent, like all stubborn and obliging machines, ground away for a while, and then came back to me with a discovery I had not considered: along with the equations and tables, many of the sources also contained comments. The agent, to whom I had given the task of reading everything, started telling me what they contained.
What’s in the comments
In most cases they were disabled LaTeX packages or purely decorative lines. There were, however, a certain number of papers in which the comments contained something decidedly more interesting, which I sorted into four categories:
- Hidden numbers (for more or less good reasons): correlations computed and not published between the measurements of an analysis, or the absolute corrections from which the differences reported in the text had been derived. In a review, even an entire table comparing methods, hidden because the authors considered the experimental conditions too different to compare them openly.
- Analysis choices: the complete list of the variables of a classification algorithm, longer than the one in the table eventually published; or an alternative result, discarded, that explained why another route had been chosen.
- Revised assumptions: in a paper comparing various options for future accelerators, the original durations of the data-taking periods, later rescaled for the final result, with a comment showing that the correction came at a later stage, during the review of the result.
- Chit-chat between colleagues: notes to a co-author asking them to check a definition, removed affiliations, names that disappeared from the author list.
Some of this “hidden” information complements the paper, some nuances it or even contradicts it, with a different number or with an assumption that the text does not state. This in itself is not so extraordinary, since the genesis of a scientific result is often more tortuous than the way it is described in the final publication. What is more intriguing, though, is that information that would by its nature remain within the circle of the authors is instead public and downloadable by anyone, even if in all likelihood almost nobody has ever done so.
The note that wasn’t hidden

To believe the LLM wiki, the most spectacular discovery would have concerned the most famous paper in the collection: Attention Is All You Need, the one that in 2017 introduced the Transformer, the architecture on which the language models are based that underpin the algorithms driving the AI revolution. According to the wiki page, the source of that paper contained a footnote with the contributions of the individual authors, from who had proposed replacing recurrent networks with attention to who had invented multi-head attention, and that note had supposedly been cut before publication, so that it never entered the official history and survived only in a comment. A nice story, but a false one: that line of the source does not start with the percent sign and is therefore not a comment, but an ordinary footnote, printed on the first page of the published paper, where anyone has been able to read it for nine years; on top of that, the quotation reported on the wiki page skipped some words without saying so. The model, perhaps carried away by enthusiasm, had in short found a true piece of information, believed it hidden when it was not, put it in the wrong place and built on it a narrative of a secret and of a story never told, making exactly the same kind of error as in the first episode, only in a more seductive form.

What to do with these open secrets?
Stunned by the experience, I rushed off to get reassurance from the chair of the ATLAS Publication Committee that we have a system for “cleaning” our papers of comments before uploading them to arXiv (and yes, we do). I then found myself mulling it over: what status does a comment have, which is not the result of the paper, because the authors chose not to publish it, but is not an error either, and is public all the same? A wiki that treats it on a par with the text thus ends up stating things that the authors decided not to state.
There is also an important difference between scientific content and people: an unpublished correlation or an extra variable are useful information, also for those working in the same field, while a name removed from the author list or a note to a colleague are quite another matter, because they tell of things between people, often colleagues I know, that have little to do with science.
The rule I chose for the wiki separates the two: the scientific content of a comment goes in, with the exact reference to the file and line and after checking that it is not already in the PDF, while changes to the author list or to affiliations, names and initials in internal notes, remarks about colleagues, referees or other groups’ work, and personal dedications all stay out.
Two wikis, one lesson
If in the first episode I described a model that added to the sources things that were not there, here I ran into one that found in the sources things that really were there and that no human reader would have seen; they are two faces of the same ability, that of reading everything without getting tired and telling it well, and the trouble is that the story always sounds convincing, whether it is true or not. The solution, in both cases, is similar: every statement must say where it comes from, and that “where from” must be verifiable by a script. For quotations from books it is enough to compare the text, while for LaTeX comments one can check that the cited line exists, that it really starts with the percent sign, and that the text does not appear in the published PDF.
When I had all the wiki’s statements about comments checked in this way, the result was starker than I expected: of the 260 “revelations” collected across 17 pages, only one in three survived intact, while the others contained wrong facts, inflated or invented interpretations, altered quotations or wrong counts. The most frequent error was always the same, similar to the story of the founding Transformer paper: a commented-out copy of material that was in fact published elsewhere, in the text, in a figure or in an appendix, presented as something “cut” or “hidden”, often with a reason invented by the model to explain the cut. In one case the model had attributed a paper to the wrong person just because they shared a surname with one of the authors, and in another it had taken the bibliographic key of a public citation, which carried a person’s name, for a private communication.
It seems to me, then, that the rule emerging from the two episodes is the same, perhaps more pronounced: in the books wiki the quotations were right nine times out of ten and the syntheses two times out of three, while here the interpretations of the comments, which are the point furthest from the published source and at the same time the most seductive to tell, held up one time in three. The script that checks that a line really is a comment is useful, but not enough, because it can confirm that the line is commented out without being able to say whether the story built on top of it is true. And for that, once again, someone has to reread.


Lascia un commento