The Reading Desk: extraction that follows its own references

A structured-extraction engine that reads several documents at once, follows the references it finds between them, and revises its own draft — cross-referencing sources the way a person does.

AAIC built the engine; the interface in these recordings is a technical demonstrator of it, not a production UI. The material shown is one curriculum corpus (OpenSciEd); the engine is domain-agnostic and runs over other document sets.

Fig. 1Reading three documents at once. The primary pass reads a teacher edition, a student procedure, and a slide deck together, and drafts a single activity — title, tags, and a rough set of steps. One step is left flagged: it names a handout the pass has not read.

The problem

Real documents rarely hold a whole thing in one place. A set of related files describes it between them, each pointing at the others by name rather than by location — a procedure that says “see the handout,” a form that assumes a reference sheet is already open. A straight-line extractor that reads one file top to bottom comes back thin, because the file it is reading defers to one it never opens.

The corpus shown here is OpenSciEd, an open-source science curriculum, and the thing extracted is a single classroom activity — described across a teacher edition, a student procedure, a handout, and a reference sheet. The material is OCR from scans: broken tables, detached headings, cross-references given as approximate titles. To recover one activity, the extractor has to read the whole set and follow the trail between the files.

Reading several sources together

The primary pass reads the main documents for one lesson at once — teacher edition, student procedure, slides — and drafts each activity in a single reading: a title, tags, the objects it needs, and its steps, every value carrying a confidence level. Where a step depends on a document the pass was not given, it is not guessed at; it is recorded as an unresolved reference for a later stage to satisfy.

Following the reference

This is the behaviour the engine is built around. A drafted step names “the Cushioning Materials Testing Procedures handout” — a title, not a file. The resolver tokenizes that phrase and every file in the unit, scores the overlap, and prefers the same lesson; a match above a threshold is read, anything below it rejected rather than forced. The recovered procedure — set up the ramp, weigh the carts, release, record, repeat — then replaces the vague step the draft carried. The extractor went back and got what it was missing, and references lead to further references: a second lookup pulls the collision-cart setup from a reference sheet the draft never named.

Fig. 2The lookup, opened up. The unresolved reference is matched to a file by token overlap — the shared tokens highlighted, the score shown against the acceptance threshold — the file is read, and the recovered steps replace the vague one the draft had.

Cleaning up what was recovered

Recovered text is written for people, not systems. A step reading “run this test 5 more times” is unrolled into explicit repeated steps, marked non-mandatory so the count stays truthful. Loose object names are matched against a kit catalog — “adhesive putty (sticky tack)” resolves to Adhesive Putty, while a brick and some textbooks match nothing and are left uncatalogued rather than invented.

Fig. 3Mapping and fanning out. Raw object names are matched to canonical kit products, with unmatched items left plainly uncatalogued; the single recovered procedure then fans out into the ten material variants the teacher edition listed.

Provenance

Because the activity is assembled from many documents rather than copied from one, where each value came from is not obvious — so the engine keeps the answer. Every field traces back to the source line that produced it, including the ones no single document held until the extractor went and found them. A reviewer can see not just what was extracted but why, and can tell a value read directly from one recovered through a lookup.

Fig. 4Tracing a field to its source. Selecting an extracted value draws a line back to the exact source line it came from — here, a tag traced to the student procedure, and a value traced to a line only the lookup recovered.

The demonstrator replays one real extraction, stage by stage, so the process can be watched rather than inferred; the engine, its prompts, and its output are unchanged by the view. What distinguishes it is the shape of the whole: an extractor that treats a document’s references as work to be done, follows them, and revises what it has already written.