AI Turned an Email and PDF Into a Checklist. Did It Keep the Meaning?

The volunteer guide gave the coordinator a deadline: send a weather update by 7 a.m. if heavy rain changes the plan. The volunteer was told to check email before traveling. Gemini turned that into a 7 a.m. deadline for the volunteer to check email.

The action was present. The person attached to the deadline had changed. That is the practical lesson from our Run #5 experiment: a polished checklist can still change what the source means.

Watch Jeremy’s full experiment on YouTube. The episode uses the actual Gemini session, document evidence, and labeled excerpts. This article accompanies that long-form episode.

What we tested—and what we did not

We wrote one fictional community Repair Cafe email thread and a two-page, selectable-text PDF. No real people, private correspondence, connected inbox, scanned-document OCR, or live event were involved. The recipient was a confirmed volunteer working the whole event, requesting vegetarian lunch and borrowing a tool kit, with no accessibility request.

The answer key and inputs were frozen before the test; Gemini did not receive the key. Source quotation checks and calendar checks verified the key separately from the model’s answer. The producer wrote the packet and assessed its meaning. There has been no independent human review of the key. Convenient paragraph labels made source checking easier than it can be in ordinary correspondence.

We tested the Gemini web interface in its Flash mode; the exact backend version was not exposed. An initial stalled session required recovery before we obtained the completed answer. We then asked for an unhinted self-audit in the same conversation. This was one tool and one packet, not an independent replication or a comparison of products. We did not measure typical time savings or establish broad AI accuracy.

Read the updates in their actual order

The October 9 PDF and quoted earlier email came first. The newer October 13 email, labeled E1–E8, replaced only the PDF’s RSVP deadline, lunch deadline, volunteer check-in time, entrance, and tool-kit collection time. All other PDF instructions still applied. The packet was framed as being read on October 14.

A reliable action list therefore needed to resolve the named updates without treating the newest email as a replacement for the entire guide. The newer instructions required a reply ALL by Friday, October 16 at 5 p.m., including confirmation and T-shirt size; a vegetarian lunch request through a form by Monday, October 19 at noon; and arrival at 8:45 a.m. through the WEST entrance on October 24. The 9:15 a.m. safety briefing had not moved. Times in this fictional packet use America/New_York.

What Gemini successfully found

Both completed answers found all 17 required action components and all four missing-information cases. Action components were counted separately from table rows: one row could contain several tasks. Finding every component does not mean every instruction was interpreted correctly.

• It applied the newer RSVP and lunch deadlines and preserved the correct response channels.
• It updated arrival to 8:45 a.m. and the WEST entrance while retaining the unchanged safety briefing.
• It separated reserving a tool kit on October 22 from collecting it on October 23, 4–6 p.m.
• It retained PDF-only requirements: safety acknowledgement, photo ID, closed-toe shoes, a water bottle, no personal power tools, cleanup, kit return, and sign-out.
• It identified the missing lunch-form link, Friday collection location, workstation assignment, and parking/travel instructions.

No required action omission was identified against the frozen key. We did not find an invented collection location or lunch link. Those successes are useful; they are also narrower than a claim that the checklist was fully correct.

The weather instruction: whose deadline is it?

The PDF’s weather paragraph, P2.4, said:

If heavy rain changes the plan, the coordinator will email volunteers by 7:00 a.m. on October 24. Check your email before traveling.

Gemini’s first-pass row 6 asked the volunteer to check email for weather updates and gave the deadline as Saturday, October 24, 2026, by 7:00 a.m., America/New_York. It cited P2.4, the relevant source paragraph, but that paragraph did not assign this deadline to the volunteer.

Coordinator: send an update by 7 a.m. if heavy rain changes the plan. Volunteer: check email before traveling. The source also supplied no alternate venue or automatic cancellation rule. A source reference can point to the right paragraph while still failing to support the checklist’s interpretation.

Sign-out: a planning choice presented as a requirement

P2.2 specified cleanup from 1:00 to 1:30 p.m. and signing out with the coordinator before leaving. P2.1 separately required returning a borrowed tool kit by 1:30 p.m.

Gemini put a 1:30 p.m. deadline on sign-out. Signing out around then is a reasonable plan for a volunteer finishing cleanup. The problem is that the source did not impose that exact sign-out deadline. It required sign-out before leaving. Keep an inferred planning choice separate from a source requirement.

What the self-audit improved—and retained

The self-audit improved presentation and explicitly supplied America/New_York for the collection window. It separated the lunch-link request into its own row. It retained the volunteer’s 7 a.m. weather-check deadline and the unsupported 1:30 p.m. sign-out deadline.

It also called the lunch-link request a missed action. The first answer had already mentioned that request in its lunch row and missing-information list. That was an inaccurate self-critique, not a newly recovered omission. An audit can make a table clearer without fixing its interpretation.

A practical check for your own action list

Ask for an action list with the person responsible, action, exact time, condition, response channel, and source passage. Ask it to flag missing details and distinguish source requirements from suggestions. Keep the original documents beside the result.

• Person: Who has to act? Do not borrow a coordinator’s deadline for a volunteer.
• Action and channel: Is it reply ALL, a direct reply, or a form? Are several components compressed into one row?
• Time and condition: Does the source say by a clock time, before traveling, or before leaving? Does a condition apply to this person’s action?
• Chronology: Which newer instruction replaces which older instruction? Which older instructions still apply?
• Source and gaps: Read the cited sentence yourself. Ask for missing links or locations instead of filling them in.

A self-audit is a useful additional pass, not proof of correctness. Verify important rows against the original wording before acting. Our fictional safety form and example email address are demonstration placeholders; no message was sent, form submitted, or purchase made during this test.

Evidence and attribution

The experiment’s primary sources are the producer-authored Email-Thread.txt (E1–E9), Volunteer-Guide-2026-10-09.pdf (P1/P2), the frozen answer key, and saved first-pass and self-audit responses from October 12, 2026. The packet’s scenario dates are fictional and are separate from the test date. Quotation checks are independent of Gemini’s output, not an independent human review. The full episode shows the document and response evidence; the source files and original recordings remain preserved in our production archive.

Gemini identifies the tool used, not an external validation of our results. For context on checking model output, see our existing explanation of AI prediction. Explore more episodes in the AI Lab Report blog.

Jeremy hosts the episode using his authorized AI presenter and voice. Music in the companion video: “Cylinder Five” by Chris Zabriskie, from Cylinders, CC BY 4.0; artist and album. The track was edited, looped, and mixed under narration.