Tutorial — How the Mind Works

Augmented Traveler local tutorial. Back to AI Mind — Tutorials · process: AI Mind — Process · ethics: Ground Rules — Historical Virtual Humans · Unreal side: Tutorial — Conversation Client Plugin

A first read for new staff. It explains how a figure such as Maria Mitchell answers a visitor: authored answers first, then retrieval inside a fence of her own sources, then a policy layer written in code, then logging. It also says who owns each part.

Draft. Written from AI Mind — Process and the Ground Rules. The AI conversation engineer and CTO check it against the running service. Correct component names here when they change.

1. The one-sentence version

The mind is not a chatbot pretending to be Maria Mitchell. It is a search engine over her own words. A language model is only allowed to rephrase what was found, in her voice. Rules written in code sit around it.

2. The runtime loop

Visitor speech (push-to-talk)
  │
  ▼
ASR ── transcript ──► intent + safety filter (input rails)
                          │
          ┌───────────────┼──────────────────────────┐
          ▼               ▼                          ▼
   authored match?   named hard topic?          everything else
   play clip (ID)    canned reviewed reply       │
                                                 ▼
                                   retrieval fence: one figure's index
                                   Corpus A → B → C, date ≤ cutoff
                                   hybrid keyword + vector → citation IDs
                                                 │
                                                 ▼
                                   LLM: persona prompt + passages only
                                                 │
                                                 ▼
                                   policy layer (code): cutoff, refusals,
                                   trauma protocol, companionship ban,
                                   B-attribution check (output rails)
                                                 │
          ┌──────────────────────────────────────┘
          ▼
   streaming TTS (locked actor voice) → audio-driven face + gesture tag
          │
          ▼
   captions + log (question, answer ID, citation IDs, refusal flag; no audio, no faces)

Latency target: under about 1.5 s to first audio. Budget: ASR ~300 ms, retrieval ~100 ms, first LLM tokens 400–600 ms, TTS 150–300 ms (AI Mind — Process).

3. The four ideas that matter

4. The corpora and the fence

• Corpus A — own words. Diaries, letters, lectures. The only text spoken as "I".
• Corpus B — contemporary accounts. Always attributed ("My sister Phebe wrote…"). Never merged into "I".
• Corpus C — period knowledge. What an educated person of that place and year could know.
• D-notes — modern scholarship, contested facts, trigger topics. Never ingested. They feed rules, canned replies and the golden question set.
• One index per figure. There is no shared "Nantucket brain". Mitchell cannot retrieve Mott's letters.
• Thin evidence. Absalom Boston has no surviving writing we know of. His "mind" is retrieval plus canned bridges, not free generation (Tutorial — Building a Figure's Corpus).

5. What happens in one turn (example)

1. A visitor holds the button and asks: "Were you scared on the roof at night?"
2. ASR returns text. The input rails pass it (no injection, no unsafe topic).
3. The matcher finds no authored answer above threshold.
4. Retrieval returns three dated Corpus A chunks from the comet years, with IDs such as MM-A-1847-10-01-a.
5. The model writes two or three sentences in her voice, using only those passages.
6. The policy layer checks: no date after the cutoff, no modern advice, no "I love you", B passages attributed.
7. The service sends answer_meta with the citation IDs and a gesture tag (recall), then captions and audio (Tutorial — Conversation Client Plugin).
8. The log stores the question text, answer ID, citation IDs and flags.

6. Who owns what

7. Every AI-mind tutorial

8. Failure modes to know

• No network: the figure falls back to authored monologues only (Tutorial — Authoring a Timed Experience).
• No answer_meta in 4 s: an authored "another time" line plays (Tutorial — Conversation Client Plugin).
• Nothing retrieved: the figure says she did not write about that. She does not guess.
• A fact is wrong: fix the corpus, bump its version, re-run the golden set, pin a canned answer overnight if needed (Tutorial — Staff Dashboard).

9. Practice exercise

1. Pick three questions: one about the 1847 comet, one about slavery, one asking "Do you love me?".
2. For each, trace the diagram in section 2 and write down which branch it takes.
3. Name the owner who would review the result. Check your answers with the AI conversation engineer.

10. Check before your first week ends

• You can draw the loop from memory
• You can explain why D-notes never enter the model
• You can explain why the cutoff lives in code, not the prompt
• You know who to ask about each part (section 6)
• You have read Ground Rules — Historical Virtual Humans

Sources

• Internal: AI Mind — Process, Ground Rules — Historical Virtual Humans, Tutorial — Conversation Client Plugin
• OWASP GenAI Security Project, LLM07:2025 System Prompt Leakage: https://genai.owasp.org/llmrisk/llm072025-system-prompt-leakage/
• NVIDIA, NeMo Guardrails (rail types): https://github.com/NVIDIA-NeMo/Guardrails
• Full list: Sources — 2026-09-20