Tutorial — Speech Out

Augmented Traveler local tutorial. Back to AI Mind — Tutorials · the actor and voice bible: Voice — Process · Unreal playback: Tutorial — Conversation Client Plugin · speech in: Tutorial — Speech In

How a live answer becomes the figure's voice. We use one contracted actor voice per figure, built into a locked synthetic voice, streamed as 24 kHz PCM16 to Unreal. It covers consent and contract, the pronunciation lexicon, prosody, latency, the fallback policy and QA.

Draft. Written from the ElevenLabs, Kokoro and Piper docs listed under Sources, and from Voice — Process. The voice director, the actor and the AI conversation engineer validate it; counsel reviews section 2. Correct setting names here after the first build.

1. The rule

Consent is not the same as public domain. The actor signs for the synthetic voice explicitly (Voice — Process step 2).

Clauses to have in the contract (counsel drafts; this is a checklist, not legal text):

Clause What it covers
Scope Synthetic voice for live answers of this figure only, on named surfaces (kiosk, headsets, phones, Pixel Streaming)
Content limits Only answers produced inside our policy layer (Tutorial — Policy Layer with Guardrails); no ads, no political content, no other characters
Retraining Whether we may rebuild the voice on a new model or vendor, and whether the actor re-approves
Version lock The approved voice version is fixed; changes need the voice director's sign-off and the actor's notice
Account ownership Who holds the cloned voice on the vendor platform (see below)
Term and revocation How long; what happens at the end (we delete the voice and fall back per section 7)
Payment Fee, buyout or residuals
Credit Wording and placement
Watermark / disclosure The actor agrees the output is labelled synthetic

ElevenLabs Professional Voice Clone (PVC) — important: the help centre says you can only create a PVC of your own voice, "even with their consent". The actor must create and verify the PVC on their own account, then share it with our account through a private sharing link. So:

  1. The contract must require the actor to create, verify and share the voice, and not withdraw it during the term.
  2. Our account never holds the only copy of the source recordings; the voice bible stays in the studio archive.
  3. Check the sharing terms with ElevenLabs before signing (unconfirmed how version changes work on a shared voice; check on first use).

3. Streaming TTS set-up

Platform: ElevenLabs class, as in AI Mind — Process. What the docs say:

Item Detail
Model Flash v2.5 is listed at about 75 ms model latency (excluding network and app). Turbo v2.5 is deprecated in favour of Flash. v3 is the most expressive.
Voice type and latency Default voices and Instant Voice Clones are faster than Professional Voice Clones on Flash v2.5; ElevenLabs says it is working on PVC latency
Output format pcm_24000 matches our protocol (24 kHz mono PCM16). pcm_44100 needs a higher tier
Endpoints HTTP streaming, or WebSocket stream-input for text arriving from the LLM. The WebSocket returns base64 audio with alignment data
Network Time to first byte about 100–150 ms from North America; api.us.elevenlabs.io opts out of global routing
Determinism seed (0–4294967295) for repeatable output in QA
Continuity previous_text / next_text help prosody across chunks
Zero retention enable_logging=false; enterprise customers only; voice cloning itself is not covered

Steps:

  1. The service receives LLM text as it streams, already checked by the policy layer.
  2. Cut at sentence or clause ends before sending to TTS. Half-sentences sound wrong.
  3. Strip anything not to be spoken: citation markers, stage directions, markup.
  4. Send over the WebSocket with the locked settings (section 5).
  5. Forward the PCM to Unreal as binary frames. Use the alignment data to time the caption chunks (Tutorial — Captions UI).

Local alternatives (offline venues):

4. Pronunciation lexicon

Owned by the voice director with the historian and dialect coach. One lexicon per figure, versioned.

Word Note
Siasconset / 'Sconset Confirm the local pronunciation with the dialect coach
Wampanoag Confirm with a Wampanoag advisor, not a dictionary
Nantucket, Madaket, Polpis, Coatue, Quidnet Island places
Atheneum Institution name
thee, thy, First Day, Fourth Month Quaker plain speech (Mitchell, Mott)
Names of ships and captains (Essex, Veeder) Whaling

On ElevenLabs:

  1. Write the lexicon as a PLS file (W3C pronunciation lexicon, XML).
  2. Use phoneme rules (IPA or CMU) where supported. The docs say phoneme tags work only with eleven_flash_v2 and eleven_v3; other models skip them.
  3. Use alias rules (respelling) where phonemes aren't supported.
  4. PLS is case sensitive: add "Sconset" and "sconset" both.
  5. Up to 3 dictionary locators per request. Each locator has a dictionary ID and a version ID; always pin the version.

5. Version lock

Log these with every answer and in the venue config:

Any change (new model, deprecation, retrained voice) means a new version: repeat section 8 before it goes live. A vendor deprecating a model is a change.

6. Prosody

7. Fallback policy: no substitute voice

Situation What happens
TTS vendor down or slow (no audio within the timeout) Authored line and clip if the question matches one; otherwise the answer shows as text captions only, with the figure in a listening idle
Offline venue Authored monologues and authored answers only, unless a contracted local voice exists
Contract ended or revoked Remove the voice; authored clips only if they are still within their licence

Never switch to a stock voice "for now". A different voice is a different person.

8. QA before a voice version goes live

  1. Lexicon test: a script with every lexicon word in two sentences. The dialect coach marks each right or wrong.
  2. 50 golden answers synthesised with the locked settings and seed (Tutorial — Evaluating Answers). The voice director listens to all 50.
  3. Loudness: match the authored monologues (the sound designer sets the target).
  4. Latency: time to first audio from the service, 50 runs, on the venue network.
  5. Watermark check: ElevenLabs says it embeds an inaudible watermark in its audio and offers an Audio Detector. Run one sample through it and file the result.
  6. Actor review: send the actor a sample reel before first opening.

9. Practice exercise

Take one Mitchell answer about the 1847 comet. Synthesise it three ways: as one block, cut at sentences, and cut at sentences with previous_text. Then add "Siasconset" and "Wampanoag" to a test lexicon and synthesise again. Note which version the voice director prefers, and why.

10. Check before a venue build

Sources