M0 and M1 are built and committed. npm test passes (48 tests) and npm run typecheck is clean. The build also succeeds. I haven't run the app or heard any audio, so the voice, the load path and the pauses still need your ears.

What changed

• M0: Vite, React and strict TypeScript scaffold in web/, with the dark theme (amber and blood accents, serif titles). The shell has a top bar with the scare slider, a Stage, and a drawer with Settings, Memories and Debug tabs. lib/idb.ts, lib/timing.ts and lib/sse.ts are in, plus the MIT LICENSE, .gitignore and a README stub.
• M1:
  • Cue code: cue/spec.ts holds every tag. cue/parser.ts is an incremental parser, and cue/segmenter.ts splits speech into sentences.
  • Voice: adapters/tts/kokoro.worker.ts and kokoroWeb.ts run Kokoro in a worker, and engine/player.ts is a minimal playback loop.
  • UI: a Play sample button, a voice picker, loading and "Summoning…" progress, and a basic DebugPanel that shows the unknown-tag count.
  • Sample: public/sample/sample.cue.txt is a draft I wrote. You approve the real one in M6.

Decisions you made, as built

• Missing or bad arguments: [pause] becomes 1 s and [light blackout] becomes 2 s, with a warning.
• Junk text: (pause 2), *sighs* and unterminated [tag are stripped and never spoken. Ordinary parentheses like "(the old one)" are still spoken.
• Style tags: the last style tag wins from that point to the end of the line.

Decisions I made that could give wrong output

Tell me if you want any of these changed.

1. Default voice is bm_george. Kokoro's voice list only has quality grades, with nothing about pitch or calmness. I picked a male British voice with a decent grade and haven't heard it. If it doesn't sound low or calm, try bm_fable, am_michael or am_fenrir in the picker.
2. WebGPU uses fp32, not q8. The plan says q8, but the kokoro-js README recommends fp32 on WebGPU, so q8 is only used on the WASM fallback. If WebGPU fails to load, it falls back to WASM with q8.
3. Blackout length is clamped to 0.2–6 s. The plan doesn't say what the limit should be, so I reused the pause clamp.
4. *...* spans are always stripped. That removes any word the model puts between asterisks for emphasis, as well as stage directions.
5. The sentence splitter is naive. "Mr." would split into two sentences.
6. slow sets Kokoro speed to 0.8. The other styles aren't emulated yet, since that's M5.

Things to know

• package.json is in web/, so run the npm commands from there. That matches the Render setup (root web).
• I installed kokoro-js (on your stack list), plus @vitejs/plugin-react and the @types packages, which I assumed come with "Vite + React + TypeScript". I wrote the IndexedDB wrapper myself, so idb-keyval isn't installed.
• The first load downloads the Kokoro model, and the production build includes a 21 MB wasm file. That's expected.

Test by hand

Run cd web && npm run dev, open http://localhost:5173 and check the following:

1. The shell renders. Click Play sample. The first run should show model download progress and then "Summoning… n/N".
2. The sample speaks line by line, and you hear real silences at the pauses. The Stage shows each line as it plays, and the Debug tab shows load and per-segment times.
3. The Settings tab shows which device it ran on (webgpu/fp32 or wasm/q8). Note which one you got and how slow the WASM path is on your laptop.
4. Switch voices in the picker and play again. Press Stop mid-sample, and it should stop cleanly.
5. Console warnings for dropped tags are expected only if the sample contains any. It shouldn't.

I'm stopping here for your testing before M2.

I haven't fixed the crackling yet. I've found the likely cause and added a way for you to confirm it. The ADR is written, and the changes are committed.

The crackling

• The voice isn't the cause: I ran the same pipeline in Node on CPU with q8. It produced clean speech statistics for af_heart and bm_george, with no NaNs, a normal level and a speech-like zero-crossing rate. So the model, the voice files and my audio conversion are fine. A missing voice also wouldn't explain every voice crackling.
• Likely cause: WebGPU with fp32. You're running on that path, and it's the only thing that differs. The suspects are the GPU driver on the AMD iGPU, or the onnxruntime-web dev build that kokoro-js pulls in. I can't run WebGPU here, so this is a strong lead and not yet proven.
• What I added:
  • A Voice engine device selector in Settings: Auto, WebGPU or WASM. Your choice is remembered.
  • Per-segment audio stats in the Debug tab.
• What to check: select WASM, reload the model and play the sample. If it's clean, WebGPU is the culprit. WASM will synthesize more slowly, so please note roughly how slow. If it's still noisy, send me the Debug tab lines. A zero-crossing rate near 0.5, NaN above 0, or a peak stuck at 1.0 would point to the audio output instead.
• If WASM works: I'd make WASM the default, with WebGPU as opt-in. I haven't made that change yet, because it's your call.

Replay starting from the start

This is expected for now. M1 playback synthesizes everything and then plays it, with no pause, resume or cache. Pause/Resume and Replay without re-synthesis are in M2, and the audio cache is in M5.

Docs/ADR.md

It holds nine entries: the decisions you approved, the ones I made that could change output, and the Kokoro device and voice issues. Each entry has the trade-off and how to debug or revert it. The device and default-voice entries (ADR-005 and ADR-006) are marked Provisional until you've listened. I'll add to it as we go.

M2 is built and committed locally as fd4cee8, but I haven't heard any of it. I haven't pushed it. npm test passes (52 tests), npm run typecheck is clean, and the build succeeds. I'll stop here for your manual testing, as CLAUDE.md requires.

I made one mistake along the way. A failed script run emptied Docs/ADR.md and that went into the first M2 commit. I restored the file from the previous commit and amended before anything was pushed. ADR-001 to ADR-009 are intact, with ADR-010 to ADR-015 added.

Done

• WebGPU fp32: removed. The voice engine is WASM q8 only, and Docs/IMPROVEMENTS.md records why and how to resume. Commit 46c380d still has the old WebGPU code.
• Push: the earlier commits are on origin/master. CLAUDE.md, PLAN.md and TASK.md still have uncommitted changes that I left alone.
• Engine:
  • Timeline: timeline.ts places events on a clock, so pauses, sound effects and lights line up with speech.
  • Audio graph: audioGraph.ts adds room reverb on the voice and lowers the drone by about 6 dB while speech plays.
  • Ambient drone: ambient.ts makes the drone procedurally.
  • Sound effects: sfx.ts has all nine named sounds plus breath, sigh, gasp and laugh.
  • Lights: effects.ts draws flicker, blackout, red and pulse as overlays on the Stage, with a reduce-flashing mode.
  • Playback: player.ts runs prerender → schedule → play, with Pause, Resume, Stop and Replay that doesn't re-synthesize.
• UI:
  • Transcript: lines fade in one at a time, and whispered lines are smaller and italic.
  • Controls: Space toggles pause.
  • Photosensitivity: a notice shows before the first play, and a "Reduce flashing" toggle is in Settings.

Voice fixes

Your "paragraph reading" complaint most likely came from sentences playing back-to-back with no gap, plus Kokoro's uneven silence padding and no reverb. What I changed (all in ADR-010):

• Sentence gap: 0.45 s of silence after every sentence.
• Silence trim: Kokoro's leading and trailing silence is trimmed, so the timeline controls all gaps.
• Speed: baseline speed is 0.92 (it was 1.0), and slow is 0.78.
• Reverb: a 2.5 s room reverb on the voice.

These are guesses. If it still sounds robotic, the next options are merging 2–3 sentences per segment for better flow, a different voice, or ElevenLabs in M5.

Test by hand

1. Play sample: the notice appears first, then the model loads and "Summoning… n/N" shows. Listen for the new gaps and reverb. Is the voice still robotic?
2. Drone and ducking: a low drone should fade in at the start, dip while the voice speaks, and fade out at the end.
3. Sound effects: creak, knock and gasp in the sample. Do they sound convincing? Which ones are bad? I haven't heard any of them.
4. Lights: flicker, then a 3 s blackout. Check they line up with the audio.
5. Pause and Space: audio, lights and transcript should all freeze together and resume together. Stop, then Replay should start immediately with no re-synthesis.
6. Reduce flashing: turn it on and replay. Flicker should become a slow dim, and the blackout should dim to about 85% over 1.5 s fades.
7. Photosensitivity notice: clear photosensitivityAck in localStorage to see it again.

Judgement calls to review

These are in ADR-011 to ADR-015, and all are provisional:

• Heartbeat: three lub-dub cycles, not the plan's two thumps, because one is barely audible.
• Pulse: [light pulse] isn't synced to the heartbeat yet.
• Non-verbals: the laugh and breath sounds are crude noise sounds.
• Reduce-flashing blackout: it dims to 85% instead of going fully black.
• Hidden tabs: the browser throttles hidden tabs, so lights can fire late there. Audio is unaffected.

Here's my read on each point, and two things I need from you before I change anything.

1. "Summoning" on replay

I think you clicked Play sample, not Replay, but I can't confirm that from here.

• Replay skips synthesis and goes straight to playback, so it shouldn't show "Summoning".
• Play sample always re-synthesizes. The audio cache that would prevent that is M5.

Both buttons sit side by side, so it's easy to hit the wrong one.

Proposal: once a story exists, Play sample should reuse the existing synthesis when the voice and script haven't changed. It would only re-synthesize if you change the voice. That is a small in-memory shortcut, not the M5 cache. M5's cache will still be needed for the LLM stories in M3, where the text is different every time.

Which button did you press? If it was Replay and "Summoning" still appeared, that's a bug and I'll dig into it.

2. Controls ruining the mood

Agreed, a pair of bordered buttons in the middle of the Stage works against the atmosphere. My proposal:

• While a story plays: the drawer slides away and the Stage fills the whole window.
• Controls: a tiny, dim pause/stop icon pair low in the bottom corner. It fades out after about 3 seconds without mouse movement and comes back when you move the mouse or press a key.
• Keyboard: Space still pauses. I'd add Esc to stop.
• Accessibility: Stop must always be reachable by keyboard, so the controls can't be hidden completely. This also fits the photosensitivity notice.
• Not playing: the Play sample and Replay buttons stay as they are.

Alternatively, I could put the controls in the top bar and leave the Stage empty. That's simpler but less immersive. I recommend the fading corner controls.

3. Italic text

That's the [whisper] styling: smaller and italic, as the plan says. I'm leaving it unless you want it changed.

4. Robotic voice

Parked for later, as you said. The notes on it are in ADR-010.

Next phase

M3 is the LLM adapter and the Director. It's the first milestone that needs your setup. The plan says not to guess the Gemma 4 model tag, so please paste the output of ollama list. If you haven't pulled the model yet, tell me which size you want to use.

So I need two answers from you:

1. Which button did you press for the replay?
2. Do you approve the fading corner controls with the drawer hidden during play?

Once I have those, I'll make the fixes, commit and push, and then start M3.

The fixes are committed and pushed (3c6978e). Typecheck is clean and all 52 tests pass. I haven't run the new controls in a browser, so please check them by hand.

• Play sample reuse: if the voice hasn't changed, Play sample now reuses the existing synthesis and skips "Summoning". Changing the voice re-synthesizes.
• Immersive mode: while a story plays or is paused, the top bar and drawer disappear. Small dim pause and stop icons sit in the bottom-right corner. They fade after 3 seconds of no mouse or key activity and come back on mouse move, key press, hover or keyboard focus.
• Keys: Space pauses and resumes, and Esc stops.
• ADR: ADR-016 records both decisions.

Please check:

1. Play, then leave the mouse alone for 3 seconds. The icons should fade, and moving the mouse should bring them back.
2. Esc should stop the story and return the drawer and top bar.
3. A second Play sample with the same voice should start immediately.

Next: M3 (LLM adapter and Director)

I haven't started M3 yet. It needs one fact from you: the exact Gemma 4 tag. When the pull finishes, paste the output of ollama list. I'll then use that tag as the default model, and it will stay editable. I also need to know whether you've set OLLAMA_ORIGINS=http://localhost:5173, and restarted Ollama afterwards. The browser can't reach Ollama otherwise, and I'll add a clear error message for that case.

I'm waiting for the model tag before I start M3.

I'd start with E4B. I haven't looked up the Gemma 4 model cards, so check the points marked below against them.

Why E4B

• Speed on your laptop: you have a Ryzen 7730U with 16 GB and no discrete GPU, so generation speed is the constraint. A 12B q4 model is about 7–8 GB of weights plus context. I'd expect only a few tokens per second on CPU/iGPU, so a 600–800 word story (1,000+ tokens) could take several minutes before the voice even starts. E4B is roughly a third of that size and should be much faster.
• Vision: M4 sends up to 4 photos through the model, one call per photo. With 12B on CPU that's likely minutes per photo. E4B keeps that tolerable, and the plan already caches the descriptions.
• Memory: the browser also runs Kokoro and the page, so 12B plus those on 16 GB leaves little headroom and risks swapping.
• The plan: CLAUDE.md already names E4B as the default.

What 12B would give you

Better instruction-following and richer prose, which matters for a model that has to emit the cue tags exactly. I'd keep it as an optional quality tier: the model field is free text, so you can switch without code changes. A decent test is to run both on the same level-3 prompt and compare.

Things to check

• Where the model comes from: the plan says to confirm the exact tag from the Ollama library. A Hugging Face GGUF (ollama run hf.co/...) needs its vision projector file as well. If it's missing, vision silently won't work. So ollama pull the official library tag if one exists, since it bundles the vision parts. I haven't verified that for these repos.
• Does it see images: check that Ollama reports vision capability for the model, for example in ollama show <tag>. M4 depends on it.
• Measure speed: run ollama run <tag> --verbose "write 5 sentences" and look at the eval rate line. Send me the tokens/s for E4B, and for 12B if you try it. Those numbers go into the write-up anyway.
• q4_0 QAT: this is quantization-aware training, so quality should hold up better than a normal q4. Fine to use.

Other options

I'd stay with the Gemma family. The plan targets Gemma 4 for the challenge, and switching families would also change how well it follows the cue tag format. A smaller model is only worth trying if E4B is too slow. In that case I'd cut the story length instead, since the vision pass is the bigger cost.

My suggestion is E4B for development and the demo, with 12B as an optional tier if its speed is acceptable.

What do ollama list and the eval rate look like for E4B? That will settle the default tag for M3.

That works, and the choice among those variants matters little. All the quants in your screenshot are within about 0.4 GB of each other (4.72 to 5.13 GB). On CPU, generation speed mostly depends on the size of the weights read per token, so I'd expect similar tokens/s across them. I haven't tested any of them, so pick on quality and compatibility.

Which one

• Q4_K_M (4.98 GB): the standard, well-tested choice. This is my default pick.
• UD-Q4_K_XL (5.13 GB): Unsloth's "dynamic" variant, which keeps more precision on the layers that matter most. Slightly better quality for about 0.15 GB more. It's a good alternative if Q4_K_M follows the cue-tag format poorly.
• Q4_K_S (4.84 GB): a bit smaller and a bit lower quality than Q4_K_M. No real reason to choose it.
• IQ4_XS and IQ4_NL: the smallest at 4.72 and 4.84 GB. They can run slower on CPU, which is the wrong trade for you.
• Q4_0 and Q4_1: older formats. Skip them unless it's Google's QAT Q4_0, which was trained for that format and should hold up better than a plain one.

So I'd compare Google's QAT q4_0 against Unsloth Q4_K_M (or UD-Q4_K_XL) on the same level-3 prompt. At around 5 GB each, either fits comfortably in your 16 GB.

The vision risk

A text GGUF alone can't read images. Vision needs a separate projector file (an mmproj file), and M4 depends on it. Before you commit to a repo:

• Ollama with hf.co/unsloth/...: it may or may not pull the projector. Check ollama show <tag> for a vision capability. I haven't verified how Ollama handles this repo.
• If Ollama doesn't expose vision: LM Studio (http://localhost:1234/v1) and a llama.cpp server (http://localhost:8080/v1) are both already presets in the plan. They let you specify the projector explicitly, and no code changes are needed.

I'd start with a text-only M3. Photos don't enter until M4, so we can settle the vision setup then.

What I need from you

1. Pull one candidate and run ollama run <tag> --verbose "write 5 sentences". Send me the eval rate line and the exact tag.
2. Whether ollama show <tag> lists vision capability.
3. Whether you've set OLLAMA_ORIGINS=http://localhost:5173 and restarted Ollama.

I'll make the tag the default in the model field, which stays editable, and then start M3.

M3 is built and committed locally (1505d25), but it has never run against a real model. npm test passes (66 tests), npm run typecheck is clean and the build succeeds. The tests use fake adapters and mocked fetch only. I haven't pushed this milestone.

What changed

• LLM adapter: OpenAICompatAdapter streams chat responses and sends the Bearer key only when one is set. Images go through as content parts, and Fetch models works. A refused connection to a local host shows the OLLAMA_ORIGINS explanation with copyable Windows and macOS/Linux commands.
• Presets: Ollama, LM Studio, llama.cpp and Custom.
• Scare profiles and prompt: director/profiles.ts has the five levels and director/prompts.ts has the director prompt. The tag list in the prompt is generated from cue/spec.ts, so it can't drift from the parser.
• Director: director/director.ts parses the story as it streams and retries once if the output breaks the format. It records time-to-first-token, generation time and tokens/s in the Debug tab.
• UI: a Tell me a story button, with the streamed text shown dimmed while it's written (tags hidden). The new Settings panel has the provider, URL, model, API key and Fetch models. The voice model loads while the story is being written.
• ADR: ADR-017 to ADR-020 cover the new decisions.

Test by hand

1. In Settings, paste the model tag from ollama list into Model. The default is blank on purpose, since I wasn't going to guess the Gemma 4 tag. If you give me the tag I'll make it the default. Fetch models should also list it.
2. Make sure OLLAMA_ORIGINS includes http://localhost:5173 and Ollama was restarted. With Ollama stopped, Tell me a story should show the helpful error with a Copy button.
3. Run Tell me a story at levels 1, 3 and 5. Check the following:
   • The draft streams in during "Summoning…".
   • The story plays, and the levels differ in length, number of effects, and ending.
   • At level 1 there are no flickers or blackouts, only pulse and calm lights.
4. The Debug tab should show the time-to-first-token, generation time and tokens/s. Please send me those numbers.
5. Open the browser console and look for any cue: dropped … warnings. They show how well the model follows the tag format.
6. Replay should work on a generated story.

Judgement calls to review

These are in the ADR:

• Light modes: limited by level through the prompt only, per the ladder you approved. The parser doesn't block a disallowed mode if the model emits one anyway.
• Prompt example: at low levels I swapped the example's lights for allowed ones, so the example doesn't contradict the rules.
• Token cap: max_tokens is 2048, my own choice. A long level-5 story could be cut off mid-sentence.
• Tokens/s: it counts stream chunks, so it's approximate.
• Failed retry: if the retry also fails, I keep whichever attempt has more spoken words. If there's no speech at all, you'll see an error.

I'm stopping here for your testing before M4, which is the photos and vision pass. Please send the model tag, the Debug numbers, and any bad stories, tag drops or errors you see.

The slow part isn't Gemma. By the time you saw "24/75", Gemma had already finished writing the story, which is why the full text was on screen. The "24/75" is the voice engine (Kokoro) turning 75 sentences into audio one at a time. I made fixes for that and for the scrolling, committed as 0265288; typecheck, the 66 tests and the build pass. I haven't tested them in a browser.

Why the voice is slow

Kokoro in the browser has been running on a single thread. I checked the library code: without cross-origin isolation, onnxruntime-web uses 1 thread, and with it, up to 4 on your machine. Our dev server wasn't sending the two headers that enable it.

Fixes

• Threads: the dev and preview servers now send those headers. Settings shows "multi-threaded" or "single-threaded" next to the device, so you can see whether it took effect.
• Progress: the Summoning screen now shows an estimated time left.
• Scrolling: the Stage scrolls when its content is taller than the window, so the Stop button can't be pushed off-screen. The streamed draft is capped to 40% of the screen height and cleared once speech synthesis starts.
• ADR: ADR-021 records this.

Before you test

Restart npm run dev. Header changes in the Vite config only take effect on a restart. Then hard-reload the page. Please check:

1. The Settings tab should say "multi-threaded". If it still says "single-threaded", tell me.
2. The model download and the Ollama calls must still work with the headers on. I expect them to, but I haven't verified it, so this is the main risk.
3. Run level 1 again and send me the Debug tab's tts segment times, the new ETA, and roughly how long the whole prerender took.

If it's still too slow

Level 5 has 600–800 words, so roughly 150 sentences. Waiting minutes before hearing anything isn't a good demo. Two options:

• Streaming playback: start speaking after the first few sentences while the rest synthesize in the background. This is the real fix, and it's on the plan's [SHOULD] list. It takes real work, since the timeline currently needs every duration up front.
• Longer segments: merge two or three sentences into one synthesis call. This probably won't cut total time much, but it might improve how natural the voice sounds, which also helps the robotic-voice problem.

I'd wait for your numbers before choosing. With a Friday 20:00 IST feature freeze, M4 to M6 still have to fit, so I'd only do streaming if the multi-threaded speed isn't enough.

I'm waiting for your test results.

I'll build streaming playback first, because M4 will sit on top of the same player. The design: the player schedules audio incrementally as each segment finishes synthesizing, starts early only once it's safe, and auto-pauses ("waiting for the voice") if the playhead ever catches up. I'll start with the timeline builder and the session.

Streaming playback and M4 are both built and committed locally (7e3e00a and f714fde). npm test passes (93 tests), npm run typecheck is clean and the build succeeds. I haven't pushed these, and none of it has run in a browser: the streaming, the vision pass and the photo reveal are all untested against a real model.

Streaming playback

• How it works: speech now plays while it's still being synthesized. The player schedules each segment as it arrives and starts once a safe buffer exists. The buffer size comes from how fast segments are being made compared with how long they play.
• If the voice falls behind: playback holds and shows "the voice is gathering itself…". It resumes by itself, and the audio, lights and text all freeze together.
• Not cached yet: Replay and Play sample reuse a story only once it has fully synthesized. Stopping partway throws the partial audio away.
• Fixes from your last run: the scrollable Stage, the prerender time estimate, and the multi-threaded WASM headers are all still in.
• Debug numbers: the Debug tab now records time to start speaking. That's the main number to send me.

M4: photos and vision

• Memories tab: add up to 4 photos by drag-drop or file picker, with thumbnails, delete, and a notes field.
• Storage and privacy: photos are downscaled to 768 px and re-encoded as JPEG. That removes EXIF, and they're stored in IndexedDB only. They go only to the model endpoint you configured, and the panel says so.
• Vision pass: it runs at "Tell me a story" time, one photo at a time at temperature 0.2. Bad JSON is retried once and then falls back to raw text. Results are cached per photo.
• Photo reveal: the best photo fades in at the climax (desaturated, red-tinted, grainy, with a brief glitch), using the plan's fade timings. Reduce flashing skips the glitch.
• No-vision models: if the model rejects images, the Stage tells you and the story uses your notes only.
• ADR: ADR-022 to ADR-024 cover streaming, the vision pass and the reveal look.

Test by hand

Restart npm run dev first.

1. Streaming: run "Tell me a story" at level 1. Speech should start well before "75/75". Note the time to start speaking value and whether you ever see the "gathering itself" hold.
2. Settings tab: it should say "multi-threaded". Does the model download still work?
3. Photos: add 2 photos of a real room and write a note. Then run a level-3 story. Check:
   • "Studying your place — photo 1 of 2" appears.
   • The story names objects that are actually in the photos.
   • Your photo fades in near the end.
4. Cache: run it again with the same photos. The vision step should be near-instant, and the Debug tab should show no new vision room_n timings.
5. Reveal: check how it looks, and that Pause freezes it. Tell me if the grain feels too flashy with Reduce flashing on.
6. Failure cases: try a text-only model, if you have one, to see the notes-only message. Also remove a photo and add another to check the slot ids.
7. Photo privacy: reload the page. Your photos and notes should still be there.

Judgement calls to review

These are in the ADR:

• Failed vision fallback: it isn't cached, so a failed analysis is retried next time instead of being pinned to that photo.
• "No vision" detection: any HTTP error from the server on an image request is treated as "model can't see". A transient 500 could be misread this way.
• Silent image ignoring: a model that accepts images but ignores them can't be detected, and would invent a room.
• Voice load contention: the voice model loads at the same time as the vision pass, so they compete for CPU.
• Reveal look: the reveal values (tint, grain, glitch timing) are guesses I haven't seen.

I'm stopping here for your testing before M5. M5 is the ElevenLabs adapter, the router and cache, and the DataFlowBadge. The plan says to test the ElevenLabs CORS question first and report before building around it, so I'd need an ElevenLabs key from you for that.

The three things you asked for are built and committed locally (6998fdd, 0057ee7, 159cf87). npm test passes (114 tests), the typecheck is clean and the build succeeds. None of it has run in a browser, and I haven't pushed anything since M3.

About the model

The server said "model does not support multimodal requests". So this gemma-4-E4B in Ollama is a text-only import. It is probably missing the vision projector file, and the Gemma family itself can read images. To fix it:

• Check the current one: run ollama show gemma-4-E4B and see whether "vision" is listed under capabilities.
• Use a build with vision: pull one that includes the projector. I can't confirm the right Ollama tag from here, so check the model page for "vision".
• Or use another server: LM Studio and llama.cpp already have presets in the app, and both load the projector explicitly.

Photos aren't needed to test the rest.

Parallel voice workers

• Toggle: Settings has a "Voice workers" selector with 1, 2 or 3. It defaults to 1 (off). Each worker is a separate copy of the model.
• Memory footprint: after the voice loads, the Debug tab records page memory for that worker count. Please compare 1, 2 and 3.
• Playback order: segments can finish out of order, but playback order stays correct. That's tested.
• Risk: each worker can use up to 4 threads, so 3 workers on your CPU may oversubscribe it. The speed test is the judge here, not my assumption.

Auto story length

• Settings: "Story length" is Auto, Short or Full. A "Run voice speed test" button times one fixed sentence across all workers and shows how many times slower than speech the voice is.
• Auto: it runs that test once, then shortens each level's word range so playback starts in roughly 90 seconds, never below 35% of the plan's range. Short is half length and Full is the plan's range.
• Debug note: the Debug tab shows the range it chose.
• Trade-off: on a slow machine all five levels get shorter, so they differ less in length. The scare differences then come from jumps, effects, ending and content ceiling.

Story quality

• Prompt: I added a story-shape section. It asks for a calm opening, three escalating wrong things, a climax, and a closing line that doesn't trail off. It also asks for a minimum number of [light] cues per level.
• Premise seeds: one of 14 original premises is picked per story and shown in the Debug tab. I wrote them myself, not copied from r/nosleep.
• Parser: it now accepts [lights pulse] and plural sfx names such as scrapes. If the model never stops the drone, the ending adds a fade-out.
• Diagnostics: the Debug tab shows whether the model finished or hit its token limit, and counts each tag type per story.

Test by hand

Restart npm run dev first.

1. Speed test: click "Run voice speed test" with 1 worker and send me the number. Then try 2 workers and 3.
2. Memory: compare the memory lines in the Debug tab for each worker count.
3. Story length: run Tell me a story on Auto. Check the "story length" and "premise" lines in Debug. Does the story now finish properly, and do the lights fire?
4. Cut-off check: if a story is cut off, the Debug tab should now say so.

The plan says to test the ElevenLabs CORS question first in M5. Whether we pick local voices or ElevenLabs for the demo depends on the speed-test numbers, and I'd like them before going on. M4's photos stay untested until you have a model that can see images.

I'm waiting on the speed-test and memory numbers from you before starting M5.

Is there a configuration for vision?

No, not for this model as it is installed. I checked it locally:

• ollama show gemma-4-E4B: its capabilities are only completion, tools and thinking. There is no vision.
• The Modelfile: it points at one file, the text GGUF. The vision projector, which is a separate file in the Unsloth repo, was never attached.
• Why the Hugging Face page says "multimodal": that describes the model family, not this Ollama import. Search results say Ollama can't take a separate mmproj file and needs a combined model. I couldn't confirm that from a primary source, so treat it as likely rather than certain.

Ways to get vision

1. Ollama's own build (easiest): the Ollama library page for gemma4 lists E4B with "Text, Image" input (audio too). Its download is 6.6–9.5 GB, bigger than your 5 GB file because it bundles the vision parts. I'd try ollama pull gemma4:e4b, but I haven't confirmed that exact tag, so copy it from the page. Then check that ollama show lists vision.
2. LM Studio or llama.cpp: both already have presets in the app and load the projector explicitly. llama-server -hf unsloth/gemma-4-E4B-it-GGUF:Q4_K_M should fetch it automatically, but I haven't tested that.
3. Hand-building a Modelfile with the projector: uncertain on this Ollama version. I wouldn't spend time on it first.

Option 1 means you'd run the official model for both the story and the photos, which also keeps one model in memory.

Something I found about speed

ollama show says the model has thinking on by default. Through Ollama's OpenAI-style endpoint, that probably means the model "thinks" first, and those tokens count toward our 2048-token cap and add delay before any story text appears. Our adapter only shows the final text, so we'd never see it. That could explain both the cut-off endings and slow generation. A search result says Ollama's /v1/chat/completions only turns thinking off when the request includes reasoning_effort: "none". I haven't verified that.

I haven't changed anything, so I don't disturb your test run. Please check these in the Debug tab after a story:

• The finish: value in the story line. If it says length, the model ran out of tokens.
• The time-to-first-token value. A long one would fit the thinking theory.

If either looks bad, I'll add a "don't think" setting. It would be on by default for the Ollama preset and not sent to other servers, since unknown parameters can trigger the same 400 error we treat as "no vision". Send me the speed-test and memory numbers when you have them.

Sources:

• Ollama library: gemma4
• Gemma 4 12B GGUF on Hugging Face (NobodyWho)
• aictrl-dev/cli issue #75 (reasoning_effort on Ollama)

Recommendation: stay in the Gemma family, but go lighter with E2B

Don't switch families, and don't try the 7 GB build.

• Memory: Windows sees 13.8 GB total and only 3.9 GB is free right now with no model loaded. The Ollama E4B at 6.6 GB or more won't fit comfortably next to the browser and the voice model. The 5 GB E4B text file from before is tight even without vision.
• E2B fits better: the Ollama library lists E2B at 4.6 GB or more, with "Text, Image" input. It's smaller and faster too. I haven't confirmed the exact tag, so copy it from the page; it's probably gemma4:e2b. After pulling it, run ollama show <tag> and check that vision is listed, then ollama ps to see its real memory use. Close other heavy apps before testing.
• Why not another family: the challenge rules only require an open-weight model. But Gemma is a featured prize category ($200, "Best Use of Gemma"), and that's one we can enter. Another family loses it. Switching also resets all the tag-format tuning we did for Gemma. I'd only switch if E2B turns out too weak.

A quick caveat

E2B will likely be a bit worse at following the cue tag format than E4B, so test the story quality too. We should test with a real story and your photos.

Time check

Submissions are due Oct 31, 11:59 PM PDT, and the feature freeze is Fri 20:00 IST. M5 (ElevenLabs, router, cache, privacy badge) and M6 (Sample mode, deploy, README) are both must-haves, and M6's Sample mode is what judges will actually use. Photo vision is a nice-to-have. So:

• Timebox E2B vision: if photos don't work well within the next hour, ship notes-only stories and use the pre-baked photo in Sample mode.
• Judging: the write-up counts most, then technical execution.

llama.cpp

llama-server isn't on my PATH. Tell me the folder it's in if you want me to try it. I won't start a heavy model load now, since your test is running and free RAM is low. Please paste the speed-test and memory numbers from your run, plus what the Debug tab says about finish: and the time to first token.

Sources:

• Ollama library: gemma4
• Hacktoberfest Open-Source AI Challenge: Week 4 rules (DEV)

The vision layer is done and committed (0e37882). npm test passes (121 tests) and the typecheck is clean, but I haven't run it in a browser.

The memory question

It isn't Ollama. I checked the process list:

• Ollama: idle, about 15 MB plus its tray app at about 22 MB, with no model loaded.
• Zen: 22 processes using roughly 2.5 GB. The 570 MB entry in your screenshot is a tab showing the Hugging Face model page.
• Edge: about 0.6 GB.
• Free RAM: about 3.6 GB of 13.8 GB.

Closing Zen's heavy tabs frees more than any model choice.

What I built

• Setting: Settings now has "Read my photos with this model" with three choices: Auto (ask the server), Yes, or No.
• Auto: it asks Ollama's /api/show whether the model lists vision, or llama.cpp's /props for the same. If the answer is no, photos are never sent, and a notice says the story is using your notes only. This works with your current model.
• Yes and No: Yes skips detection and sends photos. No skips photos.
• Check button: "Check model for image support" shows the result. It resets when you change the model or URL.
• Unknown answers: LM Studio and custom servers can't be asked. In Auto mode photos are still tried, and the earlier safety net skips them if the server refuses.

That's the end of the photo layer for now. It needs gemma4:e2b or an LM Studio / llama.cpp model with vision to actually read photos.

One question for you

In the notes-only path the photo reveal is also skipped, which is how I read the plan. But the reveal needs no vision, since it's just your own photo fading in at the climax. If you agree, I'd still fire it. That keeps the demo's best moment working with any text-only model, and the story text just wouldn't mention the photo's objects. It's a small change. Should I make it?

Next

Test one story with the new voice and length settings, send me the Debug numbers, and tell me if gemma4:e2b pulled. Then I'll start M5. The plan says to test ElevenLabs CORS before building around it, and given the speed problem on your laptop, a fast cloud voice may matter for the demo.

I'm waiting for your answers.

M5 is built and committed locally (5785aa3, 4c5a1f8). npm test passes (140 tests), the typecheck is clean, and the build succeeds. None of it has run in a browser, and the ElevenLabs path has never made a real request because it needs your key. I haven't pushed anything.

Checked

• CORS: I confirmed ElevenLabs allows browser calls from any origin, so no proxy is needed.
• Model: eleven_v4 is now the flagship (10,000 characters per request). I made it the default, and eleven_v3 and eleven_v4_turbo are selectable.

Built

• Audio cache: segments are saved on your device. A replay or re-render of a cached segment costs nothing. Settings has a "Clear audio cache" button and Debug shows the hits and misses.
• ElevenLabs: the adapter has key and voice pickers and plain-English error messages. A tag map lives in one file, styleMap.ts. A sigh, gasp or laugh right before speech becomes part of that segment.
• Voice toggle: a Voice selector sits beside the Stage buttons. Replay re-renders the same story in the other voice.
• Kokoro expression: whisper, trembling and shout are now emulated with audio effects. This should help the robotic feel a little.
• DataFlowBadge: it shows green, yellow or red in the top bar, with a "what goes where" table when clicked.
• Demo story: I drafted a new default story (about 250 spoken words, ending on the photo reveal) at web/public/sample/sample.cue.txt.
• ADR: ADR-031 to ADR-036 cover all of this.

Judgement calls to review

• Tag spellings: [nervous] is a guess for trembling. [shouting] replaces the plan's [shouts]. Whether tags change the delivery needs to be checked by ear.
• Model default: I chose eleven_v4. Say so if you want eleven_v3 as the plan said.
• The drafted story: it's a draft until you approve it.

Prerequisites for the next steps

These are what I need from you.

For M5 testing

1. ElevenLabs API key and credits. The hacktoberfest.com/my page lists promo codes. I won't claim any for you without a yes.
2. Which voice you want, if you have a preference. "Load my voices" lists your account's voices.
3. A real Kokoro vs ElevenLabs comparison on the same story. Is the voice still robotic after the emulation, and do the tags change the delivery?

For M6 (Sample mode, deploy, README) — the most important part for judges 4. sample_room.jpg: a photo of your own room, placed at web/public/sample/. 5. Approval of the demo story, or edits, so I can fit it to your photo. 6. A Render account for the static site. 7. A .tech domain and where its DNS is managed. This is the "Best Use of a .tech Domain" category. 8. Permission to push to GitHub. Several commits are still local, and Render deploys from the repo. 9. The Render headers (COOP and COEP) I'll configure so the voice stays multi-threaded when deployed.

For the write-up (it counts most) 10. A demo recording or GIF, plus screenshots, after the freeze. 11. Your Entire session saved for the "Best Use of Entire" category. I can do that with the sessions tool once you say yes. 12. Which prize categories you want: Gemma, ElevenLabs, Render, Entire, .tech.

For the local model 13. A vision-capable model. Pull gemma4:e2b (copy the exact tag from the Ollama page) and check that ollama show lists vision. Close other heavy apps first.

Time

Feature freeze is Fri 20:00 IST and submission is Sat 12:29 IST. M6 is the must-have. I'd start M6 as soon as items 4–9 are ready, and treat M5's ElevenLabs testing as parallel work.

I'm waiting on your answers to those prerequisites before starting M6.

I'll check what the DevRelay offers system shows for ElevenLabs. Checking is read-only; I won't claim anything without your yes.

I found it, and you may already have the code.

What I found

• The web page is a different offer. The "1 Month Free" card on hacktoberfest.com/my is exhausted. Your MLH account lists a separate ElevenLabs offer for Hacktoberfest 2026: "3 Months of ElevenLabs Creator Tier". It needs no credit card and no check-in, and the limit is one code per account.
• You've probably already claimed it. Your account has exactly one code redemption, made on 25 Oct 2026, at the Hacktoberfest 2026 event. This ElevenLabs pool is the only offer listed for that event, so that redemption is almost certainly it. The Week 4 event has no offers of its own.
• I can't confirm it from the redemption list. It shows when a code was claimed, but not which offer or the code itself. Only the claim call shows that.

What I need from you

Do you want me to run the claim call? Claims consume inventory, so I won't do it without a yes.

• If you already hold the code: it returns that same code and issues nothing new.
• If the 25 Oct claim was something else: it issues you the ElevenLabs code, which is what you want anyway.

Either way I'll show the code once. I won't write it into any file or commit. You'd redeem it at https://elevenlabs.io/app/agents, and then create an API key in your ElevenLabs profile for Settings.

If you've already redeemed it and have an ElevenLabs account, you can skip this and just create the API key.

I'm waiting on your yes (or the key) before continuing.

Both are done.

Your ElevenLabs code (shown once)

MLH-027B2D0E

• What it is: the 3 months of ElevenLabs Creator tier. No credit card is needed, and it doesn't stack with other codes.
• Redeem it here: https://elevenlabs.io/?coupon=MLH-027B2D0E. Sign up or log in there, and the Creator tier should apply to the account.
• Then get a key: after redeeming, open your ElevenLabs profile and create an API key. Paste it into Settings, set the voice provider to ElevenLabs, click "Load my voices", and pick a voice.

I haven't written the code into any file or commit.

Voice

Heart (af_heart) is now the default Kokoro voice, committed as 925bb2f with a note in the ADR. Typecheck and tests are unchanged and passing. If you meant an ElevenLabs voice called Heart, tell me. ElevenLabs voices come from your own account list, so I can't pick one for you yet.

Next

1. Redeem the code and send me the API key's status. Please don't paste the key itself in chat. Enter it straight into Settings.
2. Test one story in each voice and tell me how they compare. Do the audio tags change the delivery, and is [nervous] doing anything for the trembling lines?
3. Send the M6 items when ready: sample_room.jpg, approval of the draft story, the Render account and .tech domain details, and permission to push.

I'll start M6 as soon as I have those.

The .env support is committed (9744ec5). Typecheck is clean and all 140 tests pass.

How to use it

1. Create the file: copy web/.env.local.example to web/.env.local.
2. Add your key: set VITE_ELEVENLABS_API_KEY= followed by your key. I haven't touched that file, since I don't have your key.
3. Restart npm run dev: Vite only reads .env.local at startup.
4. Check Settings: with the voice provider set to ElevenLabs, it should say "Using the key from web/.env.local". Then click "Load my voices" and pick a voice.

Why it's dev-only

Vite compiles any VITE_ value into the JavaScript the browser downloads. On a deployed site, anyone could read the key from the page source and spend your credits. So the app only reads it when running npm run dev. I checked this by building for production with a dummy key set. The key value and the variable name were both absent from the output.

Other safeguards

• Git: .env.local is git-ignored, and only the empty placeholder example is committed.
• Browser storage: a key that came from .env.local is never copied into your browser's storage. A key typed into Settings is still saved there, like the plan says.
• Deployed site: each visitor enters their own key, which stays in their browser. Your key won't be on the hosted site.

Next

Test one story in each voice and tell me how they compare. Then send me the M6 items so I can start Sample mode, the deploy and the README: sample_room.jpg, approval of the draft story, the Render account and .tech domain details, and permission to push.

Only two permissions are needed. Nothing beyond Text to Speech and a read-only voice list.

What the app actually calls

• Text to Speech: POST /v1/text-to-speech/{voice} makes every spoken line.
• Voices (read): GET /v1/voices fills the voice picker when you click "Load my voices".

It never uses agents, speech-to-text, dubbing, voice cloning, history or the workspace settings. Everything else can stay off.

When you create the key

In ElevenLabs, create a restricted key (instead of an all-access one) and turn on exactly these:

1. Text to Speech: enabled.
2. Voices: read access only.

Name it something like localghost. If the key is ever leaked, the damage is limited to speech generation.

Worth setting too

• A credit quota: ElevenLabs lets you cap how many credits a key can use. I'd set one, since the key lives in your browser and in .env.local.
• No IP allowlist: you can restrict a key to certain IPs. Leave that off, since it would block the key when you demo from another network.

Housekeeping

• Copy the key right away: ElevenLabs usually shows a key only once.
• .env.local: it's git-ignored, and your IDE just added an explicit .env.local line to .gitignore, which is fine. Don't paste the key into chat or screenshots.

Sources:

• ElevenLabs: API keys
• ElevenLabs: API authentication

The key works, and both permissions are confirmed. I made no changes to the code in this turn.

• Voices permission: the voice list returned HTTP 200 with 21 premade voices.
• Text to Speech permission: two tiny test requests on eleven_v4, using the audio tag [whispers], both returned HTTP 200. They used about 48 credits in total.

Listen to the test clips first

These are the same line, "[whispers] It is on now.", in two voices:

• C:\Users\Lenovo\AppData\Local\Temp\claude\c--CloneD-Projects-localghost\50313d85-6a6b-4c95-b834-3f689cdb8eb5\scratchpad\brian_whisper.mp3 (Brian: deep, resonant, comforting)
• the same folder, lily_whisper.mp3 (Lily: velvety actress)

I can't hear them. Does the [whispers] tag actually sound whispered rather than being read aloud? That's the main thing to confirm. If a tag gets spoken out loud, tell me, and I'll change the spelling in styleMap.ts.

Next steps in the app

1. Settings: set Voice provider to ElevenLabs. It should say "Using the key from web/.env.local".
2. Voices: click "Load my voices" and pick one. For narration I'd try Brian or George (warm storyteller), and Lily for a whispery feel.
3. Run Play sample: the first run costs credits, about 1,000 characters. After that it's cached, so replays are free.
4. Compare with Kokoro: use the Voice selector next to the buttons. Do the tagged lines (whisper, slow, trembling, shout, sigh, gasp) sound different? Is it still robotic?

Then M6

M6 is the Sample mode, the deploy and the README. I'll start it once you've tried the voices. To finish it I need: sample_room.jpg, approval of the draft story, the Render account and .tech domain details, and permission to push.

Time

Today is Friday 30 Oct, about 17:10 IST. CLAUDE.md calls the freeze "Fri 2026-10-31", but 31 Oct is a Saturday. The freeze is Sat 31 Oct 20:00 IST, and the deadline is Oct 31 11:59 PM PDT = Sun 1 Nov 12:29 IST. That leaves roughly a day of work.

I'm waiting for your comparison of the two voices and for the M6 items.

M6's code is built and committed locally as e78242a. npm test passes (140 tests), the typecheck is clean, and a production build succeeds with the demo story included. I haven't run any of it in a browser, I haven't pushed, and nothing is deployed yet.

What's new

• Try it now — no setup: this is now the main button. It always uses the offline Kokoro voice, so it needs no key, and it switches the Voice selector to Kokoro. To hear the demo in ElevenLabs, switch the Voice and press Replay.
• Demo photo: your sample_room.jpg goes in web/public/sample/. Until it's there, the story plays but the photo reveal is skipped.
• render.yaml: a Render Blueprint for a static site. It builds web/, publishes web/dist, rewrites unknown paths to index.html, and sets the two headers that keep the voice multi-threaded when deployed.
• README: all ten sections from the plan, written. It also includes a table of what goes where, with the data-flow badge states.
• Smaller fixes: a ghost favicon (no more console 404), and a corrected Node requirement of 22.12+.
• ADR: ADR-038 records these decisions.

Judgement calls to review

• Gemma license: the README doesn't state it. The plan says Apache 2.0, but I couldn't verify that.
• Render config: I couldn't test it. If the deployed voice is single-threaded, or fonts or the voice download fail, suspect the two headers. Removing them makes the site work, just slower.
• README placeholders: the live URL, a screenshot or GIF, and your measured speed numbers are marked TODO.

Test by hand

Restart npm run dev, then check:

1. Try it now: use a fresh page, even with ElevenLabs selected earlier. It should load Kokoro and play the demo.
2. Voice selector: it should flip to Kokoro. Then switch to ElevenLabs and press Replay, which should re-render in that voice.
3. Photo reveal: add sample_room.jpg and check the reveal near the end.

What I need from you (the freeze is tonight at 20:00 IST)

1. sample_room.jpg in web/public/sample/.
2. Your approval of the draft demo story, or edits.
3. A yes to push to GitHub. Render deploys from the repo, and judges will need the repo link. It must be public, and I won't change its visibility.
4. Render: in the dashboard, choose New → Blueprint, select the repo, and it picks up render.yaml. Tell me the URL it gives you, and I'll check the deployed behaviour with you.
5. The .tech domain, if you're entering that category: it's optional. I can note the setup in the README once you know the domain.

The write-up counts most

Once the deploy works, I can draft the DEV post for you, covering what you built, how you read the theme, why open innovation mattered, and the Kokoro versus ElevenLabs comparison. I can also save this session with DevRelay for "Best Use of Entire". I won't publish or save anything without your yes.

I'm waiting for the photo, your story decision, and the push permission.

The story is written and committed locally (0893767). The new guard test passes along with the rest (145 tests). I haven't listened to it or seen the reveal in a browser.

The story

It's about 223 spoken words and plays for roughly 2.7 minutes, including the pauses.

Premise: You step into the room to get out of the rain. Every chair is pushed in except one. The left mirror shows the room as it once was: candles lit, a full table of pale figures all facing you. When you look back, there's a place laid at the pulled-out chair, with a card in your name written in handwriting that's still wet. A chair scrapes behind you, the lights go out, and you're sitting at the table. Your own photo then fades in, and you're told to look at the head of the table.

What it uses from your photo: the cobwebbed table, the candlesticks, the two gilt mirrors, the pulled-out chair, and the room's dust.

Effects: it uses eight sfx and light tags, which is the level-3 budget. It shows off whisper, slow and trembling, and it fades the drone out at the end.

Complete in one pass: arrival, escalation, a reveal, a twist, and an ending line that closes it, with no loose threads.

Things I did

• Renamed your photo: sample_image.jpg is now sample_room.jpg, the name the app looks for. It's small (about 50 KB), so the reveal will look a bit soft when scaled up.
• Added a guard test: it checks that the bundled story always parses cleanly, stays short, stays within the effects budget, and reveals the photo exactly once.
• ADR-039: the story and photo decisions are recorded.

For you to judge

• The last line: it tells the listener to look at the head of the table in the photo. I can't tell whether your photo actually suggests a figure there. If it doesn't, the line still works as a prompt, but you may want a different last line.
• Tone: it's implied threat, with no violence.
• Voices: please listen in both Kokoro and ElevenLabs. The [whispers] and [nervous] tags are the ones to check.

Next

Approve the story, or tell me what to change, such as a different ending or a shorter cut. Then I need the push permission and your Render details to deploy.

I'm waiting for your decision on the story and the push.

The repo is already public: GitHub answered an unauthenticated request with 200. That matters because judges can open it. It's about 01:20 IST on Saturday, which leaves about 18.7 hours to the feature freeze (Sat 20:00 IST) and about 35 hours to the deadline (Sun 12:29 IST).

Where we are

All code milestones, M0 to M6, are built and committed. Six commits are not on GitHub yet. Nothing is deployed, and the ElevenLabs and Try-it-now paths are only tested by you.

What's next, in order

1. Push to GitHub. I need your yes. The 6 commits include the demo story, the photo and the Render config. I'd also commit the explicit .env.local line your IDE added to .gitignore.
2. Deploy on Render. You do this in the dashboard: New → Blueprint, pick the repo, and it reads render.yaml. Send me the URL, and I'll check the live site with curl: the page loads, the demo files are served, unknown paths rewrite to the page, and the two isolation headers are present.
3. Check it in a browser. On a fresh browser, press Try it now. Settings should say "multi-threaded". If the voice download or fonts fail, the isolation headers are the first suspect.
4. Record the demo. About 60–90 seconds works: Try it now, the photo reveal, the Kokoro versus ElevenLabs comparison, and the privacy badge.
5. Write the DEV post. This counts most. I can draft it: what you built, how you read the theme, the privacy table, the speed and open-source story, and what I learned. You review and publish with #hf26challenge, and tick the Gemma, ElevenLabs, Render, Entire and .tech categories you actually use.

Optional, if time allows

• Domain: the .tech domain and its DNS CNAME for Render. This only matters for that category.
• A model that can read photos: pull the Ollama vision build, so your local story mode uses your photos. It isn't needed for the demo.
• README placeholders: the live URL, a screenshot or GIF, and your measured speed numbers.
• A decision I still need: when the model can't read photos, should the reveal still fire with your own photo? That keeps the best moment working with a text-only model.

Suggestion

It's after 01:00. The push takes a minute, but the deploy and testing are better done after you've slept, since there's time. If you'd rather keep going, say yes to the push and I'll do it now.

Should I push now?

Written for: DEV readers and the challenge judges, in your voice (first person). Edit it freely. Brackets mark the things only you can fill in.

The demo: no voice-over

The story is the voice-over, so narrating on top would talk over the main thing you're showing. Let the app's own audio play, and put short text captions on screen for the 3–4 moments that need explaining. The post does the technical explaining.

Format: a 60–90 second video with sound (YouTube, unlisted, embedded in the post). Add a silent 5–10 second GIF of the photo reveal for the cover or the top of the post. A GIF can't carry the experience on its own, because the experience is audio.

Shot list:

1. 0–5 s: the landing page, with the 🟢 "Fully local" badge visible. Caption: "A ghost story, set in your own place, running in your browser."
2. 5–10 s: press Try it now, accept the photosensitivity notice, and show "Summoning…". Cut the wait.
3. 10–60 s: the story full-screen, with the flicker, blackout, whispers and the photo reveal, which is the money shot. Keep the ending line.
4. 60–75 s: switch the Voice to ElevenLabs and press Replay. Play one or two whispered lines so the contrast is obvious. Caption: "Same story. Free offline voice vs. ElevenLabs."
5. 75–85 s: Memories (notes and photos), then Tell me a story with the draft text streaming in, sped up 4×. Caption: "Gemma 4, running locally through Ollama."
6. 85–90 s: click the badge to show the "what goes where" table. End card: localghost.mrituanjay.tech and "Works with the internet off."

Tools: Win+G (Xbox Game Bar) records the browser window with its sound, and Clipchamp, built into Windows, handles the trimming, captions and speed-up. Record at 1080p, keep it under 2 minutes, and use headphones while recording so nothing leaks into the mic.

Title options

1. Localghost: a ghost story that knows where you live (and runs on your laptop) ← my pick
2. Localghost: an open-source AI that tells you a horror story set in your own room
3. I built a haunted storyteller that runs entirely in the browser

Cover image: a 1000×420 frame of the photo reveal: your dining room, red-tinted and grainy, with the title in the serif font.

A small incremental parser turns that into events while the model is still streaming. Small models break formats, so the parser is lenient on purpose. It accepts [lights pulse], fixes [sfx scrapes] to scrape, and drops what it can't understand instead of crashing. If the output has no tags at all, the director retries once with a correction.

Scare levels are more than adjectives. Each level sets the length, the number of jump scares, an effects budget, which lights are allowed (level 1 gets only a gentle pulse), the kind of ending, and a content ceiling. Each story also gets a random original premise seed and a required story shape, so it ends properly instead of trailing off.

Your photos. If the model can see images, each photo goes through a "set designer" pass that lists what's physically there: the coat on the door, the dark window, the chair pulled out. Those details are written into the story, and the most unsettling photo is chosen for the climax reveal. The app asks the server whether the model supports images (the first Gemma build I pulled didn't) and falls back to your notes if it can't.

Making a free voice fast enough (the hard part). On my CPU-only laptop (Ryzen 7 7730U, 16 GB), Kokoro took around 5–11 seconds to synthesize a sentence that plays for 3–4 seconds. A few things got it usable:

• WebGPU produced pure crackle on my AMD iGPU. Gemma wasn't the problem; the culprit was the fp32 WebGPU path. I fell back to WebAssembly with 8-bit weights.
• ONNX Runtime Web was quietly running on a single thread, because the page wasn't cross-origin isolated. Two response headers (COOP + COEP: credentialless) unlocked 4 threads.
• Streaming playback: the story starts speaking while the rest is still being synthesized. It estimates how much buffer is needed, and if the playhead catches up it holds ("the voice is gathering itself…") and resumes on its own. Audio, lights and captions share one clock, so they stay in sync.
• An optional pool of voice workers, a one-time speed test that scales story length to your machine, and an audio cache, so replays are instant.

Optional premium voice. ElevenLabs (eleven_v4) is a plug-in adapter, not a requirement. The same story can be re-rendered with it from a toggle, and the whispers and sighs map to its audio tags. Kokoro then emulates those styles itself with filters, pitch wobble and soft clipping. Honestly, ElevenLabs sounds better. Kokoro does remarkably well for something free that runs entirely in a browser tab.

Why Does Open Innovation Matter?

Because the scariest input is your home.

A closed API would mean uploading photos of your bedroom to someone's server so it can make you feel unsafe in that bedroom. That's backwards. With open weights:

• Your photos never leave your machine in the default setup. A badge in the top bar always says where your data is going: 🟢 fully local, 🟡 story text sent to ElevenLabs, 🔴 photos sent to a remote model, with a click-through table of exactly what goes where.
• It works with the lights, and the internet, off. That's exactly when you want a ghost story.
• It can't disappear. No subscription, no API being retired, no keys. Clone it, run npm i, point it at Ollama.
• I could see inside it when it broke. Open models and open runtimes let me find that WebGPU was producing noise and that WebAssembly was stuck on one thread, and fix both. With a closed TTS API, "it's slow" is just the price.

Where closed did win: ElevenLabs is more expressive and much faster. So it's an optional adapter, with your key, kept in your browser.

My Agent Session

I built this with Claude Code as a pair programmer, working milestone by milestone from a written plan, testing every stage by hand, and logging decisions in an ADR file. Entire captured the sessions as checkpoints alongside the commits.

[AGENT SESSION EMBED]

The Spirit of It

[YOUR MOMENT. For example: the first time your own room faded in on screen at the end of a story, or someone you played it for. What did they say? One short, honest paragraph is enough.]

Prize Categories

• Best Use of Gemma: Gemma 4 writes every story and, when it can see, studies your photos.
• Best Use of ElevenLabs: an optional eleven_v4 voice with audio-tag direction, a restricted key and per-segment caching.
• Best Use of Render: a static site deployed from a Render Blueprint, with the cross-origin isolation headers that make the in-browser voice multi-threaded.
• Best Use of Entire: agent sessions captured as checkpoints alongside the commits.
• Best Use of a .tech Domain: https://localghost.mrituanjay.tech

Built with help from an AI coding agent (Claude Code). The design decisions, testing and judgement calls are recorded in the repo.

I've rewritten the README and committed it locally (1cb11ea, one commit ahead of GitHub, not pushed). It's documentation only, so I didn't re-run the tests.

For the 75–85 s shot: use gemma3:4b

I checked the Ollama library pages, so these sizes are confirmed:

Why gemma3:4b:

• Memory: it's the smallest Gemma that can read your photos. That matters with only about 3.6 GB free while browsers are open.
• Visible writing: Gemma 4 thinks first by default, and that thinking is hidden, so the shot would show a blank "Summoning…" for a while. Gemma 3 starts writing straight away, which is the shot you want on video.
• Still Gemma: it counts for the Gemma prize category.

Steps:

1. Run ollama pull gemma3:4b, then ollama show gemma3:4b and check that vision is listed.
2. Close heavy tabs. Zen alone was using about 2.5 GB.
3. In Settings, set Model to gemma3:4b, then press "Check model for image support".

One edit to the post: it currently says Gemma 4 E4B writes every story. If the video shows gemma3:4b reading photos, change that to something like: "Gemma 4 E4B for text-only stories; Gemma 3 4B when I want it to see my photos on a 16 GB laptop." That's honest, and it shows the app works across models.

What the README now covers

• Live URL: it links to localghost.mrituanjay.tech at the top and in Try it now.
• Full setup: the model choice by RAM, CORS commands for Windows, macOS and Linux, the app setup step by step, and LM Studio and llama.cpp as alternatives.
• ElevenLabs: the two key permissions, the credit quota, no IP allowlist, and the dev-only .env.local.
• Deploy your own: the Render Blueprint, why the two headers matter, and the custom domain CNAME.
• A controls table and 17 known gotchas. Each one is a symptom, its cause and the fix, all from problems we hit this week: CORS and restarts, the Vite port changing, a hosted page calling localhost, the missing vision projector, Gemma 4's hidden thinking, WebGPU crackle, single-threaded voice, slow first words, worker oversubscription, ElevenLabs errors, tag spellings, the .env.local restart, cache size, and private windows.

⚠️ One thing to check: the demo photo's rights

I removed a line I'd written claiming the photo was yours. It looks like a professional photo of an abandoned mansion, and it's now public in the repo and on the live site. If you downloaded it, you need a license that allows reuse, plus a credit. Fill in the README's TODO credit line, or swap in a photo you took yourself. The story isn't tied to it: any dim room with a table and a mirror works with small edits.

To finish

1. Push the README commit. Render will auto-deploy it, and it's harmless.
2. Pull gemma3:4b and record the demo.
3. Fill in the post's brackets: the video, your Spirit paragraph, the agent session, and the model line.

As before, I can stage the post as an unpublished DEV draft, save this session through DevRelay, or push the README, each only with your yes.

Does Ollama support image input natively?

Yes. Ollama accepts images two ways, but only for models whose capabilities list vision:

• The OpenAI-style endpoint /v1/chat/completions, with the image as an image_url content part holding a base64 data: URL. This is what the app uses.
• Ollama's own endpoint /api/chat, with an images: [base64…] array on the message.

I just tested the first one on your machine, with gemma3:4B and the demo photo, using exactly the app's request shape:

The object list is accurate. Its "unsettling details" came out vague ("the feeling of being watched") rather than concrete. That's fine, since the story mainly uses the objects.

A note on that CORS result: your OLLAMA_ORIGINS seems to be *. While Ollama is running, any website you visit can use it. That's fine for this weekend, but after the deadline narrow it to http://localhost:5173,https://localghost.mrituanjay.tech.

Noted on the photo; I'll leave it as is. The README still has a TODO credit line, so delete it or fill it in.

Testing steps

A. Your own story with photos (local)

1. In Settings, Story model: set Model to gemma3:4B. Set "Read my photos" to Auto, then press Check model for image support. It should say the model reports image support.
2. In the Memories tab, add one or two photos. Start with one, since each takes about 1.5 minutes the first time. Add a few notes.
3. Set Scare to 3 and press Tell me a story. Expect "Studying your place — photo 1 of 1" for about 1.5 minutes, then the draft text streaming in. Close heavy browser tabs first, because the model and the voice share your RAM.
4. Check:
   • The story names objects from the photo (mirrors, table, drapes, fireplace).
   • Your photo fades in at the climax.
   • The Debug tab shows vision room_1, the premise, and finish: stop.
5. Press Tell me a story again with the same photos. "Studying your place" should be almost instant, because the description is cached.

B. Voices 6. On the same story, switch Voice to ElevenLabs and press Replay. Compare the whisper lines. 7. Press Replay again. It should start immediately, and the Debug tab should show cache hits.

C. The live site, as a judge would see it 8. Open https://localghost.mrituanjay.tech in a private window, which acts like a first-time visitor with no cache. Press Try it now and check:

• The notice appears.
• The voice model downloads with progress.
• The dining-room story plays with flicker, blackout and the photo reveal.
• Settings says "multi-threaded".
• The badge is 🟢.

1. If you have time, try your phone too. It should play, slowly.

D. Quick checks 10. Space pauses, Esc stops, Reduce flashing turns the flicker into slow dims, and the controls fade and come back when you move the mouse.

Tip for recording

Run step 3 once before recording, so the photo description is already cached. Then the recorded run skips the 98-second wait and goes straight to writing. Also note how long the story takes to start in the Debug tab, since that number is useful for the post.

Yes, the old plan still works. Now that photos work, I'd change one thing: lead with your own photo, because that's the part no other entry will have. Here's the revised plan, with the waits handled.

The trick that removes the waits

A new story has to be generated and voiced live, which takes minutes on your laptop. So record in two takes:

• Take A (setup): add the photo, press Tell me a story, and record "Studying your place" and the draft text streaming in. You'll speed this up 4–8× in editing.
• Take B (performance): once that story has finished, press Replay. It plays instantly from the cache with no waiting. Record this one full-screen (F11) as the main performance.

Run "Tell me a story" once with your photo before Take A, so the photo description is cached and "Studying your place" takes seconds instead of 98.

Revised shot list (about 90 seconds)

Which voice for the main performance: I'd use Kokoro. It's free and offline, so it's the core of the "open" story. ElevenLabs then appears as the optional upgrade. If you'd rather the video simply sound its best, do Take B in ElevenLabs and show Kokoro in the comparison clip instead. Either is fine; just say in the post which one you used.

Be honest about the cuts: a small caption like "(waiting trimmed: about N min on a CPU laptop)" during the sped-up part reads well to judges. It's also true to your post, which talks about speed openly.

Before you hit record

• Story length: set it to Auto or Short, so the performance lasts about 1.5–2 minutes.
• Distractions: turn on Windows Focus Assist, close other tabs, and full-screen the browser.
• Recording: Win+G (Game Bar) captures the window with its sound. Wear headphones so nothing feeds back into the mic.
• For the ElevenLabs clip: the first Replay in ElevenLabs costs credits for that story, about 1–2k characters. A second Replay is free and has no wait.
• Editing: use Clipchamp for trimming, captions and speed changes. Export at 1080p, upload to YouTube as unlisted, and pull a 5–10 second GIF of the photo reveal for the cover.

One edit to the post

Change the model line to something like: "Gemma 3 4B (vision) reads my photos and writes the story locally; the app also runs Gemma 4 E4B for text-only stories." That matches what the video shows.

You don't strictly need it. Take B is insurance against three things that can go wrong in a single continuous take on your laptop:

1. The wait before the voice starts. After the text is written, Kokoro has to synthesize most of the story before playback starts, because it's slower than real time on your CPU. That can mean minutes of "Summoning…". You can cut it in editing, but it makes for a long, messy recording.
2. Holds in the middle of the story. Synthesis keeps running while the story plays. If it falls behind, playback pauses on "the voice is gathering itself…". That's fine for a user, but it breaks the tension in a video, and you can't cut it out cleanly once the music and lights are mid-scene.
3. CPU contention. Recording the screen, the browser, Kokoro and Ollama all compete for the same CPU. That makes both of the above more likely and can make the recording itself stutter.

On Replay, every line is already cached, so the story starts at once and plays straight through. The timing of the audio, lights and the photo reveal is exactly as designed.

When you can skip Take B:

• ElevenLabs: if you do the performance in ElevenLabs, it synthesizes much faster than real time. Take A will usually start in seconds and play without holds.
• A clean Take A: if Take A happened to start reasonably fast and played through without a "gathering itself" pause, just use it and trim the wait.

So record Take A first. If it played cleanly, you're done. If it didn't, press Replay and record that.

Nice work, that's a big milestone. 4:47 is fine as a complete recording, but most judges will watch the first 30–60 seconds and then decide. So trim it, while keeping the story itself intact.

Suggested edit: about 3 minutes

If you're short on time, don't edit at all. Upload the 4:47 with YouTube chapters in the description so judges can jump straight in. For example:

Use your own timestamps. YouTube needs the first one to be 0:00 and at least three chapters.

For the post

1. Upload to YouTube as unlisted, then put {% embed https://youtu.be/... %} in the Demo section, with one line under it: "Jump to 0:40 for the story; headphones recommended."
2. Make a 5–10 s GIF of the photo reveal. Put it at the top of the post or use a still as the cover. Most readers won't press play, but they'll see the GIF.
3. Fill the brackets: the model line (Gemma 3 4B for vision), your Spirit paragraph, and the agent session.
4. Push the README commit, which is still local, so the repo the judges see has the full setup guide and the live URL.

The deadline is Sun 1 Nov 12:29 IST (Oct 31, 11:59 PM PDT). Publish with time to spare, since DEV posts sometimes need a moment to show up under the challenge tag.

Once you have the YouTube link, I can do three things, each only with your yes: stage the post as an unpublished DEV draft through DevRelay, save this session to get a real agent-session embed, and push the README.

Written for: DEV readers and the challenge judges, in your voice. Two short placeholders remain that only you can fill: your Spirit paragraph and the agent-session embed.

Title: Localghost: a ghost story that knows where you live (and runs on your laptop) Cover image: a 1000×420 still of the photo reveal (your dining room, red-tinted and grainy), with the title in a serif font.

A small incremental parser turns that into events while the model is still streaming. Small models break formats, so the parser is lenient on purpose. It accepts [lights pulse], fixes [sfx scrapes] to scrape, and drops what it can't understand instead of crashing. If the output has no tags at all, the director retries once with a correction.

Scare levels are more than adjectives. Each level sets the length, the number of jump scares, an effects budget, which lights are allowed (level 1 gets only a gentle pulse), the kind of ending, and a content ceiling. Each story also gets a random original premise and a required story shape, so it ends properly instead of trailing off.

Your photos. Each photo is downscaled and stripped of EXIF in the browser, then goes through a "set designer" pass that lists what's physically there: the mirrors, the drapes, the cobwebbed table. On my CPU that takes about a minute and a half per photo the first time, and then it's cached forever. Those objects are written into the story, and the most unsettling photo is chosen for the climax reveal. The app first asks Ollama whether the model can actually see images. The first Gemma build I pulled couldn't, and instead of sending photos to a model that can't use them, it falls back to your notes.

Making a free voice fast enough (the hard part). On my CPU-only laptop (Ryzen 7 7730U, 16 GB), Kokoro took around 5–11 seconds to synthesize a sentence that plays for 3–4 seconds. A few things got it usable:

• WebGPU produced pure crackle on my AMD iGPU. The fp32 WebGPU path was the culprit, so I fell back to WebAssembly with 8-bit weights.
• ONNX Runtime Web was quietly running on a single thread, because the page wasn't cross-origin isolated. Two response headers (COOP + COEP: credentialless), set in both Vite and the Render Blueprint, unlocked 4 threads.
• Streaming playback: the story starts speaking while the rest is still being synthesized. It estimates how much buffer is needed, and if the playhead catches up it holds ("the voice is gathering itself…") and resumes on its own. Audio, lights and captions share one clock, so they stay in sync.
• An optional pool of voice workers, a one-time speed test that scales story length to your machine, and an audio cache, so replays are instant.

Optional premium voice. ElevenLabs (eleven_v4) is a plug-in adapter, not a requirement. The same story can be re-rendered with it from a toggle, and whispers and sighs map to its audio tags. Kokoro emulates those styles itself with filters, pitch wobble and soft clipping. Honestly, ElevenLabs sounds better. Kokoro does remarkably well for something free that runs entirely in a browser tab.

Why Does Open Innovation Matter?

Because the scariest input is your home.

A closed API would mean uploading photos of your bedroom to someone's server so it can make you feel unsafe in that bedroom. That's backwards. With open weights:

• Your photos never leave your machine in the default setup. A badge in the top bar always says where your data is going: 🟢 fully local, 🟡 story text sent to ElevenLabs, 🔴 photos sent to a remote model, with a click-through table of exactly what goes where.
• It works with the lights, and the internet, off. That's exactly when you want a ghost story.
• It can't disappear. No subscription, no API being retired, no keys. Clone it, run npm i, point it at Ollama.
• I could see inside it when it broke. Open models and open runtimes let me find that WebGPU was producing noise, that WebAssembly was stuck on one thread, and that my first Gemma build had no vision, and fix all three. With a closed TTS API, "it's slow" is just the price.

Where closed did win: ElevenLabs is more expressive and much faster. So it's an optional adapter, with your own restricted key, kept in your browser.

My Agent Session

I built this with Claude Code as a pair programmer, working milestone by milestone from a written plan, testing every stage by hand, and logging each decision in an ADR file. Entire captured the sessions as checkpoints alongside the commits.

The Spirit of It

Prize Categories

• Best Use of Gemma: Gemma writes every story locally through Ollama, and Gemma 3 4B reads your photos to set the story in your real room.
• Best Use of ElevenLabs: an optional eleven_v4 voice with audio-tag direction, a restricted key and per-line caching.
• Best Use of Render: a static site deployed from a Render Blueprint, with the cross-origin isolation headers that make the in-browser voice multi-threaded.
• Best Use of Entire: agent sessions captured as checkpoints alongside the commits.
• Best Use of a .tech Domain: https://localghost.mrituanjay.tech

Built with help from an AI coding agent (Claude Code). The design decisions, testing and judgement calls are recorded in the repo.

I'll write the post into a markdown file first, then load the DevRelay sessions workflow to save this session and get the real embed.

The post is saved, but I couldn't save the agent session to DEV. Claude Code's safety check blocked the upload, so the post's agent-session section is still a placeholder.

The post file

Docs/DEV_POST.md is committed locally (b8662c6, not pushed). It has the full final write-up, with the 1.5× note under the video and your YouTube link embedded. The title, cover suggestion and tags are in a comment at the top. The two TODO comments, for the session embed and the Spirit paragraph, are still for you.

Where the session upload stopped

1. Found the transcript: this whole session is a 5.9 MB file. There's also a smaller one from 29 Oct, where you set up the Entire CLI.
2. Scrubbed a copy at scratchpad/session-scrubbed.jsonl. I removed your email, the ElevenLabs promo code, your MLH user ID, and every path under your user folder (944 replacements). I also scanned for anything that looks like an API key or token and found none. The ElevenLabs key itself was never printed anywhere in the session.
3. Got an upload link from DevRelay, but the step that sends the file to DEV's storage was blocked. I didn't try another route, such as the DevRelay submit tool, because that would be the same action.

How you can proceed

• Allow it and I retry: approve this kind of upload (or add a Bash permission rule for it), then tell me to try again. I'll get a fresh upload link, since this one expires in 15 minutes, re-scrub so the latest messages are included, and send you the {% agent_session … %} embed.
• Upload it yourself: if you have the DevRelay CLI or DEV's own session upload, use the scrubbed file above. Don't use the raw one, which still has your email and the promo code in it.
• Skip it: delete the session TODO line in the post. The Entire checkpoints in the repo already show the process, and the section is optional.

Whichever you pick, publish before Sun 1 Nov 12:29 IST, and push the README and post commits so the repo matches the post.

I've added the Entire links to the post (committed locally as e38bd9b). They won't work for judges until you finish the steps below. Right now https://entire.com/gh/MJ-NJ/localghost returns "not found" to a signed-out visitor.

What I added to the post

• "My Agent Session" section: one line explaining what Entire records, plus a link to the repo's page on Entire.
• Four turning points, each linked to its commit on Entire:
  • the WebGPU crackle diagnosis and the switch to WebAssembly
  • the single-thread discovery
  • streaming playback
  • only sending photos to models that can see
• The Entire prize line: it now points at the same page.

Making the links work

1. Push: run git push origin master. That sends the last few commits plus the 2 checkpoints still on your machine; the other 26 are already on GitHub as hidden refs/entire/... refs.
2. Connect the repo on Entire's site: sign in at entire.com with GitHub and connect MJ-NJ/localghost, which may mean installing their GitHub app. I can't do this step; the Entire CLI marks it as yours.
3. Make it public: check the repo's visibility on Entire is set to public. entire repo visibility get shows it.
4. Check as a judge would: open https://entire.com/gh/MJ-NJ/localghost and one of the commit links in a private window. If they only work while you're signed in, judges won't see them.

The commit-link format comes from Entire's docs; I couldn't confirm the repo-page link myself while it returns 404. If the links still don't open publicly after these steps, you have two fallbacks:

• Narrower links: link to the checkpoint history on GitHub instead.
• The DEV embed: if you allow the transcript upload I tried earlier, I can save the session on DEV and paste in the real embed, using the scrubbed copy.

Sources:

• Entire docs: review a checkpoint
• Entire docs: sessions
• Entire docs: core concepts