# Story Intelligence Illustration Selection Technical Explanation

This document explains how Story Intelligence currently chooses illustrations, tracing the execution path from scene analysis through Story Memory, illustration matching, demand generation, snapshot construction, and browser rendering.

No code changes are described here as recommendations only; this is documentation of the current implementation.

## Complete Execution Path

```mermaid
flowchart TD
    A[PersistedStoryIntelligenceRunner] -->|Chapter prompt AI call| B[Chapter Structure JSON]
    B -->|Scene boundaries| C[Scene Intelligence prompt AI calls]
    C -->|ParsedJson saved| D[StoryIntelligenceSceneResults]
    D --> E[StoryIntelligenceVisualisationSnapshotService]
    E -->|Build scene states and Story Memory| F[Prototype ViewModel]
    F --> G[IllustrationLibraryService.ApplyApprovedIllustrationsAsync]
    G -->|Exact stable-code lookup| H[Starter/generated image if code matches]
    H --> I[StoryIntelligenceIllustrationMatchingService]
    I -->|Characters| J[Assignment revalidation and scoring]
    I -->|Locations/assets| K[Demand lookup/creation]
    J --> L[StoryIntelligenceCharacterIllustrationAssignments]
    K --> M[StoryIntelligenceIllustrationDemands and queued IllustrationLibraryItems]
    L --> N[Visualisation JSON]
    M --> N
    N --> O[Browser visualisation]
```

```mermaid
sequenceDiagram
    participant Scene as Scene Analysis
    participant Memory as Story Memory
    participant Match as Illustration Matcher
    participant Demand as Demand Generator
    participant Snap as Snapshot Builder
    participant UI as Visualisation

    Scene->>Scene: OpenAI chapter + scene prompts
    Scene->>Snap: Persisted scene result JSON
    Snap->>Memory: Replay analysed scenes in order
    Memory-->>Snap: Carried characters, relationships, POV
    Snap->>Match: Character/location/asset view model
    Match->>Match: Read library + previous assignments
    Match->>Demand: No compatible illustration
    Demand->>Demand: Upsert demand + queue planned image
    Match-->>Snap: Assigned/fallback image resolution
    Snap-->>UI: Snapshot JSON
    UI->>UI: Render imagePath/fallback
```

## Main Services and Responsibilities

### PersistedStoryIntelligenceRunner

Responsible for running Story Intelligence analysis and persisting results.

Input:

- Queued `StoryIntelligenceQueuedRun`
- Chapter source text
- Chapter Structure prompt
- Scene Intelligence prompt
- OpenAI model settings

Output:

- Persisted chapter structure result
- Persisted scene intelligence result
- Run progress/status

Database reads:

- Queued Story Intelligence run via `StoryIntelligenceRun_ClaimNextPending`
- Current run state for cancellation checks

Database writes:

- `StoryIntelligenceRuns`
- `StoryIntelligenceChapterResults`
- `StoryIntelligenceSceneResults`

AI calls:

- Chapter Structure prompt
- Scene Intelligence prompt per scene
- JSON repair retry prompt if parsing fails

Illustration role:

- None directly. It only creates the structured scene data later consumed by the visualisation and matcher.

### StoryIntelligenceVisualisationSnapshotService

Responsible for reconstructing visualisation state from persisted analysis.

Input:

- Import session ID or run ID
- Persisted run and scene-result records
- Parsed scene JSON

Output:

- `StoryIntelligenceExperiencePrototypeViewModel`
- Scene states with characters, location, assets, relationships, knowledge, activity, discoveries
- In-memory Story Memory

Database reads:

- `StoryIntelligenceBookPipelines`
- `StoryIntelligenceRuns`
- `StoryIntelligenceSceneResults`

Database writes:

- None directly.

Cached/derived values:

- `ChangeToken`
- Session change token
- Replay cursor
- In-memory `StoryMemory`
- Stable visual IDs
- Stable fallback library codes

AI calls:

- None.

### IllustrationLibraryService.ApplyApprovedIllustrationsAsync

Responsible for the first image pass, using exact stable-code lookup.

Input:

- Prototype view model with `LibraryCode` values on characters, locations and assets

Output:

- Updated image paths and `ImageResolution` objects where a matching library item exists

Database reads:

- `IllustrationLibraryItems`

Database writes:

- None.

Matching behaviour:

- Exact `StableCode` match only.
- Uses `Approved` before `Generated`.
- Then newest item first.

### StoryIntelligenceIllustrationMatchingService

Responsible for final character assignment, revalidation, scoring, and demand creation.

Input:

- Project ID
- Fully built visualisation prototype view model
- Existing illustration library
- Existing active character assignments

Output:

- Character image assignments
- Character assignment diagnostics
- Demand records when no suitable illustration exists
- Queued planned illustration items when demand threshold is reached

Database reads:

- `IllustrationLibraryItems`
- `StoryIntelligenceCharacterIllustrationAssignments`

Database writes:

- `StoryIntelligenceCharacterIllustrationAssignments`
- `StoryIntelligenceIllustrationDemands`
- `IllustrationLibraryItems` through planned/queued demand records

AI calls:

- None directly.
- It queues illustration generation. A background image-generation worker later calls OpenAI Images.

### IllustrationGenerationRunner and OpenAIIllustrationImageProvider

Responsible for generating queued illustration-library images.

Input:

- Queued `IllustrationLibraryItems`
- Prompt/specification JSON

Output:

- Stored generated image files
- Updated illustration-library item status

Database reads:

- `IllustrationLibraryItems`

Database writes:

- `IllustrationLibraryItems`

AI calls:

- `POST https://api.openai.com/v1/images/generations`

## Character Illustration Selection

Character matching is a two-layer process.

First, `StoryIntelligenceVisualisationSnapshotService` gives each character a likely starter `LibraryCode`. Then `IllustrationLibraryService.ApplyApprovedIllustrationsAsync` resolves exact library-code matches. After that, `StoryIntelligenceIllustrationMatchingService.ApplyCharacterIllustrationsAsync` performs the newer assignment and revalidation pass.

### Where Age Is Determined

Age is determined in two places.

Initial stable-code age:

- Class: `StoryIntelligenceVisualisationSnapshotService.CharacterProfile`
- Method: `DetectAgeBand`
- Used only to pick a starter code.
- Defaults to `YoungAdult`.

Final matcher age:

- Class: `StoryIntelligenceIllustrationCompatibility`
- Method: `InferCharacterEvidence`
- Produces a `CharacterEvidenceProfile`.
- Used by `StoryIntelligenceIllustrationMatchingService.Score`.

### Where Inferred Age Is Determined

Inferred age comes from:

- Explicit age terms: `13`, `14`, `15`, `16`, `17`, `18`, `19`
- Words: `toddler`, `baby`, `infant`, `child`, `schoolboy`, `schoolgirl`, `teenager`, `young adult`, `middle-aged`, `elderly`
- Role/title terms: `Mr`, `Mrs`, `Miss`, `Ms`, `Dr`, `Professor`, `teacher`, `detective`, `police officer`, `nurse`, `receptionist`, `solicitor`, `landlord`, `manager`
- Relationship terms: `mother`, `father`, `parent`, `aunt`, `uncle`, `grandmother`, `grandfather`

### Where Gender/Presentation Is Determined

Presentation is determined by `StoryIntelligenceIllustrationCompatibility.InferCharacterEvidence`.

Sources, in priority order:

1. Title presentation
2. Entity relationship/presentation words in scoped text
3. Relationship evidence
4. Pronouns
5. Hard-coded name list
6. Unknown

### Whether Names Influence Gender

Yes. Names are hard-coded.

Examples:

- Feminine: Beth, Maggie, Rosie, Grace, Zoe, Rebecca, Annie, Mary, Sharon, Catherine, Susan, Helen, Elen
- Masculine: Adam, Colin, David, Kevin, Simon, Graham, Gareth, John, Michael, Peter, Richard, Rick, George

There is also a presentation guard that can override leaked relationship/pronoun evidence using name/title evidence unless explicit contradictory evidence appears.

### Whether Titles Influence Gender

Yes.

- `Mrs`, `Miss`, `Ms`, `Lady` infer feminine.
- `Mr`, `Sir` infer masculine.
- `Dr` and `Professor` infer adult age but not masculine/feminine presentation by themselves.

### Whether Occupations Influence Age

Yes. Occupations such as teacher, driving examiner, detective, police officer, nurse, receptionist, solicitor, landlord, manager, professor and doctor infer adult age.

### Hair Colour, Eye Colour and Ethnicity

Hair colour is extracted in `StoryIntelligenceIllustrationCompatibility.HairColour`.

Recognised values include:

- Bald
- Black
- Brown/brunette/dark hair
- Blonde/fair hair
- Red/auburn/ginger
- Grey/gray
- White

Hair length is also extracted.

Eye colour is not currently extracted for matching.

Ethnicity is not currently extracted for matching. The matcher uses skin tone, not ethnicity.

### Previously Learned Evidence

Previously learned Story Memory evidence can affect the character states that enter the matcher, but there is no persistent Story Memory evidence table read by the matcher.

Previous assignment evidence is stored as JSON in `StoryIntelligenceCharacterIllustrationAssignments`, but `ListAssignmentsAsync` currently returns `EvidenceJson` and `CandidateDiagnosticsJson` as null. That means previous evidence is not rehydrated and used as a true evidence source during matching.

### Conflict Resolution

Conflict handling is mostly rule-based:

- `FirstKnown` picks the first non-unknown signal from prioritised evidence.
- The presentation guard prevents obvious gender leakage from overriding strong name/title evidence.
- Hard rejections eliminate impossible illustration candidates.
- Existing assignments are revalidated against current evidence.
- Replacement requires hard failure, material wrongness, or a better score by margin.

### Hard Rules for Characters

A candidate is rejected if:

- Named character uses Unknown Figure artwork.
- Candidate is marked unsuitable for named-character matching.
- Age is incompatible.
- Strong presentation evidence contradicts candidate presentation.
- Known hair colour contradicts candidate hair colour.
- Candidate is already assigned to another significant named character.

Additional rule:

- If character age evidence is unknown, child/teen candidate artwork is rejected until explicit child/teen evidence appears.

### Character Weighted Scoring

Final matcher score in `StoryIntelligenceIllustrationMatchingService.Score`:

- Age exact match: `+25`
- Age evidence unknown: `+12`
- Candidate age unknown: `+8` plus rejection note
- Presentation exact match with strong evidence: `+42`
- Presentation exact match without strong evidence: `+25`
- Presentation evidence unknown: half weight
- Candidate presentation unknown: one-third weight plus rejection note
- Hair colour exact match: `+20`
- Hair colour evidence unknown: `+10`
- Candidate hair colour unknown: `+6` plus rejection note
- Skin tone exact match: `+20`
- Skin tone evidence unknown: `+10`
- Candidate skin tone unknown: `+6` plus rejection note
- Hair length exact match: `+8`
- Glasses exact match: `+7`
- Facial hair exact match: `+7`
- Approved library item: `+3`

Score is clamped between `0` and `100`.

Minimum suitable score:

- `48`

Reassignment margin:

- `24`

Older stable-code scoring before the final matcher:

- Age exact: `+80`
- Compatible age: `+30`
- Age mismatch: `-45`
- Presentation exact: `+24`
- Hair colour exact: `+18`
- Hair length exact: `+10`
- Signal keyword match: `+14` each

### Why Existing Illustrations Are Retained

An existing illustration is retained when:

- Existing assignment still resolves to a public image URL.
- Existing candidate has no hard rejections.
- Existing candidate is not materially wrong for known age, presentation, hair colour or skin tone.
- No better candidate exists, or the best candidate does not beat existing score by the reassignment margin.
- Existing score is at least `48`.

### Why Existing Illustrations Are Replaced

An existing illustration is replaced when:

- It violates hard constraints.
- It is materially wrong for age, presentation, hair colour or skin tone.
- A better compatible candidate beats the existing score by at least `24`.
- The existing assignment was fallback/null and a generated valid item is now available.

### When Demand Is Created

A character demand record is created when no hard-compatible unused illustration scores at least `48`.

Demand key:

```text
character|age|presentation|hairColour|skinTone
```

Writes:

- `StoryIntelligenceIllustrationDemands`

If request count reaches threshold, currently `1`, it creates or queues:

```text
char-demand-{hash}
```

### When Generated Illustrations Become Eligible

A generated illustration becomes eligible when:

- It exists in `IllustrationLibraryItems`
- Status is `Generated` or `Approved`
- It has a valid thumbnail or file path
- A later snapshot/matcher pass reads it from the library

### Why Duplicate Portraits Sometimes Occur

Duplicate portraits can occur because:

1. Exact stable-code resolution can apply the same starter code before final assignment.
2. Duplicate assignment cleanup only handles assignment records, not every transient fallback/stable-code image.
3. Low-significance or unresolved identities can share generic fallbacks.
4. Character identity canonicalisation can split one person into multiple keys.
5. Existing duplicate assignment handling keeps the first retained item and clears later duplicates only during the matcher pass.

### Why Unknown Figure Can Still Appear

Unknown Figure can still appear when:

- The initial stable-code phase chooses `char-unknown-figure`.
- The character is unresolved/generic, so named-character hard rejection does not apply.
- The matcher records demand and falls back to an existing SVG/fallback.
- The library contains generated/approved unknown figure art used by exact code before the final assignment overrides it.

## Location Illustration Selection

### Classification

Locations are built by `StoryIntelligenceVisualisationSnapshotService.BuildLocation`.

It chooses:

1. Present-in-scene locations before mentioned-only locations.
2. More specific names.
3. Higher confidence.
4. Fallback to scene setting.

### Semantic Typing

Semantic type is determined by `StoryIntelligenceIllustrationCompatibility.LocationType`.

Examples:

- Bathroom
- Kitchen
- Bedroom
- LivingRoom
- Hallway
- Stairwell
- Hospital
- Pub
- Beach
- RailwayStation
- Canal
- Garage
- HouseExterior
- Office
- School
- WaitingRoom
- Road
- CarPark
- Shop
- StreetFurniture
- FlatInterior
- UnknownInterior

### Matching

Location matching has two phases:

1. Initial stable-code mapping in `LocationLibraryCode`.
2. Demand-based exact generated lookup in `ApplyLocationDemandAsync`.

Important current behaviour:

- Bathroom, WaitingRoom and Road intentionally return no starter code.
- This prevents borrowing wrong generic interiors.
- Generated demand selection uses exact code `loc-demand-{hash}`.

There is a `LocationCompatible` helper, but generated supporting image selection currently uses exact demand stable-code lookup rather than weighted compatibility scoring.

### Demand Creation

Location demand is created if:

- Location type is not `UnknownInterior`.
- Current image is still fallback.
- No generated demand image exists for the exact demand code.

Demand key:

```text
location|type
```

Writes:

- `StoryIntelligenceIllustrationDemands`
- `IllustrationLibraryItems` planned/queued demand item

### Reuse Rules

Locations are reusable by type/demand key. They are not unique per manuscript location.

## Asset Illustration Selection

### Classification

Assets are built by `StoryIntelligenceVisualisationSnapshotService.BuildAssets`.

It:

- Excludes mentioned-only assets.
- Excludes low-value/background objects.
- Groups by cleaned asset name.
- Picks highest-confidence entry per name.
- Takes top five by asset weight.

### Semantic Typing

Asset type is determined by `StoryIntelligenceIllustrationCompatibility.AssetType`.

Examples:

- Ambulance
- Stretcher
- BodyBag
- Bottle
- Money
- Clothing
- Document
- Jewellery
- Weapon
- Hatchback
- FamilyCar
- SportsCar
- Van
- Car
- CarPart
- Tin
- Parcel
- Rucksack
- Suitcase
- Note
- Letter
- Notebook
- Book
- Telephone
- Keys
- Photograph
- UnknownObject

### Colour Handling

Colour is determined by `AssetColour`.

Recognised values:

- Blue
- Red
- Green
- Yellow
- Black
- White
- Grey
- Brown
- Unknown

### Matching

Asset matching has two phases:

1. Initial stable-code mapping in `AssetLibraryCode`.
2. Demand-based exact generated lookup in `ApplyAssetDemandAsync`.

Starter mappings include:

- Letter/Note/Document -> `asset-folded-letter`
- Notebook/Book -> `asset-worn-notebook`
- Car -> `asset-red-classic-car`
- Keys -> `asset-old-keys`
- Telephone -> `asset-phone-recorder`
- Photograph -> `asset-photo-print`
- Rucksack -> `asset-rucksack`

Generated demand selection uses exact stable code:

```text
asset-demand-{hash}
```

There is an `AssetCompatible` helper, but generated supporting-image selection currently uses exact demand code rather than weighted compatibility scoring.

### Demand Creation

Asset demand is created if:

- Asset type is not `UnknownObject`.
- Current image is fallback.
- No generated image exists for exact demand code.

Demand key:

```text
asset|type|colour
```

Writes:

- `StoryIntelligenceIllustrationDemands`
- `IllustrationLibraryItems` planned/queued demand item

## Database Table Summary

Reads:

- `StoryIntelligenceRuns`
- `StoryIntelligenceChapterResults`
- `StoryIntelligenceSceneResults`
- `StoryIntelligenceBookPipelines`
- `IllustrationLibraryItems`
- `StoryIntelligenceCharacterIllustrationAssignments`

Writes:

- `StoryIntelligenceRuns`
- `StoryIntelligenceChapterResults`
- `StoryIntelligenceSceneResults`
- `StoryIntelligenceCharacterIllustrationAssignments`
- `StoryIntelligenceIllustrationDemands`
- `IllustrationLibraryItems`

No dedicated persistent Story Memory table is currently used in the illustration matcher.

## Cached Values Used

The system uses these cached or derived values:

- Stored `ParsedJson` in scene results
- Stored illustration library metadata/specification JSON
- Stored generated file/thumbnail paths
- Stored active character assignments
- Stored demand keys/request counts
- Snapshot `ChangeToken`
- Replay cursor index
- In-memory Story Memory during snapshot construction
- Stable hashes for visual entity IDs and demand codes

## Previous Assignments Considered

The matcher considers active previous assignments from `StoryIntelligenceCharacterIllustrationAssignments`.

Used fields include:

- `CharacterKey`
- `IllustrationLibraryItemID`
- `StableCode`
- `MatchConfidence`
- `MatchingScore`
- `ReasonChosen`
- `ReasonReassigned`

Current limitation:

- Previous `EvidenceJson` and `CandidateDiagnosticsJson` are not returned by `ListAssignmentsAsync`; they are selected as null.

## Story Memory Evidence Considered

Story Memory can influence:

- Added remembered characters
- POV identity
- Character role/relevance
- Carried relationships
- Inferred kinship relationships
- Rough presentation and age band

Story Memory does not directly persist or supply a structured evidence object to the final matcher. It affects the visual character state, and the matcher then infers evidence from that state.

## Character Evidence Ignored

Currently ignored or mostly lost for matching:

- Eye colour
- Ethnicity
- Clothing details
- Body type
- Exact age beyond broad age band
- Many action-level descriptors
- Raw manuscript text
- Prior assignment evidence JSON
- Scene observations unless transformed into role/relevance/relationship text

## Five Biggest Weaknesses

1. Two image systems compete: stable-code starter matching and final assignment matching.
2. Story Memory is rebuilt in memory and is not a durable structured evidence store.
3. Character evidence is compressed before matching.
4. Locations and assets do not have a real weighted matcher.
5. Previous evidence is stored but not reloaded as evidence for conflict resolution.

## Five Places Evidence Can Be Lost

1. AI scene JSON may omit physical details.
2. Snapshot character state drops raw action and descriptor detail.
3. Story Memory stores only simplified role/relevance/age/presentation.
4. Matcher ignores eye colour, ethnicity, clothing, build and exact age.
5. Assignment repository does not return previous evidence JSON to the matcher.

## Five Places Incorrect Assignments Can Occur

1. Initial stable-code fallback picks a plausible but wrong starter image.
2. Hard-coded name gender lists misclassify unusual or ambiguous names.
3. Relationship/family terms leak onto the wrong character.
4. Asset/location keyword order maps vague text too broadly.
5. Canonical identity splitting creates multiple assignment keys for one person.

## Five Quickest Improvements

1. Store structured evidence per canonical identity and reload it during matching.
2. Remove or demote pre-matcher stable-code character assignment for live data.
3. Add weighted matchers for locations and assets.
4. Add eye colour, ethnicity/skin-tone source, clothing and exact-age evidence fields.
5. Expand diagnostics to show winning candidate, rejected candidates and exact evidence source per entity.