PlotDirector/docs/story-intelligence/PlotDirector-Story-Intelligence-Refactor-Plan.md

PlotDirector Story Intelligence Refactor Plan

Purpose

This document records the agreed direction for the next Story Intelligence refactor.

The current system is producing genuinely useful analysis, particularly canonical scene boundaries, structural summaries, generated scene titles and scene metrics. The remaining work is primarily about establishing stronger context and canonical identity earlier, improving downstream analysis quality, and replacing the long linear review wizard with a much lighter exception-based review model.

The guiding principle is:

Establish reliable context as early as possible, preserve narrative progression, and only ask the author to resolve information PlotDirector cannot safely determine itself.

A second equally important principle is:

Later analysis must consume facts PlotDirector already knows rather than independently rediscovering them.

The work should be implemented in dependency order. Do not begin by redesigning the review UI around data that is still ambiguous or poorly contextualised.

---

Core Architectural Principles

Canonical scenes remain the structural foundation

The existing Core Import architecture remains authoritative:

• Chapters are established first.
• Canonical scenes are detected and created.
• SceneManuscriptSources contains the canonical source text.
• Structural summaries are generated once and reused.
• Optional descriptive scene titles are generated in the same structural call.
• Word markers use canonical ChapterIDs and SceneIDs.
• Story Intelligence enriches those existing scenes rather than redetecting or recreating them.

Do not reintroduce duplicate scene-boundary analysis or duplicate scene summaries.

POV must become early structural context

POV is sufficiently important that it should be established during structural analysis, before detailed Story Intelligence enrichment.

Knowing the POV character allows PlotDirector to resolve otherwise ambiguous references such as:

• Narrator
• I / me / my
• my bedroom
• my car
• my mother
• my handbag

This in turn improves character identity, locations, assets, relationships, knowledge and emotional attribution.

CharacterID represents physical identity

A canonical CharacterID represents the actual person in the story, regardless of the names, descriptions or aliases used for that person.

For example, all of the following may ultimately refer to one CharacterID:

• Margaret Grant
• Maggie
• Miss Grant
• Rusty
• Grace
• Mrs Jackson

Similarly, an initially unidentified character such as The Man in Grey may later be established as Victor Harrington.

Canonical identity is important not only for naming but for physical continuity. The same CharacterID cannot normally occupy two incompatible locations at the same time merely because different aliases are being used.

Alias identity must not erase narrative history

When two character identities are merged, PlotDirector must establish the physical/canonical truth without destroying the way the character was presented in the manuscript.

If The Man in Grey is later resolved as Victor Harrington:

• Victor becomes the canonical CharacterID.
• The Man in Grey remains an alias/reference identity.
• Earlier scene evidence that used The Man in Grey remains preserved.
• Knowledge about that identity can vary by character and scene.

The existing Knowledge system is responsible for facts such as:

Maggie learns that The Man in Grey is Victor Harrington

There is no need for a parallel identity-reveal event system if the existing Knowledge model can express this.

Relationships are progressive, not static labels

This is critical.

A canonical relationship may persist between two CharacterIDs while its state changes through the story.

For example, Maggie and Rob may progress through states such as:

• strangers / first meeting
• acquaintances
• friendship
• attraction
• romantic interest
• couple
• engaged

PlotDirector must not flatten this into a single final relationship type and lose the progression.

The relationship model must preserve:

1. Canonical relationship identity between the two CharacterIDs.
2. Relationship events / state changes tied to canonical SceneIDs.
3. Evidence supporting each change.
4. The ability to determine the relationship state at any point in the story.

A later conclusion that Maggie and Rob are a couple must not rewrite earlier scenes as though they were already a couple.

Story Intelligence should accumulate evidence over the book and identify meaningful progression.

---

Stage 1 - Establish POV as Structural Context

POV should be determined during the existing chapter-level structural analysis.

Data model

POV ultimately belongs to the Scene because mixed-POV chapters exist.

However, each chapter may have a strong default POV which is used to seed its scenes.

The desired model is:

Chapter default POV -> scene inheritance -> explicit per-scene override

The chapter default must not permanently override individual scene values.

Structural analysis

The existing chapter-level structural AI call should attempt to identify:

• default chapter POV;
• confidence/evidence;
• per-scene POV override where a change is clearly detected.

Explicit manuscript cues such as:

Narrated by Maggie

should be treated as extremely strong evidence.

Do not add separate AI calls solely for POV if it can be returned by the existing structural analysis call.

Core Import review

The chapter review/structural flow should expose the proposed POV compactly.

For example:

Chapter 6 · Danger in the Dark POV: Maggie Grant

The author should normally do nothing.

If POV cannot be established:

POV: Not detected

with a dropdown allowing the author to select a canonical character.

For mixed chapters, structural analysis may propose per-scene overrides.

Scene persistence

When canonical scenes are created:

• scenes inherit the confirmed chapter POV by default;
• structural per-scene overrides are applied where appropriate;
• provenance should distinguish generated/inherited values from explicit author changes.

Normal Chapter UI

The Chapter page should eventually expose:

POV: Maggie

or:

POV: Mixed

and provide a convenience action such as:

Set POV for all scenes: [Character]

This must protect explicit author overrides appropriately.

Downstream rule

Once a Scene has a known POV CharacterID, later Story Intelligence must treat that as authoritative context.

Narrator should not survive as a separate character candidate when the narrator is already known.

---

Stage 2 - Build Proper Character Identity Merge and Alias Support

PlotDirector needs a reusable canonical character merge mechanism.

This is not an import-only feature.

Required behaviour

Given:

Character B -> merge into Character A

PlotDirector must consolidate all identity-dependent data onto Character A.

Investigate all current and future CharacterID references before implementation.

Likely affected areas include:

• SceneCharacters
• character aliases
• relationships
• relationship events
• knowledge
• asset ownership/associations
• location associations where character-based
• evidence/provenance
• Story Intelligence review candidates
• other CharacterID-backed tables

Do not simply copy the alias string and delete the old character.

Preserve provenance

If the manuscript previously referred to Character B as The Man in Grey, preserve that reference evidence against the canonical character after merge.

The canonical physical identity becomes Character A, while the manuscript reference remains historically meaningful.

Normal Character UI

The standard Character pages should support:

Merge with another character

This operation must use the same canonical merge service used by Story Intelligence review.

Do not create a special import-only merge implementation.

Safety principle

False merges are worse than temporary duplicate identities.

When identity is genuinely uncertain, PlotDirector should prefer to keep identities separate and ask the author later.

---

Stage 3 - Redefine Character Review as Mandatory Identity Resolution

Characters are the one required Story Intelligence review prerequisite.

The purpose is no longer:

Review every character found by AI.

It becomes:

Resolve the character identities PlotDirector could not determine confidently.

Existing canonical characters are the anchor

Story Intelligence must receive and respect:

• canonical CharacterIDs;
• canonical names;
• known aliases;
• deterministic SceneCharacter mappings;
• Scene POV;
• scene evidence.

Automatic cases

Examples that should normally require no human decision:

• known canonical character matches;
• known aliases;
• punctuation variants such as Mrs Patterson / Mrs. Patterson;
• Narrator where Scene POV is known;
• obvious case/spacing/title normalisation.

Genuine review cases

Examples:

Rusty

Possible match: Maggie Grant.

Choices:

• Alias of Maggie Grant
• Keep as separate character
• Another existing character
• Ignore

Creepy Guy

If recurring/significant, this may legitimately remain an unidentified canonical character.

Choices might include:

• Keep as unidentified character
• Alias of existing character
• Ignore

Do not discard unnamed characters merely because they lack a conventional name.

Pat

If identity is known but a useful core attribute cannot be determined:

• Female
• Male
• Other/appropriate supported value
• Unknown

Only ask where the information is genuinely useful and unresolved.

Review workload

The UI should communicate something like:

91 identities resolved automatically 9 need your help

not present 100 already-resolved characters for approval.

Gate

Once required character ambiguities are resolved, the remainder of Story Intelligence review becomes available independently.

---

Stage 4 - Improve Story Intelligence Scene Context

Detailed Story Intelligence currently appears to lose contextual information that the chapter-level structural pass understands.

Each canonical scene analysis should receive a compact context package built from existing persisted data.

At minimum investigate supplying:

• BookID / ChapterID / SceneID
• chapter title
• canonical scene title
• Scenes.StructuralSummary
• full current SceneManuscriptSources.SourceText
• resolved Scene POV CharacterID/name
• structural Setting
• deterministic SceneCharacter mappings
• relevant canonical characters and aliases
• previous scene title + structural summary
• next scene title + structural summary
• Story Era
• Primary Locale

If a concise chapter structural summary already exists, consider including it.

Do not resend the complete chapter for every scene unless evidence proves that is necessary.

The aim is to provide narrative surroundings cheaply.

Established facts are facts

Later prompts should be explicitly told which supplied values are canonical/authoritative and must not be rediscovered or competed with.

Examples:

• canonical SceneID
• POV
• structural summary
• canonical character identity
• existing aliases
• structural setting where established

---

Stage 5 - Rework Location Discovery Around Structural Setting

Investigate the existing structural Scene Setting information first.

Current observations suggest it is often more realistic than the later Location extraction.

Desired approach

Use:

structural setting -> canonicalise -> additional significant location discovery -> book-wide consolidation

rather than treating every place-like noun as a new Location.

Contextual resolution

Examples:

bedroom

should become something meaningful where context supports it:

• Beth's bedroom
• Maggie's bedroom

Narrator's home

should resolve through Scene POV.

Generic references such as:

• road
• garage
• bedroom
• car park

must not automatically become useless canonical Locations.

However, generic-sounding locations can be significant. A car park may be the site of a major story event.

Therefore significance and context matter more than vocabulary alone.

Normalisation

Resolve obvious variants such as:

• garage / garages
• Mrs Patterson's house / Mrs. Patterson's house

before author review where confidence is high.

Review

Only meaningful uncertain candidates should require author attention.

Scene title, chapter, structural summary and evidence should be shown so the author has context.

---

Stage 6 - Improve Asset Discovery Using Structural Evidence

Asset Intelligence should consume what earlier structural analysis already knows.

The current six-chapter test missed Beth's Memory Tin even though structural scene information recognised it.

Investigate this specifically.

Asset significance evidence

Potential signals include:

• repeated mentions across scenes;
• presence in StructuralSummary;
• presence in generated/manual scene title;
• ownership references;
• involvement in knowledge/events;
• recurrence across chapters;
• plot significance.

An object important enough to survive structural summarisation is a useful signal that it may be a significant Asset.

Canonicalisation

Resolve obvious aliases/variants such as:

Triumph TR6 TR6

where evidence shows they are the same physical asset.

As with characters, avoid unsafe merging when uncertain.

---

Stage 7 - Make Relationships Progressive and Evidence-Based

Relationship Intelligence must operate against canonical CharacterIDs.

This stage must preserve relationship progression over time.

Canonical relationship

There should be one canonical relationship identity for a pair of characters where appropriate.

For example:

Maggie <-> Rob

Relationship events

The relationship then accumulates scene-based events and state changes.

Conceptual example:

• Scene 12: first meaningful interaction
• Scene 24: friendship deepens
• Scene 39: mutual attraction becomes evident
• Scene 51: romantic relationship begins
• Scene 88: engagement

The exact taxonomy should follow PlotDirector's existing relationship/event model rather than inventing a parallel system unnecessarily.

Do not flatten progression

If the final relationship state is Engaged, earlier scenes must not be rewritten as though Maggie and Rob were engaged throughout the book.

PlotDirector should be able to determine:

• current/final relationship state;
• relationship state at a specific scene/time;
• events that caused progression;
• evidence supporting those events.

Book-wide inference

Do not classify the relationship independently from each scene.

Use:

scene evidence -> canonical pair -> accumulate evidence -> relationship events/state progression

For example, repeated evidence of Maggie and Rosie:

• hugs;
• presents;
• social outings;
• trust;
• repeated positive interaction;

should support Friendship with reasonable confidence.

Do not overclaim romantic/familial relationships without sufficient evidence.

Review presentation

Show:

• canonical character pair;
• proposed relationship state/event;
• relevant chapter/scene;
• structural scene summary;
• concise supporting evidence;
• confidence where useful.

For large books, support grouping/pagination.

---

Stage 8 - Fix Knowledge Provenance and Presentation

The existing Knowledge system remains the correct home for what characters know and when they learn it.

Fix provenance centrally

The current Chapter 0 issue appears across multiple review categories.

Investigate and fix the common ChapterID/SceneID provenance path rather than patching each Razor page separately.

Knowledge entries should be traceable to:

• canonical ChapterID
• canonical SceneID
• scene number/title
• structural summary
• knowledge holder
• fact
• acquisition/change point

Identity knowledge

Character alias/identity revelations belong here.

Example:

Maggie learns that The Man in Grey is Victor Harrington

The canonical CharacterID establishes objective physical identity.

Knowledge records establish subjective awareness.

Do not conflate the two.

Review

Large unpaginated lists are unacceptable for full novels.

Use sensible pagination/grouping/filtering with scene context.

Only uncertain/consequential knowledge should require explicit review where possible.

---

Stage 9 - Deep Audit and Fix Plot Lines and Threads

Plot Lines and Threads are a central PlotDirector feature and require a dedicated pass.

Current positives

The AI is finding plausible narrative signals.

Current problems observed

• some proposed Plot Lines are too trivial and are really Threads;
• Plot Lines and Threads are visually mixed;
• Threads suggest assigning a related Plot Line but proposed Plot Lines are not available;
• candidates cannot merge with other candidates;
• ordering appears to follow discovery rather than narrative hierarchy;
• supporting scene evidence appears incorrect;
• a thread only present in Chapter 1 claimed support across all six chapters;
• after review, only two Plot Lines and zero Threads were created.

Definitions

Plot Line

A major sustained narrative arc.

Plot Thread

A smaller recurring question, clue, objective, subplot, secret, relationship issue, promise or unresolved matter, often belonging beneath a Plot Line.

Desired review model

Proposed Plot Lines

Show major arcs.

Each may contain:

Suggested Threads

Also allow:

Unassigned Threads

The author should be able to:

• accept;
• rename;
• reject;
• merge with existing canonical Plot Line/Thread;
• merge with another candidate;
• move a proposed Thread beneath a proposed Plot Line;
• reclassify Plot Line <-> Thread where appropriate.

Evidence

Supporting evidence must use actual canonical SceneIDs.

Only scenes that genuinely support the candidate may be listed.

Do not infer support merely because a candidate exists somewhere within the same chapter/book.

This stage should be undertaken after POV, identity and provenance foundations are reliable.

---

Stage 10 - Replace the Story Intelligence Review Wizard with a Review Centre

The linear Story Intelligence review wizard should be retired after mandatory Character Resolution.

The Core Import remains a wizard because it has genuine sequential dependencies.

Detailed Story Intelligence review does not.

Entry experience

Core Import completes and the book is immediately usable.

The author can:

• browse chapters;
• inspect scenes;
• read structural summaries;
• see scene titles;
• edit characters;
• add character portraits;
• use Word Companion/multiscreen tools.

Story Intelligence continues server-side without requiring Word to remain open.

When analysis completes, the existing email notification invites the author back.

Mandatory gate

The author first resolves required Character ambiguities.

After this, open the Review Centre.

Review Centre

Conceptually:

Analysis Status

---

Characters Resolved Locations 6 need attention Assets 3 need attention Relationships 8 suggestions Knowledge 2 need attention Metrics Available Plot Lines & Threads 7 proposals

The exact visual design should follow PlotDirector's established UI.

Independent modules

After Character Resolution, the author can review categories in any order.

They may:

• review Plot Lines first;
• leave Locations until later;
• close the browser halfway through Relationships;
• return another day.

Progress is persisted.

The book remains usable.

There is no requirement for Story Intelligence review to reach 100%.

---

Stage 11 - Change Review Philosophy to Exception Handling

The number of AI observations must not equal the number of human decisions.

This is the key scalability principle.

Three broad classes

Confident / deterministic

Resolve automatically.

Examples:

• known canonical character;
• known alias;
• punctuation/case normalisation;
• Narrator when POV is known.

Low consequence and easily editable

Save automatically.

Example:

• scene metric values.

The author can edit them later.

Ambiguous or consequential

Require review.

Examples:

• uncertain character identity;
• possible character merge;
• ambiguous significant location;
• new major Plot Line;
• uncertain relationship state change.

Target workload

A 50-chapter / 250-scene novel may generate thousands of internal observations.

The author should ideally make tens of meaningful decisions, not hundreds or thousands of repetitive approvals.

Always provide View all or equivalent for authors who want complete control.

---

Stage 12 - Standardise Busy/Save Behaviour

All review actions that perform server/database work must use one consistent busy-state pattern.

Examples:

• Create/resolve characters
• Save locations
• Save assets
• Apply relationships
• Apply knowledge
• Save Plot Lines/Threads

On activation:

• disable the triggering control;
• prevent duplicate submission;
• show a spinner/progress state;
• change the label appropriately (Saving..., Creating..., etc.);
• complete;
• return to the Review Centre or next meaningful destination.

Remove repetitive interstitial pages such as:

• Characters created
• Locations created
• Assets created

A small inline/transient confirmation is enough.

---

Stage 13 - Pagination, Grouping and Context

For categories that can still contain substantial data:

• paginate;
• filter;
• group where useful;
• preserve selections/review state across pages.

Relationships and Knowledge should particularly benefit from chapter/scene grouping.

Where a candidate is tied to manuscript evidence, display useful context:

• Chapter number/title
• Scene number/title
• StructuralSummary
• concise evidence

Pagination supports usability but is not a substitute for reducing unnecessary decisions.

---

Stage 14 - Preserve Existing Successful Features

Do not destabilise the parts that are now working well.

Preserve:

• canonical chapter/scene import;
• structural scene boundaries;
• SceneManuscriptSources;
• structural summaries as the single factual summary;
• optional descriptive scene titles;
• Word marker creation/save/acknowledgement;
• Word Companion existing-book linking;
• multiscreen SceneID following;
• deterministic SceneCharacter mapping;
• configured Scene Metric Types;
• current realistic metric scoring;
• Story Intelligence operating against canonical scenes;
• Story Era / Primary Locale direction;
• existing Knowledge architecture;
• existing relationship-event architecture where it already supports progression.

Metrics in particular are producing useful values and should not become a mandatory approval exercise.

---

Recommended Implementation Order

The work should be split into multiple focused Codex tasks.

Job 1 - POV and Character Identity Foundation

Implement/investigate:

• chapter default POV detection in structural analysis;
• scene POV inheritance and per-scene overrides;
• author POV review/editing;
• normal Chapter POV controls;
• canonical character merge;
• alias consolidation;
• provenance-preserving identity merge;
• Character Review redesigned around unresolved identity;
• propagation of canonical CharacterID references.

This is the prerequisite for everything else.

Job 2 - Story Intelligence Context and Provenance

Refactor detailed analysis so it receives canonical:

• POV;
• Setting;
• StructuralSummary;
• scene title;
• previous/next scene context;
• characters and aliases;
• Era/Locale.

Also fix the shared Chapter 0/provenance problem.

Do not yet undertake broad review UI redesign.

Job 3 - Entity and Relationship Intelligence Quality

Rework:

• Locations;
• Assets;
• Relationships;
• Knowledge.

Specifically investigate:

• reuse of structural Setting;
• Memory Tin omission;
• entity canonicalisation;
• relationship classification;
• relationship progression through scene-based events rather than a flattened final state.

Job 4 - Plot Lines and Threads

Perform a dedicated deep audit/fix covering:

• candidate evidence;
• book-wide consolidation;
• Plot Line versus Thread classification;
• hierarchy;
• candidate-to-candidate merge;
• assignment of proposed Threads to proposed Plot Lines;
• canonical persistence;
• incorrect supporting-scene claims.

Job 5 - Review Centre and Human Workload

Only after the intelligence foundations are cleaner:

• retire the linear Story Intelligence review wizard;
• retain mandatory Character Resolution;
• add independent Review Centre modules;
• implement exception-based review;
• add bulk actions;
• add pagination/grouping;
• standardise busy states;
• remove success interstitials;
• allow leaving and resuming at any time.

---

Success Criteria

The refactor is successful when:

1. Core Import gives the author a useful book quickly.
2. POV is established early enough to improve downstream interpretation.
3. Canonical CharacterIDs represent physical identity regardless of aliases.
4. Alias merges preserve manuscript/narrative history.
5. Knowledge continues to model who knows what and when.
6. Relationships preserve progression over time, not merely the final relationship type.
7. Later AI analysis reuses structural knowledge instead of rediscovering it.
8. Locations/assets/relationships receive sufficient context to avoid generic low-value candidates.
9. Plot Line and Thread evidence is trustworthy.
10. The author only has to resolve genuine uncertainty.
11. After Character Resolution, review categories are independent and resumable.
12. A full novel does not require hours of repetitive approval clicking.
13. Existing good features, particularly scene summaries, titles and metrics, remain intact.

---

Key Product Principle

PlotDirector should not demonstrate intelligence by showing the author everything it detected.

It should demonstrate intelligence by knowing what it can safely resolve itself, what changes through the story, and what genuinely requires the author's judgement.