Intent Classification Spike

Status: draft for review against RFC #654 Date: 2026-05-16 Purpose: validate the architecture proposed in #654 before any code is written. Walk a verb-context table through every manual-test scenario; identify gaps; render a verdict on whether the deterministic approach holds.

Scope

In scope:

• Verb/preposition patterns for the highest-frequency tools (create_project, create_client, log_time, allocate_hours, assign_pm, set_rate, create_liability, rename/archive verbs).
• Pronoun resolution via conversation-history scan.
• Token-boundary strategy when names are unquoted.

Out of scope:

• Implementation. No code is written in this spike.
• Response generation (prompt slimming, paraphrase format).
• Intent classification beyond entity extraction. Phase 3 (full intent classification with proposed-params block) is a follow-up exercise once Phase 1 is proven.
• Telemetry design.

Approach

For each candidate verb pattern, define:

1. Trigger — the regex/keyword that starts the pattern.
2. Slot map — for each prepositional phrase, what entity type goes there.
3. DB resolution rules — how to handle exact match, ambiguous match, no match.
4. Failure mode — what annotation is emitted when a slot can't be filled.

Then walk every row of the RFC's 12-row manual-test matrix through the table. Score each row as ✓ (clean), ⚠️ (works but reveals a required capability), or ✗ (architecture cannot express this — go back to the drawing board).

The verb-context table (v0)

This is the artifact the implementation phase would translate into code. The table is deliberately exhaustive about which preposition + verb combinations mean which entity type, because the same preposition (for, to, on) means different things in different verbs. That ambiguity is the structural bug we're solving — making it explicit in a table is the entire point.

create_project

Trigger	Pattern	Slot	Type
create [a]? [TYPE]? project [NAME]	after create [...]project up to for/as/due/,	NAME	project_name
... for [X]	after for up to as/due/,/—/end	X	client_name
... as [Y]	after as up to next boundary	Y	project_type (match against canonical set)
... [N]h or [N] hours	numeric + unit	N	hours_budget
... $[N]	dollar sign + number	N	contract_value
... due [DATE]	after due	DATE	deadline

Boundary words for project_name extraction: for, as, due, ,, ;, —, at, with, end-of-string.

Retainer / create_retainer_period disambiguation (issue #735): create_project also triggers on the object noun retainer (not just project). When the user writes create a retainer for X with Y, no project keyword is present so the between_keywords extractor returns null for project_name. A post-process step fires instead: when project_name is empty and the message matches for … with, the text between for and with is promoted to project_name (capped at the first with occurrence). client_name is captured from the text after with (the latest with occurrence wins, per the default behaviour of extractAfterPreposition). The project_type adjective fallback finds retainer in the message and injects Retainer. create_retainer_period is reserved for messages containing retainer period or a bare period noun — month alone is no longer a trigger (it false-fired on "per month" price phrasings). Explicit period creates: create a retainer period for [existing parent], create the May period for [parent].

create_client

Trigger	Pattern	Slot	Type
create [a]? client [NAME] or add [a]? client [NAME]	after client up to end / ,	NAME	client_name

log_time

Verb aliases (normalised at extraction — see Q3): log | track | record. Paraphrase always uses log.

Trigger	Pattern	Slot	Type
log [N]h or log [N] hours	numeric + unit	N	hours
... of [TASK] or on [TASK]	after of/on up to next prep / EOS	TASK	task_name
... to [Y]	after to up to next prep / EOS	Y	project_name
... for [Z]	after for up to next prep / EOS	Z	user_name (target person — defaults to current user if absent)
... [on/for]? [DATE]	recognised date or weekday	DATE	date (defaults to today)

Note (#729): on is a task_name preposition. The natural phrasing to <project> on <task> routes correctly: to → project_name, on → task_name. The legacy single-preposition form log Xh on <project> (no to) now lands in task_name rather than project_name; the resolver's missing-slot flow guides the user to supply a project.

Critical: for log_time, for X is a user, not a client. This is the inverse of create_project. The table makes the per-verb distinction explicit; the resolver must dispatch on verb, not just preposition.

allocate_hours

Two surface forms share the same verb:

Pattern A — named user (no pronoun before hours):

Trigger	Pattern	Slot	Type
allocate [N]h	numeric + unit	N	hours
... to [Z]	after to, stops at on/for	Z	user_name
... for [TASK] or as [TASK]	after for/as	TASK	task_name
... on [P]	after on	P	project_name

Pattern B — pronoun before hours (allocate <pronoun> <hours> to <project> on <task>):

Trigger	Pattern	Slot	Type
allocate [pronoun] [N]h	bare pronoun before hours token	pronoun	user_name (is_pronoun=true)
... to [P]	after to (first occurrence), stops at on/for	P	project_name
... on [TASK]	after on	TASK	task_name

The extractor detects pattern B by scanning the message between the verb alias end and the hours token for a word in PRONOUNS. If found, the pronoun fills user_name and downstream slots re-route: project_name uses to (first occurrence wins), task_name accepts on.

Again, for means task, not client. Per-verb dispatch matters.

assign_pm

Trigger	Pattern	Slot	Type
assign [Z] as PM	after assign, before as	Z	user_name
... to [P] or on [P]	after to/on	P	project_name

set_rate

Trigger	Pattern	Slot	Type
set rate for [Z]	after for	Z	user_name
... to $[N]	after to	N	rate value

create_liability

Trigger	Pattern	Slot	Type
create [a]? liability for [V]	after for, before on	V	vendor_name (user, vendor type only)
... on [P]	after on	P	project_name
... for $[N] or at $[N]	dollar amount	N	agreed_amount
... over [N]h or [N] hours	hours value	N	agreed_hours

rename_* / archive_*

These verbs need an object type because the same word ("Acme") could be a client name, project name, task name, or user surname. Two resolution paths:

1. Explicit type given ("archive the design task", "rename project X to Y") → trivial: take the object type literally, resolve in that table.
2. Pronoun or bare reference ("rename it to Y", "archive Acme") → server scans conversation history for the most-recent matching entity. Multiple matches across types → emit an ambiguous annotation; Claude presents a typed dialing menu.

Pronouns (cross-verb)

Pronoun	Default expected type	Notes
it	most-recently-mentioned project or client (by verb context)	If verb expects a user (assign her), it is wrong type — emit ambiguous
them, they	inferred from verb: allocate to them → user; create project for them → client
him, her, his	user (gendered pronouns imply person)
their	possessive — passes through to the noun it modifies

The server's pronoun-resolution algorithm:

1. Identify the expected entity type from the verb/preposition slot.
2. Scan conversation history (last N turns, where N is bounded by the existing 24h window) for entities of that type.
3. If exactly one candidate of that type, resolve to it.
4. If multiple, emit a dialing-menu annotation typed to that entity.
5. If none, emit a "no plausible referent — ask user to name explicitly" annotation.

Walkthrough of the 12 manual-test scenarios

For each row, the trace is: raw input → extractor output → resolver output → emitted annotation → expected Claude action.

---

Row 1: "Create project Rando Site for Smithsonian as T&M, 40h, due 2026-12-01"

Extractor:

• verb: create_project
• project_name: "Rando Site" (between project and for)
• client_name: "Smithsonian" (between for and as)
• project_type: "T&M" (between as and ,)
• hours_budget: 40
• deadline: 2026-12-01

Resolver:

• Smithsonian → look up in clients table → not found → auto-create permitted on this verb (matches existing findOrCreateClient behaviour in mcp-server.js).
• T&M matches canonical project_type set.

Annotation:

[Pre-resolved: verb=create_project, project_name="Rando Site", client_name="Smithsonian", project_type="T&M", hours_budget=40, deadline="2026-12-01". side_effects: ["client \"Smithsonian\" will be auto-created (not found in clients table)"]. All required params present. Call create_project with these values after confirmation paraphrase. The paraphrase MUST surface the auto-create side effect.]

Verdict: ✓ clean if the requesting user is an Owner or Manager with create_client permission. Per Q1, the confirmation paraphrase must read along the lines of "I'll create a new client Smithsonian, then create a T&M project Rando Site for them — 40h budget, due 2026-12-01" so the user sees the client creation and can cancel if they meant an existing client whose name they got slightly wrong. If the requesting user is a User (no create_client permission), the resolver falls through to capability B and emits a missing-client annotation with a menu of existing clients — see capability D for the full rule.

---

Row 2: 'Create project Rando Site for client "Smithsonian" as T&M…'

Same as Row 1. The literal client "..." is a hint that reinforces the type but doesn't change extraction — the boundary parser already lands on Smithsonian regardless. Worth keeping the explicit form working as a fallback for users who want to be unambiguous.

Verdict: ✓ clean.

---

Row 3: "log 2 hours of design to it" (preceding turn established project "Rando Site")

Extractor:

• verb: log_time
• hours: 2
• task_name: "design"
• project: pronoun "it", expected type = project (from to [Y] slot for log_time)

Resolver:

• Scan history for most-recent project → Rando Site (one turn back).
• Task design → look up in tasks table → matches Design (or returns dialing menu if multiple disciplines have a Design task).

Annotation:

[Pre-resolved: verb=log_time, hours=2, task_name="Design" (matched exactly), project_name="Rando Site" (pronoun "it" resolved from prior turn), user_name=self, date=today. Call log_time after confirmation.]

Verdict: ✓ clean.

---

Row 4: "allocate hours to them" (no user has been named in the conversation)

Extractor:

• verb: allocate_hours
• hours: missing (no number before hours)
• user: pronoun "them", expected type = user

Resolver:

• Scan history for most-recent user → none.
• hours value missing.

Annotation:

[Pre-resolved: verb=allocate_hours, MISSING user_name (pronoun "them" found no user referent in last 24h), MISSING hours value. Stop and ask the user to name the recipient and the number of hours.]

Verdict: ✓ clean. This is precisely the case the v2 fix tried and failed to handle in the prompt; the deterministic resolver handles it cleanly.

---

Row 5: "allocate 10h to @alice for design on Rando Site"

Extractor:

• verb: allocate_hours
• hours: 10
• user: @alice (mention)
• task_name: "design"
• project_name: "Rando Site"

Resolver:

• @alice → resolve Slack mention to user record.
• design → tasks table.
• Rando Site → projects table.

Annotation: standard pre-resolved block with all four slots filled.

Verdict: ✓ clean.

---

Row 6: "set rate for Sarah to $150"

Extractor:

• verb: set_rate
• user_name: "Sarah"
• rate: 150

Resolver:

• Sarah → users table. If multiple Sarahs, dialing menu.

Annotation: standard pre-resolved with potential dialing-menu fallback.

Verdict: ✓ clean.

---

Row 7: "log 4h of dev to Nike on Tuesday"

Extractor:

• verb: log_time
• hours: 4
• task_name: "dev" (alias — resolver maps to Development)
• project_name: "Nike"
• date: "Tuesday"

Resolver:

• Nike → projects table. Note: if a user is also named Nike it doesn't matter because the verb-context says this slot is a project. This is exactly the inversion the current prompt gets wrong.
• Tuesday → date module. Must resolve to the most recent past Tuesday (logging is retroactive). If today is Tuesday, today.

Annotation: standard.

Verdict: ⚠️ clean, but reveals a required capability:

Required capability A: date-resolution submodule. "Monday", "Tuesday", "yesterday", "last Friday" all need deterministic resolution rules. For logging verbs, weekday names default to the most recent past occurrence; this is consistent with existing prompt rule at lines 568+ but must be enforced in code, not prose.

---

Row 8: "assign Mark as PM" (no project mentioned)

Extractor:

• verb: assign_pm
• user_name: "Mark" (as PM)
• project: missing

Resolver:

• Mark → users table.
• Scan history for most-recent project → ideally one is found from prior turn; otherwise missing.

Annotation (project found in history):

[Pre-resolved: verb=assign_pm, user_name="Mark" (exact match, user_id=42), project_name="Rando Site" (inferred from prior turn). Confirm with user before calling.]

Annotation (no project in history):

[Pre-resolved: verb=assign_pm, user_name="Mark" (exact match), MISSING project_name. Ask user which project.]

Verdict: ⚠️ clean with a required capability:

Required capability B: missing-required-param ask-and-stop annotation. Many verbs have required slots; if any are missing after extraction + history scan, emit a "MISSING X — ask the user" annotation rather than guessing. The prompt rule for required params (in CREATING PROJECTS etc.) collapses into one general capability.

---

Row 9: 'Create a fixed price project for Acme — $50k, 200h, due 2026-12-01'

Extractor:

• verb: create_project
• project_type: "Fixed Price" (from [a fixed price] project)
• project_name: missing (user said "a fixed price project" with no name)
• client_name: "Acme"
• contract_value: 50000
• hours_budget: 200
• deadline: 2026-12-01

Resolver:

• Acme → clients table → may not exist → auto-create permitted.

Annotation:

[Pre-resolved: verb=create_project, project_type="Fixed Price", client_name="Acme", contract_value=50000, hours_budget=200, deadline="2026-12-01", MISSING project_name. Ask user what to call the project.]

Verdict: ⚠️ clean using capability B (missing-param ask).

---

Row 10: "log time for John on Acme — 3h design Monday"

Extractor (updated for #729):

• verb: log_time
• user_name: "John" (for [Z] slot for log_time → user, not client)
• task_name: "Acme" (on [Y] slot for log_time → task, not project — see #729)
• hours: 3
• date: "Monday" (date submodule → most recent past Monday)
• project_name: missing (no to preposition in this message)

Resolver: standard lookups; project_name in missing[] will prompt the user to supply a project.

Annotation: standard.

Verdict: ✓ clean. This is the row that proves the verb-context table earns its keep: for X is a user under log_time and a client under create_project. The table makes the distinction; the prompt currently does not. Note: after #729, on X is task_name — the earlier analysis that on X = project_name was superseded by the smoke evidence in that issue.

---

Row 11: "rename it to Nike Sprint Two" (preceding turn created an Agile sprint)

Extractor:

• verb: rename
• object: pronoun "it", expected type ambiguous (could rename a project, task, client, etc.)

Resolver:

• Scan history for most-recent entity of any renamable type → finds the just-created sprint (a project).
• new_name: "Nike Sprint Two".

Annotation:

[Pre-resolved: verb=rename, target_type=project, project_name="<sprint as created>", new_name="Nike Sprint Two" (pronoun "it" resolved from prior turn — most recent renamable entity). Call rename_project after confirmation.]

Verdict: ✓ clean, contingent on capability C:

Required capability C: the conversation-history scan must annotate each remembered entity with its type, so the resolver can match by type when the verb permits multiple types. Today's summary format in 09. Conversations may not preserve this — the scan must run on a structured event log, not free-text summary. This is a pre-flight check before Phase 1 starts (already flagged in RFC #654).

---

Row 12: "archive the design task" (multiple disciplines have a Design task enabled)

Extractor:

• verb: archive
• target_type: task (explicit — "the design *task*")
• task_name: "design"

Resolver:

• Look up design in tasks table → multiple matches.

Annotation:

[Pre-resolved: verb=archive, target_type=task, AMBIGUOUS task_name="design" — matched 3 tasks:
  1. Design (Design discipline)
  2. Design (Brand discipline)
  3. Design (UX discipline)
Present this dialing menu to the user.]

Verdict: ✓ clean.

---

Score summary

Row	Verdict	Notes
1	✓	The tonight-bug case. Resolves cleanly.
2	✓	Quoted-form fallback works identically.
3	✓	Pronoun-to-project from prior turn.
4	✓	Missing referent → stop-and-ask, the prompt's v2 failure case.
5	✓	All four slots filled.
6	✓	Standard user-lookup with dialing-menu fallback.
7	⚠️	Requires capability A (date submodule).
8	⚠️	Requires capability B (missing-param annotation).
9	⚠️	Same as row 8 — missing project_name.
10	✓	Per-verb preposition routing — table earns its keep here.
11	✓	Requires capability C (structured history log).
12	✓	Ambiguous lookup → typed dialing menu.

No row scored ✗. Every scenario is expressible in the architecture. Three required capabilities (A, B, C) surfaced; all three are bounded chunks of work.

Required capabilities (must be planned into Phase 1)

• A. Date-resolution submodule. lib/date-resolver.js. Inputs: bare weekday names, today/yesterday/tomorrow, ISO dates, "last Monday", relative phrases. Output: ISO date. Per-verb policy: logging defaults to most recent past; project deadlines default to next future.

• B. Missing-required-param annotation, dialing-menu-first. The pre-resolved block declares MISSING <slot> together with a numbered menu of the most plausible candidate values and a final something else — let me try again with more detail escape hatch. Freeform asks are reserved for slots with no plausible enumeration (hours, dollar amounts). This replaces the dozens of prompt-side rules that currently enumerate required fields per verb — one general affordance, table-driven. See "Resolved decisions Q4" below for the per-slot menu strategy.

• C. Structured conversation-history log. The pronoun-context scan needs to read a typed event log, not a free-text summary. Pre-flight: confirm 09. Conversations (or its successor) preserves entity references with type tags, or extend it before Phase 1 ships. Without this, pronouns can't resolve to typed entities deterministically. See index.js summary storage path. Implemented (issue #734): conversations.entities is populated from both tool inputs (ENTITY_PARAM_MAP — write tools) and tool outputs (ENTITY_OUTPUT_MAP — read tools). Read tools like get_project_info and get_project_assignments now record the PM name and project name from their result so pronouns in the next turn resolve correctly. Generic list tools (list_projects, list_users) are intentionally excluded to avoid polluting the entity set with unrelated names.

• D. Permission-gated auto-creation. Before the resolver emits any side_effects: ["X will be auto-created"] declaration, it must check that the requesting user holds the permission required for the equivalent create_X tool. Failure path: drop the auto-create plan; emit a MISSING X annotation per capability B (menu of existing entries + something else escape hatch that includes an "ask an Owner to create it first" message when the user lacks permission). This applies symmetrically to client / user / discipline / task auto-creation. The check uses the same permission tables that gate the direct create_X tool calls in mcp-server.js — no new permission model, just consistent application of the existing one.

Failure modes discovered

1. Project name with embedded for token — "Create project For The Win for Acme". The boundary parser would stop at the first for. Decision (Q2): right-greedy matching is the policy — the parser takes the rightmost for [single-token-or-known-client] as the client slot, and everything to the left of that as the project name. Quote enforcement is rejected because VERA's UX target is voice-to-text, which strips quotes. When right-greedy still gets it wrong (rare), Claude catches it in the confirmation paraphrase and the user corrects it before the tool runs.

2. Project type aliases — users say T&M, time and materials, T and M, t&m, etc. Need an alias table per project type. Bounded; no architectural issue.

3. Task aliases — dev = Development, design = Design, qa = QA. Same as above. Per-tenant if tenants use custom tasks.

4. Ambiguous referent across types — "archive Acme" where Acme is both a client and a project. Resolver emits a typed dialing menu: "Did you mean the client Acme or the project Acme?" Already handled by capability C's typed history.

5. Date phrases the submodule doesn't recognise — "next Friday after the holiday". Submodule emits MISSING date — couldn't parse "..." and falls through to ask-and-stop. Important: a date-resolver that silently returns "today" when it can't parse is the worst possible failure mode.

Observability requirements (per Tom — non-negotiable)

Every annotation the resolver emits must be reproducible from a log. Concretely, before Phase 1 implementation begins, the design must specify:

• Where the annotation log lives (filesystem? Postgres table? structured stdout?).
• The schema for one log entry: {message_id, tenant_id, raw_message, verb_extracted, slots, resolutions, fallthrough_reason}.
• The Slack command that lets a user retrieve their last annotation (!debug last).
• The retention window.

See follow-up comment on RFC #654 for the full per-phase observability checklist.

Verdict

Architecture survives the spike. Every manual-test row is expressible. Three required capabilities (A, B, C) are flagged with clear scope.

Recommended next step: open a Phase 1 implementation issue against #654 with the verb-context table from this doc as the spec, capabilities A/B/C as explicit subtasks, and observability as a non-negotiable deliverable.

Not recommended: start coding directly from the RFC without first writing the verb-context table into a code asset (e.g. lib/verb-context-table.js with one row per verb). Drift between code and prose is exactly the failure mode that bloated the prompt.

Open questions for review

1. Auto-create on create_project? The walkthrough assumed findOrCreateClient semantics are preserved (i.e. typing a new client name in create_project auto-creates the client). Confirm this is still desired. Alternative: require an explicit create_client first.
2. for token in project names. Should the table enforce quotes when a project name contains a reserved preposition, or is right-greedy parsing acceptable?
3. Per-tenant verb aliases. Some agencies say "log time" as "track time" or "record time". Is alias normalisation in scope for Phase 1, or punted to Phase 3?
4. Confidence threshold for fall-through to Claude. When the extractor has <100% confidence (e.g. ambiguous noun boundary), should it emit a "low-confidence — Claude please verify" annotation, or always commit to its best guess?

Resolved decisions (review pass, 2026-05-16)

The four open questions above were resolved during review. Recording the decisions here; the upstream sections of this doc (verb-context table, capability B, failure modes) have been amended where these decisions force a change.

Q1 → Yes, auto-create — but the paraphrase must surface it, AND the permission must be checked

When create precedes a for X project grammar and X doesn't match an existing client, auto-create X as a client. The annotation must explicitly declare the auto-creation as a side-effect, and Claude's confirmation paraphrase must surface it to the user before the tool runs. Users should never discover that a client was created by accident; the confirmation step is where they catch it.

This adds a new requirement to the annotation schema: a side_effects field listing any entities the tool call will create or modify beyond the primary action.

Permission gate (Q1 refinement): the resolver MUST NOT emit an auto-create side effect unless the requesting user holds the permission required for the equivalent create_X tool. A User has no create_client permission, so a create_project for [unknown client] request from a User must not silently provision the client. Instead, fall through to capability B: emit MISSING client_name with a dialing menu of existing clients the User can pick from, plus a something else option that returns a clear "you don't have permission to create new clients — ask [list of owners] to add this client first" response.

This generalises to every auto-creation: clients, users (auto-register on first DM), disciplines, tasks. The principle: an auto-create side effect always inherits the permission of the corresponding create_X tool, with no exceptions for "convenience". See capability D below.

Q2 → Right-greedy parsing, no quote enforcement

Quotes are nice but not enforceable. VERA users are encouraged to interact via voice-to-text, which strips quote characters. Any rule that requires quotes degrades the UX for voice users.

Decision: the boundary parser uses right-greedy matching with the boundary-word list. If a project name genuinely contains a reserved preposition ("For The Win"), the parser will get it wrong and Claude will catch it in the paraphrase step. That's an acceptable failure mode — users can correct it during confirmation. Forcing quotes is a worse failure mode because it punishes the voice-first UX VERA is designed for.

Failure mode #1 in the upstream section is updated to commit to right-greedy.

Q3 → Verb aliases normalised in Phase 1, paraphrase always uses the canonical verb

Specifically: log, track, and record all coerce to the canonical verb log for log_time. This normalisation happens at extraction time, and the paraphrase Claude shows the user uses only the canonical verb regardless of which alias the user typed. The user's input is preserved in the audit log, but the confirmation message is consistent.

Other alias families to plan for in the Phase 1 table (extend before implementation starts):

• create / add / set up / start → create
• allocate / assign hours → allocate (note: distinct from assign which is PM assignment)
• cancel / kill / stop → cancel
• archive / hide → archive

This is in scope for Phase 1, not punted. The alias table lives next to the verb-context table in the same file.

Q4 → Dialing menu of likely options, with a something else escape hatch

Replaces the original capability B framing. Instead of "MISSING X — ask the user" as freeform text, the annotation emits a dialing menu of the most plausible values plus a final something else / please try again with more detail option. The user replies with a digit; Claude resumes the action with the selection.

This is a significant refinement to capability B. Concretely, per slot type:

Slot	Menu strategy
project_type	6 enumerated options + something else
target_type for ambiguous archive/rename	each candidate type + something else
project_name (missing)	top 5 recently-touched projects for this user + something else
client_name (missing)	top 5 recently-touched clients for this tenant + something else
user_name (missing)	active users this tenant the requesting user has recently mentioned + something else
task_name (missing)	tasks enabled on the resolved project + something else
hours (missing)	no menu — must ask freeform (no plausible enumeration)
date (missing)	menu of today, yesterday, the last weekday, last week + something else
contract_value / agreed_amount (missing)	no menu — freeform

The something else option always reads as: "None of these — let me try again with more detail." It clears any partial state and re-enters the conversation. This is the affordance that lets the deterministic resolver fail gracefully without trapping the user.

Capability B in the upstream section is updated to reflect this menu-first design.