AI 機能 operating review テンプレート集

最終確認: 2026-08-01
用途: AI 機能の顧客 job、品質、人手、provider 原価、価値、継続、変更判断を同じ証拠へ接続する

書式は会計・税務・法務・個人情報・安全性の個別判断を代替しない。raw prompt、顧客文書、秘密、個人情報をこの台帳へ貼らず、権限管理された参照 ID、hash、redacted summary を使う。指標定義と判断規則の正本は 12 章とする。

使う順序

Feature decision card と job contract を介入前に凍結する。
Eligible job を、開始しなくても一行作る。
Attempt、provider request、tool effect、人手を同じ job_id へ結ぶ。
Quality と value の成熟期限を別に置く。
Weekly review は Red→欠損→価値→原価の順、monthly review は invoice→commercial contribution→cohort の順で行う。
判断後も旧定義・snapshot を上書きせず、version と restatement を残す。

書式 1: Feature decision card

Feature ID / name:
Workflow:
ICP / plan:
Customer job:
AI を使わない baseline:
AI が必要な理由:
誤りを検出・修正できる主体:
Risk tier / 最大損害 / 可逆性:

今回の決定:
[ ] AI を採用する
[ ] route / model / prompt / retrieval を変更する
[ ] automation scope を上げる
[ ] price / allowance を変更する
[ ] segment を拡張する

Primary outcome:
Grain / numerator / denominator:
Maturity window:
Minimum practical effect:

Guardrails:
- Critical failure:
- Policy / authorization:
- Fully-loaded cost:
- Latency / availability:
- Human review / founder capacity:

Stop-now rule:
Pause rule:
Unknown rule:
No-go at maturity:
Expand requirements:

Owner:
Decision date:
Rollback owner / method:
Evidence snapshot ID:

書式 2: Eligible job contract

Workflow ID:
Natural job opportunity:
One eligible opportunity → one job rule:
複数成果へ分ける規則:
一つへまとめる retry / regeneration / screen:

Eligibility evidence:
Offer event / offered_at:
Exposure event / exposed_at:
Start event:
First presented output event:
Execution deadline:
Acceptance deadline:
Value deadline:
Incident observation / reopen rule:

Execution states:
not_started | completed | no_output | abandoned | stopped_for_safety | expired

Acceptance states:
not_assessable | accepted_first_pass | accepted_after_correction | rejected | unknown

Value states:
not_due | confirmed | not_confirmed | unknown | reopened

Material correction definition:
Standard planned human review:
Nonstandard founder rescue:
Independent value definition:
Safe value definition:

書式 3: Metric contract

Field	Definition
Metric ID / version
Name
Decision supported
Grain / dedupe key
Numerator event
Denominator / eligibility
Window / time zone
Maturity
Segment
Exclusion / reason code
Missing / denominator 0
Source of truth
Owner
Stop / Pause / Change trigger
Definition effective period
Bridge from prior version

表示規則:

Rate: x / n と率を併記
Immature: 分母から除くが件数を表示
Early terminal: 成熟分母へ含める
Unknown: Green にしない
No denominator: undefined、費用総額は表示
Small n: 全 account/job、最大値、原文を表示

書式 4: Event / ledger map

Entity	Parent	Created when	Maturity / terminal
Account	—
Experiment assignment	Account		immutable
Eligible job	Account	opportunity occurs	quality/value clocks
Attempt	Job	route run starts	terminal reason
Provider request	Attempt	billable request	reported/reconciled
Provider usage line	Provider request	billable component	reconciled/restated
Tool effect	Attempt	proposal	verified/rollback
Quality review	Job/effect	independent judgment	judged/reopened
Human work	Job	work begins	minutes closed
Value event	Job/value cycle	evidence arrives	confirmed/reopened
Workflow value cycle	Account/workflow	natural cycle	mature
Commercial cycle	Account/contract	billing period	closed/restated
Invoice reconciliation	Provider/project	invoice/cost close	reconciled

必須 invariant:

eligible opportunity と job は 1:1
provider usage は attempt と job へ到達できる
source + source_event_key は一意で、訂正は元 event を上書きせず relation で結ぶ
reversal は original line へ結び、同じ economic class・通貨の反対符号。replacement は別行
tool approval は exact action hash へ結ぶ
customer value cycle と commercial cycle を分ける
raw prompt / PII は metric label にない
event time と ingested time がある
定義、価格、model、prompt、tool、policy、grader version がある

書式 5: Provider price-book row

Field	Value
Price ID
Provider / billing channel
Product / resolved model / snapshot
Modality / SKU / component
Service tier / region / batch mode / context band
Unit
Currency
List rate
Contract rate
Minimum / rounding
Free allocation / promotional credit
Effective from / to
Source URL / accessed at
Evidence snapshot hash
Owner / next verification

注意:

reasoning / thinking が output に含まれるか確認する。
cached quantity が total input に含まれるか確認する。
search、storage、session、container、gateway、egress を漏らさない。
free/promotional credit を通常単位原価の低下としない。
direct API、marketplace、地域、契約単価を混ぜない。

書式 6: Provider request / usage rows

Request row:

Field	Value
Provider request ID
Job / attempt ID
Request ordinal
Request role	primary / planned_auxiliary / validation
Failure relation type	none / app_retry / sdk_retry / fallback / repair
Retry-of / fallback-from / repair-of
Requested / resolved model
Billing channel / model / SDK version
Service tier / region / batch / context band
Started / ended / latency
Status / error class
Idempotency key

Usage line row。billable component ごとに繰り返す:

Field	Value
Usage line / provider request ID
Component / SKU	input / cached / cache_write / output / reasoning / tool / storage / other
Native quantity / native unit
Billable quantity / billable unit / unit scale
Includes component / conversion rule
Rounding rule / minimum charge
Price ID / effective version
List / contract estimate
Provider-reported cost / currency
Reconciled cost
Invoice line / reconciliation status

書式 7: Tool effect and approval

Effect ID / job / attempt:
Tool / effect class:
Target / scope / amount / count:
Argument redacted hash:
Policy / authorization version:
Approval required:
Exact action hash = H(tool + target + redacted args + policy version + effect class):

proposed_at:
policy_checked_at / result:
approval_requested_at:
approval_resolved_at / approve-reject-edit-escalate:
approval ID / approved action hash / scope / expires_at:
approver identity ref / authority:
attempted_at:
committed_at / external ID:
verification_due_at / status:
post_condition_verified_at / evidence:
rollback_at / verification:

Idempotency key:
Duplicate detected:
Approval bypass:
Unauthorized commit:
Incident ID:

書式 8: Job outcome review

Field	Value
Job / account / workflow
Eligible / offered / exposed / started at
Assignment / feature version
Attempts / provider requests / tool effects
First presented output at
Execution status
Acceptance status
Value status
Quality mature at / judged
Value mature at / judged
Incident observation due / mature / state
Policy-compliant success
Critical incident / severity
Planned review minutes
Material correction minutes
Founder rescue minutes
Customer review/correction minutes
Variable delivery cash cost
Provider/founder internal labor cost
Fully-loaded variable cost
Value evidence reference
Independent / safe value
Reopened / restatement reason

書式 9: Failure / incident classification

Incident ID / opened / detected:
Affected job / effect / account:
Severity / reversibility / scope:
Origin stage:
[ ] intake / authorization
[ ] context / retrieval
[ ] planning / policy
[ ] tool decision / control
[ ] execution / effect
[ ] output
[ ] runtime
[ ] evaluation
[ ] downstream impact

Detection stage / detector:
Failure label:
Customer/external impact:
Containment / kill switch:
Rollback / remediation:
Notification / legal-security review:
Root cause:
Regression fixture ID:
Metric restatement ID:
Reopen condition:
Owner / due / closed:

書式 10: Human work row

Field	Value
Work ID / job
Actor side	provider / founder / customer
Role
Work class	planned_review / correction / support / incident / founder_rescue / customer_review
Planned / unplanned
Active minutes
Queue/wait minutes
Internal hourly cost
Economic cost
Material edit required
Accepted after work
Evidence / note

Provider/founder の active time は cost/capacity、customer time は net value friction へ入れる。queue wait を active labor cost に足さず、latency へ別表示する。

すべての active minute は planned_review / correction / support / incident / founder_rescue / customer_review のいずれか一つへ、一度だけ分類する。契約上の標準 review は独立価値と両立し得るが、非標準 rescue は両立しない。customer minute は内部原価でなく friction として別集計する。

書式 11: Workflow value cycle

Field	Value
Account / workflow / cycle ID
Assignment / expected cycle ID
Natural cycle start / end
Expected maturity / observed-row status
Cohort entry type / date	paid / eligible / first-value / independent-value
Eligible jobs / started
Accepted first / corrected / rejected / no-output
Verified values
Safe independent values
Planned review / correction / rescue minutes
Mature at / status
On-time / eventual repeat value
Early terminal reason
Commercial cycle mapping

書式 12: Account commercial cycle

Account / contract / commercial cycle:
Period / close status / restatement:
Correction action / original line / economic class / effective period:

Gross customer revenue:
- discount:
- customer refund / credit note:
= net customer revenue:

- provider reconciled usage:
- variable infra / external API:
- payment fee:
- chargeback source transaction / dispute event / principal amount / treatment (contra_revenue | cash_cost):
- chargeback cash loss not already in contra revenue:
- dispute fee / later recovery:
- variable support cash:
= cash contribution:

- provider/founder human economic cost:
= fully-loaded contribution:

Mapped workflow value cycles:
Feature revenue allocation:
[ ] explicit add-on
[ ] predefined allocation method
[ ] unallocated

Paid / renewed / expanded / reduced / nonrenewed:
Decision reason / evidence snapshot:

同じ月額売上を複数 workflow へ複製しない。customer refund を net revenue と cost の両方で引かない。provider promotional credit は shadow/contract cost と cash paid の差として表示する。

書式 13: Experiment / rollout card

Experiment ID / hypothesis:
Decision:
Eligible unit / assignment unit:
Eligibility_at / assignment_at / followup_start:
Arms / sticky rule:
Primary outcome / maturity:
Guardrails:
Pre-treatment segments:
Baseline / paired cases:
Minimum practical effect:
Sample / natural-cycle duration:
Stop rule / spend rule:
Analysis: ITT / paired / observational association
Expected assignment×cycle spine / maturity rule:
Assigned accounts / mature expected cycles / observed cycles / missing cycles:
ITT coverage status: complete / Unknown-Pause
Multiple looks / exploration handling:
Owner / start / decision date:
Rollback:

Checks:

assignment 後の actual use / first value で treatment を再定義しない
早期解約を non-adopter にだけ入れない
account renewal を job randomization の因果効果と呼ばない
missing expected cycle を分母から落とさず、coverage 不完全なら ITT を Unknown/Pause にする
holdout を prompt 調整に使わない
critical failure は平均改善で相殺しない

この事業・機能判断を repo の case set、config hash、exact commit、PR、release artifact へ結ぶ場合は Codex delivery kitの task、eval、PR、release 書式を同じ change_id で参照する。

書式 14: Weekly AI operating review

As-of / watermark / definition version:
Feature / workflow / version / risk tier:

1. Stop-now / reopened incidents:
2. Pause conditions / missing trace / budget:
3. Mature / immature / unknown / excluded counts:
4. Eligible→started→accepted→value→independent x/n:
5. Policy-compliant success / judged coverage:
6. First-pass / correction / rescue / critical failures:
7. Cash / fully-loaded cost per safe independent value:
8. Retry/fallback / max cost / review queue:
9. Worst and best real jobs:
10. One bottleneck:

Decision:
[ ] Stop
[ ] Pause
[ ] Hold unknown
[ ] Change
[ ] No-go at maturity
[ ] Expand candidate

Change / owner / due:
Rollback / next review:
Evidence snapshot:

書式 15: Monthly invoice and commercial review

Period / provider / project / currency:
Usage estimate:
Provider-reported cost:
Invoice gross / tax / fee:
Provider credit / prepaid applied:
Cash paid:
Explicit unallocated charges:
Reconciliation delta:
Explanation / correction events:
Correction action / original line ID / same class / opposite sign / original effective period:
Reconciled-through date:

Account commercial cycles closed:
Net revenue:
Cash contribution:
Fully-loaded contribution:
Worst account / tail cost:

Mature workflow-value cohorts:
Independent repeat value x/n:
Logo renewal x/n / GRR / NRR:
Founder rescue / review / support capacity:

Price / allowance / route / segment / automation decision:
Owner / effective date / customer-contract impact:

書式 16: Model / route promotion record

Gate	Current	Candidate	Evidence	Status
Deterministic tests
Holdout compliant success
Repeated pass^k
Benign utility
Utility under attack / targeted ASR
Critical failures
Human review burden
Cash cost / accepted job
Fully-loaded cost / safe independent value
p95 latency / max
Shadow / canary
Kill switch / rollback

Decision: promote | hold | rollback | segment-only
Why:
Scope / traffic / account / authority limit:
Price-book / prompt / retrieval / tool / policy versions:
Owner / approved / effective / next review: