AI Des2Code - テストケース設計

この文書は apps/des2code/ の test contract です。internal ranking helper だけでなく、strict storage boundary と public artifact/backend fixture を検証します。

テスト戦略

Layer	Scope	External system
Unit	SQS/result schema、aligned term、candidate merge、model alias validation、family/structural/state helper	なし。typed fake のみ
Repository	scoped DocumentDB read、Atlas/DocumentDB vector pipeline、graph/catalog、S3 artifact、webhook client	collection/S3/HTTP double
Process	`process_record()` 全体の success/failure と artifact identity	injected repo、matching gateway、logger
Contract	AI artifact fixture を backend が parse して REST/MCP に公開	shared production fixture
Smoke	SQS + Lambda + DocumentDB + S3 + backend webhook/latest-artifact GET	deployed service

unit/component test は live model provider を呼びません。structured model behavior は bounded typed output で検証し、local accuracy evaluation は deterministic CI と分離します。

フィクスチャ

Fixture	Purpose
canonical `design`	512 次元 semantic/structure/region vector と component occurrence
canonical `code`	source、semantic/context metadata、2 本の 512 次元 vector
canonical `code_variation`	512 次元 visual vector を持つ rendered Storybook state
canonical `code_graph`	resolved scoped source/target dependency edge
ready/stale `code_index`	current context term、variation policy、rebuild lease
production Des2Code artifact	backend/AI shared success/failure contract
malformed legacy document	strict adapter が fail closed することを証明

fixture vector は必ず有限値 512 個です。invalid dimension test は canonical fixture を緩めず、明示的に invalid data を構築します。

ユニットテスト

Schema validation の検証

ID	Case	Expected
`U-SCH-001`	snake_case SQS body	accept
`U-SCH-002`	camelCase alias	accept して normalize
`U-SCH-003`	design/project/organization missing	reject
`U-SCH-004`	non-positive scope ID	reject
`U-SCH-005`	empty または 255 超 `request_id`	reject
`U-SCH-006`	unknown SQS body field	`extra=forbid` で reject
`U-SCH-007`	complete success artifact	alias/ISO timestamp を正しく serialize
`U-SCH-008`	failure artifact	optional request ID と bounded error を保持
`U-SCH-009`	result count と array	camelCase production contract が stable
`U-SCH-010`	selected model/embedding provider key missing	config readiness failure
`U-SCH-011`	embedding dimension != 512	config readiness failure

Retrieval merge と rerank

ID	Case	Expected
`U-MRG-001`	duplicate semantic/context/region hit	code ID で merge、axis ごとの最大値を保持
`U-MRG-002`	initial pool 外 code の variation hit	`variationSimilarity` 付き candidate を作成
`U-MRG-003`	graph neighbor	scoped source を hydrate して graph evidence を付与
`U-MRG-004`	current index context	aligned term は memory 上でのみ導出
`U-MRG-005`	deterministic rank	同じ input/evidence で stable order
`U-MRG-006`	full catalog component が top-k 外	ID validation 後 occurrence planner が選択可能
`U-MRG-007`	reranker が non-retrieved ID を返す	retrieval order から reject
`U-MRG-008`	matcher が unknown code/node ID を返す	hydrate 前に除去
`U-MRG-009`	duplicate `(nodeId, codeId)`	first valid occurrence を保持
`U-MRG-010`	identity-only component	state 不要
`U-MRG-011`	state-required component	known scoped rendered state のみ resolve
`U-MRG-012`	same-anchor unsupported alternative	lower-evidence competitor を suppress
`U-MRG-013`	empty retrieval	model call なし、empty success
`U-MRG-014`	typed bound 違反 model output	record failure、unknown identity を受け入れない

Repository テスト

ID	Case	Expected
`U-REP-001`	design fetch	`_id`, organization, project の query + strict adapter
`U-REP-002`	code index load	missing/stale reject、ready return
`U-REP-003`	Atlas vector search	inline scope filter + configured bounds
`U-REP-004`	DocumentDB exact search	leading scoped `$match`、scope ceiling、in-process cosine order
`U-REP-005`	malformed/missing result vector	fail closed
`U-REP-006`	semantic/context projection	canonical path と source field
`U-REP-007`	variation visual search	canonical visual path/state projection
`U-REP-008`	graph neighbor	scoped source/target query + unique neighbor ID
`U-REP-009`	full catalog	code/variation read が org/project scoped
`U-REP-010`	success artifact	exact `{org}/{project}/des2code/{design}/{request}.json` + object tag
`U-REP-011`	failure artifact	exact `.../{request}-failed.json`
`U-REP-012`	webhook	shared payload、API key、timeout、non-2xx delivery failure

Process テスト

ID	Case	Expected
`I-HP-001`	normal success	strict read、ready index、retrieval/matching、success artifact/webhook
`I-HP-002`	backend request ID	artifact filename と `requestId` に使用
`I-HP-003`	request ID absent	SQS message ID、次に design ID へ fallback
`I-HP-004`	design scope mismatch	retrieval なし、failed artifact/webhook、re-raise
`I-HP-005`	invalid design embedding	failed artifact/webhook、re-raise
`I-HP-006`	code index missing/stale	failed artifact/webhook、re-raise
`I-HP-007`	model success	count=array length、`codeIndexId` が current ID を pin
`I-HP-008`	success S3 write 後 webhook failure	success artifact 保持、retry 対象
`I-HP-009`	invalid body	unsafe scope/artifact key を作らない
`I-HP-010`	mixed SQS batch	failed `messageId` のみ返す
`I-HP-011`	empty candidate catalog	successful empty result を保存/通知
`I-HP-012`	backend artifact fixture	backend mapper が全 required field を accept

Smoke テスト

ID	Scenario	Verification
`S-001`	ready project success	request key に artifact、backend GET、webhook accept
`S-002`	stale project index	backend trigger が `409 CODE_INDEX_NOT_READY`、SQS send なし
`S-003`	code/variation/index rebuild	expected catalog/graph count と ready index
`S-004`	duplicate SQS delivery	同じ request object を再利用し contract valid
`S-005`	same design の second trigger	new request object、backend GET は newest `LastModified`
`S-006`	model/provider secret missing	silent model change ではなく Lambda readiness/init failure
`S-007`	strict legacy-data rejection	incompatible DocumentDB shape は matching 前に failure

カバレッジ目標

Target	Goal
`schemas.py`	public queue/result/artifact model/alias 100%
Matching helper	identity、suppression、structural、state branch
`repo.py`	両 vector provider、strict adapter、scope、S3、webhook
`service.py`	success と persistence/webhook failure ordering
Handler	partial-batch response 全体
Cross-repo contract	AI artifact fixture と backend parser の一致

CI コマンド

uv run --project apps/des2code pytest apps/des2code/__tests__
uv run pytest apps/__tests__/integration/test_production_contract_fixtures.py
uv run ruff check apps/des2code packages/des2code-core packages/models
uv run mypy apps/des2code/src packages/des2code-core/src packages/models/src

artifact/REST projection 変更時は backend contract test も実行します。deployed smoke test は AWS/DocumentDB/model-provider/backend credential を必要とするため、 pull-request unit CI には含めません。