Test Case

Test Strategy

Test type	Target	Tool	Location
API tests	Behavior of the backend routes	bun test	heineken-survey-design-backend
Unit tests	Frontend domain logic (branch evaluation, validation)	Vitest	heineken-survey-design-frontend
Unit tests	The extension's CS conversion logic	Vitest	Lab-web-extension
Integration tests	dev environment screens and preview runs	Playwright	integration-tests
Manual tests	Pushing into Creative Survey and answering on the real thing	Extension API logs + inspection	CS editing screen

Running the Tests

# backend API tests
cd heineken-survey-design-backend
bun test
bun test src/app.test.ts        # a single file
bun test -t "test name"         # filter by name

# frontend unit tests
cd heineken-survey-design-frontend
bun run test
bun run test -- lib/surveys/preview-runner.test.ts

# extension unit tests
cd Lab-web-extension
pnpm test

# integration tests against dev
cd integration-tests
bun install
bun run auth                    # first time only: Google login to dev, saving the session
bun run test                    # run 01 → 02 in sequence
bunx playwright test --grep "R13"   # a specific route only
bun run report                  # view the last run as HTML
bun run cleanup                 # delete the surveys the tests created

Backend API Test Policy

A fake store and fake auth resolver are injected into createApp(), so route behavior is verified without touching the database. The approach is to fix behavior with API tests first, then implement the repository.

const app = createApp({ store: fakeStore, resolveUser: async () => fakeUser });
const res = await app.request("/api/surveys");

Test helpers live in src/test/helpers.ts. An integration test is added separately only when verification against PostgreSQL is required.

Integration Tests (integration-tests)

The test cases in cs-import-test-cases.md are run with Playwright against the dev environment (survey-design-web.heineken.dev.4digit.ai).

Spec	Contents
00-smoke.spec.ts	Connectivity check against dev (create survey → add question → clean up). A failure means an expired login or an environment problem
01-build-survey.spec.ts	Reproduces the construction procedure through the UI and verifies CP1–CP13
02-preview-run.spec.ts	Runs the route table R1–R28 in the system preview and verifies the expected transitions

01 runs describe.serial in the order S1 → S2 → S3. The created survey's id is stored in .artifacts/survey-ref.json, which 02 and reruns of 01 reference (run 01 first if you only want to run 02).

The Test Scenario

A composite scenario: build one survey through a fixed procedure, push it into CS, then run the checklist comparison and answer walkthrough.

Section	Question	Type	Branch / visibility logic
S1 Screening	T1	Intro	—
	Q1 Gender	SA	"Prefer not to say" → screen out
	Q2 Residence	PD	"Other" → screen out
	Q3 Services used	MA	(1) exclusive setting (auto-generated) (2) "is only" → SX
S2 Operators and visibility	Q4 Colour (1–2 selections)	MA	3 rules (AND / only / has_other) + evaluation order
	Q5 Measures	MA	not_includes / not_only + visibility logic
	Q6 Satisfaction (rows × columns)	TM	Cell condition AND → SX / cell condition → SC + visibility logic (row)
	Q7 Comments (good / bad)	FA	Field-specific string match → SX / any answer → SO / answered → SO
S3 Lifestyle	Q8 Breakfast	SA	—
	Q9 Meals (sub-questions: morning/noon/night)	SA + sub-questions	Sub-question "noon" condition → SX / visibility logic
	Q10 Overall satisfaction	SA	"Other" control (Q11 auto-created)
	Q11 Other FA	FA	Q10 = other AND empty → stay (auto-generated)
	Q12 Income (personal / household)	PD multi-field	Personal ≥ 10M → SX / OR rule → SX
	Q13 Channels used (rows × columns)	TM multi-select	Exclusive setting (auto-generated per row)

Test Cases

Construction Checkpoints (CP)

Verified by 01-build-survey.spec.ts.

ID	Scenario	Expected result	Priority
CP1	Add Q3	The code is assigned as Q3 automatically (not a duplicate of Q1)	High
CP2	Add S2	Auto-generated question codes are re-assigned to non-conflicting forms such as T1-2 / Q1-2	High
CP3	Add a question after deleting one	No error, and the code is assigned from Q4 (a survey-wide sequence)	High
CP4	Duplicate a section	Question codes in the copy are re-assigned to T1-2 / Q1-2 and so on	High
CP5	Inspect the exclusive setting inside the duplicated section	The destination points at the copy's own code	High
CP6	Add an exclusive choice to Q3 and save	"Includes the exclusive choice AND is not only → stay on this question + message" is auto-generated	High
CP7	Configure "other" on Q10 and save	A follow-up FA (Q11) is auto-created, with "Q10 = other AND Q11 empty → stay" auto-generated	High
CP8	Operate the selection-count setting	The UI appears for MA only. Values persist after save. Min > max fails on save	Medium
CP9	Configure Q12 as a multi-field PD	The editor becomes "item name + newline-separated textarea". The branch screen groups by field	Medium
CP10	Configure Q13 as a multi-select TM	Exclusive rules are auto-generated per row (2). Switching back to single-select removes them	Medium
CP11	Rename Q3's exclusive choice	The auto-generated rule's conditions and message follow the new label; manual rules are untouched	Medium
CP12	Rename Q10's "other" choice	The auto-generated follow-up rule follows the change	Medium
CP13	Rename a row of Q13	The two condition rows of that row's auto rule follow the new row name	Medium

Preview Runs (R)

Verified by 02-preview-run.spec.ts. Only the difference from the base answer set (Q1=Male, Q2=Tokyo, Q3=A, Q4=Red, Q5=Measure 1, Q6=A satisfied / B neutral, Q7=empty, Q8=Yes, Q9=all at home, Q10=Satisfied, Q12=both under 4M, Q13=both rows weekday only) is listed.

ID	Difference	Expected result	Under test
R1	None	Q4=Red hides Q5's measures 2/3 → Q6 rule 2 jumps to Q8 → completes	Visibility logic, SC conversion, final steps
R2	Q1=Prefer not to say	Immediate screen out; does not become a complete respondent	equals, SX
R3	Q2=Other	Immediate screen out	Pulldown condition
R4	Q3=[A, none used]	A message appears and stays on Q3 → re-selecting allows progress	Exclusive setting (AND + self + message)
R5	Q3=none used only	Screen out	only
R6	Q4=[Red, Blue]	Screen out (rule 3 could also match, but rule 1 wins)	AND, evaluation order
R7	Q4=Green	To Q7 (skipping Q5, Q6)	only, question jump
R8	Q4=[Red, Green]	To Q6. Q6's row "Service B" is hidden	has_other, visibility logic (row)
R9	Q4=Blue	Measures 2/3 are visible in Q5 → Q5=Measure 2 goes to Q7	Visibility not matching, not_includes
R10	Q4=Blue, Q5=unanswered	To Q7 ("does not include" holds even when unanswered = CS semantics)	not_includes × unanswered
R11	Q4=Blue, Q5=[Measure 1, Measure 2]	Screen out	not_only
R12	Q6=A dissatisfied / B dissatisfied	Screen out	Matrix cell × AND
R13	Q6=A neutral / B neutral, Q7 bad = "want to cancel"	Screen out (does a partial match fire?)	FA field-specific, partial match
R14	Same, Q7 good = "cancel"	Rule 1 does not fire (field mix-up check) → rule 3 completes the survey	FA field resolution, answered
R15	Same, Q7 good = "excellent"	Survey complete	FA "any answer", SO
R16	Q9 noon = do not eat	Screen out	Sub-question-level condition
R17	Q9 morning = do not eat (noon/night at home)	Does not fire → completes	Split-target mix-up check
R18	Q8=No	"At home" disappears only for Q9 morning (kept for noon/night)	Visibility logic on a split target
R19	Q10=Other, Q11=empty	A message appears and stays → filling it in completes	"Other" control (empty detection)
R20	Q10=Satisfied, Q11=empty	Completes as-is (an empty FA passes when "other" is not selected)	The AND condition of "other" control
R21	Q4=[Red, Blue, Green]	An error keeps the respondent on Q4 (over the max of 2; validation precedes branching)	Selection limits (range)
R22	Q12 personal income ≥ 10M	Screen out	Multi-field PD condition
R23	Q12 household income ≥ 10M (personal under 4M)	Does not fire → completes (field mix-up check)	Multi-field PD target resolution
R24	Q13 store=[weekday, not used]	A message appears and stays on Q13 → re-selecting allows progress	TM exclusivity (within a row)
R25	Q13 store=[not used] only, app=[weekday]	Does not fire, completes (a selection in another row does not fire it)	Row-level TM exclusivity
R26	Q6=A neutral / B neutral, Q7=empty	"answered" does not fire; natural transition to Q8 → completes	answered × empty
R27	Q12 personal income 4M–10M	Screen out (the first branch of the OR matches)	OR decomposition
R28	Q12 household income 4M–10M (personal under 4M)	Screen out (the second branch of the OR matches)	OR decomposition

Post-push Verification in CS (I)

These require the Chrome extension and the real Creative Survey, so they are not run with Playwright. However, they can be compared automatically from the API logs the extension produces.

cd integration-tests
bun run check:cs <path to the log JSON>   # judges I-1 to I-20 and reports

How to collect the log:

1. Press Clear in the extension's "CS API log" panel
2. Run the push
3. Reload the editing screen and scroll the question list to the end (is_connect only appears on the refetch after reordering, and the list is paginated so every page must be loaded)
4. Download it and run bun run check:cs <path>

ID	What is verified	Priority
I-1	warnings is empty	High
I-2	The order is start step → T1–Q13 → complete → screen out, with labels set on the two final steps	High
I-3	Page links: Q9's three parts + Q10 + Q11 share a page; Q8 is separate (is_connect points forward)	High
I-4	Q2's pulldown choices are split into 3	High
I-5	Q3's branches: rule 1 has two AND conditions with a self destination + message; rule 2 is "is only"	High
I-6	Q4 has 3 branches in this order (order_index)	High
I-7	Q5's visibility control includes both measures 2 and 3 as targets	High
I-8	Q6's conditions specify row × column cells	High
I-9	Q7's conditions: rule 1 targets the "bad" field, rule 2 targets "any answer"	High
I-10	Q6 rule 2's destination is Q8 (section complete → head of the next section)	High
I-11	Q9's branch attaches to the last split part (night) and references the second (noon)	High
I-12	Q9's visibility control attaches to the first split part (morning) with "at home" as the target	High
I-13	Q11 shares a page with Q9–Q10, and the branch to Q11 has two conditions	Medium
I-14	Q4's selection limit is reflected (is_range: true, range_min: 1, range_max: 2)	Medium
I-15	Q12 stays one question with 2 answer_items; the condition targets the first (personal income)	Medium
I-16	Q13 has 2 branches (per row), each with two AND cell conditions and a self destination + message	Medium
I-17	Required settings are reflected on the CS side for FA / PD / TM	Medium
I-18	Q6's visibility control: condition Q4=Green, hide target the row "Service B"	Medium
I-19	Q7 has 3 branches; the third is verb 3 + empty search string + target "any answer"	Medium
I-20	Q12 has 3 logics (rule 2's OR decomposed into 2)	Medium

Notes on the Integration Tests

• dev is shared with other members. The tests create and keep a new survey every run, so clear them out with bun run cleanup when they pile up
• API authentication uses the x-user-email header; the default is the account in config.ts
• Screen operations run with headless: false (playwright.config.ts). Change this to run in CI
• Saving branches resends the whole question data the screen holds, so opening the branch panel before an auto-generated rule (such as an exclusive setting) has been applied will delete that rule. This is why BranchPanel.openFor() reloads every time
• Do not judge save completion from the button label. Deciding it disappeared before "Saving..." appeared cancels the save request on the following reload. Wait for the POST / PATCH response with waitForQuestionWrite() in lib/wait.ts before moving on

Where to Investigate a Failure

The "diagnosis table" in cs-import-test-cases.md maps symptoms to the place to fix. The main entries are:

Symptom	Likely cause	Where to look
CP1–CP5 fail	Code re-assignment is broken	backend survey-store.ts (buildUniqueQuestionCode / remapCopiedSectionCodes)
CP10 / CP11–CP13 fail	Auto rule generation and sync	frontend auto-branch-rules.ts
I-6 / R6: first match does not win	logic order_index control	extension execute-import-plan.ts, the import body
R10 fails (does not fire when unanswered)	not_includes behavior when unanswered	frontend preview-runner.ts
R13 fails (partial match does not fire)	The CS FA matching specification	frontend preview-runner.ts, evaluateFreeTextCondition
I-11 / R16–R17 fail	Sub-question index resolution	extension execute-import-plan.ts, resolveConditionTarget
R2 becomes a complete respondent	Order of complete / screen out	extension execute-import-plan.ts, setQuestionOrder
R4 fails (not sent back)	Self destination / message emission	extension execute-import-plan.ts, resolveDestinationId
I-20 / R27–R28 fail	OR decomposition into logics	extension build-import-plan.ts, conditionGroups

Not Yet Verified

Items not yet confirmed against the real Creative Survey.

• The extension's visibility POST for the second and later entries
• The placeholder separator for pulldowns
• FA partial-match behavior
• CS's interpretation of verb 4 ("is not only") in a cell condition
• The answer_type and condition representation of survival-format questions (out of scope, not collected)