End-to-end test process

The end-to-end (E2E) suite drives the real Android app on an emulator against a real, locally running Supabase backend. It checks what the unit tests and pgTAP cannot: that the screens, the cache, the backend rules, the push and sync paths and the operating system (permissions, the address book, rotation, languages, font size, the network going away) work together as the functional design describes. The suite is the executable form of this design. Every scenario names the section it verifies, and a section with no scenario is listed as a gap at the bottom of this page.

iOS is paused (see Questions and answers, “Platform order”). The scenarios are written platform-neutrally, with shared IDs and step names, so the XCUITest port can follow when iOS resumes.

What runs

Part	Where	Role
mobile/android/run-e2e.ps1	Windows host	The one command. Starts or reuses the local Supabase stack, resets it to seed.sql, boots the emulator, builds and installs the app and the test APK, pre-grants nothing the test itself must see, records video, runs the suite, collects the results and writes the report.
Local Supabase	Docker on the host, http://127.0.0.1:56321 (the emulator reaches it as 10.0.2.2)	The real schema, RLS, functions, edge functions, Realtime and Mailpit. Never staging, never production.
Edge functions	The same stack, served by the runner when nothing is behind the gateway yet	supabase start leaves the edge runtime out, so the runner serves the functions itself. A scenario that verifies an e-mail address needs verify-email-detail, and with no provider key that function sends its code to Mailpit: its address is MAILPIT_URL in the gitignored supabase/functions/.env, which the runner writes (Write-FunctionsEnv) and only supabase functions serve reads, never supabase/config.toml, because supabase config push carries that file to a hosted project and a Docker hostname must never get near one. The fallback also needs a local SUPABASE_URL (supabase/functions/_shared/email.ts, mailerFor and isLocalStack). No production path changes: a configured RESEND_API_KEY always wins.
Emulator	AVD named by -Avd (API 36, Google APIs)	A clean app install per test: Android Test Orchestrator with clearPackageData.
Instrumented suite	mobile/android/app/src/androidTest/java/nl/sefaja/incico/e2e/	The scenarios below, written with robots per screen over Compose test tags; UI Automator only for system UI (permission dialogs, the photo picker, the Contacts app, Custom Tabs).
Backend harness	e2e/harness/ in the suite	Creates each scenario’s own accounts through the Auth admin API, acts as “the other person” through the same public API the app uses (never by writing tables directly, so RLS applies), reads Mailpit for magic links and verification codes, and seeds the device address book through ContactsContract.

The service-role key of the local stack is handed to the harness as an instrumentation argument by the runner. It is never compiled into an APK and there is no service-role key for any hosted project on the PC.

An instrumented test cannot kill the app’s process, because the instrumentation runs inside it. Where a scenario asks for a process kill (E-AUTH-01), the harness closes the app and rebuilds everything a cold start rebuilds (the application container with its Supabase client and session store, the Room database, every view model) over the same files on disk, then opens the app again. The one thing this cannot catch is a write that was still only in memory when the process died.

Isolation rules

1. Every scenario creates its own people. Accounts are named e2e-<scenario>-<role>-<run>@incico.test, so scenarios never share state, can run in any order, and a failed one leaves nothing that breaks the next. The three seeded people (Anna, Ben, Cara) are read-only fixtures for search and visibility checks.
2. Every scenario starts from a clean install. The orchestrator clears the app’s data before each test, so the cache, the secure store and WorkManager start empty.
3. The other side acts through the API. When a scenario needs Ben to accept an invitation, the harness signs in as Ben and calls accept_invitation exactly as Ben’s phone would. The only scenarios that sign in to the app as two different people in turn are the ones whose subject is the second person’s screen.
4. No sleeping. Waits are waitUntil on a semantics condition with a timeout (default 10 s, 30 s for sync and Realtime), never a fixed delay.
5. Device settings are restored. A scenario that changes font scale, locale, dark mode, orientation, window size or connectivity restores it in @After, even on failure.

Running it

cd mobile\android
.\run-e2e.ps1                        # the full suite on a phone-sized emulator
.\run-e2e.ps1 -Suite smoke           # the tagged smoke subset, about 10 minutes
.\run-e2e.ps1 -Suite journey         # the @Journey subset: the three-person story, on the screen
.\run-e2e.ps1 -Suite duet            # the @Duet scenario: Anna and Ben on two emulators at once
.\run-e2e.ps1 -Area services         # one area (auth, onboarding, profile, circles, contacts, notifications, services, settings, crosscutting)
.\run-e2e.ps1 -Scenario E-SVC-03     # one scenario
.\run-e2e.ps1 -Tablet                # the same suite at 1280x800, 160 dpi (the tablet shell)
.\run-e2e.ps1 -Release               # against the R8-minified, release-signed build (needs key.properties)
.\run-e2e.ps1 -NoVideo -KeepStack    # faster local iterations

Output goes to e2e/results/android-<timestamp>/: report.md (one line per scenario with pass, fail or skip, its duration, and the design sections it covers), the JUnit XML, a screenshot and the UI hierarchy at the moment of every failure, logcat.txt, and video.mp4. e2e/results/ is gitignored.

-Suite duet is the one command that uses two devices. It boots a second emulator on port 5582 beside the first - a different AVD, because the emulator refuses a second instance of one AVD unless every instance was started read-only; -AvdB <name> names it, and without one the runner takes an AVD that emulates the same device as A’s and still has its system image, so both phones draw the same shell - installs the app and the test APK on it with adb, and runs the two halves of E-X-12 at the same time: Anna’s through Gradle on device A, Ben’s through adb shell am instrument on device B. The two scripts never talk to each other - each waits on its own screen for what the other person’s phone produced on the server, for up to two minutes - so the run shows a change made on one phone arriving on the other with nobody touching it. Each device records its own video (video-a.mp4, video-b.mp4) and writes its own logcat, the report lists both devices, and the gate passes only when both of them passed. Every other selection leaves the duet out: one half on its own waits for a partner who never arrives.

-KeepStack leaves the accounts, audit rows and notifications a run created in the local database. The pgTAP suite counts rows, so npx supabase test db fails on that leftover data with counts and orderings that look like real regressions. Run npx supabase db reset first, or run pgTAP before the E2E gate (2026-09-19: two pgTAP tests failed this way after a kept stack and passed on a reset database).

When it must be green

Moment	Suite
Before a feature branch is merged into develop	-Suite smoke plus the areas the branch touched
Before a phase is recorded as delivered	the full suite, phone and -Tablet
Before a release (the release gate in Build, sign and release)	the full suite with -Release, phone and -Tablet

The manual checklist in test.md keeps only what the suite cannot see (listed under “Not automated” below).

Failure handling

• A failing scenario is a bug in the app, the backend or the test. It is never re-run until it passes. Find which one it is from the screenshot, the hierarchy and the video, then fix it.
• A scenario that fails intermittently is quarantined with @Flaky("<reason>, <date>"). The runner reports it separately and it does not block the gate. A quarantined scenario that is not fixed within one phase is either rewritten or deleted, with the gap recorded below.
• A behaviour change that makes a scenario obsolete updates the scenario and this page in the same commit, just as a functional doc is updated with the code.

Scenario catalogue

IDs are stable. S marks the smoke subset. J marks the journey suite, the scenarios a person watches on the screen (-Suite journey). D marks the duet, the one scenario whose two halves run at the same time on two emulators, one person per phone (-Suite duet). The catalogue grows with each phase; the “Phase” column says since when a scenario can pass.

Sign-in and session (auth)

ID	Scenario	Verifies	Phase
E-AUTH-01 S	Sign in with e-mail and password; the dashboard loads, the cache fills, and the session survives a process kill	app/11 §1, app/20 local cache	2
E-AUTH-02	A wrong password shows the documented error and the “New to Incico?” offer; nothing is cached; forgot password sends a reset link that opens the reset screen	app/11 §1	2
E-AUTH-03	E-mail with code: the six-digit code from Mailpit signs in; a wrong code is refused; the link from the same mail works as an alternative	app/11 §1, platform/34 auth	2
E-AUTH-04	A new e-mail with a password creates the account after the code, and lands in onboarding; a disposable domain is refused	app/11 §1, platform/33	3
E-AUTH-05 S	Sign out empties the cache, the secure store and the background jobs; signing in as someone else shows nothing of the first person	app/20 sign-out, platform/33	2

Onboarding (onboarding)

ID	Scenario	Verifies	Phase
E-ONB-01 S	Fresh account through every guided step with one detail per category, telemetry off, All set with the right counts, the banner gone	app/11 §2	3
E-ONB-02	Skip every step; profile incomplete banner shows and leads back	app/11 §2	3
E-ONB-03	Back keeps what was typed; Finish later and relaunch resumes at the same step with the rows saved	app/11 persistence	3
E-ONB-04	Steps 2.3.1 to 2.3.3: connect device contacts in the flow, the cross-reference finds a seeded person, invite from the step, All set counts one service and one invitation	app/11 §2.3, app/15	5
E-ONB-05	A managed profile’s shortened flow	app/11 managed profile	3

Profile and details (profile)

ID	Scenario	Verifies	Phase
E-PRO-01 S	Add, edit and delete one detail of each of the six categories; the cards and circle dots follow	app/12 §3.4.1	3
E-PRO-02	Validation: an impossible phone number, a malformed address and 31 February are refused with the documented copy; the 21st phone number hits the server limit and shows errors.too_many_details	app/12 business rules	3
E-PRO-03	Custom types: create, rename, use, delete with replacement	app/12 types	3
E-PRO-04	E-mail verification with the six-digit code from Mailpit; a wrong code is refused; the badge appears	app/12 verification	3
E-PRO-05	Searchable toggles: a searchable e-mail makes the person findable by a second account, turning it off makes them unfindable	app/12 searchability, app/14 §3.1.2	4
E-PRO-06	Photos: pick from the system photo picker, reorder, delete; the uploaded file has no EXIF location and is at most 1024 px	app/12, A54, platform/33	3
E-PRO-07	Managed profiles: add, switch, edit, delete; contacts of the managed profile are told once	app/12 managed profiles	3

Circles (circles)

ID	Scenario	Verifies	Phase
E-CIR-01 S	Circle CRUD: cancel discards, a duplicate name gets “(2)” suggested, rename, colour, reorder, delete an empty circle with confirmation	app/13 (v1 scenario 2)	4
E-CIR-02	A circle with members cannot be deleted; move the members first	app/13 rules	4
E-CIR-03	The sees-matrix and “Preview as this circle” show exactly what a member of that circle receives	app/13 §3.2.1, platform/32	4
E-CIR-04	Changing what a circle sees reaches the other person’s contact card (v1 scenario 6)	app/13, platform/32 snapshots	4

Contacts, invitations and the contact card (contacts)

ID	Scenario	Verifies	Phase
E-CON-01 S	Find people by e-mail, phone and name; invite with circle, relation and message; the Sent group (v1 scenario 3)	app/14 §3.1.2	4
E-CON-02 S	Accept on the other side: the card shows exactly what the chosen circle shares, field by field (v1 scenario 4)	app/14 §3.1.3, platform/32	4
E-CON-03	Decline, cancel a sent invitation, the reminder after seven days is offered once	app/14	4
E-CON-04	Contact card actions reach the dialer, the mail app and Maps (intents checked, nothing sent)	app/14 §3.1.1	4
E-CON-05	Nickname: Cancel reverts, Save sticks, the list sorts and shows it (v1 scenario 5)	app/14 §3.1.1	4
E-CON-06	Move to circle: each “They will now see / stop seeing” line is right, a move that reveals more asks to confirm (v1 scenario 4)	app/13 moving, app/14	4
E-CON-07	Favourites: add, reorder, remove; groups, search and the index rail	app/14 §3.1	4
E-CON-08	Remove a contact: gone on both sides	app/14 rules	4
E-CON-09	Block: gone, not findable, no invitation possible; Unblock from Settings restores findability	app/14, app/17	4 (unblock 6)
E-CON-10	A detail the other person changes appears on the card without a manual refresh (Realtime)	app/20 refresh, platform/34 realtime	4

Notifications (notifications)

ID	Scenario	Verifies	Phase
E-NOT-01 S	Invitation, acceptance, removal and circle change each produce a row; the badge counts unread; mark one and mark all read	app/16 in-app list	4
E-NOT-02	Tapping a row and opening /n/{id} as an app link both land on the right contact	app/16, app/10	4
E-NOT-03	Notification permission denied: the app still works and the list still fills	app/16 Android specifics	4
E-NOT-04	Per-kind switches: a switched-off kind still lists in-app but sends no push	app/16 controls	6

Connected services (services)

ID	Scenario	Verifies	Phase
E-SVC-01 S	Device contacts: the disclosure appears before the OS dialog, configure with a circle, first sync writes the circle’s contacts into the “Incico” group (v1 scenario 7)	app/15 §3.3.2, A76	5
E-SVC-02	The eight buckets after a seeded mix (new, changed, deleted, already present, hand-edited entry) and each bucket action	app/15 §3.3.1, sync-buckets vectors	5
E-SVC-03	Allow delete off keeps a removed contact’s entry, on removes it; entries Incico did not write are never touched	app/15 rules	5
E-SVC-04	Cancel mid-sync leaves nothing half-written; the next sync finishes the job	app/15 §3.3.1.1	5
E-SVC-05	Cross-reference finds a seeded person by a hashed phone and e-mail, invite and Undo; the non-matches are counted, never listed	app/15 §3.3.1.2, platform/33	5
E-SVC-06	Only services whose circles include a changed detail show pending changes (v1 scenario 8)	app/15	5
E-SVC-07	Rename, disconnect keeping entries, disconnect removing them	app/15	5
E-SVC-08	Google Contacts through the fake provider: the explain screen, configure, sync into the fake book	app/15 providers, A74	5
E-SVC-09	The background job runs the automatic actions only, and not while signed out	app/15, A75	5

Settings and account (settings)

ID	Scenario	Verifies	Phase
E-SET-01	Theme Light, Dark and System apply at once and survive a restart	app/17	6
E-SET-02	Language: Dutch in the app, English back; the account stores it and pushes follow it	app/17, platform/35	6
E-SET-03	Contact detail preference, sort order and phone format change the Contacts list	app/17	6
E-SET-04	Signed-in devices: a second session is listed and can be signed out from here	app/17, platform/33	6
E-SET-05	Telemetry toggle, export (the link arrives in Mailpit), help and about screens, licences	app/17, platform/36	6
E-SET-06 S	Delete account: services disconnected first, typed confirmation, the person disappears from their contacts’ lists, sign-in refused afterwards (v1 scenario 9)	app/17 lifecycle, A79	6
E-SET-07	Change the sign-in e-mail with codes to both addresses	app/17 account	6

Cross-cutting (crosscutting)

ID	Scenario	Verifies	Phase
E-X-01 S	Offline: cards open from the cache in flight mode, edits say they need a connection, the banner shows and clears	app/20	2
E-X-02	Rotation keeps state on every tab (collapsed groups, search text, selection, open sheets)	app/18 state preservation (R16)	4
E-X-03	Tablet shell at 1280x800 and 1000x800: rail, list and detail panes, the empty detail line per tab	app/18	4
E-X-04	Every screen in Dutch: no English left, no raw key, nothing truncated	platform/35	3
E-X-05	Every screen at font scale 2.0 and in dark mode: nothing truncated, contrast tokens applied	platform/35, app/19	3
E-X-06	Right-to-left layout direction: nothing overlaps or is mirrored wrongly	platform/35	6
E-X-07	TalkBack semantics: every interactive node has a label, headings are marked, touch targets are at least 48 dp	platform/35	6
E-X-08	Visibility end to end: across three accounts in different circles, each card shows exactly the details the pgTAP matrix predicts	platform/32	4
E-X-09	A second device on the same account sees the first device’s edits	app/20 multi-device	4
E-X-10	Photo visibility across invitation and connection: the invitee sees the inviter’s avatar when it is shared with the chosen circle, nobody sees an unshared photo, and Storage refuses the bytes to anyone else	platform/32 photos, app/14 §3.1.3	4
E-X-11 S J	Three people on one phone in turn: find, invite, accept, the card, nickname, move with the reveal diff, remove, notifications, all on the screen	app/13, app/14, app/16	4
E-X-12 D	Two phones at once: Anna and Ben each on their own emulator, every change arriving on the other screen without a tap	app/14, app/16, app/20 Realtime	4

Not automated

These stay on the test.md checklist, because the emulator cannot show them:

• A real push arriving from FCM (the local stack has no FCM credentials; the suite checks the in-app list and the data path, not delivery).
• Google Contacts against Google itself (the suite uses the fake provider; a real account needs the OAuth client the product owner registers).
• Play Store install, in-app update prompts and the staged rollout.
• Store screenshots are produced by the suite (-Screenshots) but judged by a person.

Known gaps

A part of the functional design that no scenario reaches is listed here with the reason.

• The seven-day invitation reminder is proven only negatively: the scenario checks that the button is disabled and says why it is. The positive half needs an invitation row back-dated a week, which no scenario creates. It stays on the manual checklist.
• The circle preview is checked on the phone shell only. The tablet two-pane preview has no scenario of its own.
• Nothing drives the system photo picker. Photos reach the suite through the harness, so picking, reordering and deleting photos on the device stays on the manual checklist until the profile package.
• The phone-number assertions assume a device whose region is not the Netherlands. The national format a Dutch device shows is covered by the unit tests and the format vectors, not by a scenario.

Not built yet

These catalogue rows have no test method today. The reason for each of them is the same: next work package. The 2026-09-19 package built the multi-user process (circles, contacts, notifications, visibility and the journey); onboarding, profile and the presentation scenarios keep their own packages from the 2026-09-18 design.

E-ONB-01, E-ONB-02, E-ONB-03, E-ONB-05, E-PRO-01, E-PRO-02, E-PRO-03, E-PRO-04, E-PRO-06, E-PRO-07, E-X-02, E-X-03, E-X-04, E-X-05.

node tools/e2e-catalogue-check.mjs --allow-missing prints the same list, and the two must stay equal. The IDs are deliberately not under “Known gaps”: that heading is the check’s exempt set, and these scenarios must keep being reported until they are built.