test(storage): integration test for migrate-tier (FED-M1-08) + camelCase column fix #477

2026-04-20T01:39:56Z

jason.woltje commented

2026-04-20 01:39:56 +00:00

Summary

FED-M1-08 — End-to-end integration test that seeds a temp PGlite, runs mosaic storage migrate-tier, and verifies row counts + spot-checks against a live federated Postgres+pgvector profile.

Real bug surfaced

The integration test caught a P0 regression in M1-05 (#474) that all 32 unit tests missed because they mocked both source and target:

DrizzleMigrationSource.readTable() returns Drizzle's selected camelCase keys (emailVerified, userId, createdAt).
PostgresMigrationTarget.upsertBatch() used those keys verbatim as SQL identifiers, producing column "emailVerified" does not exist against the real federated DB.

Fix: normaliseSourceRow now applies a toSnakeCase conversion that is idempotent on already-snake_case keys (/[A-Z]/g → _<lowercase>). All 85 prior unit tests still pass; the integration test now passes against a live stack.

Test design

Seeds: users (2), teams (1), team_members (2), conversations (2), messages (5). All rows namespaced with fed-m1-08- (text PKs) or f0000xxx- UUIDs, deleted in afterAll even on test panic. No global state mutation. Test-prefix DELETE means --allow-non-empty is safe to use.

Test-utils placement

createPgliteDbWithVector and runPgliteMigrations were initially added to @mosaicstack/db public exports. Code review (sonnet) flagged this as polluting production consumers with a WASM bundle dependency. Moved to packages/storage/src/test-utils/pglite-with-vector.ts — co-located with the only consumer.

Independent review + verification

Code review (sonnet) — VERDICT: B → both items addressed in same PR
Surface verification (haiku) — claims-trustworthy across all 9 worker claims
85 storage unit tests still pass; 1 integration test gated by env var

Verification

$ pnpm --filter @mosaicstack/storage test
 Test Files  5 passed | 1 skipped (6)
      Tests  85 passed | 1 skipped (86)

$ FEDERATED_INTEGRATION=1 pnpm --filter @mosaicstack/storage test src/migrate-tier.integration.test.ts
 ✓ migrate-tier — PGlite → federated PG (1 test) 4690ms

Test plan

CI green
Manual smoke: rerun integration test against fresh federated compose

Refs #460

## Summary FED-M1-08 — End-to-end integration test that seeds a temp PGlite, runs `mosaic storage migrate-tier`, and verifies row counts + spot-checks against a live federated Postgres+pgvector profile. ## Real bug surfaced The integration test caught a P0 regression in M1-05 (#474) that all 32 unit tests missed because they mocked both source and target: - `DrizzleMigrationSource.readTable()` returns Drizzle's selected camelCase keys (`emailVerified`, `userId`, `createdAt`). - `PostgresMigrationTarget.upsertBatch()` used those keys verbatim as SQL identifiers, producing `column "emailVerified" does not exist` against the real federated DB. Fix: `normaliseSourceRow` now applies a `toSnakeCase` conversion that is idempotent on already-snake_case keys (`/[A-Z]/g` → `_<lowercase>`). All 85 prior unit tests still pass; the integration test now passes against a live stack. ## Test design Seeds: `users` (2), `teams` (1), `team_members` (2), `conversations` (2), `messages` (5). All rows namespaced with `fed-m1-08-` (text PKs) or `f0000xxx-` UUIDs, deleted in `afterAll` even on test panic. No global state mutation. Test-prefix DELETE means `--allow-non-empty` is safe to use. ## Test-utils placement `createPgliteDbWithVector` and `runPgliteMigrations` were initially added to `@mosaicstack/db` public exports. Code review (sonnet) flagged this as polluting production consumers with a WASM bundle dependency. Moved to `packages/storage/src/test-utils/pglite-with-vector.ts` — co-located with the only consumer. ## Independent review + verification - Code review (sonnet) — VERDICT: B → both items addressed in same PR - Surface verification (haiku) — claims-trustworthy across all 9 worker claims - 85 storage unit tests still pass; 1 integration test gated by env var ## Verification ``` $ pnpm --filter @mosaicstack/storage test Test Files 5 passed | 1 skipped (6) Tests 85 passed | 1 skipped (86) $ FEDERATED_INTEGRATION=1 pnpm --filter @mosaicstack/storage test src/migrate-tier.integration.test.ts ✓ migrate-tier — PGlite → federated PG (1 test) 4690ms ``` ## Test plan - [ ] CI green - [ ] Manual smoke: rerun integration test against fresh federated compose Refs #460

jason.woltje added 1 commit 2026-04-20 01:39:57 +00:00

test(storage): integration test for migrate-tier PGlite → federated PG (FED-M1-08)

ci/woodpecker/push/ci Pipeline was successful

Details

ci/woodpecker/pr/ci Pipeline was successful

Details

efdbb13ab4

Seeds a temp PGlite with representative cross-table data (users, teams,
team_members, conversations, messages), runs mosaic storage migrate-tier
against a live federated PG+pgvector profile, asserts exact row counts and
spot-checks key columns. Gated by FEDERATED_INTEGRATION=1.

Caught and fixed P0 bug in migrate-tier:

  PostgresMigrationTarget passed Drizzle-selected camelCase keys
  (emailVerified, userId, ...) verbatim as SQL identifiers, failing on
  Postgres which expects snake_case. The 32 unit tests missed this because
  both source and target were mocked. normaliseSourceRow now applies
  toSnakeCase conversion that is idempotent on already-snake_case keys.

Test infrastructure:

- packages/storage/src/test-utils/pglite-with-vector.ts: PGlite +
  @electric-sql/pglite/vector (JS-native pgvector) and migration runner.
  Co-located with storage tests rather than exposed on @mosaicstack/db
  public surface (would have polluted prod consumers with WASM bundle).
- packages/storage now declared "type": "module" (codebase convention,
  required for import.meta.url in test-utils).

afterAll cleanup deletes prefix-namespaced rows even on test panic.

Refs #460

jason.woltje merged commit 15d849c166 into main

2026-04-20 01:40:03 +00:00

jason.woltje referenced this issue from a commit

2026-04-20 01:40:03 +00:00

test(storage): integration test for migrate-tier (FED-M1-08) + camelCase column fix (#477)

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: mosaicstack/stack#477