Compare commits

..

4 Commits

Author SHA1 Message Date
Hermes Agent
7f3e513075 fix(git): select USC Gitea login for PR merges (#516)
Some checks failed
ci/woodpecker/push/ci Pipeline failed
2026-05-22 09:34:35 -05:00
755df9079e Merge pull request 'fix(db): bootstrap migrations on local-tier gateway startup' (#510) from fix/db-bootstrap-migrations into main 2026-05-04 22:13:14 +00:00
ac5650d9f9 fix(db): bootstrap migrations on local-tier gateway startup
Fresh `mosaic gateway install` (npm) left the gateway DB schema empty —
sign-in 500'd with `relation "users" does not exist`, and every entry
point (auth, bootstrap setup) failed because they all query the users
table first. Five stacked bugs on the local (PGlite) tier:

1. `packages/db/package.json` `files: ["dist"]` excluded the `drizzle/`
   SQL migrations from the published tarball.
2. `runMigrations()` only supports postgres-js — unusable for embedded
   PGlite.
3. `apps/gateway/src/database/database.module.ts` never invoked
   migrations at startup.
4. `createPgliteDb` didn't load pgvector, so migration 0001's
   `CREATE EXTENSION vector` failed.
5. Drizzle's PG migrator wraps every migration in one outer
   transaction, which trips Postgres' `check_safe_enum_use` on
   migration 0009 (`ALTER TYPE ADD VALUE 'pending'` → `SET DEFAULT
   'pending'` in the same tx).

Changes:
- Ship `drizzle/` in the published tarball.
- `createPgliteDb` loads `@electric-sql/pglite/vector`.
- New `runPgliteMigrations(handle)` walks the Drizzle journal and
  runs each statement-breakpoint chunk through PGlite's `client.exec()`
  (autocommit per statement). Records into `drizzle.__drizzle_migrations`
  for interop with the postgres-js path. Per-statement try/catch
  surfaces which statement of which migration failed.
- `DatabaseModule` runs migrations in `OnModuleInit` before
  `app.listen()`. Local tier: explicit `runPgliteMigrations` then
  `storageAdapter.migrate()`. Postgres tier: just `storageAdapter.migrate()`,
  which already calls `runMigrations(url)` internally — no double-call.
- Removed `packages/storage/src/test-utils/pglite-with-vector.ts`. The
  "intentionally not exported" rationale is moot now that migration
  0001 forces pgvector load anyway. The integration test uses
  `createPgliteDb` + `runPgliteMigrations` from `@mosaicstack/db`.

Tests: BetterAuth tables exist after migrate; idempotent (re-runs 0009);
partial-failure surfaces statement-level context and leaves no ledger row.

QA on a fresh PGlite install:
- `Applying PGlite schema migrations...` then `Initializing storage
  adapter (pglite)...` in startup log.
- `GET /api/bootstrap/status` → `{"needsSetup":true}` HTTP 200 (was 500).
- `POST /api/bootstrap/setup` reaches Zod validator (was 500).

Scope: this PR fixes the local (PGlite) tier. Postgres-tier first
install still has the outer-transaction problem and a journal ordering
bug (0009's `when` < 0008's). Documented inline as TODO and in the
scratchpad — needs a separate change with real-Postgres validation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 17:06:50 -05:00
bd83f86740 Merge pull request 'feat(federation): mTLS AuthGuard with OID-based grant resolution (FED-M3-03)' (#509) from feat/federation-m3-auth-guard into main
All checks were successful
ci/woodpecker/push/ci Pipeline was successful
ci/woodpecker/push/publish Pipeline was successful
2026-04-25 13:27:20 +00:00
14 changed files with 553 additions and 78 deletions

View File

@@ -1,8 +1,21 @@
import { mkdirSync } from 'node:fs'; import { mkdirSync } from 'node:fs';
import { homedir } from 'node:os'; import { homedir } from 'node:os';
import { join } from 'node:path'; import { join } from 'node:path';
import { Global, Inject, Module, type OnApplicationShutdown } from '@nestjs/common'; import {
import { createDb, createPgliteDb, type Db, type DbHandle } from '@mosaicstack/db'; Global,
Inject,
Logger,
Module,
type OnApplicationShutdown,
type OnModuleInit,
} from '@nestjs/common';
import {
createDb,
createPgliteDb,
runPgliteMigrations,
type Db,
type DbHandle,
} from '@mosaicstack/db';
import { createStorageAdapter, type StorageAdapter } from '@mosaicstack/storage'; import { createStorageAdapter, type StorageAdapter } from '@mosaicstack/storage';
import type { MosaicConfig } from '@mosaicstack/config'; import type { MosaicConfig } from '@mosaicstack/config';
import { MOSAIC_CONFIG } from '../config/config.module.js'; import { MOSAIC_CONFIG } from '../config/config.module.js';
@@ -39,12 +52,37 @@ export const STORAGE_ADAPTER = 'STORAGE_ADAPTER';
], ],
exports: [DB, STORAGE_ADAPTER], exports: [DB, STORAGE_ADAPTER],
}) })
export class DatabaseModule implements OnApplicationShutdown { export class DatabaseModule implements OnApplicationShutdown, OnModuleInit {
private readonly logger = new Logger(DatabaseModule.name);
constructor( constructor(
@Inject(DB_HANDLE) private readonly handle: DbHandle, @Inject(DB_HANDLE) private readonly handle: DbHandle,
@Inject(STORAGE_ADAPTER) private readonly storageAdapter: StorageAdapter, @Inject(STORAGE_ADAPTER) private readonly storageAdapter: StorageAdapter,
@Inject(MOSAIC_CONFIG) private readonly config: MosaicConfig,
) {} ) {}
// Migrations must complete before any module that injects DB starts serving
// requests. NestJS awaits onModuleInit before app.listen(), and modules that
// inject DB are initialized after this one — so all DB-dependent code sees a
// populated schema before the first HTTP request lands.
//
// Local (PGlite) tier: we run gateway-DB migrations explicitly here. The
// storage adapter writes to a separate PGlite directory and only manages its
// own KV tables, so we still call its migrate() afterwards.
//
// Postgres tier: PostgresAdapter.migrate() already calls runMigrations() on
// the same DATABASE_URL, so a single call covers both the gateway DB and
// the storage tables. We deliberately do NOT call runMigrations() here to
// avoid opening a second short-lived connection and doubling startup cost.
async onModuleInit(): Promise<void> {
if (this.config.tier === 'local') {
this.logger.log('Applying PGlite schema migrations...');
await runPgliteMigrations(this.handle);
}
this.logger.log(`Initializing storage adapter (${this.storageAdapter.name})...`);
await this.storageAdapter.migrate();
}
async onApplicationShutdown(): Promise<void> { async onApplicationShutdown(): Promise<void> {
await Promise.all([this.handle.close(), this.storageAdapter.close()]); await Promise.all([this.handle.close(), this.storageAdapter.close()]);
} }

View File

@@ -0,0 +1,53 @@
# t_3a368a52 — Gitea login selection for USC repos
## Objective
Fix Mosaic git wrapper behavior so `git.uscllc.com` repositories use the USC Gitea/tea login instead of the Mosaic Stack login during PR merge operations.
## Issue / tracking
- Kanban: `t_3a368a52`
- Gitea issue: `#516` (`http://git.mosaicstack.dev/mosaicstack/stack/issues/516`)
- Branch: `fix/t_3a368a52-gitea-usc-login`
## Scope
- In scope: Mosaic framework git wrapper scripts under `packages/mosaic/framework/tools/git/` and matching framework docs.
- Out of scope: U-Connect source, PR #1905 contents, Authentik settings, smoke credentials, and runtime infrastructure manifests.
## Root cause
`pr-merge.sh` always built the Gitea merge command with `--login ${GITEA_LOGIN:-mosaicstack}`. In a `git.uscllc.com/USC/uconnect` repo with no explicit `GITEA_LOGIN`, this selected the `mosaicstack` tea login even though the remote host requires the `usc` login. While validating `pr-metadata.sh`, I also found that `load_credentials` preserves existing env vars; an ambient `GITEA_TOKEN` for a different account could override host-specific credential loading unless the lookup clears Gitea env vars inside the credential-loader subshell.
## Plan
1. Add regression coverage for host → tea login selection.
2. Add shared `get_gitea_login(host)` helper in `detect-platform.sh`.
3. Update `pr-merge.sh` to derive the tea login from the current remote host.
4. Document the host mapping in framework `TOOLS.md`.
5. Validate with safe fake-`tea` merge command captures; do not perform a real merge.
## Evidence log
- Reproduced old behavior safely from `/src/uconnect` with fake `tea`: PR #1905 command used `--login mosaicstack` for repo `USC/uconnect`.
- RED test: `bash packages/mosaic/framework/tools/git/tests/gitea-login-selection.test.sh` failed because `get_gitea_login` did not exist.
- RED test extension: same test failed with `expected 'usc-token', got 'ambient-wrong-token'`, proving ambient `GITEA_TOKEN` could override host-specific USC credentials.
- GREEN test: `bash packages/mosaic/framework/tools/git/tests/gitea-login-selection.test.sh` passed after adding host mapping and clearing Gitea env vars in the credential-loader subshell.
- Syntax check: `bash -n packages/mosaic/framework/tools/git/detect-platform.sh packages/mosaic/framework/tools/git/pr-merge.sh packages/mosaic/framework/tools/git/tests/gitea-login-selection.test.sh` passed.
- Metadata validation from `/src/uconnect` using the fixed wrapper source and `MOSAIC_CREDENTIALS_FILE=/src/jarvis-brain/credentials.json`:
- PR #1905: `number=1905 state=open base=main head=edith/t_39ce717c-authentik-smoke-gate mergeable=True`.
- PR #1869: `number=1869 state=closed base=main head=fix/t_6f492e4a-cert-renewal-malformed-crt mergeable=True`.
- Safe fake-`tea` merge validation from `/src/uconnect` using the fixed wrapper source and `MOSAIC_CREDENTIALS_FILE=/src/jarvis-brain/credentials.json`:
- PR #1905 command captured `pr merge 1905 --style squash --repo USC/uconnect --login usc` and exited through fake `tea` with code 42; no merge was attempted.
- PR #1869 command captured `pr merge 1869 --style squash --repo USC/uconnect --login usc` and exited through fake `tea` with code 42; no merge was attempted.
- `ci-queue-wait.sh --purpose merge -B main -t 5 -i 1` from `/src/uconnect` resolved `platform=gitea`, branch `main`, SHA `49f0bce75c242eee19472ed367295658da9e56fc`, state `unknown`, exit 0.
- Final shell regression: `bash packages/mosaic/framework/tools/git/tests/gitea-login-selection.test.sh` passed, including `pr-merge.sh` fake-`tea` argv capture for USC login selection and a negative metacharacter login override test.
- Final syntax check: `bash -n packages/mosaic/framework/tools/git/detect-platform.sh packages/mosaic/framework/tools/git/pr-merge.sh packages/mosaic/framework/tools/git/pr-metadata.sh packages/mosaic/framework/tools/git/tests/gitea-login-selection.test.sh` passed.
- Independent review initially found the changed `pr-merge.sh` path still used string-built `eval`; remediated by switching GitHub/Gitea merge execution to argv arrays, validating numeric PR numbers, and rejecting unsupported characters in explicit `GITEA_LOGIN` overrides.
- Workspace gates: `pnpm typecheck`, `pnpm lint`, and `pnpm format:check` passed after dependency install.
## Current blocker/risk
`ci-queue-wait.sh` still reports `state=unknown` for U-Connect main because the Gitea commit status payload does not classify into success/failure/pending/no-status. This task fixed the wrong tea login selection path; it did not alter CI status semantics.
Full `pnpm test` remains blocked by unrelated gateway database setup in this Kanban workspace: gateway tests fail with `PostgresError: relation "messages" does not exist` (`42P01`) even after starting Postgres/Valkey with Docker Compose. Jaeger also fails to start because host port `16686` is already allocated. The targeted wrapper regression and repo type/lint/format gates pass.

View File

@@ -42,6 +42,7 @@
"access": "public" "access": "public"
}, },
"files": [ "files": [
"dist" "dist",
"drizzle"
] ]
} }

View File

@@ -1,10 +1,12 @@
import { PGlite } from '@electric-sql/pglite'; import { PGlite } from '@electric-sql/pglite';
import { vector } from '@electric-sql/pglite/vector';
import { drizzle } from 'drizzle-orm/pglite'; import { drizzle } from 'drizzle-orm/pglite';
import * as schema from './schema.js'; import * as schema from './schema.js';
import type { DbHandle } from './client.js'; import type { DbHandle } from './client.js';
export function createPgliteDb(dataDir: string): DbHandle { export function createPgliteDb(dataDir: string): DbHandle {
const client = new PGlite(dataDir); // pgvector extension is required by migration 0001 (insights.embedding column).
const client = new PGlite(dataDir, { extensions: { vector } });
const db = drizzle(client, { schema }); const db = drizzle(client, { schema });
return { return {
db: db as unknown as DbHandle['db'], db: db as unknown as DbHandle['db'],

View File

@@ -1,6 +1,6 @@
export { createDb, type Db, type DbHandle } from './client.js'; export { createDb, type Db, type DbHandle } from './client.js';
export { createPgliteDb } from './client-pglite.js'; export { createPgliteDb } from './client-pglite.js';
export { runMigrations } from './migrate.js'; export { runMigrations, runPgliteMigrations } from './migrate.js';
export * from './schema.js'; export * from './schema.js';
export * from './federation.js'; export * from './federation.js';
export { export {

View File

@@ -0,0 +1,70 @@
import { mkdtempSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { sql } from 'drizzle-orm';
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
import { createPgliteDb } from './client-pglite.js';
import { runPgliteMigrations } from './migrate.js';
import type { DbHandle } from './client.js';
interface PgliteExec {
exec(query: string): Promise<unknown>;
}
describe('runPgliteMigrations', () => {
let dataDir: string;
let handle: DbHandle;
beforeEach(() => {
dataDir = mkdtempSync(join(tmpdir(), 'mosaic-db-migrate-test-'));
handle = createPgliteDb(dataDir);
});
afterEach(async () => {
await handle.close();
rmSync(dataDir, { recursive: true, force: true });
});
it('creates the BetterAuth tables required by the gateway', async () => {
await runPgliteMigrations(handle);
const result = (await handle.db.execute(sql`
SELECT table_name FROM information_schema.tables
WHERE table_schema = 'public'
ORDER BY table_name
`)) as unknown as { rows: Array<{ table_name: string }> };
const tables = result.rows.map((r) => r.table_name);
// Auth tables — required for sign-in / bootstrap to function.
expect(tables).toContain('users');
expect(tables).toContain('sessions');
expect(tables).toContain('accounts');
expect(tables).toContain('verifications');
// Schema sanity check — admin token table consumed by mosaic gateway config.
expect(tables).toContain('admin_tokens');
});
it('is idempotent — running twice does not error', async () => {
await runPgliteMigrations(handle);
await expect(runPgliteMigrations(handle)).resolves.toBeUndefined();
});
it('surfaces statement-level error context on failure and leaves no ledger row', async () => {
// Pre-create a `users` table that conflicts with migration 0000's CREATE TABLE,
// forcing it to fail without IF NOT EXISTS.
const client = (handle.db as unknown as { $client: PgliteExec }).$client;
await client.exec('CREATE TABLE users (sentinel text)');
await expect(runPgliteMigrations(handle)).rejects.toThrow(
/migration hash=[a-f0-9]+ statement #\d+ failed/,
);
// Ledger should be empty — partial application must not pretend to be complete.
const ledger = (await handle.db.execute(
sql`SELECT count(*)::int AS count FROM drizzle.__drizzle_migrations`,
)) as unknown as { rows: Array<{ count: number }> };
expect(ledger.rows[0]?.count).toBe(0);
});
});

View File

@@ -1,18 +1,109 @@
import { dirname, resolve } from 'node:path'; import { dirname, resolve } from 'node:path';
import { fileURLToPath } from 'node:url'; import { fileURLToPath } from 'node:url';
import { drizzle } from 'drizzle-orm/postgres-js'; import { sql } from 'drizzle-orm';
import { migrate } from 'drizzle-orm/postgres-js/migrator'; import { drizzle as drizzlePostgres } from 'drizzle-orm/postgres-js';
import { migrate as migratePostgres } from 'drizzle-orm/postgres-js/migrator';
import { readMigrationFiles } from 'drizzle-orm/migrator';
import postgres from 'postgres'; import postgres from 'postgres';
import { DEFAULT_DATABASE_URL } from './defaults.js'; import { DEFAULT_DATABASE_URL } from './defaults.js';
import type { DbHandle } from './client.js';
interface PgliteExecutor {
exec(query: string): Promise<unknown>;
}
interface ExecuteRows<T> {
rows: T[];
}
function migrationsFolder(): string {
const here = dirname(fileURLToPath(import.meta.url));
return resolve(here, '../drizzle');
}
export async function runMigrations(url?: string): Promise<void> { export async function runMigrations(url?: string): Promise<void> {
const connectionString = url ?? process.env['DATABASE_URL'] ?? DEFAULT_DATABASE_URL; const connectionString = url ?? process.env['DATABASE_URL'] ?? DEFAULT_DATABASE_URL;
const sql = postgres(connectionString, { max: 1 }); const sqlClient = postgres(connectionString, { max: 1 });
const db = drizzle(sql); const db = drizzlePostgres(sqlClient);
const __dirname = dirname(fileURLToPath(import.meta.url));
try { try {
await migrate(db, { migrationsFolder: resolve(__dirname, '../drizzle') }); // TODO: postgres-tier first-install also fails because (a) Drizzle wraps every
// migration in one transaction (breaks 0009's ALTER TYPE ADD VALUE → SET DEFAULT
// sequence) and (b) drizzle/meta/_journal.json has 0009 ordered before 0008,
// which the postgres-js migrator skips by `created_at < folderMillis`. The
// PGlite path below sidesteps both. A follow-up should either share the
// per-statement loop (see runPgliteMigrations) or fix the journal ordering.
await migratePostgres(db, { migrationsFolder: migrationsFolder() });
} finally { } finally {
await sql.end(); await sqlClient.end();
}
}
// Apply Drizzle migrations against an embedded PGlite database.
//
// We don't reuse drizzle's pglite migrator because it wraps ALL migrations in
// one outer transaction, which breaks Postgres' `check_safe_enum_use` rule —
// e.g. migration 0009 does `ALTER TYPE ADD VALUE 'pending'` then references
// `'pending'` as a default in the same tx. PGlite's `exec()` runs each
// statement under the Simple Query protocol, autocommitting between them.
//
// We still write to the standard `drizzle.__drizzle_migrations` ledger so the
// result is interoperable with `runMigrations()` on a postgres-backed deploy
// (modulo the journal-ordering bug noted above).
//
// We skip-by-hash rather than skip-by-folderMillis (which is what Drizzle's
// postgres-js migrator does). That's deliberate — out-of-order timestamps in
// `_journal.json` won't silently drop migrations.
//
// Failure model: each statement autocommits, and the ledger row is written
// only after all statements in a migration succeed. A crash mid-migration
// leaves the prefix applied with no ledger entry, so the next boot will
// replay those statements and fail loudly on "already exists". Recovery:
// drop the partially-applied objects, or insert the migration's hash into
// `drizzle.__drizzle_migrations` manually. The error log identifies which
// statement of which migration was the culprit.
export async function runPgliteMigrations(handle: DbHandle): Promise<void> {
const client = (handle.db as unknown as { $client?: PgliteExecutor }).$client;
if (!client || typeof client.exec !== 'function') {
throw new Error('runPgliteMigrations: handle.db is not backed by a PGlite client');
}
await client.exec('CREATE SCHEMA IF NOT EXISTS drizzle');
await client.exec(`
CREATE TABLE IF NOT EXISTS drizzle.__drizzle_migrations (
id SERIAL PRIMARY KEY,
hash text NOT NULL,
created_at bigint
)
`);
const appliedRows = (await handle.db.execute(
sql`SELECT hash FROM drizzle.__drizzle_migrations`,
)) as unknown as ExecuteRows<{ hash: string }>;
const applied = new Set(appliedRows.rows.map((r) => r.hash));
const migrations = readMigrationFiles({ migrationsFolder: migrationsFolder() });
for (const migration of migrations) {
if (applied.has(migration.hash)) continue;
// Run each statement-breakpoint chunk in its own exec() call so PGlite
// commits between statements — this is what lets `ALTER TYPE ADD VALUE`
// become visible before a subsequent statement references the new value.
for (const [stmtIdx, stmt] of migration.sql.entries()) {
const trimmed = stmt.trim();
if (!trimmed) continue;
try {
await client.exec(trimmed);
} catch (err) {
const cause = err instanceof Error ? err.message : String(err);
throw new Error(
`runPgliteMigrations: migration hash=${migration.hash} statement #${stmtIdx} failed: ${cause}\n` +
`Statement: ${trimmed.slice(0, 200)}${trimmed.length > 200 ? '…' : ''}`,
{ cause: err },
);
}
}
await handle.db.execute(
sql`INSERT INTO drizzle.__drizzle_migrations (hash, created_at) VALUES (${migration.hash}, ${migration.folderMillis})`,
);
} }
} }

View File

@@ -9,7 +9,7 @@ All tool suites are located at `~/.config/mosaic/tools/`.
### Git Wrappers (Use First) ### Git Wrappers (Use First)
Mosaic wrappers at `~/.config/mosaic/tools/git/*.sh` handle platform detection and edge cases. Always use these before raw CLI commands. Mosaic wrappers at `~/.config/mosaic/tools/git/*.sh` handle platform detection and edge cases. Always use these before raw CLI commands. For self-hosted Gitea, the shared credential helper selects API credentials by remote host (`git.mosaicstack.dev``gitea-mosaicstack`, `git.uscllc.com``gitea-usc`), and the PR merge wrapper selects the matching tea login (`git.mosaicstack.dev``mosaicstack`, `git.uscllc.com``usc`) unless `GITEA_LOGIN` is explicitly set to a safe tea login override.
```bash ```bash
# Issues # Issues

View File

@@ -91,6 +91,31 @@ get_remote_host() {
return 1 return 1
} }
# Resolve the tea login name for the given Gitea host.
# Priority: explicit caller override → known Mosaic host mapping → no forced login.
get_gitea_login() {
local host="$1"
if [[ -n "${GITEA_LOGIN:-}" ]]; then
echo "$GITEA_LOGIN"
return 0
fi
case "$host" in
git.mosaicstack.dev)
echo "mosaicstack"
return 0
;;
git.uscllc.com)
echo "usc"
return 0
;;
*)
return 1
;;
esac
}
# Resolve a Gitea API token for the given host. # Resolve a Gitea API token for the given host.
# Priority: Mosaic credential loader → GITEA_TOKEN env → ~/.git-credentials # Priority: Mosaic credential loader → GITEA_TOKEN env → ~/.git-credentials
get_gitea_token() { get_gitea_token() {
@@ -104,6 +129,10 @@ get_gitea_token() {
local token local token
token=$( token=$(
source "$cred_loader" source "$cred_loader"
# load_credentials preserves pre-existing env vars by design. Clear
# Gitea env in this subshell so host-specific credential lookup wins
# over an ambient token for a different Gitea instance.
unset GITEA_TOKEN GITEA_URL
case "$host" in case "$host" in
git.mosaicstack.dev) load_credentials gitea-mosaicstack 2>/dev/null ;; git.mosaicstack.dev) load_credentials gitea-mosaicstack 2>/dev/null ;;
git.uscllc.com) load_credentials gitea-usc 2>/dev/null ;; git.uscllc.com) load_credentials gitea-usc 2>/dev/null ;;

View File

@@ -69,6 +69,11 @@ if [[ -z "$PR_NUMBER" ]]; then
usage usage
fi fi
if [[ ! "$PR_NUMBER" =~ ^[0-9]+$ ]]; then
echo "Error: PR number must be numeric." >&2
exit 1
fi
if [[ "$MERGE_METHOD" != "squash" ]]; then if [[ "$MERGE_METHOD" != "squash" ]]; then
echo "Error: Mosaic policy enforces squash merge only. Received '$MERGE_METHOD'." >&2 echo "Error: Mosaic policy enforces squash merge only. Received '$MERGE_METHOD'." >&2
exit 1 exit 1
@@ -94,19 +99,31 @@ REPO=$(get_repo_name)
case "$PLATFORM" in case "$PLATFORM" in
github) github)
CMD="gh pr merge $PR_NUMBER --squash" CMD=(gh pr merge "$PR_NUMBER" --squash)
[[ "$DELETE_BRANCH" == true ]] && CMD="$CMD --delete-branch" [[ "$DELETE_BRANCH" == true ]] && CMD+=(--delete-branch)
eval "$CMD" "${CMD[@]}"
;; ;;
gitea) gitea)
CMD="tea pr merge $PR_NUMBER --style squash --repo $OWNER/$REPO --login ${GITEA_LOGIN:-mosaicstack}" HOST=$(get_remote_host) || {
echo "Error: Could not determine remote host." >&2
exit 1
}
CMD=(tea pr merge "$PR_NUMBER" --style squash --repo "$OWNER/$REPO")
GITEA_TEA_LOGIN=$(get_gitea_login "$HOST" || true)
if [[ -n "$GITEA_TEA_LOGIN" ]]; then
if [[ ! "$GITEA_TEA_LOGIN" =~ ^[A-Za-z0-9._-]+$ ]]; then
echo "Error: Gitea tea login contains unsupported characters." >&2
exit 1
fi
CMD+=(--login "$GITEA_TEA_LOGIN")
fi
# Delete branch after merge if requested # Delete branch after merge if requested
if [[ "$DELETE_BRANCH" == true ]]; then if [[ "$DELETE_BRANCH" == true ]]; then
echo "Note: Branch deletion after merge may need to be done separately with tea" >&2 echo "Note: Branch deletion after merge may need to be done separately with tea" >&2
fi fi
eval "$CMD" "${CMD[@]}"
;; ;;
*) *)
echo "Error: Could not detect git platform" >&2 echo "Error: Could not detect git platform" >&2

View File

@@ -0,0 +1,97 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
source "$SCRIPT_DIR/detect-platform.sh"
fail() {
echo "FAIL: $*" >&2
exit 1
}
assert_eq() {
local expected="$1"
local actual="$2"
local message="$3"
if [[ "$actual" != "$expected" ]]; then
fail "$message: expected '$expected', got '$actual'"
fi
}
unset GITEA_LOGIN || true
assert_eq "usc" "$(get_gitea_login git.uscllc.com)" "USC Gitea host should select usc tea login"
assert_eq "mosaicstack" "$(get_gitea_login git.mosaicstack.dev)" "Mosaic Gitea host should select mosaicstack tea login"
GITEA_LOGIN="custom-login"
export GITEA_LOGIN
assert_eq "custom-login" "$(get_gitea_login git.uscllc.com)" "Explicit GITEA_LOGIN should override host default"
unset GITEA_LOGIN || true
unknown_login="$(get_gitea_login git.example.invalid || true)"
assert_eq "" "$unknown_login" "Unknown Gitea hosts should not force a mismatched login"
TEST_WORKDIR="${TEST_WORKDIR:-$SCRIPT_DIR/tests/.tmp-gitea-login-selection}"
rm -rf "$TEST_WORKDIR"
mkdir -p "$TEST_WORKDIR"
trap 'rm -rf "$TEST_WORKDIR"' EXIT
cat > "$TEST_WORKDIR/credentials.json" <<'JSON'
{
"gitea": {
"mosaicstack": {
"url": "https://git.mosaicstack.dev",
"token": "mosaic-token"
},
"usc": {
"url": "https://git.uscllc.com",
"token": "usc-token"
}
}
}
JSON
export MOSAIC_CREDENTIALS_FILE="$TEST_WORKDIR/credentials.json"
GITEA_TOKEN="ambient-wrong-token"
GITEA_URL="https://git.mosaicstack.dev"
export GITEA_TOKEN GITEA_URL
assert_eq "usc-token" "$(get_gitea_token git.uscllc.com)" "Host-specific credential lookup should ignore ambient mismatched GITEA_TOKEN"
assert_eq "mosaic-token" "$(get_gitea_token git.mosaicstack.dev)" "Host-specific credential lookup should select Mosaic token for Mosaic host"
FAKEBIN="$TEST_WORKDIR/fakebin"
REPO_DIR="$TEST_WORKDIR/repo"
CAPTURE_FILE="$TEST_WORKDIR/tea-args.txt"
mkdir -p "$FAKEBIN" "$REPO_DIR"
cat > "$FAKEBIN/python3" <<'SH'
#!/usr/bin/env bash
cat >/dev/null
printf 'main\n'
SH
chmod +x "$FAKEBIN/python3"
cat > "$FAKEBIN/tea" <<'SH'
#!/usr/bin/env bash
printf '%s\n' "$@" > "$TEA_CAPTURE_FILE"
SH
chmod +x "$FAKEBIN/tea"
(
cd "$REPO_DIR"
git init -q
git remote add origin https://git.uscllc.com/USC/uconnect.git
PATH="$FAKEBIN:$PATH" TEA_CAPTURE_FILE="$CAPTURE_FILE" "$SCRIPT_DIR/pr-merge.sh" --skip-queue-guard -n 1905
)
assert_eq $'pr\nmerge\n1905\n--style\nsquash\n--repo\nUSC/uconnect\n--login\nusc' "$(cat "$CAPTURE_FILE")" "pr-merge should pass USC tea login as isolated argv entries"
PWNED_FILE="$TEST_WORKDIR/pwned"
if (
cd "$REPO_DIR"
PATH="$FAKEBIN:$PATH" TEA_CAPTURE_FILE="$CAPTURE_FILE" GITEA_LOGIN="bad;touch $PWNED_FILE" "$SCRIPT_DIR/pr-merge.sh" --skip-queue-guard -n 1905 >/dev/null 2>&1
); then
fail "pr-merge should reject GITEA_LOGIN values with shell metacharacters"
fi
if [[ -e "$PWNED_FILE" ]]; then
fail "pr-merge executed shell metacharacters from GITEA_LOGIN"
fi
echo "gitea-login-selection tests passed"

View File

@@ -16,8 +16,15 @@ import fs from 'node:fs/promises';
import os from 'node:os'; import os from 'node:os';
import path from 'node:path'; import path from 'node:path';
import { users, teams, teamMembers, conversations, messages } from '@mosaicstack/db'; import {
import { createPgliteDbWithVector, runPgliteMigrations } from './test-utils/pglite-with-vector.js'; users,
teams,
teamMembers,
conversations,
messages,
createPgliteDb,
runPgliteMigrations,
} from '@mosaicstack/db';
import postgres from 'postgres'; import postgres from 'postgres';
import { afterAll, describe, expect, it } from 'vitest'; import { afterAll, describe, expect, it } from 'vitest';
@@ -102,11 +109,8 @@ describe.skipIf(!run)('migrate-tier — PGlite → federated PG', () => {
/* ---- 1. Create a temp PGlite db ---------------------------------- */ /* ---- 1. Create a temp PGlite db ---------------------------------- */
pgliteDataDir = await fs.mkdtemp(path.join(os.tmpdir(), 'fed-m1-08-')); pgliteDataDir = await fs.mkdtemp(path.join(os.tmpdir(), 'fed-m1-08-'));
const handle = createPgliteDbWithVector(pgliteDataDir); const handle = createPgliteDb(pgliteDataDir);
await runPgliteMigrations(handle);
// Run Drizzle migrations against PGlite.
// eslint-disable-next-line @typescript-eslint/no-explicit-any
await runPgliteMigrations(handle.db as any);
/* ---- 2. Seed representative data --------------------------------- */ /* ---- 2. Seed representative data --------------------------------- */

View File

@@ -1,52 +0,0 @@
/**
* Test-only helpers for creating a PGlite database with the pgvector extension
* and running Drizzle migrations against it.
*
* These are intentionally NOT exported from @mosaicstack/db to avoid pulling
* the WASM vector bundle into the public API surface.
*/
import { createRequire } from 'node:module';
import { dirname, resolve } from 'node:path';
import { PGlite } from '@electric-sql/pglite';
import { vector } from '@electric-sql/pglite/vector';
import { drizzle } from 'drizzle-orm/pglite';
import { migrate as migratePglite } from 'drizzle-orm/pglite/migrator';
import type { PgliteDatabase } from 'drizzle-orm/pglite';
import * as schema from '@mosaicstack/db';
import type { DbHandle } from '@mosaicstack/db';
/**
* Create a PGlite DB handle with the pgvector extension loaded.
* Required for running Drizzle migrations that include `CREATE EXTENSION vector`.
*/
export function createPgliteDbWithVector(dataDir: string): DbHandle {
const client = new PGlite(dataDir, { extensions: { vector } });
const db = drizzle(client, { schema });
return {
db: db as unknown as DbHandle['db'],
close: async () => {
await client.close();
},
};
}
/**
* Run Drizzle migrations against an already-open PGlite database handle.
* Resolves the migrations folder from @mosaicstack/db's installed location.
*
* @param db A PgliteDatabase instance (from drizzle-orm/pglite).
*/
export async function runPgliteMigrations(
// eslint-disable-next-line @typescript-eslint/no-explicit-any
db: PgliteDatabase<any>,
): Promise<void> {
// Resolve @mosaicstack/db package root to locate its drizzle migrations folder.
const _require = createRequire(import.meta.url);
const dbPkgMain = _require.resolve('@mosaicstack/db');
// dbPkgMain → …/packages/db/dist/index.js → dirname = dist/
// go up one level from dist/ to find the sibling drizzle/ folder
const migrationsFolder = resolve(dirname(dbPkgMain), '../drizzle');
await migratePglite(db, { migrationsFolder });
}

View File

@@ -0,0 +1,125 @@
# fix(db): bootstrap migrations on local-tier gateway startup
## Problem
Fresh `mosaic gateway install` (npm-installed) leaves the gateway DB schema empty:
```
relation "users" does not exist
```
Sign-in 500s, `auth users create` says "Not signed in", `admin/bootstrap setup`
also fails — every entry point queries `users` before doing anything else.
## Scope
This PR fixes the **local (PGlite) tier** end-to-end. The postgres-tier path
has additional pre-existing bugs (see "Known issues, out of scope" below) and
needs a separate change with real Postgres validation.
## Root causes addressed (5 stacked bugs on the local-tier path)
1. **`packages/db/package.json` `files: ["dist"]`** — the `drizzle/` SQL
migrations folder is excluded from the published tarball. Even if a
migrate runner existed, it would have nothing to apply.
2. **`packages/db/src/migrate.ts`** only supports `postgres-js`. Local-tier
gateways use embedded PGlite, which can't be reached over a postgres wire
protocol — so `runMigrations()` is unusable for the local tier.
3. **`apps/gateway/src/database/database.module.ts`** never invokes
migrations at startup. The module creates the DB handle and storage
adapter, but no consumer calls `.migrate()` on either. `mosaic storage
migrate` CLI even claims "pglite runs schema setup automatically on first
connection via `adapter.migrate()`" — but `adapter.migrate()` is only
called by tests, never at runtime.
4. **`createPgliteDb` does not load the pgvector extension.** Migration 0001
declares `CREATE EXTENSION IF NOT EXISTS vector;` for the
`insights.embedding` column. Bare PGlite has no pgvector — the migration
fails on extension control file lookup.
5. **Drizzle's PG migrator wraps every migration in one outer transaction.**
Migration 0009 does `ALTER TYPE grant_status ADD VALUE 'pending'` and then
`ALTER TABLE federation_grants ALTER COLUMN status SET DEFAULT 'pending'`.
Postgres' `check_safe_enum_use` rejects the second statement because the
new enum value isn't committed yet. Splitting the migration into two
files doesn't help — drizzle batches all migrations into one outer tx.
## Fix
- `packages/db/package.json` — ship `drizzle/` in `files`.
- `packages/db/src/client-pglite.ts` — load `@electric-sql/pglite/vector`.
- `packages/db/src/migrate.ts` — add `runPgliteMigrations(handle)`. Walks the
Drizzle journal and runs each statement-breakpoint chunk through PGlite's
`client.exec()` (Simple Query protocol → autocommit per statement). Writes
to the standard `drizzle.__drizzle_migrations` ledger so the result is
interoperable with `runMigrations()` on a postgres-backed deployment.
Per-statement try/catch surfaces which statement of which migration failed
and the ledger row is only written on full success.
- `packages/db/src/index.ts` — re-export.
- `apps/gateway/src/database/database.module.ts` — implement `OnModuleInit`:
- Local tier → `runPgliteMigrations(handle)`, then `storageAdapter.migrate()`
(the local storage adapter has its own kv tables in a separate PGlite dir).
- Postgres tier → `storageAdapter.migrate()` only, since
`PostgresAdapter.migrate()` already calls `runMigrations(url)` against
the same DATABASE_URL — we deliberately don't double-call.
NestJS awaits `onModuleInit` before `app.listen()`, so DB-dependent modules
see a populated schema before any HTTP traffic is accepted.
- `packages/storage/src/test-utils/pglite-with-vector.ts`**deleted**.
The "intentionally not exported" rationale is moot now that migration 0001
forces pgvector load anyway. `migrate-tier.integration.test.ts` switched
to `createPgliteDb` + `runPgliteMigrations` from `@mosaicstack/db`.
## Tests
`packages/db/src/migrate.test.ts`:
- Verifies `runPgliteMigrations` creates the BetterAuth tables (the original
failure mode).
- Idempotence (transitively re-runs migration 0009).
- Partial-failure: pre-creates a conflicting `users` table, asserts the
thrown error includes statement context (`hash=… statement #N failed`)
and that no ledger row was written.
## QA evidence
End-to-end on a fresh PGlite install:
- `[DatabaseModule] Applying PGlite schema migrations...` then
`Initializing storage adapter (pglite)...` in startup log.
- `GET /api/bootstrap/status``{"needsSetup":true}` HTTP 200 (was 500
with `relation "users" does not exist`).
- `POST /api/bootstrap/setup` with empty body → HTTP 400 with Zod
validation error (was 500), confirming the request reached the
validator past the table-existence check.
## Known issues, out of scope (file separately)
- **Postgres-tier first install is still broken.** `runMigrations()` uses
Drizzle's `migratePostgres`, which has the same outer-transaction problem
as PGlite's migrator. A fresh standalone-tier install would also fail at
migration 0009. Inline TODO in `migrate.ts:31-35` flags this. Fixing it
needs either (a) a shared per-statement loop reused for both drivers, or
(b) splitting migration 0009.
- **`drizzle/meta/_journal.json` has 0009 ordered before 0008** (`when`
values `1745280000000` < `1776822435828`). `migratePostgres` skips by
`created_at < folderMillis`, so on a postgres deployment that already
applied 0008, 0009 would be skipped forever. Our hash-based skip in the
PGlite path sidesteps this.
- **No advisory lock around the migration loop.** Two gateway processes
pointed at the same DATABASE_URL would race. PGlite is single-process by
file lock so the local tier is fine; postgres-tier deployments should add
`pg_advisory_lock(<deterministic-id>)` around the loop in a follow-up.
- **`mosaic storage migrate` CLI message is misleading** — it claims
"automatic on first connection via adapter.migrate()" but the adapter
doesn't self-migrate. With this PR the gateway invokes it explicitly, but
the CLI message could still be tightened.
- **Crash mid-migration leaves a partial-state PGlite DB without a ledger
row.** Detected loudly on next boot (the replay errors on "already
exists"), but recovery is manual (drop the partially-applied objects or
insert the migration hash into `drizzle.__drizzle_migrations`). A robust
fix would add a "started_at" column to a sidecar table to detect
half-applied state and refuse to start with actionable guidance.