Comprehensive documentation for the speech services module:
- docs/SPEECH.md: Architecture, API reference, WebSocket protocol,
environment variables, provider configuration, Docker setup,
GPU VRAM budget, and frontend integration examples
- apps/api/src/speech/AGENTS.md: Module structure, provider pattern,
how to add new providers, gotchas, and test patterns
- README.md: Speech capabilities section with quick start
Fixes#406
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implements the SpeechSettings component with four sections:
- STT settings (enable/disable, language preference)
- TTS settings (enable/disable, voice selector, tier preference, auto-play, speed control)
- Voice preview with test button
- Provider status with health indicators
Also adds Slider UI component and getHealthStatus API client function.
30 unit tests covering all sections, toggles, voice loading, and PDA-friendly design.
Fixes#404
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implements AudioPlayer inline component with play/pause, progress bar,
speed control (0.5x-2x), download, and duration display. Adds
TextToSpeechButton "Read aloud" component that synthesizes text via
the speech API and integrates AudioPlayer for playback. Includes
useTextToSpeech hook with API integration, audio caching, and
playback state management. All 32 tests passing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add SpeechGateway with Socket.IO namespace /speech for real-time
streaming transcription. Supports start-transcription, audio-chunk,
and stop-transcription events with session management, authentication,
and buffer size rate limiting. Includes 29 unit tests covering
authentication, session lifecycle, error handling, cleanup, and
client isolation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add SpeechController with POST /api/speech/transcribe for audio
transcription and GET /api/speech/health for provider status.
Uses AudioValidationPipe for file upload validation and returns
results in standard { data: T } envelope.
Includes 10 unit tests covering transcribe with options, error
propagation, and all health status combinations.
Fixes#392
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add fallback-tier TTS provider using Piper via OpenedAI Speech for
ultra-lightweight CPU-only synthesis. Maps 6 standard OpenAI voice
names (alloy, echo, fable, onyx, nova, shimmer) to Piper voices.
Update factory to use the new PiperTtsProvider class, replacing the
inline stub. Includes 37 unit tests covering provider identity,
voice mapping, and voice listing.
Fixes#395
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Create AudioValidationPipe for MIME type and file size validation,
TextValidationPipe for TTS text input validation, and DTOs for
transcribe/synthesize endpoints. Includes 36 unit tests.
Fixes#398
Add ChatterboxSynthesizeOptions interface with referenceAudio and
emotionExaggeration fields, and comprehensive unit tests (26 tests)
covering voice cloning, emotion control, clamping, graceful degradation,
and cross-language support.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract KokoroTtsProvider from factory into its own module with:
- Full voice catalog of 54 built-in voices across 8 languages
- Voice metadata parsing from ID prefix (language, gender, accent)
- Exported constants for supported formats and speed range
- Comprehensive unit tests (48 tests)
- Fix lint/type errors in chatterbox provider (Prettier + unsafe cast)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add the BaseTTSProvider abstract class and TTS provider factory that were
part of the tiered TTS architecture but missed from the previous commit.
- BaseTTSProvider: abstract base with synthesize(), listVoices(), isHealthy()
- tts-provider.factory: creates Kokoro/Chatterbox/Piper providers from config
- 30 tests (22 base provider + 8 factory)
Refs #391
Add abstract BaseTTSProvider class that implements common OpenAI-compatible
TTS logic using the OpenAI SDK with configurable baseURL. Includes synthesize(),
listVoices(), and isHealthy() methods. Create TTS provider factory that
dynamically registers Kokoro (default), Chatterbox (premium), and Piper
(fallback) providers based on configuration. Update SpeechModule to use
the factory for TTS_PROVIDERS injection token.
Also fixes lint error in speaches-stt.provider.ts (Array<T> -> T[]).
30 tests added (22 base provider + 8 factory), all passing.
Fixes#391
Add docker-compose.speech.yml with three speech services:
- Speaches (STT via Whisper + basic TTS) on port 8090
- Kokoro-FastAPI (default TTS) on port 8880
- Chatterbox TTS (premium, GPU-required) on port 8881 behind
the premium-tts profile
All services include health checks, connect to the mosaic-internal
network, and follow existing naming/labeling conventions. Makefile
targets added: speech-up, speech-down, speech-logs.
Fixes#399
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add SpeechConfig with typed configuration and startup validation for
STT (Whisper/Speaches), TTS default (Kokoro), TTS premium (Chatterbox),
and TTS fallback (Piper/OpenedAI). Includes registerAs factory for
NestJS ConfigModule integration, .env.example documentation, and 51
unit tests covering all validation paths.
Refs #401
Database: 6 models in the Prisma schema had no CREATE TABLE migration:
cron_schedules, workspace_llm_settings, quality_gates, task_rejections,
token_budgets, llm_usage_logs. Same root cause as the federation tables.
CORS: Health check requests (Docker, load balancers) don't send Origin
headers. The CORS config was rejecting these in production, causing
/health to return 500 and Docker to mark the container as unhealthy.
Requests without Origin headers are not cross-origin per the CORS spec
and should be allowed through.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Added CSRF_SECRET to docker-compose.swarm.portainer.yml (the active
Portainer deployment) and both example compose files. Also added
ENCRYPTION_KEY to the example files where it was missing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Both env vars were missing from the API service environment in
docker-compose.prod.yml and docker-compose.build.yml, causing the
CSRF_SECRET check to fail at startup even when set in .env.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Standalone Synapse + Element Web deployment for Docker Swarm/Portainer.
Separate infrastructure from Mosaic Stack (same pattern as Authentik).
Includes: Synapse, Element Web, dedicated PostgreSQL, optional coturn.
Traefik labels match existing Stack conventions.
Refs #387
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Federation is optional and should not prevent the app from starting
when DEFAULT_WORKSPACE_ID is not set. Changed from throwing (crash)
to logging a warning. The endpoint-level validation in the controller
still rejects requests when federation is unconfigured.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The NestJS tsconfig compiles to CommonJS (module: "CommonJS") but
package.json had "type": "module", causing Node.js v24 to treat the
CJS output as ESM and fail with "exports is not defined in ES module
scope" at startup.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Federation models (FederationConnection, FederatedIdentity,
FederationMessage) and their enums were defined in the Prisma schema
but never had CREATE TABLE migrations. This caused the
20260203_add_federation_event_subscriptions migration to fail with
"relation federation_messages does not exist".
Adds new migration 20260202200000 to create the 3 missing enums,
3 missing tables, all indexes, and foreign keys. Removes the
now-redundant ALTER TABLE from the 20260203 migration since
event_type is created with the table.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The base openbao image's docker-entrypoint.sh injects -dev-root-token-id
and -dev-listen-address flags when it sees 'server' as $1, causing the
server to exit immediately (code 0). Override entrypoint with dumb-init
and call bao directly to avoid the dev-mode flag injection.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The CMD exec form drops everything after & in the healthcheck URL,
causing uninitcode=200 and sealedcode=200 params to be lost. Without
them, OpenBao returns 501 when uninitialized, healthcheck fails, and
Swarm kills the container before the init sidecar can reach it.
Switch to CMD-SHELL with single-quoted URL to preserve query params.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
npx is unavailable in production image since npm is removed.
Use ./node_modules/.bin/prisma directly instead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
BullMQ requires noeviction to prevent silent job data loss. With
allkeys-lru, Valkey could evict keys BullMQ depends on for job tracking.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add docker-entrypoint.sh that runs prisma migrate deploy before
starting the app, ensuring all tables exist on deployment
- Add "type": "module" to package.json to eliminate Node.js ESM
reparsing warning for eslint.config.js
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Pass BETTER_AUTH_SECRET through all 6 docker-compose files to API container
- Fix BullModule to parse VALKEY_URL instead of VALKEY_HOST/VALKEY_PORT,
matching all other Redis consumers in the codebase
- Migrate Prisma encryption from removed $use() middleware to $extends()
client extensions (Prisma 6.x compatibility), keeping extends PrismaClient
pattern with only account and llmProviderInstance getters overridden
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
AuthGuard used across federation controllers depends on AuthService,
which requires AuthModule to be imported. Matches pattern used by
TasksModule, ProjectsModule, and CredentialsModule.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CsrfGuard is already applied globally via APP_GUARD in AppModule.
The explicit @UseGuards(CsrfGuard) on FederationController caused a
DI error because CsrfService is not provided in FederationModule.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Node.js 24 (Krypton) entered Active LTS on 2026-02-09. Update all
Dockerfiles, CI pipelines, and engine constraint from node:20-alpine
to node:24-alpine. Corrected .trivyignore: tar CVEs come from Next.js
16.1.6 bundled tar@7.5.2 (not npm). Orchestrator and API images are
clean; web image needs Next.js upstream fix.
Fixes#367
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two Trivy fixes:
1. Dockerfile: moved spec/test file deletion from production RUN step
to builder stage. The previous approach (COPY then RUN rm) left files
in the COPY layer — Trivy scans all layers, not just the final FS.
Now spec files are deleted in builder BEFORE COPY to production.
2. .trivyignore: added 3 tar CVEs (CVE-2026-23745/23950/24842) with
documented rationale. tar@7.5.2 is bundled inside npm which ships
with node:20-alpine. Not upgradeable — not our dependency. npm is
already removed from all production images.
Verified: local Trivy scan passes (exit code 0, 0 findings)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add build-shared step to web.yml so lint/typecheck/test can resolve
@mosaic/shared types (same fix previously applied to api.yml)
- Remove compiled .spec.js/.test.js files from orchestrator production
image to prevent Trivy secret scanning false positives from test
fixtures (fake AWS keys and RSA private keys in secret-scanner tests)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- docker/postgres/Dockerfile: build gosu from source with Go 1.26 via
multi-stage build (eliminates 1 CRITICAL + 5 HIGH Go stdlib CVEs)
- apps/{api,web,orchestrator}/Dockerfile: remove npm from production
images (eliminates 5 HIGH CVEs in npm's bundled cross-spawn/glob/tar)
- .trivyignore: trimmed from 16 to 5 CVEs (OpenBao only — 4 false
positives from Go pseudo-version + 1 real Go stdlib waiting on upstream)
Fixes#363
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
All 16 suppressed CVEs are in upstream binaries/packages we don't control:
- Go stdlib CVEs in openbao bin/bao (Go 1.25.6) and postgres gosu (Go 1.24.6)
- OpenBao CVE false positives (Trivy reads Go pseudo-version, we run 2.5.0)
- npm bundled cross-spawn/glob/tar CVEs in node:20-alpine base image
Updated all 6 Trivy scan steps across 5 pipelines to use --ignorefile.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three fixes for the coordinator pipeline:
1. Use bandit.yaml config file (-c bandit.yaml) so global skips
and exclude_dirs are respected in CI.
2. Upgrade pip to >=25.3 in the install step so pip-audit doesn't
fail on the stale pip 24.0 bundled with python:3.11-slim.
3. Clean up nosec inline comments to bare "# nosec BXXX" format,
moving explanations to a separate comment line above. This
prevents bandit from misinterpreting trailing text as test IDs.
Fixes#365
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The lint and typecheck steps fail because @mosaic/shared isn't built.
Add a build-shared step that compiles the shared package before lint
and typecheck run, both of which now depend on build-shared in
addition to prisma-generate.
Fixes#364
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
gosu doesn't publish proper Go module semver tags, so
`go install github.com/tianon/gosu@v1.19` fails with "no matching
versions". Replace the multi-stage golang builder with
`COPY --from=tianon/gosu /gosu /usr/local/bin/gosu`, which pulls the
pre-built binary from the official tianon/gosu Docker image. This image
is rebuilt with recent Go toolchains, so it still addresses the Go
stdlib CVEs documented in the Dockerfile comments.
Fixes#363
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
