Commit Graph

12 Commits

Author SHA1 Message Date
c221b63d14 fix: Resolve CI typecheck failures and improve type safety
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
Fixes CI pipeline failures caused by missing Prisma Client generation and TypeScript type safety issues. Added Prisma generation step to CI pipeline, installed missing type dependencies, and resolved 40+ exactOptionalPropertyTypes violations across service layer.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-30 20:39:03 -06:00
Jason Woltje
3ec2059470 feat: add semantic search with pgvector (closes #68, #69, #70)
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
ci/woodpecker/pr/woodpecker Pipeline failed
Issues resolved:
- #68: pgvector Setup
  * Added pgvector vector index migration for knowledge_embeddings
  * Vector index uses HNSW algorithm with cosine distance
  * Optimized for 1536-dimension OpenAI embeddings

- #69: Embedding Generation Pipeline
  * Created EmbeddingService with OpenAI integration
  * Automatic embedding generation on entry create/update
  * Batch processing endpoint for existing entries
  * Async generation to avoid blocking API responses
  * Content preparation with title weighting

- #70: Semantic Search API
  * POST /api/knowledge/search/semantic - pure vector search
  * POST /api/knowledge/search/hybrid - RRF combined search
  * POST /api/knowledge/embeddings/batch - batch generation
  * Comprehensive test coverage
  * Full documentation in docs/SEMANTIC_SEARCH.md

Technical details:
- Uses OpenAI text-embedding-3-small model (1536 dims)
- HNSW index for O(log n) similarity search
- Reciprocal Rank Fusion for hybrid search
- Graceful degradation when OpenAI not configured
- Async embedding generation for performance

Configuration:
- Added OPENAI_API_KEY to .env.example
- Optional feature - disabled if API key not set
- Falls back to keyword search in hybrid mode
2026-01-30 15:19:13 -06:00
Jason Woltje
10a812aedc fix: code review cleanup
- Add missing dependencies: ioredis, adm-zip, archiver, gray-matter, @types/multer, @types/archiver
- Fix import statements: use default imports for AdmZip, archiver, gray-matter
- Remove unused imports: ArrayMinSize
- Fix export types: use 'export type' for type-only exports
- Replace 'any' types with proper types:
  - AuthUser for user parameters
  - ExportEntry interface for entry data
  - unknown for frontmatter parsing parameters
  - Record<string, unknown> for dynamic objects
- Add security improvements:
  - File upload size limit: 50MB max
  - File type validation in FileInterceptor
  - Path traversal protection in zip extraction
  - Zip bomb protection: max 1000 files, 100MB uncompressed
- Fix exactOptionalPropertyTypes issues: use conditional spreading for optional fields
2026-01-30 00:15:44 -06:00
Jason Woltje
c4c15ee87e feat: add markdown import/export (closes #77, #78)
- Add POST /api/knowledge/import endpoint for .md and .zip files
- Add GET /api/knowledge/export endpoint with markdown/json formats
- Import parses frontmatter (title, tags, status, visibility)
- Export includes frontmatter in markdown format
- Add ImportExportActions component with drag-and-drop UI
- Add import progress dialog with success/error summary
- Add export dropdown with format selection
- Include comprehensive test suite
- Support bulk import with detailed error reporting
2026-01-30 00:05:15 -06:00
Jason Woltje
6b776a74d2 feat: add Valkey integration for task queue (closes #98)
- Add ioredis package dependency for Redis-compatible operations
- Create ValkeyModule as global NestJS module
- Implement ValkeyService with task queue operations:
  - enqueue(task): Add tasks to FIFO queue
  - dequeue(): Get next task and update to PROCESSING status
  - getStatus(taskId): Retrieve task metadata and status
  - updateStatus(taskId, status): Update task state (COMPLETED/FAILED)
  - getQueueLength(): Monitor queue depth
  - clearQueue(): Queue management utility
  - healthCheck(): Verify Valkey connectivity
- Add TaskDto, EnqueueTaskDto, UpdateTaskStatusDto interfaces
- Implement TaskStatus enum (PENDING/PROCESSING/COMPLETED/FAILED)
- Add comprehensive test suite with in-memory Redis mock (20 tests)
- Integrate ValkeyModule into app.module.ts
- Valkey Docker Compose service already configured in docker-compose.yml
- VALKEY_URL environment variable already in .env.example
- Add detailed README with usage examples and API documentation

Technical Details:
- Uses FIFO queue (RPUSH/LPOP for strict ordering)
- Task metadata stored with 24-hour TTL
- Lifecycle hooks for connection management (onModuleInit/onModuleDestroy)
- Automatic retry with exponential backoff on connection errors
- Global module - no explicit imports needed

Tests verify:
- Connection initialization and health checks
- FIFO enqueue/dequeue behavior
- Status lifecycle transitions
- Concurrent task handling
- Queue management operations
- Complete task processing workflows
2026-01-29 23:25:33 -06:00
Jason Woltje
abbf886483 fix: resolve TypeScript errors in migrated components 2026-01-29 22:00:14 -06:00
Jason Woltje
287a0e2556 feat(knowledge): add markdown rendering (KNOW-004)
- Install marked, marked-highlight, marked-gfm-heading-id, sanitize-html
- Create markdown utility with GFM support (tables, task lists, strikethrough)
- Add code syntax highlighting with highlight.js
- Implement XSS sanitization for security
- Update knowledge service to use markdown renderer
- Add comprehensive test suite (34 tests, all passing)
- Generate IDs for headers for deep linking
- Cache rendered HTML in database for performance
2026-01-29 16:57:57 -06:00
Jason Woltje
f07f04404d feat(knowledge): add tag management API (KNOW-003)
- Add Tag DTOs (CreateTagDto, UpdateTagDto) with validation
- Implement TagsService with CRUD operations
- Add TagsController with authenticated endpoints
- Support automatic slug generation from tag names
- Add workspace isolation for tags
- Include entry count in tag responses
- Add findOrCreateTags method for entry creation/update
- Implement comprehensive test coverage (29 tests passing)

Endpoints:
- GET /api/knowledge/tags - List workspace tags
- POST /api/knowledge/tags - Create tag
- GET /api/knowledge/tags/:slug - Get tag by slug
- PUT /api/knowledge/tags/:slug - Update tag
- DELETE /api/knowledge/tags/:slug - Delete tag
- GET /api/knowledge/tags/:slug/entries - List entries with tag

Related: KNOW-003
2026-01-29 16:13:40 -06:00
Jason Woltje
973502f26e feat(#37-41): Add domains, ideas, relationships, agents, widgets schema
Schema additions for issues #37-41:

New models:
- Domain (#37): Life domains (work, marriage, homelab, etc.)
- Idea (#38): Brain dumps with pgvector embeddings
- Relationship (#39): Generic entity linking (blocks, depends_on)
- Agent (#40): ClawdBot agent tracking with metrics
- AgentSession (#40): Conversation session tracking
- WidgetDefinition (#41): HUD widget registry
- UserLayout (#41): Per-user dashboard configuration

Updated models:
- Task, Event, Project: Added domainId foreign key
- User, Workspace: Added new relations

New enums:
- IdeaStatus: CAPTURED, PROCESSING, ACTIONABLE, ARCHIVED, DISCARDED
- RelationshipType: BLOCKS, BLOCKED_BY, DEPENDS_ON, etc.
- AgentStatus: IDLE, WORKING, WAITING, ERROR, TERMINATED
- EntityType: Added IDEA, DOMAIN

Migration: 20260129182803_add_domains_ideas_agents_widgets
2026-01-29 12:29:21 -06:00
Jason Woltje
6a038d093b feat(#4): Implement Authentik OIDC authentication with BetterAuth
- Integrated BetterAuth library for modern authentication
- Added Session, Account, and Verification database tables
- Created complete auth module with service, controller, guards, and decorators
- Implemented shared authentication types in @mosaic/shared package
- Added comprehensive test coverage (26 tests passing)
- Documented type sharing strategy for monorepo
- Updated environment configuration with OIDC and JWT settings

Key architectural decisions:
- BetterAuth over Passport.js for better TypeScript support
- Separation of User (DB entity) vs AuthUser (client-safe subset)
- Shared types package to prevent FE/BE drift
- Factory pattern for auth config to use shared Prisma instance

Ready for frontend integration (Issue #6).

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Fixes #4
2026-01-28 17:26:34 -06:00
Jason Woltje
99afde4f99 feat(#2): Implement PostgreSQL 17 + pgvector database schema
Establishes multi-tenant database layer with vector similarity search for AI-powered memory features. Includes Docker infrastructure, Prisma ORM integration, NestJS services, and shared types across the monorepo.

Key changes:
- Docker: PostgreSQL 17 + pgvector v0.7.4, Valkey cache
- Schema: 8 models (User, Workspace, Task, Event, Project, ActivityLog, MemoryEmbedding) with RLS preparation
- NestJS: PrismaModule, DatabaseModule, EmbeddingsService
- Shared: Type-safe enums, constants, and database types

Fixes #2

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-28 16:06:34 -06:00
Jason Woltje
92e20b1686 feat(#1): Set up monorepo scaffold with pnpm workspaces + TurboRepo
Implements the foundational project structure including:
- pnpm workspaces configuration
- TurboRepo for build orchestration
- NestJS 11.1.12 API (apps/api)
- Next.js 16.1.6 web app (apps/web)
- Shared packages (config, shared, ui)
- TypeScript strict mode configuration
- ESLint + Prettier setup
- Vitest for unit testing (19 passing tests)

Fixes #1

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 13:31:33 -06:00