feat(memory): add operator retrieval plugin
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful
All checks were successful
ci/woodpecker/pr/ci Pipeline was successful
This commit is contained in:
27
docs/scratchpads/tess-m4-003-operator-plugins.md
Normal file
27
docs/scratchpads/tess-m4-003-operator-plugins.md
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
# TESS-M4-003 — Operator Plugin Foundations
|
||||||
|
|
||||||
|
- **Task:** TESS-M4-003 / TESS-MEM-001
|
||||||
|
- **Branch/base:** `feat/tess-operator-plugins` rebased on `origin/main` `76325ca3`
|
||||||
|
- **Scope:** first leaf-package memory/retrieval slice only; no durable inbox ownership, gateway integration, or Mosaic catalog implementation.
|
||||||
|
|
||||||
|
## Handoff
|
||||||
|
|
||||||
|
Coder4's uncommitted implementation was preserved first in commit `5b99c821` before review. The completion pass corrected the contract so namespace is injected configuration rather than caller-selected scope data, storage keys include tenant/owner/session via collision-safe tuple encoding, and malformed runtime scope values fail closed.
|
||||||
|
|
||||||
|
## Delivered boundary
|
||||||
|
|
||||||
|
- `OperatorMemoryPlugin` exposes `capture`, `search`, `recent`, `stats`, and `startupContext` through `MemoryAdapter` only.
|
||||||
|
- Scope is server-derived `{tenantId, ownerId, sessionId}`; adapter and namespace are configuration, not operation input.
|
||||||
|
- Capture redacts before persistence and records configured instance/namespace/source provenance.
|
||||||
|
- Wildcard retrieval is documented at the `MemoryAdapter` boundary and implemented by the keyword adapter.
|
||||||
|
- Startup context uses a bounded 64-result candidate window, then prioritizes project and flat-file provenance before slicing the configured output limit.
|
||||||
|
- No `Tess` identity is hardcoded in storage keys or defaults; tests use configured `Nova`.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
- `pnpm --filter @mosaicstack/memory test` — PASS (32 tests)
|
||||||
|
- `pnpm --filter @mosaicstack/memory typecheck` — PASS
|
||||||
|
- `pnpm --filter @mosaicstack/memory lint` — PASS
|
||||||
|
- `pnpm --filter @mosaicstack/memory build` — PASS
|
||||||
|
- Codex code review — APPROVE after remediation
|
||||||
|
- Codex security review — no findings after runtime scope-validation remediation
|
||||||
@@ -2,9 +2,9 @@
|
|||||||
|
|
||||||
## Memory/retrieval slice — TESS-MEM-001
|
## Memory/retrieval slice — TESS-MEM-001
|
||||||
|
|
||||||
Introduce a transport-neutral `OperatorMemoryPlugin` in `packages/memory`. The plugin receives a server-derived `{tenantId, ownerId, sessionId}` scope and delegates to a registered `MemoryAdapter`; adapter and namespace are injected configuration, never caller input. Its operations are `capture`, `search`, `recent`, `stats`, and `startupContext`. Results carry configured instance, provenance, and namespace metadata. Capture/redaction occurs before adapter persistence; startup context is bounded and ordered so project/flat-file truth can take precedence over retrieved material.
|
Introduce a transport-neutral `OperatorMemoryPlugin` in `packages/memory`. The plugin receives a server-derived `{tenantId, ownerId, sessionId}` scope and delegates to a registered `MemoryAdapter`; adapter and namespace are injected configuration, never caller input. Its operations are `capture`, `search`, `recent`, `stats`, and `startupContext`. Results carry configured instance, provenance, and namespace metadata. Capture/redaction occurs before adapter persistence; startup context uses a bounded candidate window ordered so project/flat-file truth takes precedence within returned material.
|
||||||
|
|
||||||
Registration remains replaceable-adapter based: `registerOperatorMemoryAdapter(kind, factory)` and `createOperatorMemoryPlugin(config)` resolve a configured adapter. Identity and namespace are configuration/scope data; no interaction-agent name is embedded in keys or defaults.
|
Registration remains replaceable-adapter based: the existing `registerMemoryAdapter(kind, factory)` / `createMemoryAdapter(config)` seam supplies the injected adapter to `createOperatorMemoryPlugin(config)`. Identity and namespace are configuration data; no interaction-agent name is embedded in keys or defaults.
|
||||||
|
|
||||||
## Remaining plugin foundations — TESS-PLG-001
|
## Remaining plugin foundations — TESS-PLG-001
|
||||||
|
|
||||||
@@ -14,4 +14,4 @@ Registration remains replaceable-adapter based: `registerOperatorMemoryAdapter(k
|
|||||||
|
|
||||||
## First implementation boundary
|
## First implementation boundary
|
||||||
|
|
||||||
The first PR slice should add the operator-memory plugin contract, config-driven factory registration, scope isolation, provenance-bearing retrieval, and tests for namespace isolation plus a differently named configured instance. Durable inbox/outbox remains owned by the existing `DurableSessionCoordinator`; this plugin only supplies bounded context/capture at lifecycle boundaries.
|
The first PR slice should add the operator-memory plugin contract, configuration-injected adapter seam, scope isolation, provenance-bearing retrieval, and tests for namespace isolation plus a differently named configured instance. Durable inbox/outbox remains owned by the existing `DurableSessionCoordinator`; this plugin only supplies bounded context/capture at lifecycle boundaries.
|
||||||
|
|||||||
@@ -274,6 +274,20 @@ describe('KeywordAdapter', () => {
|
|||||||
expect(results).toHaveLength(1);
|
expect(results).toHaveLength(1);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('should return all scoped insights for the explicit wildcard query', async () => {
|
||||||
|
await adapter.storeInsight({
|
||||||
|
userId: 'u1',
|
||||||
|
content: 'A literal * marker is still ordinary content',
|
||||||
|
source: 'chat',
|
||||||
|
category: 'technical',
|
||||||
|
relevanceScore: 0.7,
|
||||||
|
});
|
||||||
|
|
||||||
|
const results = await adapter.searchInsights('u1', '*');
|
||||||
|
expect(results).toHaveLength(4);
|
||||||
|
expect(results.every((result) => result.score === 1)).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
it('should return empty for empty query', async () => {
|
it('should return empty for empty query', async () => {
|
||||||
const results = await adapter.searchInsights('u1', ' ');
|
const results = await adapter.searchInsights('u1', ' ');
|
||||||
expect(results).toHaveLength(0);
|
expect(results).toHaveLength(0);
|
||||||
|
|||||||
@@ -132,19 +132,23 @@ export class KeywordAdapter implements MemoryAdapter {
|
|||||||
opts?: { limit?: number; embedding?: number[] },
|
opts?: { limit?: number; embedding?: number[] },
|
||||||
): Promise<InsightSearchResult[]> {
|
): Promise<InsightSearchResult[]> {
|
||||||
const limit = opts?.limit ?? 10;
|
const limit = opts?.limit ?? 10;
|
||||||
const words = query
|
const normalizedQuery = query.trim();
|
||||||
.toLowerCase()
|
const matchAll = normalizedQuery === '*';
|
||||||
.split(/\s+/)
|
const words = matchAll
|
||||||
.filter((w) => w.length > 0);
|
? []
|
||||||
|
: normalizedQuery
|
||||||
|
.toLowerCase()
|
||||||
|
.split(/\s+/)
|
||||||
|
.filter((word) => word.length > 0);
|
||||||
|
|
||||||
if (words.length === 0) return [];
|
if (words.length === 0 && !matchAll) return [];
|
||||||
|
|
||||||
const rows = await this.storage.find<InsightRecord>(INSIGHTS, { userId });
|
const rows = await this.storage.find<InsightRecord>(INSIGHTS, { userId });
|
||||||
|
|
||||||
const scored: InsightSearchResult[] = [];
|
const scored: InsightSearchResult[] = [];
|
||||||
for (const row of rows) {
|
for (const row of rows) {
|
||||||
const content = row.content.toLowerCase();
|
const content = row.content.toLowerCase();
|
||||||
let score = 0;
|
let score = matchAll ? 1 : 0;
|
||||||
for (const word of words) {
|
for (const word of words) {
|
||||||
if (content.includes(word)) score++;
|
if (content.includes(word)) score++;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -55,6 +55,24 @@ describe('OperatorMemoryPlugin', () => {
|
|||||||
]);
|
]);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('rejects an incomplete runtime scope before it can produce a shared storage key', async () => {
|
||||||
|
const memory = adapter();
|
||||||
|
const plugin = createOperatorMemoryPlugin({
|
||||||
|
adapter: memory,
|
||||||
|
instanceId: 'Nova',
|
||||||
|
namespace: 'operator-memory',
|
||||||
|
redact: (value) => value,
|
||||||
|
});
|
||||||
|
|
||||||
|
await expect(
|
||||||
|
plugin.capture(
|
||||||
|
{ tenantId: 'tenant-a', ownerId: 'owner-a', sessionId: ' ' },
|
||||||
|
{ content: 'note', source: 'test', category: 'note' },
|
||||||
|
),
|
||||||
|
).rejects.toThrow('Operator memory session ID is required');
|
||||||
|
expect(memory.storeInsight).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
it('uses a differently named configured instance in retrieval provenance', async () => {
|
it('uses a differently named configured instance in retrieval provenance', async () => {
|
||||||
const memory = adapter();
|
const memory = adapter();
|
||||||
memory.searchInsights.mockResolvedValue([
|
memory.searchInsights.mockResolvedValue([
|
||||||
@@ -76,6 +94,31 @@ describe('OperatorMemoryPlugin', () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('orders startup context with project and flat-file truth before retrieved material', async () => {
|
||||||
|
const memory = adapter();
|
||||||
|
memory.searchInsights.mockResolvedValue([
|
||||||
|
{ id: 'retrieval', content: 'retrieval', score: 1 },
|
||||||
|
{ id: 'flat-file', content: 'flat-file', score: 1, metadata: { source: 'flat-file' } },
|
||||||
|
{ id: 'project', content: 'project', score: 1, metadata: { source: 'project' } },
|
||||||
|
]);
|
||||||
|
const plugin = createOperatorMemoryPlugin({
|
||||||
|
adapter: memory,
|
||||||
|
instanceId: 'Nova',
|
||||||
|
namespace: 'operator-memory',
|
||||||
|
maxStartupContext: 2,
|
||||||
|
redact: (value) => value,
|
||||||
|
});
|
||||||
|
|
||||||
|
const context = await plugin.startupContext({
|
||||||
|
tenantId: 'tenant-a',
|
||||||
|
ownerId: 'owner-a',
|
||||||
|
sessionId: 'session-a',
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(context.map((result) => result.id)).toEqual(['project', 'flat-file']);
|
||||||
|
expect(memory.searchInsights).toHaveBeenCalledWith(expect.any(String), '*', { limit: 64 });
|
||||||
|
});
|
||||||
|
|
||||||
it('redacts content before adapter persistence and records configured provenance metadata', async () => {
|
it('redacts content before adapter persistence and records configured provenance metadata', async () => {
|
||||||
const memory = adapter();
|
const memory = adapter();
|
||||||
const plugin = createOperatorMemoryPlugin({
|
const plugin = createOperatorMemoryPlugin({
|
||||||
|
|||||||
@@ -1,5 +1,7 @@
|
|||||||
import type { Insight, InsightSearchResult, MemoryAdapter } from './types.js';
|
import type { Insight, InsightSearchResult, MemoryAdapter } from './types.js';
|
||||||
|
|
||||||
|
const STARTUP_CONTEXT_CANDIDATE_LIMIT = 64;
|
||||||
|
|
||||||
/** Immutable server-derived boundary; callers never choose an adapter namespace. */
|
/** Immutable server-derived boundary; callers never choose an adapter namespace. */
|
||||||
export interface OperatorMemoryScope {
|
export interface OperatorMemoryScope {
|
||||||
readonly tenantId: string;
|
readonly tenantId: string;
|
||||||
@@ -38,8 +40,44 @@ export interface OperatorMemoryPlugin {
|
|||||||
}
|
}
|
||||||
|
|
||||||
function scopedUserId(scope: OperatorMemoryScope, namespace: string): string {
|
function scopedUserId(scope: OperatorMemoryScope, namespace: string): string {
|
||||||
|
const normalizedScope = normalizeScope(scope);
|
||||||
// JSON tuple encoding avoids delimiter collisions between independently scoped IDs.
|
// JSON tuple encoding avoids delimiter collisions between independently scoped IDs.
|
||||||
return JSON.stringify([namespace, scope.tenantId, scope.ownerId, scope.sessionId]);
|
return JSON.stringify([
|
||||||
|
namespace,
|
||||||
|
normalizedScope.tenantId,
|
||||||
|
normalizedScope.ownerId,
|
||||||
|
normalizedScope.sessionId,
|
||||||
|
]);
|
||||||
|
}
|
||||||
|
|
||||||
|
function normalizeScope(scope: OperatorMemoryScope): OperatorMemoryScope {
|
||||||
|
if (typeof scope !== 'object' || scope === null) {
|
||||||
|
throw new Error('Operator memory scope is required');
|
||||||
|
}
|
||||||
|
return Object.freeze({
|
||||||
|
tenantId: requiredScopeId(scope.tenantId, 'tenant ID'),
|
||||||
|
ownerId: requiredScopeId(scope.ownerId, 'owner ID'),
|
||||||
|
sessionId: requiredScopeId(scope.sessionId, 'session ID'),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function requiredScopeId(value: unknown, field: string): string {
|
||||||
|
if (typeof value !== 'string' || value.trim().length === 0) {
|
||||||
|
throw new Error(`Operator memory ${field} is required`);
|
||||||
|
}
|
||||||
|
return value.trim();
|
||||||
|
}
|
||||||
|
|
||||||
|
function compareStartupContext(left: OperatorMemoryResult, right: OperatorMemoryResult): number {
|
||||||
|
return (
|
||||||
|
startupSourcePriority(left.provenance.source) - startupSourcePriority(right.provenance.source)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function startupSourcePriority(source: string): number {
|
||||||
|
if (source === 'project') return 0;
|
||||||
|
if (source === 'flat-file') return 1;
|
||||||
|
return 2;
|
||||||
}
|
}
|
||||||
|
|
||||||
function normalizeConfig(config: OperatorMemoryConfig): OperatorMemoryConfig {
|
function normalizeConfig(config: OperatorMemoryConfig): OperatorMemoryConfig {
|
||||||
@@ -82,7 +120,7 @@ export function createOperatorMemoryPlugin(config: OperatorMemoryConfig): Operat
|
|||||||
).map(mapResult);
|
).map(mapResult);
|
||||||
return {
|
return {
|
||||||
async capture(scope, input) {
|
async capture(scope, input) {
|
||||||
return config.adapter.storeInsight({
|
return pluginConfig.adapter.storeInsight({
|
||||||
userId: scopedUserId(scope, pluginConfig.namespace),
|
userId: scopedUserId(scope, pluginConfig.namespace),
|
||||||
content: pluginConfig.redact(input.content),
|
content: pluginConfig.redact(input.content),
|
||||||
source: input.source,
|
source: input.source,
|
||||||
@@ -106,7 +144,11 @@ export function createOperatorMemoryPlugin(config: OperatorMemoryConfig): Operat
|
|||||||
};
|
};
|
||||||
},
|
},
|
||||||
async startupContext(scope) {
|
async startupContext(scope) {
|
||||||
return search(scope, '*', pluginConfig.maxStartupContext);
|
const maxStartupContext = pluginConfig.maxStartupContext ?? 8;
|
||||||
|
// Prioritize authoritative sources within a bounded candidate window.
|
||||||
|
const candidateLimit = Math.max(maxStartupContext, STARTUP_CONTEXT_CANDIDATE_LIMIT);
|
||||||
|
const context = await search(scope, '*', candidateLimit);
|
||||||
|
return [...context].sort(compareStartupContext).slice(0, maxStartupContext);
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -49,6 +49,10 @@ export interface MemoryAdapter {
|
|||||||
// Insights
|
// Insights
|
||||||
storeInsight(insight: NewInsight): Promise<Insight>;
|
storeInsight(insight: NewInsight): Promise<Insight>;
|
||||||
getInsight(id: string): Promise<Insight | null>;
|
getInsight(id: string): Promise<Insight | null>;
|
||||||
|
/**
|
||||||
|
* Searches within one scoped user ID. The reserved `*` query returns scoped
|
||||||
|
* recent/all results rather than performing backend-specific wildcard parsing.
|
||||||
|
*/
|
||||||
searchInsights(
|
searchInsights(
|
||||||
userId: string,
|
userId: string,
|
||||||
query: string,
|
query: string,
|
||||||
|
|||||||
Reference in New Issue
Block a user