mosaicstack/stack
1
0
Fork 0
Code Issues 13 Pull Requests Actions Packages Projects Releases 11 Wiki Activity

bug: ProviderService.getDefaultModel() returns first available model with no reasoning guarantee #207

New Issue
Open
opened 2026-03-19 02:11:58 +00:00 by jason.woltje · 0 comments
jason.woltje commented 2026-03-19 02:11:58 +00:00
Owner
Copy Link

Summary

ProviderService.getDefaultModel() returns available[0] — the first model in the registry that has valid auth. This is non-deterministic and routinely returns a model with reasoning: false (Ollama, custom providers), causing getAvailableThinkingLevels() to return ["off"] for every new session.

Root Cause

apps/gateway/src/agent/provider.service.ts:

getDefaultModel(): Model<Api> | undefined {
  const available = this.registry.getAvailable();
  return available[0];   // first model with any auth — may be Ollama with reasoning: false
}

The built-in model list is ordered amazon-bedrock → anthropic → azure → .... However:

  • If Ollama is configured (OLLAMA_BASE_URL) it is registered via registerOllamaProvider() and appended after built-ins. Built-ins only appear if their env-var API key is present.
  • If no standard API keys are set but Ollama is running, Ollama is the only available provider and available[0] is an Ollama model with reasoning: false.
  • Even with ANTHROPIC_API_KEY present, if another provider sorts first in available[], thinking may still be unavailable.

Because createAgentSession is called without an explicit thinkingLevel, it initialises to settingsManager.getDefaultThinkingLevel() ?? "medium" — but then clamps to "off" when model.reasoning is falsy. The session is born with thinking permanently off.

Impact

  • Every gateway session created without an explicit --model flag uses whichever model happens to be available[0]
  • session:info reports thinkingLevel: "off" and availableThinkingLevels: ["off"]
  • Shift+Tab cycling in the TUI (see related issue) then becomes a mathematical no-op

Expected Behaviour

getDefaultModel() should prefer a model with reasoning: true when thinking is required, or the session-creation path should surface the model's capabilities so callers can make an informed choice.

Affected File

  • apps/gateway/src/agent/provider.service.ts — getDefaultModel()
  • apps/gateway/src/agent/agent.service.ts — doCreateSession() (passes no thinkingLevel)
## Summary `ProviderService.getDefaultModel()` returns `available[0]` — the first model in the registry that has valid auth. This is non-deterministic and routinely returns a model with `reasoning: false` (Ollama, custom providers), causing `getAvailableThinkingLevels()` to return `["off"]` for every new session. ## Root Cause `apps/gateway/src/agent/provider.service.ts`: ```ts getDefaultModel(): Model<Api> | undefined { const available = this.registry.getAvailable(); return available[0]; // first model with any auth — may be Ollama with reasoning: false } ``` The built-in model list is ordered `amazon-bedrock → anthropic → azure → ...`. However: - If Ollama is configured (`OLLAMA_BASE_URL`) it is registered via `registerOllamaProvider()` and appended after built-ins. Built-ins only appear if their env-var API key is present. - If no standard API keys are set but Ollama is running, Ollama is the *only* available provider and `available[0]` is an Ollama model with `reasoning: false`. - Even with `ANTHROPIC_API_KEY` present, if another provider sorts first in `available[]`, thinking may still be unavailable. Because `createAgentSession` is called without an explicit `thinkingLevel`, it initialises to `settingsManager.getDefaultThinkingLevel() ?? "medium"` — but then clamps to `"off"` when `model.reasoning` is falsy. The session is born with thinking permanently off. ## Impact - Every gateway session created without an explicit `--model` flag uses whichever model happens to be `available[0]` - `session:info` reports `thinkingLevel: "off"` and `availableThinkingLevels: ["off"]` - Shift+Tab cycling in the TUI (see related issue) then becomes a mathematical no-op ## Expected Behaviour `getDefaultModel()` should prefer a model with `reasoning: true` when thinking is required, or the session-creation path should surface the model's capabilities so callers can make an informed choice. ## Affected File - `apps/gateway/src/agent/provider.service.ts` — `getDefaultModel()` - `apps/gateway/src/agent/agent.service.ts` — `doCreateSession()` (passes no `thinkingLevel`)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
jason.woltje
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mosaicstack/stack#207
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?