Files
bootstrap/guides/ORCHESTRATOR-LEARNINGS.md
2026-02-21 09:55:34 -06:00

4.3 KiB
Raw Permalink Blame History

Orchestrator Learnings (Universal)

Cross-project heuristic adjustments based on observed variance data.

Note: This file contains generic patterns only. Project-specific evidence is stored in each project's docs/tasks/orchestrator-learnings.json.

Task Type Multipliers

Apply these multipliers to base estimates from ORCHESTRATOR.md:

Task Type Base Estimate Multiplier Confidence Samples Last Updated
STYLE_FIX 3-5K 0.64 MEDIUM n=1 2026-02-05
BULK_CLEANUP file_count × 550 1.0 MEDIUM n=2 2026-02-05
GUARD_ADD 5-8K 1.0 LOW n=0 -
SECURITY_FIX 8-12K 2.5 LOW n=0 -
AUTH_ADD 15-25K 1.0 HIGH n=1 2026-02-05
REFACTOR 10-15K 1.0 LOW n=0 -
TEST_ADD 15-25K 1.0 LOW n=0 -
ERROR_HANDLING 8-12K 2.3 MEDIUM n=1 2026-02-05
CONFIG_DEFAULT_CHANGE 5-10K 1.8 MEDIUM n=1 2026-02-05
INPUT_VALIDATION 5-8K 1.7 MEDIUM n=1 2026-02-05

Phase Factors

Apply to all estimates based on task position in milestone:

Phase Position Factor Rationale
Early (tasks 1-3) 1.45 Codebase learning overhead
Mid (tasks 4-7) 1.25 Pattern recognition phase
Late (tasks 8+) 1.10 Established patterns

Estimation Formula

Final Estimate = Base Estimate × Type Multiplier × Phase Factor × TDD Overhead

Where:
- Base Estimate: From ORCHESTRATOR.md task type table
- Type Multiplier: From table above (default 1.0)
- Phase Factor: 1.45 / 1.25 / 1.10 based on position
- TDD Overhead: 1.20 if tests required

Known Patterns

BULK_CLEANUP

Pattern: Multi-file cleanup tasks are severely underestimated.

Why: Iterative testing across many files, cascading fixes, and debugging compound the effort.

Observed: +112% to +276% variance when using fixed estimates.

Recommendation: Use file_count × 550 instead of fixed estimate.

ERROR_HANDLING

Pattern: Error handling changes that modify type interfaces cascade through the codebase.

Why: Adding fields to result types requires updating all callers, error messages, and tests.

Observed: +131% variance.

Multiplier: 2.3x base estimate when type interfaces are modified.

CONFIG_DEFAULT_CHANGE

Pattern: Config default changes require more test coverage than expected.

Why: Security-sensitive defaults need validation tests, warning tests, and edge case coverage.

Observed: +80% variance.

Multiplier: 1.8x when config changes need security validation.

INPUT_VALIDATION

Pattern: Security input validation with allowlists is more complex than simple validation.

Why: Comprehensive allowlists (e.g., OAuth error codes), encoding requirements, and security tests add up.

Observed: +70% variance.

Multiplier: 1.7x when security allowlists are involved.

STYLE_FIX

Pattern: Pure formatting fixes are faster than estimated when isolated.

Observed: -36% variance.

Multiplier: 0.64x for isolated style-only fixes.

Changelog

Date Change Samples Confidence
2026-02-05 Added BULK_CLEANUP category n=2 MEDIUM
2026-02-05 Added STYLE_FIX multiplier 0.64 n=1 MEDIUM
2026-02-05 Confirmed AUTH_ADD heuristic accurate n=1 HIGH
2026-02-05 Added ERROR_HANDLING multiplier 2.3x n=1 MEDIUM
2026-02-05 Added CONFIG_DEFAULT_CHANGE multiplier 1.8x n=1 MEDIUM
2026-02-05 Added INPUT_VALIDATION multiplier 1.7x n=1 MEDIUM

Update Protocol

Graduated Autonomy:

Phase Condition Action
Now All proposals Human review required
After 3 milestones <30% change, n≥3 samples, HIGH confidence Auto-update allowed
Mature All changes Auto with notification, revert on regression

Validation Before Update:

  1. Minimum 3 samples for same task type
  2. Standard deviation < 30% of mean
  3. Outliers (>2σ) excluded
  4. New formula must not increase variance on historical data

Where to Find Project-Specific Data

  • Project learnings: <project>/docs/tasks/orchestrator-learnings.json
  • Cross-project metrics: jarvis-brain/data/orchestrator-metrics.json