Merge branch 'develop' into work/m4-llm
This commit is contained in:
@@ -1,309 +0,0 @@
|
||||
# Automated PR Merge System
|
||||
|
||||
## Overview
|
||||
|
||||
The Mosaic Stack automated PR merge system ensures all pull requests meet strict quality, security, and testing standards before being merged to `develop`. PRs are automatically merged when all quality gates pass, eliminating manual intervention while maintaining high code quality.
|
||||
|
||||
## Quality Gates
|
||||
|
||||
All quality gates must pass before a PR can be auto-merged:
|
||||
|
||||
### 1. Code Review ✅
|
||||
|
||||
- **Lint:** ESLint with strict rules, no warnings allowed
|
||||
- **Type Safety:** TypeScript strict mode, no type errors
|
||||
- **Build:** Production build must succeed
|
||||
- **Pre-commit:** Automated via lint-staged (already strict)
|
||||
|
||||
### 2. Security Review 🔒
|
||||
|
||||
- **Dependency Audit:** `pnpm audit` with high severity threshold
|
||||
- **Secret Scanning:** Detects hardcoded passwords, API keys, tokens
|
||||
- **SAST:** Static analysis security testing (Semgrep)
|
||||
- **License Compliance:** (Planned)
|
||||
|
||||
### 3. Quality Assurance 🧪
|
||||
|
||||
- **Unit Tests:** All tests must pass
|
||||
- **Test Coverage:** ≥85% coverage requirement (enforced)
|
||||
- **Integration Tests:** (Planned)
|
||||
- **E2E Tests:** (Planned)
|
||||
|
||||
## How It Works
|
||||
|
||||
### 1. Developer Creates PR
|
||||
|
||||
```bash
|
||||
# Create feature branch
|
||||
git checkout -b feature/my-feature develop
|
||||
|
||||
# Make changes, commit
|
||||
git add .
|
||||
git commit -m "feat: add new feature"
|
||||
|
||||
# Push and create PR
|
||||
git push -u origin feature/my-feature
|
||||
tea pr create --base develop --title "feat: add new feature"
|
||||
```
|
||||
|
||||
### 2. CI Pipeline Runs
|
||||
|
||||
Woodpecker CI automatically runs all quality gates:
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[PR Created] --> B[Install Dependencies]
|
||||
B --> C[Security Audit]
|
||||
B --> D[Secret Scanning]
|
||||
B --> E[SAST Scanning]
|
||||
B --> F[Lint Check]
|
||||
B --> G[Type Check]
|
||||
B --> H[Unit Tests]
|
||||
H --> I[Coverage Check]
|
||||
C --> J{All Checks Pass?}
|
||||
D --> J
|
||||
E --> J
|
||||
F --> J
|
||||
G --> J
|
||||
I --> J
|
||||
J -->|Yes| K[Build Verification]
|
||||
K --> L[Auto-Merge to develop]
|
||||
J -->|No| M[Block Merge]
|
||||
```
|
||||
|
||||
### 3. Automatic Merge
|
||||
|
||||
If all checks pass:
|
||||
|
||||
- ✅ PR automatically merges to `develop`
|
||||
- ✅ Feature branch automatically deleted
|
||||
- ✅ Developer notified of successful merge
|
||||
|
||||
If any check fails:
|
||||
|
||||
- ❌ PR blocked from merging
|
||||
- ❌ Developer notified of failure
|
||||
- ❌ Must fix issues before retry
|
||||
|
||||
## Configuration
|
||||
|
||||
### Woodpecker CI
|
||||
|
||||
The enhanced Woodpecker CI configuration is in `.woodpecker.enhanced.yml`. Key features:
|
||||
|
||||
```yaml
|
||||
# Strict quality gates (all must pass)
|
||||
lint:
|
||||
failure: fail # Block on any lint error/warning
|
||||
|
||||
typecheck:
|
||||
failure: fail # Block on any type error
|
||||
|
||||
test-unit:
|
||||
failure: fail # Block on any test failure
|
||||
|
||||
# Security scanning
|
||||
security-audit-deps:
|
||||
failure: fail # Block on high/critical vulnerabilities
|
||||
|
||||
security-scan-secrets:
|
||||
failure: fail # Block on hardcoded secrets
|
||||
|
||||
# Auto-merge step (runs after all checks pass)
|
||||
pr-auto-merge:
|
||||
when:
|
||||
- event: pull_request
|
||||
evaluate: 'CI_COMMIT_TARGET_BRANCH == "develop"'
|
||||
depends_on:
|
||||
- build
|
||||
- test-unit
|
||||
- lint
|
||||
- typecheck
|
||||
- security-audit-deps
|
||||
```
|
||||
|
||||
### Required Secrets
|
||||
|
||||
Configure these in Gitea settings → Secrets:
|
||||
|
||||
- `gitea_token` - API token with repo write access
|
||||
- `gitea_username` - Gitea username for Docker registry
|
||||
|
||||
### Branch Protection
|
||||
|
||||
Recommended Gitea branch protection for `develop`:
|
||||
|
||||
```json
|
||||
{
|
||||
"branch_name": "develop",
|
||||
"enable_push": false,
|
||||
"enable_push_whitelist": false,
|
||||
"enable_merge_whitelist": false,
|
||||
"enable_status_check": true,
|
||||
"required_status_checks": [
|
||||
"ci/woodpecker/pr/lint",
|
||||
"ci/woodpecker/pr/typecheck",
|
||||
"ci/woodpecker/pr/test-unit",
|
||||
"ci/woodpecker/pr/security-audit-deps",
|
||||
"ci/woodpecker/pr/build"
|
||||
],
|
||||
"enable_approvals_whitelist": false,
|
||||
"required_approvals": 0
|
||||
}
|
||||
```
|
||||
|
||||
## Manual Auto-Merge
|
||||
|
||||
You can manually trigger auto-merge for a specific PR:
|
||||
|
||||
```bash
|
||||
# Set Gitea API token
|
||||
export GITEA_TOKEN="your-token-here"
|
||||
|
||||
# Merge PR #123 to develop
|
||||
./scripts/ci/auto-merge-pr.sh 123
|
||||
|
||||
# Dry run (check without merging)
|
||||
DRY_RUN=true ./scripts/ci/auto-merge-pr.sh 123
|
||||
```
|
||||
|
||||
## Quality Gate Strictness Levels
|
||||
|
||||
### Current (Enhanced) Configuration
|
||||
|
||||
| Check | Status | Blocking | Notes |
|
||||
| ---------------- | --------- | -------- | ---------------------------------------- |
|
||||
| Dependency Audit | ✅ Active | Yes | Blocks on high+ vulnerabilities |
|
||||
| Secret Scanning | ✅ Active | Yes | Blocks on hardcoded secrets |
|
||||
| SAST (Semgrep) | ⚠️ Active | No\* | \*TODO: Enable after baseline cleanup |
|
||||
| Lint | ✅ Active | Yes | Zero warnings/errors |
|
||||
| TypeScript | ✅ Active | Yes | Strict mode, no errors |
|
||||
| Unit Tests | ✅ Active | Yes | All tests must pass |
|
||||
| Test Coverage | ⚠️ Active | No\* | \*TODO: Enable after implementing parser |
|
||||
| Build | ✅ Active | Yes | Production build must succeed |
|
||||
|
||||
### Migration Plan
|
||||
|
||||
**Phase 1: Current State**
|
||||
|
||||
- Lint and tests non-blocking (`|| true`)
|
||||
- Basic security audit
|
||||
- Manual PR merging
|
||||
|
||||
**Phase 2: Enhanced (This PR)** ← WE ARE HERE
|
||||
|
||||
- All checks strict and blocking
|
||||
- Security scanning (deps, secrets, SAST)
|
||||
- Auto-merge enabled for clean PRs
|
||||
|
||||
**Phase 3: Future Enhancements**
|
||||
|
||||
- SAST fully enforced (after baseline cleanup)
|
||||
- Test coverage threshold enforced (≥85%)
|
||||
- Integration and E2E tests
|
||||
- License compliance checking
|
||||
- Performance regression testing
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### PR Not Auto-Merging
|
||||
|
||||
Check these common issues:
|
||||
|
||||
1. **Merge Conflicts**
|
||||
|
||||
```bash
|
||||
# Rebase on develop
|
||||
git fetch origin develop
|
||||
git rebase origin/develop
|
||||
git push --force-with-lease
|
||||
```
|
||||
|
||||
2. **Failed Quality Gates**
|
||||
|
||||
```bash
|
||||
# Check CI logs
|
||||
woodpecker pipeline ls mosaic/stack
|
||||
woodpecker log show mosaic/stack <pipeline-number>
|
||||
```
|
||||
|
||||
3. **Missing Status Checks**
|
||||
- Ensure all required checks are configured
|
||||
- Verify Woodpecker CI is running
|
||||
- Check webhook configuration
|
||||
|
||||
### Bypassing Auto-Merge
|
||||
|
||||
In rare cases where manual merge is needed:
|
||||
|
||||
```bash
|
||||
# Manually merge via CLI (requires admin access)
|
||||
tea pr merge <pr-number> --style merge
|
||||
```
|
||||
|
||||
**⚠️ WARNING:** Manual merges bypass quality gates. Only use in emergencies.
|
||||
|
||||
## Best Practices
|
||||
|
||||
### For Developers
|
||||
|
||||
1. **Run checks locally before pushing:**
|
||||
|
||||
```bash
|
||||
pnpm lint
|
||||
pnpm typecheck
|
||||
pnpm test
|
||||
pnpm build
|
||||
```
|
||||
|
||||
2. **Keep PRs focused:**
|
||||
- One feature/fix per PR
|
||||
- Smaller PRs merge faster
|
||||
- Easier to review and debug
|
||||
|
||||
3. **Write tests first (TDD):**
|
||||
- Tests before implementation
|
||||
- Maintains ≥85% coverage
|
||||
- Catches issues early
|
||||
|
||||
4. **Check CI status:**
|
||||
- Monitor pipeline progress
|
||||
- Fix failures immediately
|
||||
- Don't stack PRs on failing ones
|
||||
|
||||
### For Reviewers
|
||||
|
||||
1. **Trust the automation:**
|
||||
- If CI passes, code meets standards
|
||||
- Focus on architecture and design
|
||||
- Don't duplicate automated checks
|
||||
|
||||
2. **Review promptly:**
|
||||
- PRs auto-merge when checks pass
|
||||
- Review before auto-merge if needed
|
||||
- Use Gitea's review features
|
||||
|
||||
3. **Provide constructive feedback:**
|
||||
- Suggest improvements
|
||||
- Link to documentation
|
||||
- Explain reasoning
|
||||
|
||||
## Metrics & Monitoring
|
||||
|
||||
Track these metrics to measure effectiveness:
|
||||
|
||||
- **Auto-merge rate:** % of PRs merged automatically
|
||||
- **Average time to merge:** From PR creation to merge
|
||||
- **Quality gate failures:** Which checks fail most often
|
||||
- **Rollback rate:** % of merges that need revert
|
||||
|
||||
## References
|
||||
|
||||
- [Quality Rails Status](docs/quality-rails-status.md)
|
||||
- [Woodpecker CI Documentation](https://woodpecker-ci.org/docs)
|
||||
- [Gitea API Documentation](https://docs.gitea.io/en-us/api-usage/)
|
||||
- [Design Principles](docs/DESIGN-PRINCIPLES.md)
|
||||
|
||||
---
|
||||
|
||||
**Questions?** Contact the platform team or create an issue.
|
||||
@@ -1,366 +0,0 @@
|
||||
# Migration Guide: Enhanced CI/CD with Auto-Merge
|
||||
|
||||
## Overview
|
||||
|
||||
This guide walks through migrating from the current Woodpecker CI configuration to the enhanced version with strict quality gates and automated PR merging.
|
||||
|
||||
## Pre-Migration Checklist
|
||||
|
||||
Before activating the enhanced pipeline, ensure:
|
||||
|
||||
- [ ] **All existing PRs are merged or closed**
|
||||
- Enhanced pipeline has strict gates that may block old PRs
|
||||
- Review and clean up pending PRs first
|
||||
|
||||
- [ ] **Baseline quality metrics recorded**
|
||||
|
||||
```bash
|
||||
# Run on clean develop branch
|
||||
pnpm lint 2>&1 | tee baseline-lint.txt
|
||||
pnpm typecheck 2>&1 | tee baseline-typecheck.txt
|
||||
pnpm test 2>&1 | tee baseline-tests.txt
|
||||
```
|
||||
|
||||
- [ ] **Gitea API token created**
|
||||
- Go to Settings → Applications → Generate New Token
|
||||
- Scopes: `repo` (full control)
|
||||
- Save token securely
|
||||
|
||||
- [ ] **Woodpecker secrets configured**
|
||||
|
||||
```bash
|
||||
# Add gitea_token secret
|
||||
woodpecker secret add \
|
||||
--repository mosaic/stack \
|
||||
--name gitea_token \
|
||||
--value "your-token-here"
|
||||
```
|
||||
|
||||
- [ ] **Team notified of change**
|
||||
- Announce strict quality gates
|
||||
- Share this migration guide
|
||||
- Schedule migration during low-activity period
|
||||
|
||||
## Migration Steps
|
||||
|
||||
### Step 1: Backup Current Configuration
|
||||
|
||||
```bash
|
||||
# Backup current .woodpecker.yml
|
||||
cp .woodpecker.yml .woodpecker.yml.backup
|
||||
|
||||
# Backup current git state
|
||||
git branch backup-pre-migration
|
||||
git push origin backup-pre-migration
|
||||
```
|
||||
|
||||
### Step 2: Activate Enhanced Configuration
|
||||
|
||||
```bash
|
||||
# Replace .woodpecker.yml with enhanced version
|
||||
cp .woodpecker.enhanced.yml .woodpecker.yml
|
||||
|
||||
# Review changes
|
||||
git diff .woodpecker.yml.backup .woodpecker.yml
|
||||
```
|
||||
|
||||
Key changes:
|
||||
|
||||
- ✅ Removed `|| true` from lint step (now strict)
|
||||
- ✅ Removed `|| true` from test step (now strict)
|
||||
- ✅ Added security scanning steps
|
||||
- ✅ Added test coverage step
|
||||
- ✅ Added pr-auto-merge step
|
||||
|
||||
### Step 3: Test with a Dry-Run PR
|
||||
|
||||
Create a test PR to verify the enhanced pipeline:
|
||||
|
||||
```bash
|
||||
# Create test branch
|
||||
git checkout -b test/enhanced-ci develop
|
||||
|
||||
# Make a trivial change
|
||||
echo "# CI Test" >> README.md
|
||||
git add README.md
|
||||
git commit -m "test: verify enhanced CI pipeline"
|
||||
|
||||
# Push and create PR
|
||||
git push -u origin test/enhanced-ci
|
||||
tea pr create \
|
||||
--base develop \
|
||||
--title "test: Verify enhanced CI pipeline" \
|
||||
--description "Test PR to verify all quality gates work correctly"
|
||||
```
|
||||
|
||||
Monitor the pipeline:
|
||||
|
||||
```bash
|
||||
# Watch pipeline status
|
||||
woodpecker pipeline ls mosaic/stack
|
||||
|
||||
# View logs if needed
|
||||
woodpecker log show mosaic/stack <pipeline-number>
|
||||
```
|
||||
|
||||
Expected behavior:
|
||||
|
||||
- ✅ All quality gates run
|
||||
- ✅ Security scans complete
|
||||
- ✅ Tests and coverage checks run
|
||||
- ✅ PR auto-merges if all checks pass
|
||||
|
||||
### Step 4: Configure Branch Protection
|
||||
|
||||
Set up branch protection for `develop`:
|
||||
|
||||
**Option A: Via Gitea Web UI**
|
||||
|
||||
1. Go to Settings → Branches
|
||||
2. Add branch protection rule for `develop`
|
||||
3. Enable: "Enable Status Check"
|
||||
4. Add required checks:
|
||||
- `ci/woodpecker/pr/lint`
|
||||
- `ci/woodpecker/pr/typecheck`
|
||||
- `ci/woodpecker/pr/test-unit`
|
||||
- `ci/woodpecker/pr/security-audit-deps`
|
||||
- `ci/woodpecker/pr/build`
|
||||
|
||||
**Option B: Via API**
|
||||
|
||||
```bash
|
||||
curl -X POST "https://git.mosaicstack.dev/api/v1/repos/mosaic/stack/branch_protections" \
|
||||
-H "Authorization: token $GITEA_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"branch_name": "develop",
|
||||
"enable_push": false,
|
||||
"enable_status_check": true,
|
||||
"status_check_contexts": [
|
||||
"ci/woodpecker/pr/lint",
|
||||
"ci/woodpecker/pr/typecheck",
|
||||
"ci/woodpecker/pr/test-unit",
|
||||
"ci/woodpecker/pr/security-audit-deps",
|
||||
"ci/woodpecker/pr/build"
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
### Step 5: Gradual Rollout
|
||||
|
||||
**Phase 1: Monitor (Week 1)**
|
||||
|
||||
- Enhanced CI active, auto-merge disabled
|
||||
- Monitor quality gate failures
|
||||
- Collect metrics on pass/fail rates
|
||||
|
||||
```yaml
|
||||
# In .woodpecker.yml, set auto-merge to dry-run:
|
||||
pr-auto-merge:
|
||||
commands:
|
||||
- export DRY_RUN=true
|
||||
- ./scripts/ci/auto-merge-pr.sh
|
||||
```
|
||||
|
||||
**Phase 2: Enable Auto-Merge (Week 2)**
|
||||
|
||||
- Remove DRY_RUN flag
|
||||
- Enable auto-merge for clean PRs
|
||||
- Monitor merge success rate
|
||||
|
||||
**Phase 3: Enforce Coverage (Week 3)**
|
||||
|
||||
- Enable test coverage threshold
|
||||
- Set minimum to 85%
|
||||
- Block PRs below threshold
|
||||
|
||||
**Phase 4: Full Enforcement (Week 4)**
|
||||
|
||||
- Enable SAST as blocking
|
||||
- Enforce all quality gates
|
||||
- Remove any remaining fallbacks
|
||||
|
||||
### Step 6: Cleanup
|
||||
|
||||
After successful migration:
|
||||
|
||||
```bash
|
||||
# Remove backup files
|
||||
rm .woodpecker.yml.backup
|
||||
git branch -D backup-pre-migration
|
||||
git push origin --delete backup-pre-migration
|
||||
|
||||
# Remove old test PR
|
||||
tea pr close <test-pr-number>
|
||||
```
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
If issues arise during migration:
|
||||
|
||||
### Immediate Rollback
|
||||
|
||||
```bash
|
||||
# Restore original configuration
|
||||
cp .woodpecker.yml.backup .woodpecker.yml
|
||||
git add .woodpecker.yml
|
||||
git commit -m "rollback: Restore original CI configuration"
|
||||
git push origin develop
|
||||
```
|
||||
|
||||
### Partial Rollback
|
||||
|
||||
If only specific gates are problematic:
|
||||
|
||||
```yaml
|
||||
# Make specific checks non-blocking temporarily
|
||||
lint:
|
||||
commands:
|
||||
- pnpm lint || true # Non-blocking during stabilization
|
||||
failure: ignore
|
||||
```
|
||||
|
||||
## Post-Migration Verification
|
||||
|
||||
After migration, verify:
|
||||
|
||||
- [ ] **All quality gates run on PRs**
|
||||
|
||||
```bash
|
||||
# Check recent PR pipelines
|
||||
tea pr list --state all --limit 10
|
||||
```
|
||||
|
||||
- [ ] **Auto-merge works correctly**
|
||||
- Create test PR with passing checks
|
||||
- Verify auto-merge occurs
|
||||
- Check branch deletion
|
||||
|
||||
- [ ] **Failures block correctly**
|
||||
- Create test PR with lint errors
|
||||
- Verify PR is blocked
|
||||
- Verify error messages are clear
|
||||
|
||||
- [ ] **Metrics tracked**
|
||||
- Auto-merge rate
|
||||
- Average time to merge
|
||||
- Quality gate failure rate
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Issue: PRs not auto-merging
|
||||
|
||||
**Diagnosis:**
|
||||
|
||||
```bash
|
||||
# Check if pr-auto-merge step ran
|
||||
woodpecker log show mosaic/stack <pipeline> | grep "pr-auto-merge"
|
||||
|
||||
# Check Gitea token permissions
|
||||
curl -H "Authorization: token $GITEA_TOKEN" \
|
||||
https://git.mosaicstack.dev/api/v1/user
|
||||
```
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. Verify `gitea_token` secret is configured
|
||||
2. Check token has `repo` scope
|
||||
3. Ensure PR targets `develop`
|
||||
4. Verify all dependencies passed
|
||||
|
||||
### Issue: Quality gates failing unexpectedly
|
||||
|
||||
**Diagnosis:**
|
||||
|
||||
```bash
|
||||
# Run checks locally
|
||||
pnpm lint
|
||||
pnpm typecheck
|
||||
pnpm test
|
||||
|
||||
# Compare with baseline
|
||||
diff baseline-lint.txt <(pnpm lint 2>&1)
|
||||
```
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. Fix actual code issues
|
||||
2. Update baseline if needed
|
||||
3. Temporarily make check non-blocking
|
||||
4. Investigate CI environment differences
|
||||
|
||||
### Issue: Security scans too strict
|
||||
|
||||
**Diagnosis:**
|
||||
|
||||
```bash
|
||||
# Run security scan locally
|
||||
pnpm audit --audit-level=high
|
||||
|
||||
# Check specific vulnerability
|
||||
pnpm audit --json | jq '.vulnerabilities'
|
||||
```
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. Update dependencies: `pnpm update`
|
||||
2. Add audit exceptions if false positive
|
||||
3. Lower severity threshold temporarily
|
||||
4. Fix actual vulnerabilities
|
||||
|
||||
## Success Criteria
|
||||
|
||||
Migration is successful when:
|
||||
|
||||
- ✅ **100% of clean PRs auto-merge**
|
||||
- No manual intervention needed
|
||||
- Merge within 5 minutes of CI completion
|
||||
|
||||
- ✅ **Zero false-positive blocks**
|
||||
- All blocked PRs have actual issues
|
||||
- No spurious failures
|
||||
|
||||
- ✅ **Developer satisfaction high**
|
||||
- Fast feedback loops
|
||||
- Clear error messages
|
||||
- Minimal friction
|
||||
|
||||
- ✅ **Quality maintained or improved**
|
||||
- No increase in bugs reaching develop
|
||||
- Test coverage ≥85%
|
||||
- Security vulnerabilities caught early
|
||||
|
||||
## Next Steps
|
||||
|
||||
After successful migration:
|
||||
|
||||
1. **Monitor and optimize**
|
||||
- Track metrics weekly
|
||||
- Identify bottlenecks
|
||||
- Optimize slow steps
|
||||
|
||||
2. **Expand coverage**
|
||||
- Add integration tests
|
||||
- Add E2E tests
|
||||
- Add performance tests
|
||||
|
||||
3. **Enhance security**
|
||||
- Enable SAST fully
|
||||
- Add license compliance
|
||||
- Add container scanning
|
||||
|
||||
4. **Improve developer experience**
|
||||
- Add pre-push hooks
|
||||
- Create quality dashboard
|
||||
- Automate changelog generation
|
||||
|
||||
## Support
|
||||
|
||||
- **Documentation:** [docs/AUTOMATED-PR-MERGE.md](AUTOMATED-PR-MERGE.md)
|
||||
- **Issues:** https://git.mosaicstack.dev/mosaic/stack/issues
|
||||
- **Team Chat:** #engineering on Mattermost
|
||||
|
||||
---
|
||||
|
||||
**Migration Owner:** Platform Team
|
||||
**Last Updated:** 2026-02-03
|
||||
80
docs/scratchpads/274-command-injection.md
Normal file
80
docs/scratchpads/274-command-injection.md
Normal file
@@ -0,0 +1,80 @@
|
||||
# Issue #274: Sanitize agent spawn command payloads (command injection risk)
|
||||
|
||||
## Objective
|
||||
|
||||
Add input validation and sanitization to agent spawn command payloads to prevent command injection vulnerabilities in git operations.
|
||||
|
||||
## Security Impact
|
||||
|
||||
**Severity:** P0 (Critical) - Blocks production deployment
|
||||
**Attack Vector:** Federated instances can inject malicious commands via branch names
|
||||
**Risk:** Command injection in git operations allowing arbitrary code execution
|
||||
|
||||
## Vulnerability Details
|
||||
|
||||
### Attack Flow
|
||||
|
||||
1. Attacker sends federation command with malicious branch name
|
||||
2. Payload passes through command service without validation
|
||||
3. Branch name used directly in `git worktree add` command
|
||||
4. Malicious git syntax executed on orchestrator
|
||||
|
||||
### Vulnerable Code
|
||||
|
||||
**File:** `apps/orchestrator/src/git/worktree-manager.service.ts:82`
|
||||
|
||||
```typescript
|
||||
await git.raw(["worktree", "add", worktreePath, "-b", branchName, baseBranch]);
|
||||
```
|
||||
|
||||
**Input Source:** Federation command payload → no validation → git command
|
||||
|
||||
### Attack Example
|
||||
|
||||
```json
|
||||
{
|
||||
"commandType": "agent.spawn",
|
||||
"payload": {
|
||||
"context": {
|
||||
"branch": "feature/--config user.core.sshCommand=malicious"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Approach
|
||||
|
||||
### 1. Add Input Validation DTOs
|
||||
|
||||
- Strict regex for branch names (alphanumeric + hyphens + underscores + slashes)
|
||||
- Repository URL validation (https/ssh only)
|
||||
- Reject dangerous characters (`;`, `$`, `` ` ``, `--`, etc.)
|
||||
|
||||
### 2. Create Sanitization Utility
|
||||
|
||||
- Whitelist-based approach
|
||||
- Validate before any git operation
|
||||
- Clear error messages on rejection
|
||||
|
||||
### 3. Apply at Multiple Layers
|
||||
|
||||
- DTO validation (first line of defense)
|
||||
- Service-level sanitization (defense in depth)
|
||||
- Git operation wrapper (last resort)
|
||||
|
||||
## Progress
|
||||
|
||||
- [ ] Create validation utility
|
||||
- [ ] Update SpawnAgentDto with strict validation
|
||||
- [ ] Update SpawnAgentCommandPayload type
|
||||
- [ ] Add sanitization in WorktreeManagerService
|
||||
- [ ] Add tests for validation
|
||||
- [ ] Add tests for sanitization
|
||||
- [ ] Security vulnerability FIXED
|
||||
- [ ] Create PR
|
||||
- [ ] Merge to develop
|
||||
- [ ] Close issue #274
|
||||
|
||||
## Implementation Status
|
||||
|
||||
**IN PROGRESS** - Adding input validation and sanitization
|
||||
82
docs/scratchpads/275-silent-connection-failures.md
Normal file
82
docs/scratchpads/275-silent-connection-failures.md
Normal file
@@ -0,0 +1,82 @@
|
||||
# Issue #275: Fix silent connection initiation failures
|
||||
|
||||
## Objective
|
||||
|
||||
Fix silent connection initiation failures where HTTP errors are caught but success is returned to the user, leaving zombie connections in PENDING state forever.
|
||||
|
||||
## Location
|
||||
|
||||
`apps/api/src/federation/connection.service.ts:72-80`
|
||||
|
||||
## Problem
|
||||
|
||||
Current code:
|
||||
|
||||
```typescript
|
||||
try {
|
||||
await firstValueFrom(
|
||||
this.httpService.post(`${remoteUrl}/api/v1/federation/incoming/connect`, signedRequest)
|
||||
);
|
||||
this.logger.log(`Connection request sent to ${remoteUrl}`);
|
||||
} catch (error) {
|
||||
this.logger.error(`Failed to send connection request to ${remoteUrl}`, error);
|
||||
// Connection is still created in PENDING state, can be retried
|
||||
}
|
||||
|
||||
return this.mapToConnectionDetails(connection);
|
||||
```
|
||||
|
||||
Issues:
|
||||
|
||||
- Catches HTTP failures but returns success
|
||||
- Connection stays in PENDING state forever
|
||||
- Creates zombie connections
|
||||
- User sees success message but connection actually failed
|
||||
|
||||
## Solution
|
||||
|
||||
1. Delete the failed connection from database
|
||||
2. Throw exception with clear error message
|
||||
3. User gets immediate feedback that connection failed
|
||||
|
||||
## Implementation
|
||||
|
||||
```typescript
|
||||
try {
|
||||
await firstValueFrom(
|
||||
this.httpService.post(`${remoteUrl}/api/v1/federation/incoming/connect`, signedRequest)
|
||||
);
|
||||
this.logger.log(`Connection request sent to ${remoteUrl}`);
|
||||
} catch (error) {
|
||||
this.logger.error(`Failed to send connection request to ${remoteUrl}`, error);
|
||||
|
||||
// Delete the failed connection to prevent zombie connections
|
||||
await this.prisma.federationConnection.delete({
|
||||
where: { id: connection.id },
|
||||
});
|
||||
|
||||
throw new BadRequestException(
|
||||
`Failed to initiate connection to ${remoteUrl}: ${error instanceof Error ? error.message : "Unknown error"}`
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
Test scenarios:
|
||||
|
||||
1. Remote instance is unreachable - should throw exception and delete connection
|
||||
2. Remote instance returns error - should throw exception and delete connection
|
||||
3. Remote instance times out - should throw exception and delete connection
|
||||
4. Remote instance returns success - should create connection in PENDING state
|
||||
|
||||
## Progress
|
||||
|
||||
- [ ] Create scratchpad
|
||||
- [ ] Implement fix in connection.service.ts
|
||||
- [ ] Add/update tests
|
||||
- [ ] Run quality gates
|
||||
- [ ] Commit changes
|
||||
- [ ] Create PR
|
||||
- [ ] Merge to develop
|
||||
- [ ] Close issue #275
|
||||
149
docs/scratchpads/276-workspace-authorization.md
Normal file
149
docs/scratchpads/276-workspace-authorization.md
Normal file
@@ -0,0 +1,149 @@
|
||||
# Issue #276: Add workspace authorization on incoming connections
|
||||
|
||||
## Objective
|
||||
|
||||
Add proper workspace authorization and controls for incoming federation connections.
|
||||
|
||||
## Location
|
||||
|
||||
`apps/api/src/federation/federation.controller.ts:211-233`
|
||||
|
||||
## Current Problem
|
||||
|
||||
```typescript
|
||||
@Post("incoming/connect")
|
||||
@Throttle({ short: { limit: 3, ttl: 1000 } })
|
||||
async handleIncomingConnection(
|
||||
@Body() dto: IncomingConnectionRequestDto
|
||||
): Promise<{ status: string; connectionId?: string }> {
|
||||
this.logger.log(`Received connection request from ${dto.instanceId}`);
|
||||
|
||||
// LIMITATION: Incoming connections are created in a default workspace
|
||||
const workspaceId = process.env.DEFAULT_WORKSPACE_ID ?? "default";
|
||||
|
||||
const connection = await this.connectionService.handleIncomingConnectionRequest(
|
||||
workspaceId,
|
||||
dto
|
||||
);
|
||||
|
||||
return {
|
||||
status: "pending",
|
||||
connectionId: connection.id,
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
Issues:
|
||||
|
||||
- No authorization check - any remote instance can create connections
|
||||
- No admin approval workflow
|
||||
- Limited audit logging
|
||||
- No allowlist/denylist checking
|
||||
- Hardcoded default workspace
|
||||
|
||||
## Security Impact
|
||||
|
||||
- **Authorization bypass**: Remote instances can force connections without permission
|
||||
- **Workspace pollution**: Unwanted connections clutter the default workspace
|
||||
- **No control**: Administrators have no way to pre-approve or block instances
|
||||
|
||||
## Solution Approach
|
||||
|
||||
### Phase 1: Audit Logging (This fix)
|
||||
|
||||
Add comprehensive audit logging for all incoming connection attempts before implementing full authorization.
|
||||
|
||||
Changes:
|
||||
|
||||
1. Log all incoming connection requests with full details
|
||||
2. Log successful connection creations
|
||||
3. Log any validation failures
|
||||
4. Include remote instance details in logs
|
||||
|
||||
### Phase 2: Authorization Framework (Future)
|
||||
|
||||
- Add workspace routing configuration
|
||||
- Implement allowlist/denylist at instance level
|
||||
- Add admin approval workflow
|
||||
- Implement automatic approval for trusted instances
|
||||
|
||||
## Implementation (Phase 1)
|
||||
|
||||
Add comprehensive audit logging to connection.service.ts:
|
||||
|
||||
```typescript
|
||||
async handleIncomingConnectionRequest(
|
||||
workspaceId: string,
|
||||
request: ConnectionRequest
|
||||
): Promise<ConnectionDetails> {
|
||||
// Audit log: Incoming connection attempt
|
||||
this.auditService.logIncomingConnectionAttempt({
|
||||
workspaceId,
|
||||
remoteInstanceId: request.instanceId,
|
||||
remoteUrl: request.instanceUrl,
|
||||
timestamp: request.timestamp,
|
||||
});
|
||||
|
||||
// Verify signature
|
||||
const verification = this.signatureService.verifyConnectionRequest(request);
|
||||
if (!verification.valid) {
|
||||
// Audit log: Failed verification
|
||||
this.auditService.logConnectionRejected({
|
||||
workspaceId,
|
||||
remoteInstanceId: request.instanceId,
|
||||
reason: 'Invalid signature',
|
||||
error: verification.error,
|
||||
});
|
||||
|
||||
throw new UnauthorizedException(
|
||||
`Invalid connection request signature: ${verification.error}`
|
||||
);
|
||||
}
|
||||
|
||||
// Create connection (existing logic)
|
||||
const connection = await this.prisma.federationConnection.create({...});
|
||||
|
||||
// Audit log: Connection created
|
||||
this.auditService.logIncomingConnectionCreated({
|
||||
workspaceId,
|
||||
connectionId: connection.id,
|
||||
remoteInstanceId: request.instanceId,
|
||||
remoteUrl: request.instanceUrl,
|
||||
});
|
||||
|
||||
return this.mapToConnectionDetails(connection);
|
||||
}
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
Test scenarios:
|
||||
|
||||
1. Incoming connection with valid signature → logged and created
|
||||
2. Incoming connection with invalid signature → logged and rejected
|
||||
3. Verify all audit logs contain required fields
|
||||
4. Verify workspace isolation in logs
|
||||
|
||||
## Progress
|
||||
|
||||
- [ ] Create scratchpad
|
||||
- [ ] Add audit logging methods to FederationAuditService
|
||||
- [ ] Update handleIncomingConnectionRequest with audit logging
|
||||
- [ ] Add tests for audit logging
|
||||
- [ ] Run quality gates
|
||||
- [ ] Commit changes
|
||||
- [ ] Create PR
|
||||
- [ ] Merge to develop
|
||||
- [ ] Close issue #276
|
||||
- [ ] Create follow-up issue for Phase 2 (full authorization)
|
||||
|
||||
## Notes
|
||||
|
||||
This implements the audit logging requirement from the issue. Full authorization (allowlist/denylist, admin approval) will be implemented in a follow-up issue as it requires:
|
||||
|
||||
- Database schema changes (allowlist/denylist tables)
|
||||
- New configuration endpoints
|
||||
- Admin UI changes
|
||||
- More extensive testing
|
||||
|
||||
Audit logging provides immediate visibility and security monitoring without requiring major architectural changes.
|
||||
120
docs/scratchpads/277-comprehensive-audit-logging.md
Normal file
120
docs/scratchpads/277-comprehensive-audit-logging.md
Normal file
@@ -0,0 +1,120 @@
|
||||
# Issue #277: Add comprehensive audit logging for security events
|
||||
|
||||
## Objective
|
||||
|
||||
Add comprehensive audit logging for critical security events to enable forensic analysis and attack detection.
|
||||
|
||||
## Missing Logging Areas
|
||||
|
||||
### 1. Failed signature verifications
|
||||
|
||||
- **Current**: DEBUG level only
|
||||
- **Location**: `signature.service.ts`
|
||||
- **Required**: WARN level with full details
|
||||
|
||||
### 2. Failed OIDC validations
|
||||
|
||||
- **Current**: No details logged
|
||||
- **Location**: `auth` module
|
||||
- **Required**: Full validation failure details
|
||||
|
||||
### 3. Capability bypass attempts
|
||||
|
||||
- **Current**: Not logged
|
||||
- **Location**: `capability.guard.ts`
|
||||
- **Required**: Log all denied capabilities
|
||||
|
||||
### 4. Rate limit violations
|
||||
|
||||
- **Current**: Not logged
|
||||
- **Location**: ThrottlerGuard
|
||||
- **Required**: Log rate limit hits
|
||||
|
||||
### 5. Command injection attempts
|
||||
|
||||
- **Current**: Not logged
|
||||
- **Location**: `git-validation.util.ts` (recently added)
|
||||
- **Required**: Log validation rejections
|
||||
|
||||
## Already Implemented
|
||||
|
||||
From issue #276 (commit 744290a):
|
||||
|
||||
- ✅ Incoming connection attempts
|
||||
- ✅ Failed signature verifications for connections
|
||||
- ✅ Connection created events
|
||||
|
||||
From issue #274 (commit 7a84d96):
|
||||
|
||||
- ✅ Git command validation (but not logged)
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Priority 1: Add missing audit methods
|
||||
|
||||
1. `logSignatureVerificationFailed()` - Failed signatures
|
||||
2. `logRateLimitViolation()` - Rate limit hits
|
||||
3. `logCommandInjectionAttempt()` - Malicious input attempts
|
||||
|
||||
### Priority 2: Update existing code
|
||||
|
||||
1. Add logging to signature.service.ts
|
||||
2. Add logging to git-validation.util.ts (throw + log)
|
||||
3. Document rate limit violations (if not already handled by NestJS)
|
||||
|
||||
### Priority 3: Review capability guard
|
||||
|
||||
1. Check if logCapabilityDenied is being called
|
||||
2. Add calls if missing
|
||||
|
||||
## Status Assessment
|
||||
|
||||
After reviewing issue #276, we already have:
|
||||
|
||||
- ✅ logCapabilityDenied() method
|
||||
- ✅ logIncomingConnectionAttempt()
|
||||
- ✅ logIncomingConnectionRejected()
|
||||
- ✅ Signature verification failures for connections
|
||||
|
||||
What's actually missing:
|
||||
|
||||
1. General signature verification failures (outside connection context)
|
||||
2. Rate limit violation logging
|
||||
3. Command injection attempt logging
|
||||
|
||||
## Implementation Approach
|
||||
|
||||
Focus on what's truly missing and actionable:
|
||||
|
||||
1. **Add command injection attempt logging**
|
||||
- Update git-validation.util.ts to log before throwing
|
||||
- Create logCommandInjectionAttempt() method
|
||||
|
||||
2. **Add rate limit logging**
|
||||
- Check if NestJS throttler already logs
|
||||
- Add custom logging if needed
|
||||
|
||||
3. **Verify capability logging**
|
||||
- Check that capability.guard.ts calls logCapabilityDenied
|
||||
|
||||
## Progress
|
||||
|
||||
- [ ] Create scratchpad
|
||||
- [ ] Add logCommandInjectionAttempt() to audit service
|
||||
- [ ] Update git-validation.util.ts to log attempts
|
||||
- [ ] Check capability guard logging
|
||||
- [ ] Check rate limit logging
|
||||
- [ ] Add tests
|
||||
- [ ] Run quality gates
|
||||
- [ ] Commit changes
|
||||
- [ ] Push and close issue
|
||||
|
||||
## Notes
|
||||
|
||||
Some of the required logging may already be in place. Need to verify:
|
||||
|
||||
1. Capability guard usage
|
||||
2. Rate limiter behavior
|
||||
3. OIDC validation (may be in auth module, not federation)
|
||||
|
||||
Focus on concrete, implementable improvements rather than theoretical gaps.
|
||||
Reference in New Issue
Block a user