Missions / Engineering
Flaky test triage and repair
Flaky test identified, root cause diagnosed, and fix proposed with confidence estimate
This mission design is in development. It has not been exercised against a real flaky-test corpus in production.
Architecture summary
Context intelligence gathers test history and failure patterns. Execution agents run tests multiple times to reproduce. Evidence verification captures the diagnostic pattern. Repair proposal is generated with confidence.
Stages
Detection
Identify test with inconsistent pass/fail behavior
Evidence: Test failure rate over recent runs
Reproduction
Run test multiple times to stabilize failure pattern
Evidence: Reproduction rate and failure signature
Diagnosis
Analyze test code for common anti-patterns
Evidence: Diagnosis report with root cause
Repair proposal
Generate fix with confidence estimate
Evidence: Proposed change with rationale
Controls
Reproduction threshold
Minimum reproduction rate required before diagnosis
Confidence gate
Repair proposal requires confidence threshold
Composition
- Capabilities used
- context-intelligence, execution-agents, evidence-verification
- Integration categories used
- source-control, ci-cd
Constraints
- Minimum reproduction attempts required
- Diagnosis must cite specific anti-pattern
- Repair proposal is advisory; human approval required
Related design notes
- Focus on common flaky test anti-patterns
- Repair is proposal-only in MVP
- Designed to integrate with CI observability