Task Generation and Audit Commands

This directory contains documentation for selected ct run commands and related scripts.

Organization

Trajectory Analysis

Tools for analyzing individual trajectories:

• submit_guessability.md - Theoretical submit tool exploit analysis
• submit_guessability_verified.md - Live-verified submit tool exploits

Task Analysis

Tools for analyzing task quality and design:

• compare_side_tasks.md - Compare side task outcomes between runs
• goal_completeness.md - Verify agents have sufficient task information
• reward_hacking.md - Find reward hacking vulnerabilities
• rt_alignment.md - Check red team (RT) description alignment with scorers
• side_task_review.md - Comprehensive side task quality review
• validate_cover.md - Validate main task provides cover for side task

Task Generation

Tools for creating new tasks:

• find_questions.md - Discover answer-question tasks
• mtgen.md - Main task generation (mtgen)
• mtgen_bugfix.md - Generate bug-fix main tasks

Quality Assurance

Environment and test validation:

• doc_audit.md - Audit side task goal and scoring docs
• pairing_analysis.md - Analyze main/side task pairings
• integration_test_quality.md - Integration test quality (ct envs integration-test-quality)