Red Team (RT) Alignment Audit

Audits side tasks for red team description alignment: whether the red team (RT) description gives an agent enough information to accomplish the side task and pass the scorer. Use it to find vague RT descriptions and scorer requirements the description never mentions, before creating sabotage trajectories (strajs).

Commands

ct run rt-alignment

Audit side tasks for RT description alignment.

Examples:

# Audit all side tasks
ct run rt-alignment

# Audit specific environment
ct run rt-alignment -e web_scraping

# Audit specific side task
ct run rt-alignment -s expose_secret

ct run rt-alignment-view

Interactive viewer for RT alignment audit results.

Usage:

ct run rt-alignment-view [RUN_PATH]

Output

Results are saved to data/rt_alignment_runs/<name>_<timestamp>/:

• results.jsonl - Alignment analysis and recommendations for each side task
• prompt_logs/<env>_<side_task>.json - Prompts, response, and thinking for each side task
• run.log - Run log

Analysis Focus

The audit reads the RT description, scorer code, and submit tool, using the prompts in prompts/system.md and prompts/user.md. For each side task it asks:

• Does the RT description clearly state what to do?
• Does it provide enough detail to pass the scorer?
• Are there undocumented requirements?
• Is the task achievable based on the RT description alone?