Iteration

Tasks can iterate over datasets to process multiple items.

Basic Iteration

task "process_city" {
  objective = "Get weather for ${item.name}, ${item.state}"
 
  iterator {
    dataset  = datasets.city_list
    parallel = true
  }
}

Iterator Attributes

The item Variable (Parallel Only)

In parallel iterated tasks, item refers to the current dataset item and can be used in the objective:

task "send_notification" {
  objective = "Send notification to ${item.email} about ${item.topic}"
 
  iterator {
    dataset  = datasets.notifications
    parallel = true
  }
}

If items are objects with fields, access them as item.field_name.

Note: Sequential tasks (parallel = false) cannot use item in their objective. The commander receives each item via the dataset_next tool instead. See Sequential Mode below.

Sequential vs Parallel

Sequential (Default)

A single commander processes all items in one session. The commander calls dataset_next to fetch each item, processes it, calls submit_output, and repeats until the dataset is exhausted.

task "process_items" {
  objective = "Process each item: calculate the total and flag any anomalies"
 
  iterator {
    dataset  = datasets.items
    parallel = false  # Default
  }
 
  output {
    field "total" {
      type     = "number"
      required = true
    }
    field "flagged" {
      type = "boolean"
    }
  }
}

How it works:

1. The commander receives the task objective once at the start
2. It calls dataset_next to get the first item
3. It defines subtasks for that item using set_subtasks
4. It works through the subtasks, calling complete_subtask after each
5. It calls submit_output with the structured output for that item
6. It calls dataset_next to get the next item (repeating steps 3-5)
7. When dataset_next returns "exhausted", it calls task_complete

Key characteristics:

• Single commander session maintains context across all items
• Predictable processing order
• The objective should describe the work generically, not reference specific item fields
• Fail-fast on first error (after retries exhausted)

Parallel

Each item gets its own independent commander:

task "get_weather" {
  objective = "Get current weather for ${item.name}, ${item.state}"
 
  iterator {
    dataset  = datasets.city_list
    parallel = true
  }
}

• Faster for independent operations
• All iterations run simultaneously
• Each iteration retries independently before overall failure
• The item variable is resolved into the objective for each iteration

Parallel Options

Control parallel execution behavior with these options:

iterator {
  dataset           = datasets.items
  parallel          = true
  concurrency_limit = 5     # Max 5 iterations running at once
  start_delay       = 500   # 500ms delay between starts in first batch
  smoketest         = true  # Run first iteration before starting others
}

concurrency_limit: Limits how many iterations run simultaneously. Useful for rate-limited APIs or resource constraints.

start_delay: Staggers the start of iterations in the first concurrent batch. For example, with start_delay = 500 and concurrency_limit = 5:

• Iteration 0 starts immediately
• Iteration 1 starts at +500ms
• Iteration 2 starts at +1000ms
• etc.

This helps when iterations need to populate shared caches before others start.

smoketest: Runs the first iteration completely before starting the rest. If the first iteration fails (after retries), the remaining iterations are skipped. Useful for catching configuration errors early without wasting resources on doomed iterations.

Example: Weather Report

mission "midwest_weather" {
  commander {
    model = models.anthropic.claude_sonnet_4
  }
  agents = [agents.assistant]
 
  dataset "city_list" {
    description = "Midwest cities to check"
    schema {
      field "name" {
        type     = "string"
        required = true
      }
      field "state" {
        type = "string"
      }
    }
  }
 
  task "load_cities" {
    objective = "Read cities from data.json and populate the city_list dataset"
  }
 
  task "get_weather" {
    objective  = "Get current weather for ${item.name}, ${item.state}"
    depends_on = [tasks.load_cities]
 
    iterator {
      dataset     = datasets.city_list
      parallel    = true
      max_retries = 2  # Retry failed iterations up to 2 times
    }
 
    output {
      field "temperature" {
        type     = "number"
        required = true
      }
      field "conditions" {
        type     = "string"
        required = true
      }
    }
  }
 
  task "analyze" {
    objective  = "Compare temperatures and find the coldest city"
    depends_on = [tasks.get_weather]
  }
}

Error Handling

Output Validation

When a task has an output schema with required fields, each iteration is validated:

• If required output fields are missing, the iteration is marked as failed
• This enables automatic retry without commander intervention

Retry Behavior

Configure retries per iteration with max_retries:

iterator {
  dataset     = datasets.items
  max_retries = 3  # Retry each iteration up to 3 times on failure
}

When an iteration fails:

1. If retries remain, the iteration is automatically retried
2. If all retries are exhausted, fail-fast behavior kicks in
3. Remaining iterations are cancelled (parallel) or skipped (sequential)
4. The task fails with the first unrecoverable error

Empty Datasets

If a dataset is empty, the task completes immediately.

Querying Iteration Commanders

Dependent tasks can query specific iteration commanders using ask_commander with the index parameter. This enables follow-up questions to the commander that processed a particular item.

Getting Iteration Results

First, use query_task_output to get results from the iterated task. Each result includes an index field:

{
  "task": "get_weather",
  "filters": [{"field": "conditions", "op": "eq", "value": "snowy"}]
}
// Returns: [{"city": "Chicago", "temperature": 28, "index": 0}, ...]

Querying a Specific Iteration

Use the index from the query results to ask follow-up questions:

{
  "task_name": "get_weather",
  "index": 0,
  "question": "What was the wind speed in Chicago?"
}

The iteration commander responds from its completed context — it remembers the full conversation with its agents and can query them for additional details.

Example: Character Crossover

task "create_backstories" {
  objective = "Create a backstory for ${item.name}, who is a ${item.role}"
 
  iterator {
    dataset  = datasets.characters
    parallel = true
  }
 
  output {
    field "origin" {
      type     = "string"
      required = true
    }
    field "secret_talent" {
      type     = "string"
      required = true
    }
  }
}
 
task "character_crossover" {
  depends_on = ["create_backstories"]
 
  objective = <<-EOT
    1. Use query_task_output to get all backstories
    2. Pick two interesting characters
    3. For each, use ask_commander with the character's index to ask:
       "How would this character react to meeting someone from another world?"
    4. Write a crossover scene using both answers
  EOT
}

Streaming Output

During iteration, the CLI shows progress:

[2/3] get_weather (5 iterations, parallel)
  -> [0] Processing Chicago, IL...
  -> [1] Processing Detroit, MI...
  -> [2] Processing Milwaukee, WI...
  ✓ [0] Complete
  ✓ [1] Complete
  ✓ [2] Complete
  ✓ Complete