Project Dynamo Update: Project Unpaused & New Tasks Available
First, thank you all for your patience while we worked through a number of updates and improvements to the project. We've made several important changes and clarifications to the task creation and evaluation process:


Key Updates


The reference task.toml is now a single canonical block, identical between the "Write instruction.md and task.toml" step and the "task.toml reference" page:


Top-level artifacts
[task]
[metadata] (grouped into pre-seeded / fixed-dataset / fellow-set fields)
[verifier]
[agent]
[environment] (including mcp_servers)



avg_at_8 has been removed as a task.toml field everywhere. The pass@8 acceptance requirements are now covered under the evaluation process described below.
Updated Difficulty Bar & Evaluation Pipeline


The acceptance bar has been raised:


A task must now produce at least 4 valid failures out of 8 trials, measured using GPT-5.4 at extra-high reasoning effort.

Only genuine failures count.
Timeouts, infrastructure issues, and instruction/verifier misalignment do not count.
Any failure caused by misalignment is considered blocking and must be fixed before proceeding.


Stage 1 — Pass@2


Two trials are run.

At least one valid failure is required.

Includes an LLM analysis of the runs.


Stage 2 — Pass@8


Triggered only after Pass@2 succeeds.

Eight new trials are executed.

At least 4 valid failures are required.


Submissions are blocked until all checks have been successfully cleared.


Additional Changes


[verifier].environment_mode should be left unset.

The verifier runs inside the task's single environment/Dockerfile image (canonical TB2).

Harbor overlays tests/ at verification time.

There is no separate tests/Dockerfile.

Category and subcategory are now pre-seeded by the Dynamo team and should not be modified.


Contributors only set:

task_objective
artifact_type



Both should use lowercase snake_case values from .dynamo/diversity-taxonomy.toml.

instruction.md now has a 1,500-token limit.

Any structured-output schema should be documented directly in the prompt or in a referenced specification file.

The separate structured-output line has been removed.


Added a new "Common errors and examples" subsection containing four worked failure cases, including:



Problem description

Code excerpts


Guidance on how to avoid the issue


Added a new "Before you submit: quick self-check" section with a five-item checklist based on the common failure examples.

Added a fourth justification question (with an accompanying callout) in the proposal step.


📚 Full update details can be found here:

https://project-dynamo.learn.joinhandshake.com/updates


2-Day Push Incentive


We're also kicking off a focused 2-day push to increase high-quality task submissions.

For this period, there will be an additional $50 incentive for every task that reaches the RTD layer.

We appreciate everyone's contributions and are looking forward to seeing the next wave of Dynamo tasks.


Thank you, and happy building!

Common Rejection Reasons
Almost every rejected task fails for one of five reasons. In each case the task looks hard, but the model failed because something was unfair, not because the problem was genuinely difficult.
The five most common reasons tasks get rejected:

Grading on a rule you never told the agent (the most common, about half of all rejections). The verifier checks a format, ordering, or tie-break the instructions never stated.
Task files that contradict your instructions. A data file or reference solution follows a different rule than the prompt.
Instructions that read two ways. The grader silently accepts only one reading.
The only hard part is a mistake. Disclose or fix it honestly and the task becomes trivial.
Decoys with no way out. The task points at the wrong answer with nothing the agent can use to correct it.

The test to apply before submitting: if writing the deciding rule down plainly makes the task easy, the difficulty was fake. A good task stays hard even when every rule is clearly stated.
Full guidance and examples in the instructions sheet below.

https://project-dynamo.learn.joinhandshake.com/stump-the-model/why-tasks-get-rejected

Ackno

PR Review Walkthrough + Automated Review Check
We've added a new Automated Review check that runs on every PR before it reaches reviewers. It provides a PASS/FAIL verdict, blocking issues, and advisory notes to help identify potential problems early.

We've also added an Automated Review section to the task instructions for attempters that explains what the check does, what each section means, and how to address blocking issues. Reviewers may find it helpful as a reference as well.
Link - https://project-dynamo.learn.joinhandshake.com/submit/automated-review

Reminder: The Automated Review is a floor, not a ceiling. Use it as a guide, but always rely on your own judgment when reviewing tasks.

Ackno

Project Dynamo Terms
These are the terms you agreed to for this project.

We are excited to welcome you to Project Dynamo. Your participation in this Project is governed by the Handshake AI Contractor Agreement and these Project Terms. Start date: July 1, 2026

Estimated end date: August 1, 2026, with the possibility of extension.

Location requirement: You must be located in, and all work must be completed from India, Mexico, Argentina or Brazil.

Time commitment: Maximum 40 hours per week. For any number of hours over 40 hours per week, you must receive prior written approval from the Project Lead. Reach out to your Project Lead directly for approval via Slack.

Pay rate: $5 per hr + $80 per accepted Task.

Maximum handling time per task: If your project stipulates a Maximum Handling Time per task, it will be found in the project instructions. You will only be paid for your time spent on a Task up to the Maximum Handling Time. If you exceed the Maximum Handling Time on any Task, you are still required to complete and successfully submit that Task before you will be paid your fee for the Task. If you fail to complete Tasks or repeatedly exceed the Maximum Handling Time, Handshake may, in its sole discretion, remove you from the applicable Project or the Program.

Task limit (number of Tasks that can be worked on concurrently): If your project has a task limit, it will be found in the annotation platform or project communications. If you work on more than that number of Tasks at once, you will not be compensated for any additional Tasks completed concurrently. Task Quality, Submission and Acceptance: An accepted task is defined as one that has been moved to the Ready to Deliver (RTD) stage in the Handshake AI annotation platform. Tasks that reach this stage will require: a complete attempt (in conjunction with project guidelines), a successful pass of Review Layer 1 (R1), and a successful pass of Review Layer 2 (R2). Passing R1 and R2 will occur once you've integrated all relevant feedback from your reviewers into your task. Failure to integrate this feedback may result in your task failing at R1 or R2. Tasks that do not reach RTD will not be compensated, without exception. Handshake may, in its sole discretion, remove you from the Project or the Program if your attempted tasks repeatedly fail R1 or R2. Effect of a pause or termination of the project on active tasks: Handshake retains sole discretion to end the project at any time. In this scenario, you may have tasks that have already been started across the attempt, R1 and R2 stages that have not yet been moved to RTD at the effective time of project end. In these situations, if the task has passed the R1 layer, the task will be compensated up to 50% of your standard payout per task. Additional Payment Terms: Payment timing: Payments are typically issued weekly on Wednesday for the previous week's work. Handshake reserves the right to adjust the payment schedule as needed and will notify you of any change. Other payments: Any incentives, bonuses or additional stipends will be earned and paid according to the terms shared by the Project Lead. Withholding of Payment by Handshake: Handshake reserves the right to withhold payment if your account is temporarily deactivated or permanently deactivated on the basis of a Material Breach or Violation of your Contractor Agreement. Handshake may modify these Project Terms at any time. Please refer to the Contractor Agreement for more information about the Handshake AI Program