MetaCurate:
Evolving Curation Filters for High-Quality Agentic Coding Data

Donghoon Kim^1*, Seunghyuk Oh^1*, Kevin Galim¹, Minseo Kim², Minjae Lee¹, Younghyun Kim¹, Wooseok Seo¹,
Hyung Il Koo¹, Wonjun Kang¹

¹FuriosaAI ²University of California, Berkeley ^*Equal contribution

Supervised fine-tuning (SFT) on agent trajectories is an effective way to improve coding agents, but selecting useful training examples requires more than identifying successful runs. A resolved patch may follow inefficient reasoning or unnecessary tool use, while a failed run may still contain useful intermediate steps. Training pipelines therefore rely on rules and qualitative assessments to filter trajectories, yet designing these filters typically requires substantial human expertise, and validating improvements through repeated training and evaluation is expensive.

We introduce MetaCurate, a training-free framework that evolves executable trajectory curation filters using pairwise LLM judgments. Disagreements between the filter’s rankings and the judge’s preferences provide targeted feedback for jointly revising its rules, rubric, and aggregation policy. The resulting filter scores trajectories independently, so pairwise comparisons are required only during search. Starting from a blank filter, MetaCurate autonomously discovers a curation strategy that achieves competitive downstream performance without task-specific handwritten criteria.

We validate downstream utility through controlled SFT experiments on our uncurated trajectory pool and the Nemotron v3 SWE dataset, with random selection and hand-authored filtering as baselines. In our experiments, MetaCurate achieves the highest mean SWE-bench Verified score among the compared methods, including hand-authored rule-based filtering and a training-loop baseline whose search costs 3.5× as much.

Method

MetaCurate uses pairwise judge preferences as supervision, transferring comparison feedback into a reusable pointwise filter. Once fixed, the filter scores the full pool without further pairwise judgments.

Concordance

We evaluate each candidate filter by comparing its trajectory rankings with the judge’s pairwise preferences for SFT usefulness.

C_F(P) = 1⁄|P| Σ_(x≻y)∈P κ_F(x, y)

Each filter F scores a trajectory x individually, producing s_F(x). For a judged pair (x ≻ y) in a preference set P, κ_F(x, y) equals 1 if s_F(x) > s_F(y), ½ for a tie, and 0 otherwise.

Results

Qwen 3.5 9B fine-tuned on 1,000 selected trajectories.
SWE-bench Verified resolved rate (%), mean ± SD over three evaluations of each checkpoint.

Dataset

We generate and release a corpus of 4,286 coding-agent trajectories with DeepSeek V4 Pro from 1,415 instance IDs, designed to support experiments with a range of curation strategies.

@article{kim2026metacurate,
  title   = {MetaCurate: Evolving Curation Filters for High-Quality Agentic Coding Data},
  author  = {Kim, Donghoon and Oh, Seunghyuk and Galim, Kevin and Kim, Minseo and
             Lee, Minjae and Kim, Younghyun and Seo, Wooseok and Koo, Hyung Il and Kang, Wonjun},
  journal = {arXiv preprint arXiv:[ARXIV ID]},
  year    = {2026}
}