skilled-proposer — a GEPA instruction proposer with anti-overfitting meta-prompt and skill injection

TL;DR

skilled-proposer is a drop-in replacement for GEPA’s default instruction proposer inside DSPy, from Drew Breunig / cmpnd-ai. GEPA’s stock proposer instructs the reflection LLM to include “niche and domain specific factual information” from trajectories in the rewritten prompt, which the README argues encourages copying entities, numbers, and gold answers from training examples straight into the instruction. The SkilledProposer replaces that meta-prompt with a three-step procedure — infer the task, diagnose the failure, extract the general rule that would have prevented it — plus practical controls: reference SKILL.md files, extra guidance strings, and word/token length budgets enforced by prompt constraint → compression pass → truncation.

Key claims

• The stock GEPA proposer’s meta-prompt instruction to include “niche and domain specific factual information” from execution traces can cause training-example entities and numbers to leak into the rewritten prompt, degrading held-out performance [README §Why].
• The replacement meta-prompt uses a three-step procedure — task inference from examples, failure diagnosis with rule extraction, then instruction rewrite from those rules — and states one explicit principle against overfitting to trace specifics [README §Why].
• Skills are passed as reference material to the reflector LLM, following the Agent Skills SKILL.md layout: directories, markdown files, inline strings, or Skill(name, content, description) objects are all accepted [README §Skills].
• Loading a skill directory reads only its top-level SKILL.md; subfolders like models/ and references/ are not auto-loaded — the developer must pass them explicitly as separate entries, because the reflection LM has no filesystem access at proposal time [README §Skills with subfolders].
• Length budgets are enforced by a three-stage cascade: prompt-level constraint, then compression pass, then hard truncation; token counts use litellm’s tokenizer with a ~4 chars/token fallback [README §Options].
• v0.1 is text-only: dspy.Image and other rich values are stringified in reflective examples, so the reflection model cannot see them; multimodal support is planned for v0.2 [README §Limits].

Method

SkilledProposer is a callable that plugs into dspy.GEPA(instruction_proposer=...). Each proposal call, the reflector LLM receives (a) the reflective examples GEPA has selected for that component, (b) a <skill> block per loaded skill file, (c) any additional_instructions guidance string, (d) the replacement three-step meta-prompt, and (e) the length budget as a constraint. The proposer then post-processes the model’s output: if over budget, a compression pass rewrites the instruction; if still over budget, truncate. Failed proposals default to on_error="keep" — log and retain the current instruction, so a long optimization run survives flaky reflection calls; on_error="raise" is available for development. For the standalone gepa package (which does not set a DSPy LM context), a prompt_model=dspy.LM(...) argument is required.

Results

The repo is v0.1 with no benchmark numbers reported; the argument is a mechanism-level claim about overfitting in the stock proposer plus example workflow. Drew Breunig’s launch tweet frames the release as workshopping the concept with GEPA/DSPy practitioners rather than a finished benchmark result. The shipped example skill (skills/prompt-engineering) is a prompt-optimization guide that the reflection model can apply when rewriting instructions.

Why it’s interesting

This sits directly downstream of GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning (the GEPA paper) — same reflection-based prompt-mutation loop, but a targeted intervention on the proposer sub-prompt that GEPA uses to elicit mutations. The GEPA paper’s Key claims describe the reflector as being asked to “diagnose problems” from trajectories and propose an updated instruction; the skilled-proposer README is arguing that the specific wording of that ask matters a lot, and that GEPA’s default wording biases toward overfitting. The concept connection is Reasoning RL — GEPA is the wiki’s canonical non-RL alternative for adapting LLMs to downstream tasks, and this repo is a community-driven refinement of the meta-prompt that does the “learning” work. Contrast with Data-driven prompt optimizer (Vertex AI) (Vertex AI’s productized batch prompt optimizer), which uses score-based rather than trace-reflection-based mutation and does not expose the mutation meta-prompt for customization. The overfitting-prevention concern is also a recurring theme in Segmenting Robot Video into Actionable Subtasks (WGO-Bench), which uses a GEPA-tuned “completed-events” evaluator prompt for VLM-based robot-video segmentation.

See also

• GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning — the underlying GEPA algorithm this proposer swaps into; targets exactly the reflection-based mutation step described there.
• Reasoning RL — the concept where GEPA and its “prompt optimization as RL alternative” framing lives.
• Data-driven prompt optimizer (Vertex AI) — productized counterpart with a simpler score-based mutation signal and no user-facing meta-prompt.
• Segmenting Robot Video into Actionable Subtasks (WGO-Bench) — real-world GEPA use case (VLM prompt-tuning for robot-video segmentation) illustrating why overfitting matters in practice.