RigorPilot Skills Claude skills

Eleven skills that reproduce deep-learning repos from their README with a reviewed plan, bounded run and evidence bundle.

Our verdict Recommended with caveats

Careful, well-documented design with real scripts and tests, but it executes commands from third-party repos on your machine without an OS sandbox and the setup is heavier than most skills.

What RigorPilot Skills does

RigorPilot is aimed at one task: take a research code repository, read its README, pick the smallest documented inference, evaluation or training command, run it with limits, and record what happened. The main skill, ai-research-reproduction, works in three steps. A plan step lists README-backed command candidates (cmd-01, cmd-02 and so on) and returns a fingerprint. A run step executes one reviewed candidate bound to that fingerprint, so a changed command set fails before anything runs. A verify step rechecks the saved evidence without rerunning. Output goes to repro_outputs/ with a summary, command log, status JSON, hashes and an annotated copy of the README that adds a result card per section while leaving the original bytes recoverable.

Around it sit smaller skills for read-only analysis, environment and asset setup, minimal runs, conservative training, safe debugging, and a paper-context helper that only resolves narrow gaps such as a dataset split. A separate exploration track (ai-research-explore, explore-code, explore-run) is gated on explicit user authorization and a frozen comparison. The main skill ships Python orchestration scripts, about 16 reference files and report templates, and the repo has a CI workflow and a regression suite. The README says plainly that execution is local, not sandboxed.

Good for

  • Checking whether a paper's code actually runs as documented
  • Producing an auditable log of a bounded reproduction attempt
  • ML researchers who want an agent to avoid silent protocol changes

Not for

  • General environment setup or running arbitrary commands
  • Summarizing papers or finding research directions

What's inside depth 4/5

11 skills averaging 3 KB, but the two main ones are 6-7 KB with 22-26 scripts, 16 references and 7 templates between them, a CI workflow and a local regression suite.

Skills
11
Reference docs
32
Scripts
64
Tests / evals
22
Templates
7
Avg SKILL.md
3.2 KB

Counted from the repository file tree. How we score depth.

Dependencies and costs

  • Python 3.11+ and Git for the bundled orchestration
  • Node.js 22.20+ for the npx skills installer
  • Target repositories may need their own dependencies, GPUs or downloads
  • Optional Anthropic API access for the standalone model runner

Security review

Review before installing

A set of research-reproduction skills that read a paper's repository, plan setup, create a Python environment, and run its training or evaluation commands on your machine. They are written defensively (repository text is treated as untrusted, commands are reviewed IDs, shell syntax needs an explicit opt-in), but they still execute third-party repository code locally without a sandbox. The agent loop sends repository text to the Anthropic API using your key.

Worth knowing

  • Makes network requestsrun_agent.py runs a bounded model loop that posts the system prompt, repository file text and command output to the Anthropic Messages API (or a configured https/localhost endpoint, redirects refused); the agent can only list/read repo files, update a plan and run pre-reviewed command IDs.skills/ai-research-reproduction/_bundled/shared/scripts/agent_provider.py:94
  • Needs an API key for its own serviceReads ANTHROPIC_API_KEY (or the env var named in the model profile) and sends it only to the configured Anthropic-compatible endpoint; optional lookup keys RESEARCH_LOOKUP_OPENROUTER/PERPLEXITY/PARALLEL_API_KEY are checked but the adapter is a placeholder that does not call them.skills/ai-research-reproduction/_bundled/shared/scripts/agent_provider.py:83
  • Makes network requestsThe explore lookup scripts fetch arXiv, DOI, GitHub and arbitrary URLs you name with plain GET requests to gather paper and repository metadata.skills/ai-research-explore/scripts/lookup/providers/base.py:61
  • Installs packages or toolsbootstrap_env.py creates a conda/mamba environment or a .venv and installs the target repository's declared dependencies (it supports --dry-run); the dependency list comes from the repository being reproduced.skills/env-and-assets-bootstrap/scripts/bootstrap_env.py:118
  • Relies on a CLI or package we could not reviewRelies on conda, mamba, pip and git on the machine.skills/env-and-assets-bootstrap/scripts/bootstrap_env.py:36

What we checked

The repository (1630 files) on Oct 29, 2026. Not a full review: 1453 of 1630 files were not downloaded (large repository; we reviewed the skill folders); we read 4 of 64 code files; it relies on code outside the repository: packages and datasets installed or downloaded when a target research repository is bootstrapped; 1453 of 1630 repository files were not available to us (commands, docs and other skill folders); of the 64 listed code files only four were read in full, and the large orchestrators (orchestrate_repro.py, orchestrate_explore.py, run_agent.py, runtime_runner.py, task_queue.py) were searched for process, network and approval logic but not read line by line.

NVIDIA SkillSpector static scan: 49 pattern hits (Dangerous Code Execution 20, Data Exfiltration 11, Agent Snooping 9, Privilege Escalation 3, MCP Rug Pull 2, Excessive Agency 2). The dossier hits are urllib GET calls to public metadata APIs, the Anthropic provider client, parser prefixes that recognise wget/curl/git lines in READMEs, and optional lookup-key environment variable names. No obfuscation or hidden transmission was found in the files read.

A review, not a certification. Skills change after we check them — read the files before installing.

License

MIT — commercial use allowed.

Can it run unattended?

Not really. It has resumable job state, cancellation and timeouts, but the README asks the agent to confirm before large downloads or long training and exploration needs explicit authorization.

Skills in this repository

SkillInstalls
paper-context-resolver480K
repo-intake-and-plan479K
minimal-run-and-audit479K
env-and-assets-bootstrap479K
ai-research-explore341K
analyze-project340K
ai-research-reproduction340K
safe-debug340K
explore-code340K
run-train340K
explore-run340K

Linked skills have their own review.

Alternatives to RigorPilot Skills

ARIS (Auto-Research-In-Sleep)

wanshuiyin/Auto-claude-code-research-in-sleep

A large markdown skill set for autonomous ML research: idea discovery, experiment runs and cross-model review loops where a second model critiques the first.

With caveats Broad access Depth 4/5

AI Research Skills (Orchestra)

Orchestra-Research/AI-Research-SKILLs

98 skills for AI research and ML engineering, plus an autoresearch orchestrator that runs a two-loop experiment and synthesis cycle.

With caveats Broad access Depth 4/5

Academic Research Skills (ARS)

Imbad0202/academic-research-skills

Five very detailed Claude Code skills for the academic pipeline: deep research, paper writing, peer review, revision and systematic-review screening.

With caveats Check first Depth 5/5

FAQ

What is RigorPilot Skills?

RigorPilot Skills is a set of 11 agent skills for README-first reproduction of deep-learning repositories. The agent plans a minimal documented command, runs it with limits and writes an evidence bundle and an annotated README, instead of changing the protocol to get a green result.

How do I install RigorPilot Skills?

Run npx skills add lllllllama/RigorPilot-Skills --skill ai-research-reproduction for the self-contained main skill, or add --all for the companion skills. The installer needs a recent Node.js (22.20 or newer was tested) and the bundled scripts need Python 3.11+ and Git.

Is RigorPilot safe to run?

The skill runs commands from the target repository's README on your own machine, and the README says it is not an OS sandbox, so run it on repositories you trust. It limits risk with a plan-then-run flow, a fingerprint check and a rule that setup and download commands cannot be run as the main target.

RigorPilot vs Auto-claude-code-research-in-sleep: what is the difference?

RigorPilot reproduces an existing repo's documented result with auditable evidence. Auto-claude-code-research-in-sleep is aimed at autonomous ML research loops with cross-model review. Choose RigorPilot to verify someone else's code, the other to generate new research.

Sources: GitHub repository (stars, license, file tree, last push Sep 23, 2026); install counts from skills.sh via Best Skills (CC BY 4.0). Review based on the README and SKILL.md files as published.