← All skills

evaluation-bigcode-evaluation-harness

CommunityMIT

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.

The ZIP contains only this skill. It lives on GitHub Releases of the public orlandoneto/agent-skills repo — Ontoom is the showcase. A daily crawler refreshes the catalog.

How to install

  1. Download the ZIP and extract the skill folder.
  2. Cursor: .agents/skills/aitmpl-evaluation-bigcode-evaluation-harness/ or .cursor/skills/aitmpl-evaluation-bigcode-evaluation-harness/
  3. Claude Code: .claude/skills/aitmpl-evaluation-bigcode-evaluation-harness/
  4. Other AIs: .agents/skills/aitmpl-evaluation-bigcode-evaluation-harness/
cursorclaude-codecodexgeminiany

Other skills in this collection

Other collections

  • evaluation-lm-evaluation-harness

    Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

  • evaluation-nemo-evaluator

    Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.

  • evm-contract-scaffold

    Scaffolds a Solidity smart contract project — Foundry-first with Hardhat alternative, OpenZeppelin-based token patterns (ERC-20/721/1155), testing, and deployment. Use when starting a new Solidity project, standardizing an existing one, adding a new contract to an existing repo, or migrating from Hardhat to Foundry (or back).

  • fine-tuning-axolotl

    Expert guidance for fine-tuning LLMs with Axolotl - YAML configs, 100+ models, LoRA/QLoRA, DPO/KTO/ORPO/GRPO, multimodal support

  • fine-tuning-llama-factory

    Expert guidance for fine-tuning LLMs with LLaMA-Factory - WebUI no-code, 100+ models, 2/3/4/5/6/8-bit QLoRA, multimodal support

  • fine-tuning-peft

    Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.