evaluation-bigcode-evaluation-harness
ComunidadMITEvaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.
El ZIP contiene solo este skill. Está en GitHub Releases del repo público orlandoneto/agent-skills — Ontoom es la vitrina. El crawler diario actualiza el catálogo.
Cómo instalar
- Descarga el ZIP y extrae la carpeta del skill.
- Cursor:
.agents/skills/aitmpl-evaluation-bigcode-evaluation-harness/o.cursor/skills/aitmpl-evaluation-bigcode-evaluation-harness/ - Claude Code:
.claude/skills/aitmpl-evaluation-bigcode-evaluation-harness/ - Otras IAs:
.agents/skills/aitmpl-evaluation-bigcode-evaluation-harness/
Otros skills de esta colección
- evaluation-lm-evaluation-harness
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
- evaluation-nemo-evaluator
Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.
- evm-contract-scaffold
Scaffolds a Solidity smart contract project — Foundry-first with Hardhat alternative, OpenZeppelin-based token patterns (ERC-20/721/1155), testing, and deployment. Use when starting a new Solidity project, standardizing an existing one, adding a new contract to an existing repo, or migrating from Hardhat to Foundry (or back).
- fine-tuning-axolotl
Expert guidance for fine-tuning LLMs with Axolotl - YAML configs, 100+ models, LoRA/QLoRA, DPO/KTO/ORPO/GRPO, multimodal support
- fine-tuning-llama-factory
Expert guidance for fine-tuning LLMs with LLaMA-Factory - WebUI no-code, 100+ models, 2/3/4/5/6/8-bit QLoRA, multimodal support
- fine-tuning-peft
Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.