← All skills

inference-serving-tensorrt-llm

CommunityMIT

Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.

The ZIP contains only this skill. It lives on GitHub Releases of the public orlandoneto/agent-skills repo — Ontoom is the showcase. A daily crawler refreshes the catalog.

How to install

  1. Download the ZIP and extract the skill folder.
  2. Cursor: .agents/skills/aitmpl-inference-serving-tensorrt-llm/ or .cursor/skills/aitmpl-inference-serving-tensorrt-llm/
  3. Claude Code: .claude/skills/aitmpl-inference-serving-tensorrt-llm/
  4. Other AIs: .agents/skills/aitmpl-inference-serving-tensorrt-llm/
cursorclaude-codecodexgeminiany

Other skills in this collection

Other collections

  • inference-serving-vllm

    Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

  • infrastructure-lambda-labs

    Reserved and on-demand GPU cloud instances for ML training and inference. Use when you need dedicated GPU instances with simple SSH access, persistent filesystems, or high-performance multi-node clusters for large-scale training.

  • infrastructure-modal

    Serverless GPU cloud platform for running ML workloads. Use when you need on-demand GPU access without infrastructure management, deploying ML models as APIs, or running batch jobs with automatic scaling.

  • infrastructure-skypilot

    Multi-cloud orchestration for ML workloads with automatic cost optimization. Use when you need to run training or batch jobs across multiple clouds, leverage spot instances with auto-recovery, or optimize GPU costs across providers.

  • integration-checker

    Verify cross-component wiring and data flow.

  • internal-comms

    A set of resources to help me write all kinds of internal communications, using the formats that my company likes to use. Claude should use this skill whenever asked to write some sort of internal communications (status reports, leadership updates, 3P updates, company newsletters, FAQs, incident reports, project updates, etc.).