← Todos los skills

inference-serving-tensorrt-llm

ComunidadMIT

Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.

El ZIP contiene solo este skill. Está en GitHub Releases del repo público orlandoneto/agent-skills — Ontoom es la vitrina. El crawler diario actualiza el catálogo.

Cómo instalar

  1. Descarga el ZIP y extrae la carpeta del skill.
  2. Cursor: .agents/skills/aitmpl-inference-serving-tensorrt-llm/ o .cursor/skills/aitmpl-inference-serving-tensorrt-llm/
  3. Claude Code: .claude/skills/aitmpl-inference-serving-tensorrt-llm/
  4. Otras IAs: .agents/skills/aitmpl-inference-serving-tensorrt-llm/
cursorclaude-codecodexgeminiany

Otros skills de esta colección

Otras colecciones

  • inference-serving-vllm

    Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

  • infrastructure-lambda-labs

    Reserved and on-demand GPU cloud instances for ML training and inference. Use when you need dedicated GPU instances with simple SSH access, persistent filesystems, or high-performance multi-node clusters for large-scale training.

  • infrastructure-modal

    Serverless GPU cloud platform for running ML workloads. Use when you need on-demand GPU access without infrastructure management, deploying ML models as APIs, or running batch jobs with automatic scaling.

  • infrastructure-skypilot

    Multi-cloud orchestration for ML workloads with automatic cost optimization. Use when you need to run training or batch jobs across multiple clouds, leverage spot instances with auto-recovery, or optimize GPU costs across providers.

  • integration-checker

    Verify cross-component wiring and data flow.

  • internal-comms

    A set of resources to help me write all kinds of internal communications, using the formats that my company likes to use. Claude should use this skill whenever asked to write some sort of internal communications (status reports, leadership updates, 3P updates, company newsletters, FAQs, incident reports, project updates, etc.).