← Todos los skills

inference-serving-sglang

ComunidadMIT

Fast structured generation and serving for LLMs with RadixAttention prefix caching. Use for JSON/regex outputs, constrained decoding, agentic workflows with tool calls, or when you need 5× faster inference than vLLM with prefix sharing. Powers 300,000+ GPUs at xAI, AMD, NVIDIA, and LinkedIn.

El ZIP contiene solo este skill. Está en GitHub Releases del repo público orlandoneto/agent-skills — Ontoom es la vitrina. El crawler diario actualiza el catálogo.

Cómo instalar

  1. Descarga el ZIP y extrae la carpeta del skill.
  2. Cursor: .agents/skills/aitmpl-inference-serving-sglang/ o .cursor/skills/aitmpl-inference-serving-sglang/
  3. Claude Code: .claude/skills/aitmpl-inference-serving-sglang/
  4. Otras IAs: .agents/skills/aitmpl-inference-serving-sglang/
cursorclaude-codecodexgeminiany

Otros skills de esta colección

Otras colecciones

  • inference-serving-tensorrt-llm

    Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.

  • inference-serving-vllm

    Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

  • infrastructure-lambda-labs

    Reserved and on-demand GPU cloud instances for ML training and inference. Use when you need dedicated GPU instances with simple SSH access, persistent filesystems, or high-performance multi-node clusters for large-scale training.

  • infrastructure-modal

    Serverless GPU cloud platform for running ML workloads. Use when you need on-demand GPU access without infrastructure management, deploying ML models as APIs, or running batch jobs with automatic scaling.

  • infrastructure-skypilot

    Multi-cloud orchestration for ML workloads with automatic cost optimization. Use when you need to run training or batch jobs across multiple clouds, leverage spot instances with auto-recovery, or optimize GPU costs across providers.

  • integration-checker

    Verify cross-component wiring and data flow.