inference-serving-sglang
ComunidadMITFast structured generation and serving for LLMs with RadixAttention prefix caching. Use for JSON/regex outputs, constrained decoding, agentic workflows with tool calls, or when you need 5× faster inference than vLLM with prefix sharing. Powers 300,000+ GPUs at xAI, AMD, NVIDIA, and LinkedIn.
El ZIP contiene solo este skill. Está en GitHub Releases del repo público orlandoneto/agent-skills — Ontoom es la vitrina. El crawler diario actualiza el catálogo.
Cómo instalar
- Descarga el ZIP y extrae la carpeta del skill.
- Cursor:
.agents/skills/aitmpl-inference-serving-sglang/o.cursor/skills/aitmpl-inference-serving-sglang/ - Claude Code:
.claude/skills/aitmpl-inference-serving-sglang/ - Otras IAs:
.agents/skills/aitmpl-inference-serving-sglang/
Otros skills de esta colección
- inference-serving-tensorrt-llm
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.
- inference-serving-vllm
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
- infrastructure-lambda-labs
Reserved and on-demand GPU cloud instances for ML training and inference. Use when you need dedicated GPU instances with simple SSH access, persistent filesystems, or high-performance multi-node clusters for large-scale training.
- infrastructure-modal
Serverless GPU cloud platform for running ML workloads. Use when you need on-demand GPU access without infrastructure management, deploying ML models as APIs, or running batch jobs with automatic scaling.
- infrastructure-skypilot
Multi-cloud orchestration for ML workloads with automatic cost optimization. Use when you need to run training or batch jobs across multiple clouds, leverage spot instances with auto-recovery, or optimize GPU costs across providers.
- integration-checker
Verify cross-component wiring and data flow.