← Todos los skills

tokenization-sentencepiece

ComunidadMIT

Language-independent tokenizer treating text as raw Unicode. Supports BPE and Unigram algorithms. Fast (50k sentences/sec), lightweight (6MB memory), deterministic vocabulary. Used by T5, ALBERT, XLNet, mBART. Train on raw text without pre-tokenization. Use when you need multilingual support, CJK languages, or reproducible tokenization.

El ZIP contiene solo este skill. Está en GitHub Releases del repo público orlandoneto/agent-skills — Ontoom es la vitrina. El crawler diario actualiza el catálogo.

Cómo instalar

  1. Descarga el ZIP y extrae la carpeta del skill.
  2. Cursor: .agents/skills/aitmpl-tokenization-sentencepiece/ o .cursor/skills/aitmpl-tokenization-sentencepiece/
  3. Claude Code: .claude/skills/aitmpl-tokenization-sentencepiece/
  4. Otras IAs: .agents/skills/aitmpl-tokenization-sentencepiece/
cursorclaude-codecodexgeminiany

Origen / créditos: davila7/claude-code-templates

Otros skills de esta colección

Claude Code Templates

  • using-git-worktrees

    Use when starting feature work that needs isolation from current workspace or before executing implementation plans - creates isolated git worktrees with smart directory selection and safety verification

  • using-neon

    Guides and best practices for working with Neon Serverless Postgres. Covers getting started, local development with Neon, choosing a connection method, Neon features, authentication (@neondatabase/auth), PostgREST-style data API (@neondatabase/neon-js), Neon CLI, and Neon's Platform API/SDKs. Use for any Neon-related questions.

  • using-superpowers

    Use when starting any conversation - establishes how to find and use skills, requiring Skill tool invocation before ANY response including clarifying questions

  • vercel-deployment

    Expert knowledge for deploying to Vercel with Next.js Use when: vercel, deploy, deployment, hosting, production.

  • verification-before-completion

    Use when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always

  • viral-generator-builder

    Expert in building shareable generator tools that go viral - name generators, quiz makers, avatar creators, personality tests, and calculator tools. Covers the psychology of sharing, viral mechanics, and building tools people can't resist sharing with friends. Use when: generator tool, quiz maker, name generator, avatar creator, viral tool.