sentencepiece

Name: sentencepiece
Rating: 1.3 (1 reviews)
Author: zechenzhangAGI

26.2

"Language-independent tokenizer treating text as raw Unicode. Supports BPE and Unigram algorithms. Fast (50k sentences/sec), lightweight (6MB memory), deterministic vocabulary. Used by T5, ALBERT, XLNet, mBART. Train on raw text without pre-tokenization. Use when you need multilingual support, CJK languages, or reproducible tokenization."

Also in: javascript ci cd networking

Third-Party Agent Skill: Review the code before installing. Agent skills execute in your AI assistant's environment and can access your files. Learn more about security

Installation for Agentic Skill

View all platforms →

Claude Code (CLI) Fast

skilz install zechenzhangAGI/AI-research-SKILLs/sentencepiece

OpenCode (CLI) Fast

skilz install zechenzhangAGI/AI-research-SKILLs/sentencepiece --agent opencode

OpenAI Codex (CLI) Native

skilz install zechenzhangAGI/AI-research-SKILLs/sentencepiece --agent codex

Gemini CLI (Project) Project

skilz install zechenzhangAGI/AI-research-SKILLs/sentencepiece --agent gemini

First time? Install Skilz: pip install skilz

Works with 14 AI coding assistants

Cursor, Aider, Copilot, Windsurf, Qwen, Kimi, and more...

View All Agents

For Claude Desktop Easy

Download Agent Skill ZIP

Extract and copy to ~/.claude/skills/ then restart Claude Desktop

Manual Installation

1. Clone the repository:

git clone https://github.com/zechenzhangAGI/AI-research-SKILLs

2. Copy the agent skill directory:

cp -r AI-research-SKILLs/02-tokenization/sentencepiece ~/.claude/skills/

View on GitHub

Need detailed installation help? Check our platform-specific guides:

Claude Desktop Guide Claude Code Guide Troubleshooting

Related Agentic Skills

mastering-github-cli

by SpillwaveSolutions

GitHub CLI (gh) command reference for repository search, code discovery, CI/CD monitoring, workflow authoring, and automation. Triggers on "gh" comman...

100

TECHgithub

skill-scanner

by SpillwaveSolutions

Discover, clone, and analyze Claude Code skills from GitHub repositories. Build a searchable catalog of skills with scoring and categorization.

100

TECHgithub› automation

doc-coauthoring

by anthropics

Guide users through a structured workflow for co-authoring documentation. Use when user wants to write documentation, proposals, technical specs, deci...

TECHgithub

Marketplace

+markdown

skill-creator

by anthropics

Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Claude...

github

Marketplace

+testing+ci cd

Agentic Skill Details

Owner: zechenzhangAGI (GitHub)
Repository: AI-research-SKILLs
Type: Technical
Meta-Domain: development
Primary Domain: github
Market Score: 26.2

Agentic Skill Grades →

Browse Category

More development Agentic Skills

Report Security Issue

Found a security vulnerability in this agent skill?