AI Token & Cost Estimator
Welcome to the AI Token & Cost Estimator, an advanced playground to estimate, analyze, and debug LLM tokens and API consumption costs.
Features
- Interactive Tokenizer Visualizer: Supports cl100k_base, LLaMA, and any custom HuggingFace
tokenizer.jsonconfig. - Cost Estimations: Calculate prompt and completion token counts and pricing dynamically.
- Cross-Platform: Runs natively on Linux, Windows, macOS, and in-browser via WebAssembly.
Technology Stack
Powered by Rust, egui, wasm-bindgen, and HuggingFace tokenizers.