Tech.us Releases Tokle-3M, Top Performing Small Language Model

SAN JOSE, CA, UNITED STATES, October 9, 2026 /CourierPR/ -- Tech.us has released Tokle-3M, a small language model with 2.91 million parameters that has achieved remarkable results in the field of artificial intelligence. As of October 1, 2026, Tokle-3M ranks first among models under 3 million parameters on the Open SLM Leaderboard, scoring 8.91 on the Intelligence Index. This performance is particularly impressive, as it also holds the highest arithmetic score in its class at 40.80% on ArithMark-3.

According to the company, Tokle-3M's success is attributed to a unique training approach called Static Pairwise Attention Bias (SPAB). During the first stage of training, SPAB uses statistical patterns in the training data to highlight relationships between words or tokens, which are then integrated into the model's attention process. This guidance helps the model focus on useful relationships earlier in the training process. In the second stage, the SPAB information is removed, and the model retains the benefits of this guidance in its final form.

Tech.us emphasizes that Tokle-3M is not intended to replace large, general-purpose language models but rather to explore the potential of more efficient, compact models. The model was pre-trained on English datasets with a 512-token context window, utilizing rotary position embeddings (RoPE) that can be extended at inference time without changing the architecture. The company notes that performance beyond 512 tokens has not been evaluated.

The research behind Tokle-3M focuses on models in a comparable size range, aiming to understand how training methods can significantly impact the performance of compact models. The model has been evaluated across HellaSwag, ARC-Easy, ARC-Challenge, PIQA, and ArithMark-3 using zero-shot normalized accuracy. Measured against the leaderboard's trend line for performance versus model size, Tokle-3M scores 1.80 standard deviations above what its parameter count would predict, marking a significant deviation from the norm.

Tech.us sees the potential for models like Tokle-3M in specific business environments where organizations need to control how models are trained, deployed, and adapted. The company's broader aim is to understand how focused training, compact architecture, and deployment control can make AI more practical for these environments. With task-specific fine-tuning, models in this class could support narrow, well-defined workloads such as text classification, intent detection, interpreting device commands, and triaging system logs, particularly in offline, embedded, or on-device settings where data must remain on local hardware.

Tokle-3M is now available on Hugging Face, along with a model card that describes its training, evaluation, and limitations. Tech.us, headquartered in San Jose, California, is an AI consulting and implementation company that designs and builds production-ready AI, agentic AI, and custom software systems for mid-market and enterprise organizations. Its AI research explores practical approaches to model efficiency, architecture, and deployment alongside its work delivering AI systems for real-world business applications.