Sakana AI and NVIDIA have introduced TwELL, a new approach using CUDA kernels that achieves 20.5% inference and 21.9% tr...

Sakana AI and NVIDIA have introduced TwELL, a new approach using CUDA kernels that achieves 20.5% inference and 21.9% training speedup in large language models. The technique targets feedforward layers, which account for over two-thirds of model parameters and 80% of FLOPs, by inducing over 99% sparsity and translating it into real GPU throughput gains through new sparse data formats. https://www.marktechpost.com/2026/05/11/sakana-ai-and-nvidia-introduce-twell-with-cuda-kernels-for-20-5-inference-and-21-9-training-speedup-in-llms/ #AIagent #AI #GenAI #Infrastructure