You've set up a local LLM inference node. The model loads. The first tokens stream in at 20 t/s....
Why your GPU reports 75 C while your VRAM is cooking at 105 C – the telemetry gap that kills LLM inference
Related
We turned agent conversations into git commits (and it's actually useful)
Ever wished your agent conversations were as trackable as your code? Gitlord makes it real. What it...
Every Word I Say Gets Tokenized. This Library Does It 1000x Faster.
GigaToken is a drop-in replacement for HuggingFace Tokenizers and tiktoken that’s up to 1000x faster. I looked inside from an AI perspective.
I Built 41 Repos in Five Months. The Green Checkmarks Were Lying.
Originally published at devopsdiary.blog. Post R24 in "The Quiet Years" series. The draft title for...