A field report on running Google's Gemma-4 on AWS Inferentia2: mixed attention heads, the vLLM / optimum-neuron / NxD dead-ends, and the neuronx-cc compiler limits.
Porting Gemma-4 (2B / 4B / 12B) to AWS Inferentia2
Related
Building AI Agents in PHP: Tool Calling with Laravel
The problem: chat that can only talk The chatbot from earlier in this series can answer...
Keep Your AI Agent Traces on Your Machine: A Local-First Approach
Adding tracing to an AI agent changes more than the debugging experience. It also creates a new data...
Five Hours Of Interviewing - Zero Questions About The Job
This spring I interviewed with a company whose product I genuinely respect — an internet-intelligence...