Rolling your own tool-calling loop on a HF chat template? Watch the stop token: splice a tool result back in and re-gene...

Rolling your own tool-calling loop on a HF chat template? Watch the stop token: splice a tool result back in and re-generate, and the model can re-emit the same stop token your delta already added - silently doubled in the KV cache. Byte-diff vs the canonical render to catch it. #LLM #AIagents

Read Original

Related