GitHub Trending repo Repositories Jun 30

jiahongsigma/Efficient-LLM-Inference-Serving-Systems: Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → parallelism → vLLM/SGLang, with GPU labs on open models.

by jiahongsigma

Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → parallelism → vLLM/SGLang, with GPU labs on open models.

Read Original
AI Hardware LLM

Metadata

Stars
11
Forks
1
Language
Python
Watchers
11
License
MIT

Related

GitHub Trending repo 8h ago

ramyadharshinimurugaprakash-commits/Titanic-Survival-Prediction-Random-Forest: A machine learning project that predicts passenger survival on the Titanic using the Random Forest Classifier. The project includes data preprocessing, label encoding, feature scaling, model training, prediction, and performance evaluation using Python and Scikit-learn.

A machine learning project that predicts passenger survival on the Titanic using the Random Forest Classifier. The project includes data preprocessing, label encoding, feature scal...

39