Chapter 10
LLM Serving
LLM Serving
2024
OSDI
- Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve(https://www.usenix.org/conference/osdi24/presentation/agrawal)
- ServerlessLLM: Low-Latency Serverless Inference for Large Language Models(https://www.usenix.org/conference/osdi24/presentation/fu)
- InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management(https://www.usenix.org/conference/osdi24/presentation/lee)
- Llumnix: Dynamic Scheduling for Large Language Model Serving(https://www.usenix.org/conference/osdi24/presentation/sun-biao)
- DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving(https://www.usenix.org/conference/osdi24/presentation/zhong-yinmin)
- dLoRA: Dynamically Orchestrating Requests and Adapters for LoRA LLM Serving(https://www.usenix.org/conference/osdi24/presentation/wu-bingyang)
- Parrot: Efficient Serving of LLM-based Applications with Semantic Variable(https://www.usenix.org/conference/osdi24/technical-sessions)
- USHER: Holistic Interference Avoidance for Resource Optimized ML Inference(https://www.usenix.org/conference/osdi24/presentation/shubha)
- Fairness in Serving Large Language Models(https://www.usenix.org/conference/osdi24/presentation/sheng)