AI System School

💫💫💫 System for Machine Learning, LLM (Large Language Model), GenAI (Generative AI)

Updates:

• Video Tutorials(https://youtu.be/ChD1_aVZJ0g?si=Kg-yB3F4Iea0Xp9J)(https://www.bilibili.com/video/BV1ZwYUerEtL/)(http://xhslink.com/MmrjcT)
• We are preparing a new website(https://letsgoai.pro/) for this repo!!!

Path to System for AI(./paper/mlsys-whitepaper.pdf)

A curated list of research in machine learning systems. Link to the code if available is also present. Now we have a team to maintain this project. You are very welcome to pull request by using our template.

System for AI (Ordered by Category)

ML / DL Infra

• Data Processing
• Training System
• Inference System
• Machine Learning Infrastructure

LLM Infra

• LLM Training
• LLM Serving

Domain-Specific Infra

• Video System
• AutoML System
• Edge AI
• GNN System
• Federated Learning System
• Deep Reinforcement Learning System

System for ML/LLM Conference

Conference

• OSDI
• SOSP
• SIGCOMM
• NSDI
• MLSys
• ATC
• Eurosys
• Middleware
• SoCC
• TinyML

General Resources

• Survey
• Book
• Video
• Course
• Blog

Survey

• Toward Highly Available, Intelligent Cloud and ML Systems(http://sysnetome.com/Talks/cguo_netai_2018.pdf)
• A curated list of awesome System Designing articles, videos and resources for distributed computing, AKA Big Data.(https://github.com/madd86/awesome-system-design)
• awesome-production-machine-learning: A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning(https://github.com/EthicalML/awesome-production-machine-learning)
• Opportunities and Challenges Of Machine Learning Accelerators In Production(https://www.usenix.org/system/files/opml19papers-ananthanarayanan.pdf)
  • Ananthanarayanan, Rajagopal, et al. "
  • 2019 {USENIX} Conference on Operational Machine Learning (OpML 19). 2019.
• How (and How Not) to Write a Good Systems Paper(https://www.usenix.org/legacy/events/samples/submit/advice_old.html)
• Applied machine learning at Facebook: a datacenter infrastructure perspective(https://research.fb.com/wp-content/uploads/2017/12/hpca-2018-facebook.pdf)
  • Hazelwood, Kim, et al. (HPCA 2018)
• Infrastructure for Usable Machine Learning: The Stanford DAWN Project
  • Bailis, Peter, Kunle Olukotun, Christopher Ré, and Matei Zaharia. (preprint 2017)
• Hidden technical debt in machine learning systems(https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf)
  • Sculley, David, et al. (NIPS 2015)
• End-to-end arguments in system design(http://web.mit.edu/Saltzer/www/publications/endtoend/endtoend.pdf)
  • Saltzer, Jerome H., David P. Reed, and David D. Clark.
• System Design for Large Scale Machine Learning(http://shivaram.org/publications/shivaram-dissertation.pdf)
• Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications(https://arxiv.org/pdf/1811.09886.pdf)
  • Park, Jongsoo, Maxim Naumov, Protonu Basu et al. arXiv 2018
  • Summary: This paper presents a characterizations of DL models and then shows the new design principle of DL hardware.
• A Berkeley View of Systems Challenges for AI(https://arxiv.org/pdf/1712.05855.pdf)

Book

• Computer Architecture: A Quantitative Approach(http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.115.1881&rep=rep1&type=pdf)
• Distributed Machine Learning Patterns(https://www.manning.com/books/distributed-machine-learning-patterns)
• Streaming Systems(https://www.oreilly.com/library/view/streaming-systems/9781491983867/)
• Kubernetes in Action (start to read)(https://www.oreilly.com/library/view/kubernetes-in-action/9781617293726/)
• Machine Learning Systems: Designs that scale(https://www.manning.com/books/machine-learning-systems)
• Trust in Machine Learning(https://www.manning.com/books/trust-in-machine-learning)
• Automated Machine Learning in Action(https://www.manning.com/books/automated-machine-learning-in-action)
• Machine Learning Systems: Principles and Practices of Engineering Artificially Intelligent Systems(https://mlsysbook.ai/)

Video

• ScalaDML2020: Learn from the best minds in the machine learning community.(https://info.matroid.com/scaledml-media-archive-preview)
• Jeff Dean: "Achieving Rapid Response Times in Large Online Services" Keynote - Velocity 2014(https://www.youtube.com/watch?v=1-3Ahy7Fxsc)
• From Research to Production with PyTorch(https://www.infoq.com/presentations/pytorch-torchscript-botorch/#downloadPdf/)
• Introduction to Microservices, Docker, and Kubernetes(https://www.youtube.com/watch?v=1xo-0gCVhTU)
• ICML Keynote: Lessons Learned from Helping 200,000 non-ML experts use ML(https://slideslive.com/38916584/keynote-lessons-learned-from-helping-200000-nonml-experts-use-ml)
• Adaptive & Multitask Learning Systems(https://www.amtl-workshop.org/schedule)
• System thinking. A TED talk.(https://www.youtube.com/watch?v=_vS_b7cJn2A)
• Flexible systems are the next frontier of machine learning. Jeff Dean(https://www.youtube.com/watch?v=Jnunp-EymJQ&list=WL&index=12)
• Is It Time to Rewrite the Operating System in Rust?(https://www.youtube.com/watch?v=HgtRAbE1nBM&list=WL&index=17&t=0s)
• InfoQ: AI, ML and Data Engineering(https://www.youtube.com/playlist?list=PLndbWGuLoHeYsZk6VpCEj_SSd9IFgjJ-2)
  • Start to watch.
• Netflix: Human-centric Machine Learning Infrastructure(https://www.infoq.com/presentations/netflix-ml-infrastructure?utm_source=youtube&utm_medium=link&utm_campaign=qcontalks)
• SysML 2019:(https://www.youtube.com/channel/UChutDKIa-AYyAmbT45s991g/videos)
• ScaledML 2019: David Patterson, Ion Stoica, Dawn Song and so on(https://www.youtube.com/playlist?list=PLRM2gQVaW_wWXoUnSfZTxpgDmNaAS1RtG)
• ScaledML 2018: Jeff Dean, Ion Stoica, Yangqing Jia and so on(https://www.youtube.com/playlist?list=PLRM2gQVaW_wW9KAxcibxdqY_TDyvmEjzm)(https://www.matroid.com/blog/post/slides-and-videos-from-scaledml-2018)
• A New Golden Age for Computer Architecture History, Challenges, and Opportunities. David Patterson(https://www.youtube.com/watch?v=uyc_pDBJotI&t=767s)
• How to Have a Bad Career. David Patterson (I am a big fan)(https://www.youtube.com/watch?v=Rn1w4MRHIhc)
• SysML 18: Perspectives and Challenges. Michael Jordan(https://www.youtube.com/watch?v=4inIBmY8dQI&t=26s)
• SysML 18: Systems and Machine Learning Symbiosis. Jeff Dean(https://www.youtube.com/watch?v=Nj6uxDki6-0)
• AutoML Basics: Automated Machine Learning in Action. Qingquan Song, Haifeng Jin, Xia Hu(https://www.youtube.com/watch?v=9KpieG0B7VM)

Course

• CS692 Seminar: Systems for Machine Learning, Machine Learning for Systems(https://github.com/guanh01/CS692-mlsys)
• Topics in Networks: Machine Learning for Networking and Systems, Autumn 2019(https://people.cs.uchicago.edu/~junchenj/34702-fall19/syllabus.html)
• CS6465: Emerging Cloud Technologies and Systems Challenges(http://www.cs.cornell.edu/courses/cs6465/2019fa/)
• CS294: AI For Systems and Systems For AI.(https://github.com/ucbrise/cs294-ai-sys-sp19) (Strong Recommendation)(https://ucbrise.github.io/cs294-ai-sys-fa19/)
• CSE 599W: System for ML.(https://github.com/tqchen)(http://dlsys.cs.washington.edu/)
• EECS 598: Systems for AI (W'21).(https://www.mosharaf.com/)(https://github.com/mosharaf/eecs598/tree/w21-ai)
• Tutorial code on how to build your own Deep Learning System in 2k Lines(https://github.com/tqchen/tinyflow)
• CSE 291F: Advanced Data Analytics and ML Systems.(http://cseweb.ucsd.edu/classes/wi19/cse291-f/)
• CSci 8980: Machine Learning in Computer Systems(http://www-users.cselabs.umn.edu/classes/Spring-2019/csci8980/)
• Mu Li (MxNet, Parameter Server): Introduction to Deep Learning(https://courses.d2l.ai/berkeley-stat-157/index.html)(https://www.d2l.ai/)
• 10-605: Machine Learning with Large Datasets.(https://10605.github.io/fall2020/index.html)
• CS 329S: Machine Learning Systems Design.(https://stanford-cs329s.github.io/index.html)

Blog

• Parallelizing across multiple CPU/GPUs to speed up deep learning inference at the edge(https://aws.amazon.com/blogs/machine-learning/parallelizing-across-multiple-cpu-gpus-to-speed-up-deep-learning-inference-at-the-edge/)
• Building Robust Production-Ready Deep Learning Vision Models in Minutes(https://medium.com/google-developer-experts/building-robust-production-ready-deep-learning-vision-models-in-minutes-acd716f6450a)
• Deploy Machine Learning Models with Keras, FastAPI, Redis and Docker(https://medium.com/@shane.soh/deploy-machine-learning-models-with-keras-fastapi-redis-and-docker-4940df614ece)
• How to Deploy a Machine Learning Model -- Creating a production-ready API using FastAPI + Uvicorn(https://towardsdatascience.com/how-to-deploy-a-machine-learning-model-dc51200fe8cf)(https://github.com/MaartenGr/ML-API)
• Deploying a Machine Learning Model as a REST API(https://towardsdatascience.com/deploying-a-machine-learning-model-as-a-rest-api-4a03b865c166)
• Continuous Delivery for Machine Learning(https://martinfowler.com/articles/cd4ml.html)
• Kubernetes CheatSheets In A4(https://github.com/HuaizhengZhang/cheatsheet-kubernetes-A4)
• A Gentle Introduction to Kubernetes(https://medium.com/faun/a-gentle-introduction-to-kubernetes-4961e443ba26)
• Train and Deploy Machine Learning Model With Web Interface - Docker, PyTorch & Flask(https://github.com/imadelh/ML-web-app)
• Learning Kubernetes, The Chinese Taoist Way(https://github.com/caicloud/kube-ladder)
• Data pipelines, Luigi, Airflow: everything you need to know(https://towardsdatascience.com/data-pipelines-luigi-airflow-everything-you-need-to-know-18dc741449b7)
• The Deep Learning Toolset — An Overview(https://medium.com/luminovo/the-deep-learning-toolset-an-overview-b71756016c06)
• Summary of CSE 599W: Systems for ML(http://jcf94.com/2018/10/04/2018-10-04-cse559w/)
• Polyaxon, Argo and Seldon for Model Training, Package and Deployment in Kubernetes(https://medium.com/analytics-vidhya/polyaxon-argo-and-seldon-for-model-training-package-and-deployment-in-kubernetes-fa089ba7d60b)
• Overview of the different approaches to putting Machine Learning (ML) models in production(https://medium.com/analytics-and-data/overview-of-the-different-approaches-to-putting-machinelearning-ml-models-in-production-c699b34abf86)
• Being a Data Scientist does not make you a Software Engineer(https://towardsdatascience.com/being-a-data-scientist-does-not-make-you-a-software-engineer-c64081526372) Architecting a Machine Learning Pipeline(https://towardsdatascience.com/architecting-a-machine-learning-pipeline-a847f094d1c7)
• Model Serving in PyTorch(https://pytorch.org/blog/model-serving-in-pyorch/)
• Machine learning in Netflix(https://medium.com/@NetflixTechBlog)
• SciPy Conference Materials (slides, repo)(https://github.com/deniederhut/Slides-SciPyConf-2018)
• 继Spark之后，UC Berkeley 推出新一代AI计算引擎——Ray(http://www.qtmuniao.com/2019/04/06/ray/)
• 了解/从事机器学习/深度学习系统相关的研究需要什么样的知识结构？(https://www.zhihu.com/question/315611053/answer/623529977)
• Learn Kubernetes in Under 3 Hours: A Detailed Guide to Orchestrating Containers(https://www.freecodecamp.org/news/learn-kubernetes-in-under-3-hours-a-detailed-guide-to-orchestrating-containers-114ff420e882/)(https://github.com/rinormaloku/k8s-mastery)
• data-engineer-roadmap: Learning from multiple companies in Silicon Valley. Netflix, Facebook, Google, Startups(https://github.com/hasbrain/data-engineer-roadmap)
• TensorFlow Serving + Docker + Tornado机器学习模型生产级快速部署(https://zhuanlan.zhihu.com/p/52096200?utm_source=wechat_session&utm_medium=social&utm_oi=38612796178432)
• Deploying a Machine Learning Model as a REST API(https://towardsdatascience.com/deploying-a-machine-learning-model-as-a-rest-api-4a03b865c166)
• Colossal-AI: A Unified Deep Learning System for Big Model Era(https://medium.com/@hpcaitech/train-18-billion-parameter-gpt-models-with-a-single-gpu-on-your-personal-computer-8793d08332dc)(https://github.com/hpcaitech/ColossalAI)
• Data Engineer Roadmap(https://www.scaler.com/blog/data-engineer-roadmap/)
• Build an ML Framework from Scratch(https://haifengjin.com/build-an-ml-framework-from-scratch/)(https://github.com/haifeng-jin/readable-ml-framework)