Chapter 14

Video System

Tools

VideoFlow: Python framework that facilitates the quick development of complex video analysis applications and other series-processing based applications in a multiprocessing environment.(https://github.com/videoflow/videoflow)
VidGear: Powerful Multi-Threaded OpenCV and FFmpeg based Turbo Video Processing Python Library with unique State-of-the-Art Features.(https://github.com/abhiTronix/vidgear)
NVIDIA DALI: A library containing both highly optimized building blocks and an execution engine for data pre-processing in deep learning applications(https://github.com/NVIDIA/DALI)
TensorStream: A library for real-time video stream decoding to CUDA memory(https://github.com/Fonbet/argus-tensor-stream)
C++ image processing library with using of SIMD: SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, AVX, AVX2, AVX-512, VMX(Altivec)(https://github.com/ermig1979/Simd)
Pretrained image and video models for Pytorch.(https://github.com/alexandonian/pretorched-x)
LiveDetect - Live video client to DeepDetect.(https://github.com/jolibrain/livedetect)

Video Analysis Papers

Server-Driven Video Streaming for Deep Learning Inference(https://kuntaidu.github.io/assets/doc/DDS.pdf)
- Kuntai Du, Ahsan Pervaiz, Xin Yuan, Aakanksha Chowdhery, Qizheng Zhang, Henry Hoffmann, Junchen Jiang (SIGCOMM2020)
Reducto: On-Camera Filtering for ResourceEfficient Real-Time Video Analytics(http://web.cs.ucla.edu/~harryxu/papers/li-sigcomm20.pdf)
- Yuanqi Li, Arthi Padmanabhan, Pengzhan Zhao, Yufei Wang, Guoqing Harry Xu, Ravi Netravali. (SIGCOMM2020)
Fu, Daniel Y., et al. "Rekall: Specifying video events using compositions of spatiotemporal labels." arXiv preprint arXiv:1910.02993 (2019).(https://arxiv.org/pdf/1910.02993.pdf)
Puffer: Puffer is a Stanford University research study about using machine learning to improve video-streaming algorithms. Please visit(https://github.com/StanfordSNR/puffer)
Visual Road: A Video Data Management Benchmark(http://db.cs.washington.edu/projects/visualroad/)
- Brandon Haynes, Amrita Mazumdar, Magdalena Balazinska, Luis Ceze, Alvin Cheung (SIGMOD 2019)
CaTDet: Cascaded Tracked Detector for Efficient Object Detection from Video(http://www.sysml.cc/doc/2019/111.pdf)
- Mao, Huizi, Taeyoung Kong, and William J. Dally. (SysML2019)
Live Video Analytics at Scale with Approximation and Delay-Tolerance(https://www.microsoft.com/en-us/research/wp-content/uploads/2017/02/videostorm_nsdi17.pdf)
- Zhang, Haoyu, Ganesh Ananthanarayanan, Peter Bodik, Matthai Philipose, Paramvir Bahl, and Michael J. Freedman. (NSDI 2017)
Chameleon: scalable adaptation of video analytics(http://people.cs.uchicago.edu/~junchenj/docs/Chameleon_SIGCOMM_CameraReady.pdf)
- Jiang, Junchen, et al. (SIGCOMM 2018)
- Summary: Configuration controller for balancing accuracy and resource. Golden configuration is a good design. Periodic profiling often exceeded any resource savings gained by adapting the configurations.
Kang, Daniel, Peter Bailis, and Matei Zaharia. "Blazeit: Fast exploratory video queries using neural networks." arXiv preprint arXiv:1805.01046 (2018).(https://arxiv.org/pdf/1805.01046.pdf)
Noscope: optimizing neural network queries over video at scale(https://arxiv.org/pdf/1703.02529)(https://github.com/stanford-futuredata/noscope)
- Kang, Daniel, John Emmons, Firas Abuzaid, Peter Bailis, and Matei Zaharia. (VLDB2017)
- Summary: Information cache + difference detection model + small detection model + sequence optimizer
SVE: Distributed video processing at Facebook scale(http://www.cs.princeton.edu/~wlloyd/papers/sve-sosp17.pdf)
- Huang, Qi, et al. (SOSP2017)
Scanner: Efficient Video Analysis at Scale(http://graphics.stanford.edu/papers/scanner/poms18_scanner.pdf)(https://github.com/scanner-research/scanner)
- Poms, Alex, Will Crichton, Pat Hanrahan, and Kayvon Fatahalian (SIGGRAPH 2018)
A cloud-based large-scale distributed video analysis system(https://ai.google/research/pubs/pub45631)
- Wang, Yongzhe, et al. (ICIP 2016)
Rosetta: Large scale system for text detection and recognition in images(https://research.fb.com/wp-content/uploads/2018/10/Rosetta-Large-scale-system-for-text-detection-and-recognition-in-images.pdf)
- Borisyuk, Fedor, Albert Gordo, and Viswanath Sivakumar. (KDD 2018)

Video Streaming Papers

Neural-Enhanced Live Streaming: Improving Live Video Ingest via Online Learning(http://ina.kaist.ac.kr/~livenas/livenas_sigcomm2020.pdf)
- Jaehong Kim, Youngmok Jung, Hyunho Yeo, Juncheol Ye, and Dongsu Han (SIGCOMM2020)
Learning in situ: a randomized experiment in video streaming(https://www.usenix.org/system/files/nsdi20-paper-yan.pdf)
- Francis Y. Yan and Hudson Ayers, Stanford University; Chenzhi Zhu, Tsinghua University; Sadjad Fouladi, James Hong, Keyi Zhang, Philip Levis, and Keith Winstein, Stanford University (NSDI2020)
CSI: Inferring Mobile ABR Video Adaptation Behavior under HTTPS and QUIC(https://dl.acm.org/doi/abs/10.1145/3342195.3387558)
- Shichang Xu (University of Michigan), Subhabrata Sen (AT&T Labs Research), Z. Morley Mao (University of Michigan) (Eurosys2020)
Reconstructing proprietary video streaming algorithms(https://www.usenix.org/conference/atc20/presentation/gruener)
- Maximilian Grüner, Melissa Licciardello, and Ankit Singla, ETH Zürich (ATC2020)
Neural adaptive content-aware internet video delivery.(https://www.usenix.org/system/files/osdi18-yeo.pdf)(https://github.com/kaist-ina/NAS_public)
- Yeo, H., Jung, Y., Kim, J., Shin, J. and Han, D., 2018. (OSDI 2018)
- Summary: Combine video super-resolution and ABR

Chapter 14

Video System

Tools

VideoFlow: Python framework that facilitates the quick development of complex video analysis applications and other series-processing based applications in a multiprocessing environment.(https://github.com/videoflow/videoflow)
VidGear: Powerful Multi-Threaded OpenCV and FFmpeg based Turbo Video Processing Python Library with unique State-of-the-Art Features.(https://github.com/abhiTronix/vidgear)
NVIDIA DALI: A library containing both highly optimized building blocks and an execution engine for data pre-processing in deep learning applications(https://github.com/NVIDIA/DALI)
TensorStream: A library for real-time video stream decoding to CUDA memory(https://github.com/Fonbet/argus-tensor-stream)
C++ image processing library with using of SIMD: SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, AVX, AVX2, AVX-512, VMX(Altivec)(https://github.com/ermig1979/Simd)
Pretrained image and video models for Pytorch.(https://github.com/alexandonian/pretorched-x)
LiveDetect - Live video client to DeepDetect.(https://github.com/jolibrain/livedetect)

Video Analysis Papers

Server-Driven Video Streaming for Deep Learning Inference(https://kuntaidu.github.io/assets/doc/DDS.pdf)
- Kuntai Du, Ahsan Pervaiz, Xin Yuan, Aakanksha Chowdhery, Qizheng Zhang, Henry Hoffmann, Junchen Jiang (SIGCOMM2020)
Reducto: On-Camera Filtering for ResourceEfficient Real-Time Video Analytics(http://web.cs.ucla.edu/~harryxu/papers/li-sigcomm20.pdf)
- Yuanqi Li, Arthi Padmanabhan, Pengzhan Zhao, Yufei Wang, Guoqing Harry Xu, Ravi Netravali. (SIGCOMM2020)
Fu, Daniel Y., et al. "Rekall: Specifying video events using compositions of spatiotemporal labels." arXiv preprint arXiv:1910.02993 (2019).(https://arxiv.org/pdf/1910.02993.pdf)
Puffer: Puffer is a Stanford University research study about using machine learning to improve video-streaming algorithms. Please visit(https://github.com/StanfordSNR/puffer)
Visual Road: A Video Data Management Benchmark(http://db.cs.washington.edu/projects/visualroad/)
- Brandon Haynes, Amrita Mazumdar, Magdalena Balazinska, Luis Ceze, Alvin Cheung (SIGMOD 2019)
CaTDet: Cascaded Tracked Detector for Efficient Object Detection from Video(http://www.sysml.cc/doc/2019/111.pdf)
- Mao, Huizi, Taeyoung Kong, and William J. Dally. (SysML2019)
Live Video Analytics at Scale with Approximation and Delay-Tolerance(https://www.microsoft.com/en-us/research/wp-content/uploads/2017/02/videostorm_nsdi17.pdf)
- Zhang, Haoyu, Ganesh Ananthanarayanan, Peter Bodik, Matthai Philipose, Paramvir Bahl, and Michael J. Freedman. (NSDI 2017)
Chameleon: scalable adaptation of video analytics(http://people.cs.uchicago.edu/~junchenj/docs/Chameleon_SIGCOMM_CameraReady.pdf)
- Jiang, Junchen, et al. (SIGCOMM 2018)
- Summary: Configuration controller for balancing accuracy and resource. Golden configuration is a good design. Periodic profiling often exceeded any resource savings gained by adapting the configurations.
Kang, Daniel, Peter Bailis, and Matei Zaharia. "Blazeit: Fast exploratory video queries using neural networks." arXiv preprint arXiv:1805.01046 (2018).(https://arxiv.org/pdf/1805.01046.pdf)
Noscope: optimizing neural network queries over video at scale(https://arxiv.org/pdf/1703.02529)(https://github.com/stanford-futuredata/noscope)
- Kang, Daniel, John Emmons, Firas Abuzaid, Peter Bailis, and Matei Zaharia. (VLDB2017)
- Summary: Information cache + difference detection model + small detection model + sequence optimizer
SVE: Distributed video processing at Facebook scale(http://www.cs.princeton.edu/~wlloyd/papers/sve-sosp17.pdf)
- Huang, Qi, et al. (SOSP2017)
Scanner: Efficient Video Analysis at Scale(http://graphics.stanford.edu/papers/scanner/poms18_scanner.pdf)(https://github.com/scanner-research/scanner)
- Poms, Alex, Will Crichton, Pat Hanrahan, and Kayvon Fatahalian (SIGGRAPH 2018)
A cloud-based large-scale distributed video analysis system(https://ai.google/research/pubs/pub45631)
- Wang, Yongzhe, et al. (ICIP 2016)
Rosetta: Large scale system for text detection and recognition in images(https://research.fb.com/wp-content/uploads/2018/10/Rosetta-Large-scale-system-for-text-detection-and-recognition-in-images.pdf)
- Borisyuk, Fedor, Albert Gordo, and Viswanath Sivakumar. (KDD 2018)

Video Streaming Papers

Neural-Enhanced Live Streaming: Improving Live Video Ingest via Online Learning(http://ina.kaist.ac.kr/~livenas/livenas_sigcomm2020.pdf)
- Jaehong Kim, Youngmok Jung, Hyunho Yeo, Juncheol Ye, and Dongsu Han (SIGCOMM2020)
Learning in situ: a randomized experiment in video streaming(https://www.usenix.org/system/files/nsdi20-paper-yan.pdf)
- Francis Y. Yan and Hudson Ayers, Stanford University; Chenzhi Zhu, Tsinghua University; Sadjad Fouladi, James Hong, Keyi Zhang, Philip Levis, and Keith Winstein, Stanford University (NSDI2020)
CSI: Inferring Mobile ABR Video Adaptation Behavior under HTTPS and QUIC(https://dl.acm.org/doi/abs/10.1145/3342195.3387558)
- Shichang Xu (University of Michigan), Subhabrata Sen (AT&T Labs Research), Z. Morley Mao (University of Michigan) (Eurosys2020)
Reconstructing proprietary video streaming algorithms(https://www.usenix.org/conference/atc20/presentation/gruener)
- Maximilian Grüner, Melissa Licciardello, and Ankit Singla, ETH Zürich (ATC2020)
Neural adaptive content-aware internet video delivery.(https://www.usenix.org/system/files/osdi18-yeo.pdf)(https://github.com/kaist-ina/NAS_public)
- Yeo, H., Jung, Y., Kim, J., Shin, J. and Han, D., 2018. (OSDI 2018)
- Summary: Combine video super-resolution and ABR