ILSVRC優勝を支える 深層学習プラットフォーム 株式会社センスタイムジャパン 代表取締役 勞世竑 1 What is ILSVRC センスタイムジャパン ILSVRC: ImageNet Large Scale Visual Recognition Challenge • Olympics in computer vision research. • Evaluates algorithms for object detection and image classification at large scale. 2 ILSVRC2016 object detection センスタイムジャパン • 200 clases • Training data 400,000 objects • Testing >100,000 Crutch Bee Sunglasses Horizontal bar Aspect ratio Watercraft Burrito Cart Monkey Deformability Low 3 High Challenges • Intra-class variation (person) センスタイムジャパン 3 Keys to win the challenge センスタイムジャパン 1. A smart algorithm good enough to deal with those variations 2. Enough big data for training 3. A powerful training platform to improving the training efficiency 5 Our Integrated Solutions センスタイムジャパン Deep Link • • • • C P U Storage Communucation Scheduling Monitoring Parrots Parrots PPL PPL G P U G P U G P U G P U C P U G P U G P U G P U G P U 7 Overall Architectures センスタイムジャパン Deep Learning Monitoring Parrots Parrots Learning Framework PPL Library Communication Module Backup Virtualization Cluster monitoring & management GPU Clusters DeepLink Cache Storage 8 DeepLink Clusters センスタイムジャパン • Designed for Deep Learning Software Hardware Codesign Maximize respective strengths while ensuring optimal cooperation. Highperformance Hardware • High speed interconnects • High performance GPU computing • Efficient distributed storage Customized Middlewares • Distributed storage & cache system (optimized for small files) • Distributed deep learning framework • Task scheduling & monitoring 9 Parrots for large scale training センスタイムジャパン • A New Deep Learning Framework for the Future Efficiency 効率性 Scalability 大規模対応 Flexibility 拡張性 10 Feature Matrix センスタイムジャパン Parrots Caffe TensorFlow Chainer Multi-GPU support Y Y L L Distributed training (multinodes) Y N L N Model parallelism Planned N L L Concurrent feeding Y N Y N Parallel data preprocessing Y N Y N RNN support Y N Y Y Variable input size Y Y Y Y Dynamic network structure Y N Y Y Partial flow execution Y N Y N Block composition Y N Y Y Customized layer Y Y Y Y Customized updating policy Y N Y Y Interoperability with Caffe Y Y N N Can directly load Caffe models and provide tools to convert Caffe to Parrots L: low-level support. Non-trivial coding is required to make it happen. 11 PPL is a library for fast DL computing センスタイムジャパン • The efficiency of a deep network essentially depends on how fast we can compute • PPL (Parrots Primitive Library) is to provide highly optimized functions for such computation. Convolution Matrix product Pooling Activation Batch normalization Others … 12 ILSVRC2016 Results センスタイムジャパン 1. By using the power of the DL platform we are able to train very large scale deep learning networks efficiently 2. Chinese Univ of HongKong and SenseTime won on Object Detection and Scene Parsing 14 Summary センスタイムジャパン • Technical infrastructure is crucial • We have solutions across the entire stack • DeepLink: sophisticated and versatile cluster training platform • Parrots: deep learning framework (efficient, scalable, and flexible) • PPL: computation modules (supporting CPU, GPU, and embedded devices) • Top-notch performance • Looking forward to collaborating with you 15 センスタイムジャパン Thanks for attention 16
© Copyright 2024 Paperzz
