A Statistical Distribution-Based Deep Neural Network Model – a new perspective on effective learning
Abstract:
The impressive results achieved by deep neural networks (DNNs) in various tasks, computer vision in particular, such as image recognition, object detection and image segmentation, have sparked the recent surging interests in artificial intelligence (AI) from both the industry and the academia alike. The wide adoption of DNN models in real-time applications has, however, brought up a need for more effective training of an easily parallelizable DNN model for low latency and high throughput. This is particularly challenging because of DNN's deep structures. To address this challenge, we observe that most of existing DNN models operate on deterministic numbers and process one single frame of image at a time, and may not fully utilize the temporal and contextual correlation typically present in multiple channels of the same image or adjacent frames from a video. Based on well-established statistical timing analysis foundations from the EDA domain, we propose a novel statistical distribution-based DNN model that extends existing DNN architectures but operates directly on correlated distributions rather than deterministic numbers. This new perspective of training DNN has resulted in surprising effects on achieving not only improved learning accuracy, but also reduced latency and increased high throughputs. Preliminary experimental results on various tasks, including 3D Cardiac Cine MRI segmentation, showed a great potential of this new type of statistical distribution-based DNN model, which warrants further investigation.
Bio:
Dr. Jinjun Xiong is currently Senior Researcher and Program Director for AI and Hybrid Clouds Systems at the IBM Thomas J. Watson Research Center. He co-founded and co-directs the IBM-Illinois Center for Cognitive Computing Systems Research (C3SR.com). His recent research interests are on across-stack AI systems research, which include AI applications, algorithms, tooling and computer architectures. Many of his research results have been adopted in IBM’s products and tools. He published more than 100 peer-reviewed papers in top AI conferences and systems conferences. His publication won seven Best Paper Awards and eight Nominations for Best Paper Awards. He also won top awards from various international competitions, including the recent champion for the IEEE GraphChallenge on accelerating sparse neural networks, and champions for the DAC'19 Systems Design Contest on designing an object detection neural network for edge FPGA and GPU devices.
Microsoft Teams meeting
Join on your computer or mobile app
Click here to join the meeting
Or call in (audio only)
+1 630-556-7958,,821911144# United States, Big Rock
Phone Conference ID: 821 911 144#