Edge Enhanced Scalable and Real-time Video Stream Analytics

With the rise and adoption of Internet of Things devices, CCTV cameras, and smart objects, a large number of data streams with high volume and velocity are being produced that need to be accurately analyzed in real-time. State-of-the-art deep neural networks (DNNs) can produce accurate results but are computationally complex to train and perform inference. Consequently, data streams are sent to a resource-rich cloud platform to perform analytics. This cloud-only approach has limited ability to provide real-time and scalable analytics for applications such as real-time object recognition, telesurgery and autonomous vehicles due to high latency and bandwidth consumption. These limitations arise from the network connecting the data source to the cloud. To resolve these issues with cloud-only analytics, this thesis builds on the core idea that stream analytics can be moved to the edge of the network to do partial and collaborative processing with the cloud platform.

The methodology consists of two parts. In the first part, the stream analytics pipeline for deep learning-based object recognition is broken down into stages based on task complexity and ease of decomposition. The stages are then distributed over the participating edge and cloud resources using the stage-to-resource mapping algorithm while satisfying the user quality of service requirements, i.e., deadline time. In the second analytics part, a deep neural network model is decomposed into groups of layer segments which are distributed over the computing hierarchy, from the edge to the cloud resource. The main aim of the first part is to efficiently distribute the static stages over the resources, while the second part further improves the first part by breaking the compute and time-intensive machine learning stages dynamically over the resources using a binary search-inspired reinforcement learning algorithm. The second part focuses on accelerating the inference part of the pre-trained DNNs. The proposed methodology has three main advantages for stream analytics compared to the cloud-only case. Firstly, it provides a scalable approach to real-time stream analytics without compromising on the accuracy of the analytics. Secondly, it accelerates the inference of deep neural network models to provide real-time performance. Thirdly, it reduces the bandwidth between the edge and the cloud platform.

An extensive set of experiments were performed to prove the efficacy of the proposed approach. The experiments show that decomposing the stream analytics pipeline of object recognition and DNN models over the edge to the cloud continuum can significantly accelerate the analytics performance and reduce the bandwidth on the cloud.