Published Paper


Design of an Integrated Adaptive and Resource-Optimized Deep Learning Architecture for Real-Time Streaming Data Analysis

1 Rushikesh M. Shete; 2 Virendra K. Sharma
Department of Computer Science and Engineering, Bhagwant University, Ajmer (305004) Rajasthan, India
Page: 1787-1813
Published on: 2025 December

Abstract

The large quantity and speed of streaming data increasingly demand intelligent systems able to analyze events in real-time while being able to adapt to changes in data distributions and operate under constrained computational budgets. Shortcomings related to significant latencies, inability to integrate various classifiers to handle multi-modal streams, and inability to use resources efficiently on-edge devices and deployments are factors limiting the current approaches based on deep learning for streaming data analysis. In this scenario, we propose a framework for high-resolution integrated deep learning for very high-velocity streaming scenarios with five interconnected novel approaches, which include Dynamic Streaming Aware Graph Embedding Transformer (DSGET) for scalable, real-time temporal feature extraction, Continual Drift Adaptive Meta Learning Framework (CDAML) for fast adaptability to distributional shifts, Hierarchical Parameter Sharing Compression Network (HPSCN) for very effective resource utilization through temporal weight reuse, Multi-Modal Incremental Knowledge Integration Engine (MIKIE) for adaptive cross-modal fusion without full retraining, and Streaming Real Time Benchmark and Feedback Optimization Module (SRBFOM) for continuous in-operation evaluation and self-optimizations. The components for closed-loop pipelining where each output from each stage feeds the next stage form a closed-loop pipeline comprising these components. Experimental analysis shows about 40% lower processing latency and more than a 60% model size compression, with respect to drift recovery time reduced by 45% and an improvement in multi-modal predictive accuracy of 5-7% relative to state-of-the-art methods. The proposed architecture stands to unite scalability, adaptability, and computational efficiency that allow deployment in both cloud and edge environments for mission-critical real-time analytics. Further, this establishes a solid next-generation foundation for streaming deep learning systems with advancement in state-of-the-art adaptive, resource-efficient, and highly accurate streaming data analysis.

 

PDF