Edge-to-Cloud Data Integration Models for Industrial IoT Applications
Abstract
Data integration is a crucial requirement for Industrial Internet of Things (IIoT) applications involving very large data volumes generated by multiple sensors and devices. With the aim of timesensitive access to valuable information, both distributed and centralized edge-cloud paradigms are considered. Edge-to-cloud data integration applies the principles of data ingestion, analysis, and storage across the entire edge-cloud ecosystem. Ephemeral data management reflects the transient nature of edge-centred processing that leverages the temporal proximity of computation and data source. Data quality, security, and compliance encompass privacy-preserving mechanisms. Further recognised aspects of an appropriate data integration model are computational offload and resource allocation over the edge-cloud infrastructure. The analysis is illustrated by means of representative solutions. Edge-cloud integration of Industrial IoT data bridges the requirements of latency-sensitive use cases with the need for system-wide orchestration and management.
The Internet of Things (IoT) and the Industrial Internet of Things (IIoT) comprise a large number of devices and sensors that continuously generate an unprecedented amount of data. The data can be collected, analysed, and aggregated in or around the places of generation, such as factories, power plants, or automatically guided vehicle systems, and used for autonomous fault detection and localization, predictive maintenance, resource utilization optimization, and process improvement. For low-latency use cases, such as anomaly detection requiring high frequency computation (not necessarily high processing requirements), data can be processed at the edge (the proximity of the sensor) for real-time responses. However, many use cases with high latency tolerance (e.g. predictive maintenance and resource utilization optimization) require data to be sent to the cloud, where an orchestration mechanism executes the appropriate tasks.
Article Information
Journal |
International Journal of Science, Research and Technology |
|---|---|
Volume (Issue) |
Vol. 5 No. 6 (2022): International Journal of Science, Research and Technology (IJSRAT) |
DOI |
|
Pages |
8852-8866 |
Published |
December 5, 2022 |
| Copyright |
All rights reserved |
Open Access |
This work is licensed under a Creative Commons Attribution 4.0 International License. |
How to Cite |
Mallesham Goli (%2022). Edge-to-Cloud Data Integration Models for Industrial IoT Applications. International Journal of Science, Research and Technology , Vol. 5 No. 6 (2022): International Journal of Science, Research and Technology (IJSRAT) , pp. 8852-8866. https://doi.org/10.15662/IJSRAT.2022.0506002 |
References
[2] Inala, R. Advancing Group Insurance Solutions Through Ai-Enhanced Technology Architectures And Big Data Insights.
[3] Chiang, M., & Zhang, T. (2016). Fog and IoT: An overview of research opportunities. IEEE Internet of Things Journal, 3(6), 854–864.
[4] Bonomi, F., Milito, R., Zhu, J., & Addepalli, S. (2012). Fog computing and its role in the Internet of Things. MCC Workshop, 13–16.
[5] Rongali, S. K. (2022). AI-Driven Automation in Healthcare Claims and EHR Processing Using MuleSoft and Machine Learning Pipelines. Available at SSRN 5763022.
[6] Gubbi, J., Buyya, R., Marusic, S., & Palaniswami, M. (2013). Internet of Things (IoT): A vision, architectural elements, and future directions. Future Generation Computer Systems, 29(7), 1645–1660.
[7] Gottimukkala, V. R. R. (2020). Energy-Efficient Design Patterns for Large-Scale Banking Applications Deployed on AWS Cloud. power, 9(12).
[8] Botta, A., De Donato, W., Persico, V., & Pescapé, A. (2016). Integration of cloud computing and IoT. Future Generation Computer Systems, 56, 684–700.
[9] Aitha, A. R. (2022). Cloud Native ETL Pipelines for Real Time Claims Processing in Large Scale Insurers. Available at SSRN 5532601.
[10] Gill, S. S., Tuli, S., Xu, M., et al. (2019). Transformative effects of IoT, blockchain and AI. IEEE Internet of Things Journal, 6(2), 2674–2689.
[11] Yandamuri, U. S. (2022). Cloud-Based Data Integration Architectures for Scalable Enterprise Analytics. International Journal of Intelligent Systems and Applications in Engineering, 10, 472-483.
[12] Zaharia, M., Xin, R. S., Wendell, P., et al. (2016). Apache Spark. Communications of the ACM, 59(11), 56–65.
[13] Amistapuram, K. Energy-Efficient System Design for High-Volume Insurance Applications in Cloud-Native Environments. International Journal of Innovative Research in Electrical, Electronics, Instrumentation and Control Engineering (IJIREEICE), DOI, 10.
[14] Carbone, P., Katsifodimos, A., Ewen, S., et al. (2015). Apache Flink. IEEE Data Engineering Bulletin, 38(4), 28–38.
[15] Varri, D. B. S. (2022). AI-Driven Risk Assessment And Compliance Automation In Multi-Cloud Environments. Available at SSRN 5774924.
[16] Stonebraker, M., Çetintemel, U., & Zdonik, S. (2005). The 8 requirements of real-time stream processing. ACM SIGMOD Record, 34(4), 42–47.
[17] Segireddy, A. R. (2020). Cloud Migration Strategies for High-Volume Financial Messaging Systems.
[18] Newman, S. (2021). Building microservices (2nd ed.). O’Reilly Media.
[19] Garapati, R. S. (2022). Web-Centric Cloud Framework for Real-Time Monitoring and Risk Prediction in Clinical Trials Using Machine Learning. Current Research in Public Health, 2, 1346.
[20] Fielding, R. T. (2000). Architectural styles and the design of network-based software architectures. Doctoral dissertation.
[21] Davuluri, P. N. Event-Driven Compliance Systems: Modernizing Financial Crime Detection Without Machine Intelligence.
[22] Buyya, R., Broberg, J., & Goscinski, A. (2011). Cloud computing: Principles and paradigms. Wiley.
[23] Vadisetty, R., Polamarasetti, A., Guntupalli, R., Raghunath, V., Jyothi, V. K., & Kudithipudi, K. (2021). Privacy-Preserving Gen AI in Multi-Tenant Cloud Environments. Sateesh kumar and Raghunath, Vedaprada and Jyothi, Vinaya Kumar and Kudithipudi, Karthik, Privacy-Preserving Gen AI in Multi-Tenant Cloud Environments (January 20, 2021).
[24] Pahl, C. (2015). Containerization and the PaaS cloud. IEEE Cloud Computing, 2(3), 24–31.
[25] Gottimukkala, V. R. R. (2022). Licensing Innovation in the Financial Messaging Ecosystem: Business Models and Global Compliance Impact. International Journal of Scientific Research and Modern Technology, 1(12), 177-186.
[26] Erl, T. (2005). Service-oriented architecture. Prentice Hall.
[27] Vadisetty, R., Polamarasetti, A., Guntupalli, R., Raghunath, V., Jyothi, V. K., & Kudithipudi, K. (2022). AI-Driven Cybersecurity: Enhancing Cloud Security with Machine Learning and AI Agents. Sateesh kumar and Raghunath, Vedaprada and Jyothi, Vinaya Kumar and Kudithipudi, Karthik, AI-Driven Cybersecurity: Enhancing Cloud Security with Machine Learning and AI Agents (February 07, 2022).
[28] Gilbert, S., & Lynch, N. (2002). Brewer’s conjecture. ACM SIGACT News, 33(2), 51–59.
[29] Siva Hemanth Kolla. (2022). Knowledge Retrieval Systems for Enterprise Service Environments. International Journal of Intelligent Systems and Applications in Engineering, 10(3s), 495–506. Retrieved from https://ijisae.org/index.php/IJISAE/article/view/8037
[30] Lamport, L. (1978). Time, clocks, and ordering of events. Communications of the ACM, 21(7), 558–565.
[31] Zhang, Y., Yu, R., Nekovee, M., et al. (2017). Software-defined and virtualization-based fog computing. IEEE Communications Magazine, 55(8), 36–43.
[32] Inala, R. (2022). Engineering Data Products for Investment Analytics: The Role of Product Master Data and Scalable Big Data Solutions. International Journal of Scientific Research and Modern Technology, 155-171.
[33] Sarkar, S., & Misra, S. (2016). Theoretical modelling of fog computing. IEEE Transactions on Computers, 65(2), 350–363.
[34] Aitha, A. R. (2022). Deep Neural Networks for Property Risk Prediction Leveraging Aerial and Satellite Imaging. International Journal of Communication Networks and Information Security (IJCNIS), 14(3), 1308-1318.
[35] Xu, X., Chen, Y., & Li, J. (2018). QoS-aware resource management. IEEE Access, 6, 69128–69141.
[36] Mao, Y., You, C., Zhang, J., Huang, K., & Letaief, K. (2017). Mobile edge computing. IEEE Communications Surveys & Tutorials, 19(4), 2322–2358.
[37] Aitha, A. R. (2021). Optimizing Data Warehousing for Large Scale Policy Management Using Advanced ETL Frameworks.
[38] Varghese, B., & Buyya, R. (2018). Next generation cloud computing. IT Professional, 20(3), 38–47.
[39] Satyanarayanan, M., et al. (2019). Edge analytics in the Internet of Things. IEEE Pervasive Computing, 18(2), 70–75.
[40] Chen, M., Mao, S., & Liu, Y. (2014). Big data survey. Mobile Networks and Applications, 19(2), 171–209.
[41] Segireddy, A. R. (2022). Terraform and Ansible in Building Resilient Cloud-Native Payment Architectures. International Journal of Intelligent Systems and Applications in Engineering, 10, 444-455.
[42] Manyika, J., et al. (2011). Big data. McKinsey Global Institute.
[43] McAfee, A., & Brynjolfsson, E. (2012). Big data. Harvard Business Review, 90(10), 60–68.
[44] Amistapuram, K. (2022). Fraud Detection and Risk Modeling in Insurance: Early Adoption of Machine Learning in Claims Processing. Available at SSRN 5741982.
[45] Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning. Springer.
[46] Rongali, S. K. (2020). Predictive Modeling and Machine Learning Frameworks for Early Disease Detection in Healthcare Data Systems. Current Research in Public Health, 1(1), 1-15.
[47] Vapnik, V. (1998). Statistical learning theory. Wiley.
[48] Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.
[49] Varri, D. B. S. (2021). Cloud-Native Security Architecture for Hybrid Healthcare Infrastructure. Available at SSRN 5785982.
[50] Abadi, M., et al. (2016). TensorFlow. OSDI, 265–283.
[51] Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection. ACM Computing Surveys, 41(3), 1–58.
[52] Breunig, M. M., et al. (2000). LOF. SIGMOD, 93–104.
[53] Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2008). Isolation forest. ICDM, 413–422.
[54] Yandamuri, U. S. (2022). Big Data Pipelines for Cross-Domain Decision Support: A Cloud-Centric Approach. International Journal of Scientific Research and Modern Technology, 227.
[55] Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). Why should I trust you? KDD, 1135–1144.
[56] Lundberg, S. M., & Lee, S.-I. (2017). SHAP. NeurIPS, 4765–4774.
[57] Rudin, C. (2019). Stop explaining black box models. Nature Machine Intelligence, 1, 206–215.
[58] Davuluri, P. N. (2020). Improving Data Quality and Lineage in Regulated Financial Data Platforms. Finance and Economics, 1(1), 1-14.
[59] Polyzotis, N., Roy, S., Whang, S. E., & Zinkevich, M. (2018). Data management challenges in ML. SIGMOD Record, 47(2), 34–43.
[60] Zinkevich, M., et al. (2017). ML: The high interest credit card. Google Research.
[61] Garapati, R. S. (2022). AI-Augmented Virtual Health Assistant: A Web-Based Solution for Personalized Medication Management and Patient Engagement. Available at SSRN 5639650.
[62] Cavoukian, A. (2011). Privacy by design. IPC Ontario.
[63] Solove, D. J., & Schwartz, P. M. (2018). Information privacy law. Wolters Kluwer.
[64] Nagabhyru, K. C. (2022). Bridging Traditional ETL Pipelines with AI Enhanced Data Workflows: Foundations of Intelligent Automation in Data Engineering. Available at SSRN 5505199.
[65] ISO/IEC. (2018). ISO/IEC 27018.
[66] Rongali, S. K. (2021). Cloud-Native API-Led Integration Using MuleSoft and .NET for Scalable Healthcare Interop-erability. Journal for ReAttach Therapy and Developmental Diversities, 4(2), 181-192.
[67] Sakimura, N., et al. (2014). OpenID Connect Core 1.0.
[68] Cameron, K. (2005). The laws of identity. Microsoft.
[69] Varri, D. B. S. (2022). A Framework for Cloud-Integrated Database Hardening in Hybrid AWS-Azure Environments: Security Posture Automation Through Wiz-Driven Insights. International Journal of Scientific Research and Modern Technology, 1(12), 216-226.
[70] Chaum, D. (1985). Security without identification. Communications of the ACM, 28(10), 1030–1044.
[71] Vadisetty, R., Polamarasetti, A., Guntupalli, R., Rongali, S. K., Raghunath, V., Jyothi, V. K., & Kudithipudi, K. (2021). Legal and Ethical Considerations for Hosting GenAI on the Cloud. International Journal of AI, BigData, Computational and Management Studies, 2(2), 28-34.
[72] Xu, Y., et al. (2019). Dynamic resource allocation in fog computing. IEEE Access, 7, 118217–118230.
[73] Segireddy, A. R. (2021). Containerization and Microservices in Payment Systems: A Study of Kubernetes and Docker in Financial Applications. Universal Journal of Business and Management, 1(1), 1-17.
[74] Deng, R., Lu, R., Lai, C., Luan, T. H., & Liang, H. (2016). Optimal workload allocation. IEEE Transactions on Vehicular Technology, 66(8), 7287–7299.
[75] Ramesh Inala. (2022). Cross-Domain MDM Integration Using AI-Driven Data Governance: A Case Study In Financial Technology Architecture. Migration Letters, 19(2), 280–304. Retrieved from https://migrationletters.com/index.php/ml/article/view/11982
[76] Roman, R., Lopez, J., & Mambo, M. (2018). Mobile edge computing security. IEEE Internet of Things Journal, 5(6), 4504–4516.
[77] Amistapuram, K. (2021). Digital Transformation in Insurance: Migrating Enterprise Policy Systems to .NET Core. Universal Journal of Computer Sciences and Communications, 1(1), 1-17.
[78] Weber, R. H. (2010). Internet of Things security. Computer Law & Security Review, 26(1), 23–30.
[79] Yandamuri, U. S. (2021). A Comparative Study of Traditional Reporting Systems versus Real-Time Analytics Dashboards in Enterprise Operations. Universal Journal of Business and Management.
[80] Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019). Edge intelligence. IEEE Access, 7, 47630–47646.
[81] Kolla, S. H. (2021). Rule-Based Automation for IT Service Management Workflows. Online Journal of Engineering Sciences, 1(1), 1–14. Retrieved from https://www.scipublications.com/journal/index.php/ojes/article/view/1360
[82] van der Aalst, W. (2016). Process mining. Springer.
[83] Gottimukkala, V. R. R. (2021). Digital Signal Processing Challenges in Financial Messaging Systems: Case Studies in High-Volume SWIFT Flows.
[84] Augusto, A., et al. (2019). Automated discovery of process models. ACM Computing Surveys, 52(5), 1–43.
[85] Davuluri, P. N. (2020). Event-Driven Architectures for Real-Time Regulatory Monitoring in Global Banking.
[86] Fischer, M. J., Lynch, N. A., & Paterson, M. S. (1985). Impossibility of consensus. Journal of the ACM, 32(2), 374–382.