Securing Enterprise Data Pipelines Through Zero Trust Architecture and Continuous Compliance Monitoring in Distributed Environments
Abstract
Enterprise data pipelines are increasingly distributed across hybrid cloud platforms, multi-cloud infrastructures, edge environments, and interconnected enterprise applications, creating complex security and compliance challenges. Traditional perimeter-based security mechanisms are insufficient to protect sensitive information moving between diverse systems, processing services, and storage environments. This research proposes an integrated security framework that combines Zero Trust Architecture (ZTA) with continuous compliance monitoring to strengthen enterprise data pipeline protection. The framework incorporates identity-centric access control, continuous authentication, least-privilege authorization, encryption, behavioral analytics, policy enforcement, and automated compliance assessment throughout the data lifecycle. It evaluates user identities, workload credentials, device conditions, contextual risk indicators, and resource sensitivity before granting access to pipeline components. Continuous monitoring mechanisms assess configuration changes, access activities, data transfers, and security events against organizational policies and regulatory requirements. A risk-based orchestration mechanism prioritizes security violations and initiates corrective actions, including access revocation, workload isolation, policy remediation, and audit evidence collection. The proposed methodology establishes a reproducible evaluation model using representative enterprise pipeline workloads, simulated security incidents, and compliance scenarios. Performance indicators include unauthorized access detection, policy violation identification, remediation latency, processing overhead, and compliance coverage. The framework aims to improve data confidentiality, operational resilience, and regulatory accountability while preserving pipeline efficiency. This approach provides a foundation for secure, adaptive, and continuously governed enterprise data engineering across distributed environments.Article Information
Journal |
International Journal of Science, Research and Technology |
|---|---|
Volume (Issue) |
Vol. 6 No. 5 (2023): International Journal of Science, Research and Technology (IJSRAT) |
DOI |
|
Pages |
10724-10732 |
Published |
October 23, 2023 |
| Copyright |
All rights reserved |
Open Access |
This work is licensed under a Creative Commons Attribution 4.0 International License. |
How to Cite |
Suryanti Awang (%2023). Securing Enterprise Data Pipelines Through Zero Trust Architecture and Continuous Compliance Monitoring in Distributed Environments. International Journal of Science, Research and Technology , Vol. 6 No. 5 (2023): International Journal of Science, Research and Technology (IJSRAT) , pp. 10724-10732. https://doi.org/10.15662/IJSRAT.2023.0605027 |
References
1. Christoforou, A., Andreou, A. S., Garriga, M., & Baresi, L. (2022). Adopting microservice architecture: A decision support model based on genetically evolved multi-layer FCM. Applied Soft Computing, 114, Article 108066. https://doi.org/10.1016/j.asoc.2021.108066
2. Vimal Raja, G. (2022). Leveraging machine learning for real-time short-term snowfall forecasting using multisource atmospheric and terrain data integration. International Journal of Multidisciplinary Research in Science, Engineering and Technology, 5(8), 1336-1339.
3. Gadige, C. D. (2023). API-led connectivity in enterprise CRM: A reference architecture for Salesforce and heterogeneous enterprise systems. International Research Journal of Innovative Engineering, 7(2), 12125–12135.
4. Sugumar, R. (2021). Generative AI Pipelines for Safety Validation of Autonomous Driving Models. International Journal of Advanced Research in Computer Science & Technology (IJARCST), 4(5), 5466-5469.
5. Tarigoppula, S. (2021). Policy-driven autonomous infrastructure provisioning: A framework for enterprise cloud governance using infrastructure as code. International Journal of Emerging Trends in Engineering and Management Research, 6(5), 10361–10369.
6. Gunasekaran, R. M. N. (2023). Operational Challenges in Basel IV Credit Risk Compliance. The Eastasouth Journal of Information System and Computer Science, 1(01), 169-178.
7. Ramasamy, M. (2023). Cloud-native control plane design for modern network automation systems. International Journal of Research Publications in Engineering, Technology and Management (IJRPETM), 6(4), 9082–9091.
8. Sudhan, S. K. H. H., & Kumar, S. S. (2016). Gallant Use of Cloud by a Novel Framework of Encrypted Biometric Authentication and Multi Level Data Protection. Indian Journal of Science and Technology, 9, 44.
9. Polamarasetty, V. K. (2021). Modernizing SAP sales and distribution systems through ABAP-based enterprise solutions. International Journal of Research and Applied Innovations, 4(2), 4925–4930.
10. Chinnam, N. B. (2023). Modernizing retail fulfillment operations through automated load execution and microservices. International Journal of Computer Technology and Electronics Communication, 6(4), 7367–7371.
11. Neela, S. (2022). Semantic Interoperability in Real-Time Enterprise Integration Using Middleware Abstractions. American International Journal of Computer Science and Technology, 4(6), 56-66.
12. Venkatasalam, K., Rajendran, P., & Thangavel, M. (2019). Improving the accuracy of feature selection in big data mining using accelerated flower pollination (AFP) algorithm. Journal of medical systems, 43(4), 96.
13. Basavala, S. R. (2023). Automated threat modeling and vulnerability prioritization for secure software development. International Journal of Research and Applied Innovations, 6(1), 8414–8424.
14. Jayaraman, S., Rajendran, S., & P, S. P. (2019). Fuzzy c-means clustering and elliptic curve cryptography using privacy preserving in cloud. International Journal of Business Intelligence and Data Mining, 15(3), 273-287.
15. Reddy, K. V. (2023). Reconfigurable FPGA-based acceleration of RTL simulation and co-simulation for cloud datacenter workloads. International Journal of Emerging Trends in Engineering and Management Research, 8(2), 13252–13265.
16. Kundavaram, R. M. R. (2022). Cloud-based data analysis identifying key financial influencers at scale. European Economic Letters (EEL), 12 (2), 234–241.
17. Chinthamani, R. (2022). Event-driven microservices for scalable retail fulfillment optimization. International Journal of Research and Applied Innovations, 5(2), 6790–6795.
18. Kollu, R. K. (2023). CRM architecture deep dive: Building for scale on Salesforce. International Journal of Engineering & Extended Technologies Research, 5(5), 7309–7312.
19. Bandari, U., & Thilagaraj, S. V. (2022). From central BI to federated self-service: A platform-as-a-product reference architecture for data mesh for a multinational manufacturer. International Journal of Science, Research and Technology, 5(3), 7784–7794.
20. Selvarajan, K. (2022). Architecting scalable self-service data platforms for enterprise analytics. International Journal of Science, Research and Technology (IJSRAT), 5(2), 7427–7436.
21. Padmanabham, S. (2023). Building resilient Banking platforms using Event-Driven Microseconds and cloud Native Architecture. International Journal of Research and Applied Innovations, 6(4), 9284-9290.
22. Mathew, A. R. (2019). Cyber-infrastructure connections and smart gird security. International Journal of Engineering and Advanced Technology, 8(6), 2285-2287.
23. Raja, G. V., & Mali, R. K. (2021). Federated learning frameworks for privacy-preserving artificial intelligence applications. International Journal of Research Publications in Engineering, Technology and Management (IJRPETM), 4(3), 4946-4950.
24. Bitragunta, S. L. V., & Paramasivan, M. (2023). Midterm dynamic simulation for the governance of reserves in systems with elevated renewable energy integration. Journal of Artificial Intelligence, 1(1), 1956–1962.
25. Pochincharla, S., & Devineni, A. (2023). Automated compliance-driven patch management and security hardening in multi-cloud banking infrastructure. International Journal of Advances in Signal and Image Sciences, 57–62.
26. Soundappan, S. J. (2022). Distributed Cloud Architecture for Intelligent Performance Optimization and Proactive Real-Time Threat Detection. International Research Journal of Innovative Engineering, 6(6), 11426-11432.
27. Puram, S. (2021). Reliable and Scalable Mobile Technology Architecture: A Formal Model, Synchronization Protocol, and Reference Design. The USA Journals TAJET, 3(08), 33-48.
28. Koganti, H. (2021). Machine learning-driven performance anomaly detection and auto-tuning in distributed Java full-stack systems: A comprehensive review. International Journal of Research Publications in Engineering, Technology and Management, 4(3), 4977–4986.
29. Gopinathan, V. R. (2022). Advanced Predictive Decision Intelligence Using Machine Learning for Strategic Enterprise Management Platforms. International Journal of Research and Applied Innovations, 5(6), 8162-8169.
30. Kalabhavi, V. (2023). Automated quote generation using AI for product configuration, pricing and external order integration in SAP CPQ. International Research Journal of Innovative Engineering, 7(3), 12503–12515.
31. Yadavalli, V. R. (2022). Continuous trust scoring for cross-layer access governance in S/4HANA, Fiori, and SAP BTP. International Journal of Engineering & Extended Technologies Research, 4(1), 4347–4357.
32. Mathew, A. (2021). Artificial intelligence and cognitive computing for 6G communications & networks. International Journal of Computer Science and Mobile Computing, 10(3), 26-31.
33. Mysula, S. (2022). Architectural modernization of legacy enterprise integration systems for scalable and reliable business operations. International Journal of Research and Applied Innovations, 5(2), 6796–6801.
34. Shaik, N. (2023). Zero downtime, zero trust: Automating certificate security at scale. International Journal of Emerging Trends in Engineering and Management Research (IJETEMR), 8(4), 13973–13988.
35. Bandari, U., & Thilagaraj, S. V. (2022). From central BI to federated self-service: A platform-as-a-product reference architecture for data mesh for a multinational manufacturer. International Journal of Science, Research and Technology, 5(3), 7784–7794.
36. Bhattacharjee, B., Khan, M. A. N., Enayet, M. A., Gazi, M. S., Tasnim, M., Jakir, T., ... & Himeluzzaman, M. (2021). Adversarial Machine Learning and Cognitive AI for Autonomous Defense against Advanced Cyber Threats. International Journal of Computer Technology and Electronics Communication, 4(6), 4327-4337.
37. Hannousse, A., & Yahiouche, S. (2021). Securing microservices and microservice architectures: A systematic mapping study. Computer Science Review, 41, Article 100415. https://doi.org/10.1016/j.cosrev.2021.100415
2. Vimal Raja, G. (2022). Leveraging machine learning for real-time short-term snowfall forecasting using multisource atmospheric and terrain data integration. International Journal of Multidisciplinary Research in Science, Engineering and Technology, 5(8), 1336-1339.
3. Gadige, C. D. (2023). API-led connectivity in enterprise CRM: A reference architecture for Salesforce and heterogeneous enterprise systems. International Research Journal of Innovative Engineering, 7(2), 12125–12135.
4. Sugumar, R. (2021). Generative AI Pipelines for Safety Validation of Autonomous Driving Models. International Journal of Advanced Research in Computer Science & Technology (IJARCST), 4(5), 5466-5469.
5. Tarigoppula, S. (2021). Policy-driven autonomous infrastructure provisioning: A framework for enterprise cloud governance using infrastructure as code. International Journal of Emerging Trends in Engineering and Management Research, 6(5), 10361–10369.
6. Gunasekaran, R. M. N. (2023). Operational Challenges in Basel IV Credit Risk Compliance. The Eastasouth Journal of Information System and Computer Science, 1(01), 169-178.
7. Ramasamy, M. (2023). Cloud-native control plane design for modern network automation systems. International Journal of Research Publications in Engineering, Technology and Management (IJRPETM), 6(4), 9082–9091.
8. Sudhan, S. K. H. H., & Kumar, S. S. (2016). Gallant Use of Cloud by a Novel Framework of Encrypted Biometric Authentication and Multi Level Data Protection. Indian Journal of Science and Technology, 9, 44.
9. Polamarasetty, V. K. (2021). Modernizing SAP sales and distribution systems through ABAP-based enterprise solutions. International Journal of Research and Applied Innovations, 4(2), 4925–4930.
10. Chinnam, N. B. (2023). Modernizing retail fulfillment operations through automated load execution and microservices. International Journal of Computer Technology and Electronics Communication, 6(4), 7367–7371.
11. Neela, S. (2022). Semantic Interoperability in Real-Time Enterprise Integration Using Middleware Abstractions. American International Journal of Computer Science and Technology, 4(6), 56-66.
12. Venkatasalam, K., Rajendran, P., & Thangavel, M. (2019). Improving the accuracy of feature selection in big data mining using accelerated flower pollination (AFP) algorithm. Journal of medical systems, 43(4), 96.
13. Basavala, S. R. (2023). Automated threat modeling and vulnerability prioritization for secure software development. International Journal of Research and Applied Innovations, 6(1), 8414–8424.
14. Jayaraman, S., Rajendran, S., & P, S. P. (2019). Fuzzy c-means clustering and elliptic curve cryptography using privacy preserving in cloud. International Journal of Business Intelligence and Data Mining, 15(3), 273-287.
15. Reddy, K. V. (2023). Reconfigurable FPGA-based acceleration of RTL simulation and co-simulation for cloud datacenter workloads. International Journal of Emerging Trends in Engineering and Management Research, 8(2), 13252–13265.
16. Kundavaram, R. M. R. (2022). Cloud-based data analysis identifying key financial influencers at scale. European Economic Letters (EEL), 12 (2), 234–241.
17. Chinthamani, R. (2022). Event-driven microservices for scalable retail fulfillment optimization. International Journal of Research and Applied Innovations, 5(2), 6790–6795.
18. Kollu, R. K. (2023). CRM architecture deep dive: Building for scale on Salesforce. International Journal of Engineering & Extended Technologies Research, 5(5), 7309–7312.
19. Bandari, U., & Thilagaraj, S. V. (2022). From central BI to federated self-service: A platform-as-a-product reference architecture for data mesh for a multinational manufacturer. International Journal of Science, Research and Technology, 5(3), 7784–7794.
20. Selvarajan, K. (2022). Architecting scalable self-service data platforms for enterprise analytics. International Journal of Science, Research and Technology (IJSRAT), 5(2), 7427–7436.
21. Padmanabham, S. (2023). Building resilient Banking platforms using Event-Driven Microseconds and cloud Native Architecture. International Journal of Research and Applied Innovations, 6(4), 9284-9290.
22. Mathew, A. R. (2019). Cyber-infrastructure connections and smart gird security. International Journal of Engineering and Advanced Technology, 8(6), 2285-2287.
23. Raja, G. V., & Mali, R. K. (2021). Federated learning frameworks for privacy-preserving artificial intelligence applications. International Journal of Research Publications in Engineering, Technology and Management (IJRPETM), 4(3), 4946-4950.
24. Bitragunta, S. L. V., & Paramasivan, M. (2023). Midterm dynamic simulation for the governance of reserves in systems with elevated renewable energy integration. Journal of Artificial Intelligence, 1(1), 1956–1962.
25. Pochincharla, S., & Devineni, A. (2023). Automated compliance-driven patch management and security hardening in multi-cloud banking infrastructure. International Journal of Advances in Signal and Image Sciences, 57–62.
26. Soundappan, S. J. (2022). Distributed Cloud Architecture for Intelligent Performance Optimization and Proactive Real-Time Threat Detection. International Research Journal of Innovative Engineering, 6(6), 11426-11432.
27. Puram, S. (2021). Reliable and Scalable Mobile Technology Architecture: A Formal Model, Synchronization Protocol, and Reference Design. The USA Journals TAJET, 3(08), 33-48.
28. Koganti, H. (2021). Machine learning-driven performance anomaly detection and auto-tuning in distributed Java full-stack systems: A comprehensive review. International Journal of Research Publications in Engineering, Technology and Management, 4(3), 4977–4986.
29. Gopinathan, V. R. (2022). Advanced Predictive Decision Intelligence Using Machine Learning for Strategic Enterprise Management Platforms. International Journal of Research and Applied Innovations, 5(6), 8162-8169.
30. Kalabhavi, V. (2023). Automated quote generation using AI for product configuration, pricing and external order integration in SAP CPQ. International Research Journal of Innovative Engineering, 7(3), 12503–12515.
31. Yadavalli, V. R. (2022). Continuous trust scoring for cross-layer access governance in S/4HANA, Fiori, and SAP BTP. International Journal of Engineering & Extended Technologies Research, 4(1), 4347–4357.
32. Mathew, A. (2021). Artificial intelligence and cognitive computing for 6G communications & networks. International Journal of Computer Science and Mobile Computing, 10(3), 26-31.
33. Mysula, S. (2022). Architectural modernization of legacy enterprise integration systems for scalable and reliable business operations. International Journal of Research and Applied Innovations, 5(2), 6796–6801.
34. Shaik, N. (2023). Zero downtime, zero trust: Automating certificate security at scale. International Journal of Emerging Trends in Engineering and Management Research (IJETEMR), 8(4), 13973–13988.
35. Bandari, U., & Thilagaraj, S. V. (2022). From central BI to federated self-service: A platform-as-a-product reference architecture for data mesh for a multinational manufacturer. International Journal of Science, Research and Technology, 5(3), 7784–7794.
36. Bhattacharjee, B., Khan, M. A. N., Enayet, M. A., Gazi, M. S., Tasnim, M., Jakir, T., ... & Himeluzzaman, M. (2021). Adversarial Machine Learning and Cognitive AI for Autonomous Defense against Advanced Cyber Threats. International Journal of Computer Technology and Electronics Communication, 4(6), 4327-4337.
37. Hannousse, A., & Yahiouche, S. (2021). Securing microservices and microservice architectures: A systematic mapping study. Computer Science Review, 41, Article 100415. https://doi.org/10.1016/j.cosrev.2021.100415