Skip to main content

Dependency-Aware AI Pipeline Orchestration Through Intelligent Recovery for Secure Cloud Operations Automation

Abstract

Modern cloud operations increasingly depend on complex AI and data pipelines containing multiple interconnected tasks, services, models, and infrastructure resources. Failures in one component can propagate through dependency chains, resulting in pipeline interruption, inconsistent outputs, security exposure, and inefficient resource utilization. This study proposes a dependency-aware AI pipeline orchestration framework that combines dependency modeling, intelligent failure detection, predictive recovery, and security-aware cloud automation. The framework represents pipeline tasks as dependency graphs and continuously analyzes execution states, service health, resource conditions, and operational telemetry to identify failure patterns and determine recovery priorities. Machine learning models are incorporated to classify failures and predict the probability of task-level disruption, while an intelligent recovery mechanism dynamically selects actions such as retry, restart, rollback, rescheduling, dependency isolation, or alternative resource allocation. Security policies are integrated into orchestration decisions to prevent unauthorized recovery actions and preserve data and workload integrity. The proposed methodology evaluates orchestration efficiency using recovery time, failure detection accuracy, pipeline completion rate, resource utilization, and security-policy compliance. Experimental analysis is designed to compare dependency-aware recovery with conventional static retry and rule-based orchestration approaches. The framework aims to improve resilience, automation, operational continuity, and secure execution of AI pipelines across dynamic cloud environments.

References

1. Mohammed, A. S., Pradhan, M. R., & others. (2021). Machine learning with big data analytics for cloud security. Computers and Electrical Engineering, 96, 107527. https://doi.org/10.1016/j.compeleceng.2021.107527
2. Panker, T., & Nissim, N. (2021). Leveraging malicious behavior traces from volatile memory using machine learning methods for trusted unknown malware detection in Linux cloud environments. Knowledge-Based Systems, 226, 107095. https://doi.org/10.1016/j.knosys.2021.107095
3. Sandhu, Y. S. (2025). Scalable pipeline orchestration through intelligent dependency-aware recovery and performance optimization. International Journal of Future Innovative Science and Technology (IJFIST), 8(5), 15712–15722.
4. Ambalakannu, M. (2024). The emergence of AI-powered data analytics revolutionizing business intelligence. International Journal of Future Innovative Science and Technology (IJFIST), 7(6), 13955.
5. Kalakoti, M. K. R. (2025). Provider-agnostic infrastructure as code: A modular framework for secure multi-tenant cloud automation. Journal of international crisis and risk communication research, 188-197.
6. Puram, S. (2024). AI-Enabled Software-Defined Vehicles: An Intelligent Cloud-Edge Architecture for Adaptive, Secure, and Resilient Automotive Software Systems. The USA Journals TAJAS, 6(03), 23-33.
7. Mathew, A., & Alex, H. (2025). Federated Learning for Secure Genomic Research: Privacy-Preserving AI Solutions for Precision Medicine. Science and Technology: Developments and Applications, 9, 36-43.
8. Reddy, K. V. (2024). Optimizing gate-level simulation performance through cloud-based distributed computing. International Journal of Research Publications in Engineering, Technology and Management, 7(6), 11747–11752.
9. Chinnam, N. B. (2022). Engineering real-time supply chain visibility and ETA intelligence platforms for large-scale retail distribution networks. International Journal of Research Publications in Engineering, Technology and Management (IJRPETM), 5(6), 7918–7922.
10. Pothuri, M. K. (2025). From legacy systems to scalable cloud platforms: Building modern data pipelines with data engineering strategies, scaling trust, compliance, and performance in public health. International Journal of Innovative Research in Engineering & Multidisciplinary Physical Sciences, 13(5). https://doi.org/10.37082/IJIRMPS.v13.i5.232724
11. Ravichandran, S., & Kandasamy, V. (2025). Optimized Attention Augmented Residual Convolutional Neural Network with Fa-Resnet for Fabric Defect Detection. Journal of Control Engineering and Applied Informatics, 27(4), 3-15.
12. Venkatasalam, K., Rajendran, P., & Thangavel, M. (2019). Improving the accuracy of feature selection in big data mining using accelerated flower pollination (AFP) algorithm. Journal of medical systems, 43(4), 96.
13. Kalabhavi, V. (2025). Integrating Trade Promotion Management With SAP CRM For Enhanced Brand Spend Optimization: A Case Study In The Consumer-Packaged Goods Industry. Frontiers in Emerging Artificial Intelligence and Machine Learning, 2(09), 17-22.
14. Ahuja, D. (2025, August). Intelligent Failure Prediction in CI/CD Pipelines Using Efficient Machine Learning Techniques. In 2025 5th Asian Conference on Innovation in Technology (ASIANCON) (pp. 1-7). IEEE.
15. Aragani, V. M., Maroju, P. K., & Mudunuri, L. N. R. (2024, December). Implementing Non-functional Production Regression Testing with Kubernetes. In International Conference on Information and Communication Technology for Competitive Strategies (pp. 231-244). Singapore: Springer Nature Singapore.
16. Bitragunta, S. L. V., & Paramasivan, M. (2023). Midterm dynamic simulation for the governance of reserves in systems with elevated renewable energy integration. Journal of Artificial Intelligence, 1(1), 1956–1962.
17. Panda, M. (2025). Agentic AI: A paradigm for autonomous decision-making. International Journal of Innovative Research in Science, Engineering and Technology, 14(1), 568–575.
18. Selvarajan, K. (2025). AI-powered big data platforms for enterprise analytics. International Journal of Computer Technology and Electronics Communication, 8(2), 10490–10496.
19. Ramasamy, M. (2024). From configuration management to autonomous operations: Evolution of network controller architectures. International Research Journal of Innovative Engineering, 8(5), 15413–15423.
20. Shaik, N. (2023). Zero downtime, zero trust: Automating certificate security at scale. International Journal of Emerging Trends in Engineering and Management Research (IJETEMR), 8(4), 13973–13988.
21. Tarigoppula, S. (2023). Adaptive cloud cost optimization through predictive auto-scaling and infrastructure intelligence. International Journal of Future Innovative Science and Technology, 6(2), 10278–10290.
22. Kollu, R. K. (2023). CRM architecture deep dive: Building for scale on Salesforce. International Journal of Engineering & Extended Technologies Research, 5(5), 7309–7312.
23. Yadavalli, V. R. (2025). Incident memory graphs for predictive AMS triage and self-improving SAP operations. International Journal of Science, Research and Technology, 8(2), 13913–13927.
24. Sahu, S. (2024). Digital Governance Framework for Salesforce Data Cloud in Healthcare Insurance Platforms. International Journal of Innovations in Science, Engineering And Management, 120-128.
25. Yadavalli, V. R. (2024). Hypercare signal fusion for early instability detection in SAP S/4HANA and EWM go-lives. International Research Journal of Innovative Engineering, 8(5), 15423–15437.
26. Gudipuri, N. (2024). Intent-Aware Enterprise Architecture Using Persistent AI Agents for Dynamic FinTech Service Composition. The Eastasouth Journal of Information System and Computer Science, 2(01), 151-161.
27. Pochincharla, S., & Devineni, A. (2023). Automated compliance-driven patch management and security hardening in multi-cloud banking infrastructure. International Journal of Advances in Signal and Image Sciences, 57–62.
28. Basavala, S. R. (2022). Security assessment framework for microservices-based applications using automated penetration testing. International Journal of Emerging Trends in Engineering and Management Research, 7(4), 12179–12190.
29. Bandari, U., & Gajendran, R. (2024). From data hub to data marketplace: Retrieval-augmented generation over governed KPI products in a federated customer experience platform. International Journal of Research Publications in Engineering, Technology and Management, 7(3), 10616–10626.
30. Madabathula, L. (2024). Metadata-driven multi-tenant data ingestion for cloud-native pipelines. International Journal of Computer Technology and Electronics Communication, 7(6), 9857-9865.
31. Karrothu, A. (2024). A cloud-native architecture for scalable real-time security telemetry ingestion and threat signal processing. Journal of Computational Analysis and Applications (JoCAAA), 33(8), 8137–8154. https://eudoxuspress.com/index.php/pub/article/view/5220
32. Challa, A., & Konatham, M. R. (2024). Self-Healing CI/CD Pipelines with Feedback-Loop Automation: Building Fault-Tolerant CI/CD Systems Using Anomaly Detection and Automated Rollback Logic. Int. J. Intell. Syst. Appl. Eng, 12(23s), 3217.
33. Srikanth, V., Arpitha, K., Akbar, S., Patil, S., Bhosale, Y. H., & Pathak, M. (2025). Artificial Intelligence-Based Monitoring and Forecasting of Urban Air Pollution in Smart Cities. Int. J. Environ. Sci, 11, 2025.
34. Gajjela, H., & Kari, M. (2024). Secure Multi-Tenant AI Architecture Enhances Enterprise AI Adoption Through Tenant Trust Using PLS-SEM in India’s Salesforce Cloud Industry. International Journal of Engineering Science & Humanities, 14(2), 253-271.
35. Kandibedala, B., Shrestha, G., Pyayt, A., Ivanov, T., & Gubanov, M. (2025, May). Scalable tabular hierarchical metadata classification in heterogeneous structured large-scale datasets using contrastive learning. In 2025 IEEE 41st International Conference on Data Engineering (ICDE) (pp. 1841-1854). IEEE.
36. Soundappan, S. J. (2022). Distributed Cloud Architecture for Intelligent Performance Optimization and Proactive Real-Time Threat Detection. International Research Journal of Innovative Engineering, 6(6), 11426-11432.
37. Sugumar, R. (2021). Generative AI Pipelines for Safety Validation of Autonomous Driving Models. International Journal of Advanced Research in Computer Science & Technology (IJARCST), 4(5), 5466-5469.
38. Mathew, A. (2021). Artificial intelligence and cognitive computing for 6G communications & networks. International Journal of Computer Science and Mobile Computing, 10(3), 26-31.
39. Jayaraman, S., Rajendran, S., & P, S. P. (2019). Fuzzy c-means clustering and elliptic curve cryptography using privacy preserving in cloud. International Journal of Business Intelligence and Data Mining, 15(3), 273-287.
40. Rajasoundaran, S., Prabu, A. V., Routray, S., Santhosh Kumar, S. V. N., Malla, P. P., Maloji, S., Mukherjee, A., & Ghosh, U. (2021). Machine learning based deep job exploration and secure transactions in virtual private cloud systems. Computers & Security, 109, 102379. https://doi.org/10.1016/j.cose.2021.102379