Designing a Multi-Criteria Decision Framework for Enterprise Language Model Adaptation
Abstract
Enterprises do not choose among prompt engineering, retrieval-augmented generation (RAG), fine-tuning, and hybrids in the abstract. They choose deployable combinations of model, context, behaviour control, orchestration, and serving under task-specific constraints. This paper presents E-LADF 2.0, a design-science decision artefact for configuration-level feasibility, value functions, and robustness. It begins with a registered acceptance test, diagnoses the locked baseline, freezes a causally plausible configuration shortlist, applies noncompensatory feasibility gates, and then combines anchored 0-to-100 value functions with analytic hierarchy process weights. A bounded uncertainty analysis perturbs criterion weights and performance values. A configuration is recommended only when its worst-case advantage exceeds a predeclared practical difference; otherwise the framework returns ‘not separated’ and prescribes a targeted pilot. Five constructed scenarios directly supply every partial value, criterion weight, and uncertainty input used in the arithmetic and, by design, exercise prompt, RAG, PEFT, hybrid, and abstention outcomes. They demonstrate execution, traceability, and edge-case handling, not empirical effectiveness. The contribution is a configuration-level procedure that separates constraints from preferences, replaces universal archetype scores with case-specific partial values, and treats uncertainty as a decision outcome.
Article Information
Journal |
International Journal of Science, Research and Technology |
|---|---|
Volume (Issue) |
Vol. 8 No. 6 (2025): International Journal of Science, Research and Technology (IJSRAT) |
DOI |
|
Pages |
15449-15465 |
Published |
December 17, 2025 |
| Copyright |
All rights reserved |
Open Access |
This work is licensed under a Creative Commons Attribution 4.0 International License. |
How to Cite |
Karan Nisar (%2025). Designing a Multi-Criteria Decision Framework for Enterprise Language Model Adaptation. International Journal of Science, Research and Technology , Vol. 8 No. 6 (2025): International Journal of Science, Research and Technology (IJSRAT) , pp. 15449-15465. https://doi.org/10.15662/IJSRAT.2025.0806021 |
References
[2] O. Ovadia, M. Brief, M. Mishaeli, and O. Elisha, “Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs,” in Proc. 2024 Conf. Empirical Methods in Natural Language Processing, 2024, pp. 237-250. doi: 10.18653/v1/2024.emnlp-main.15
[3] H. Soudani, E. Kanoulas, and F. Hasibi, “Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledge,” in Proc. 2024 ACM SIGIR Conf. Research and Development in Information Retrieval in the Asia Pacific Region, 2024, pp. 12-22. doi: 10.1145/3673791.3698415
[4] T. Zhang, S. G. Patil, N. Jain, S. Shen, M. Zaharia, I. Stoica, and J. E. Gonzalez, “RAFT: Adapting Language Model to Domain Specific RAG,” in Proc. 1st Conf. Language Modeling, 2024. OpenReview
[5] S. Jeong, J. Baek, S. Cho, S. J. Hwang, and J. Park, “Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity,” in Proc. 2024 NAACL-HLT, vol. 1, 2024, pp. 7036-7050. doi: 10.18653/v1/2024.naacl-long.389
[6] A. Asai, Z. Wu, Y. Wang, A. Sil, and H. Hajishirzi, “Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection,” in Proc. 12th Int. Conf. Learning Representations, 2024. OpenReview
[7] Z. Li, C. Li, M. Zhang, Q. Mei, and M. Bendersky, “Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach,” in Proc. 2024 EMNLP Industry Track, 2024, pp. 881-893. doi: 10.18653/v1/2024.emnlp-industry.66
[8] N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang, “Lost in the Middle: How Language Models Use Long Contexts,” Transactions of the Association for Computational Linguistics, vol. 12, pp. 157-173, 2024. doi: 10.1162/tacl_a_00638
[9] Y. Bai et al., “LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-Context Multitasks,” in Proc. 63rd Annual Meeting of the Association for Computational Linguistics, vol. 1, 2025, pp. 3639-3664. doi: 10.18653/v1/2025.acl-long.183
[10] R. Pryzant, D. Iter, J. Li, Y. T. Lee, C. Zhu, and M. Zeng, “Automatic Prompt Optimization with ‘Gradient Descent’ and Beam Search,” in Proc. 2023 Conf. Empirical Methods in Natural Language Processing, 2023, pp. 7957-7968. doi: 10.18653/v1/2023.emnlp-main.494
[11] C. Yang, X. Wang, Y. Lu, H. Liu, Q. V. Le, D. Zhou, and X. Chen, “Large Language Models as Optimizers,” in Proc. 12th Int. Conf. Learning Representations, 2024. OpenReview
[12] O. Khattab et al., “DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines,” in Proc. 12th Int. Conf. Learning Representations, 2024. OpenReview
[13] S. Geng, M. Josifoski, M. Peyrard, and R. West, “Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning,” in Proc. 2023 Conf. Empirical Methods in Natural Language Processing, 2023, pp. 10932-10952. doi: 10.18653/v1/2023.emnlp-main.674
[14] L. Beurer-Kellner, M. Fischer, and M. Vechev, “Guiding LLMs the Right Way: Fast, Non-Invasive Constrained Generation,” in Proc. 41st Int. Conf. Machine Learning, PMLR, vol. 235, 2024, pp. 3658-3673. PMLR
[15] D. Banerjee, T. Suresh, S. Ugare, S. Misailovic, and G. Singh, “CRANE: Reasoning with Constrained LLM Generation,” in Proc. 42nd Int. Conf. Machine Learning, PMLR, vol. 267, 2025, pp. 2836-2857. PMLR
[16] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-Rank Adaptation of Large Language Models,” in Proc. 10th Int. Conf. Learning Representations, 2022. OpenReview
[17] T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “QLoRA: Efficient Finetuning of Quantized LLMs,” in Advances in Neural Information Processing Systems 36, 2023. doi: 10.52202/075280-0441
[18] X. Zhang and J. Wu, “Dissecting Learning and Forgetting in Language Model Finetuning,” in Proc. 12th Int. Conf. Learning Representations, 2024. OpenReview
[19] D. Biderman et al., “LoRA Learns Less and Forgets Less,” Transactions on Machine Learning Research, 2024. OpenReview
[20] L. Chen, M. Zaharia, and J. Zou, “FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance,” Transactions on Machine Learning Research, 2024. OpenReview
[21] I. Ong, A. Almahairi, V. Wu, W.-L. Chiang, T. Wu, J. E. Gonzalez, M. W. Kadous, and I. Stoica, “RouteLLM: Learning to Route LLMs with Preference Data,” in Proc. 13th Int. Conf. Learning Representations, 2025. OpenReview
[22] I. Gim, G. Chen, S.-S. Lee, N. Sarda, A. Khandelwal, and L. Zhong, “Prompt Cache: Modular Attention Reuse for Low-Latency Inference,” Proceedings of Machine Learning and Systems, vol. 6, 2024. MLSys
[23] Y. Sheng et al., “S-LoRA: Serving Thousands of Concurrent LoRA Adapters,” Proceedings of Machine Learning and Systems, vol. 6, 2024. MLSys
[24] Z. Pan et al., “LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression,” in Findings of the Association for Computational Linguistics: ACL 2024, 2024, pp. 963-981. doi: 10.18653/v1/2024.findings-acl.57
[25] L. Zheng et al., “SGLang: Efficient Execution of Structured Language Model Programs,” in Advances in Neural Information Processing Systems 37, 2024. doi: 10.52202/079017-2000
[26] K. Nisar, “Prompting, Retrieval, and Fine-Tuning: Foundations of Enterprise Language Model Adaptation,” International Journal of Engineering & Extended Technologies Research, vol. 6, no. 6, pp. 9310-9319, 2024. doi: 10.15662/IJEETR.2024.0606030
[27] K. Nisar, “Quantifying the Cost and Risk of Enterprise LLM Adaptation Strategies,” International Journal of Research Publications in Engineering, Technology and Management, vol. 8, no. 3, pp. 12162-12169, 2025. Journal article
[28] S. Es, J. James, L. Espinosa Anke, and S. Schockaert, “RAGAs: Automated Evaluation of Retrieval Augmented Generation,” in Proc. 18th Conf. European Chapter of the Association for Computational Linguistics: System Demonstrations, 2024, pp. 150-158. doi: 10.18653/v1/2024.eacl-demo.16
[29] J. Saad-Falcon, O. Khattab, C. Potts, and M. Zaharia, “ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems,” in Proc. 2024 NAACL-HLT, vol. 1, 2024, pp. 338-354. doi: 10.18653/v1/2024.naacl-long.20
[30] J. Chen, H. Lin, X. Han, and L. Sun, “Benchmarking Large Language Models in Retrieval-Augmented Generation,” Proc. AAAI Conf. Artificial Intelligence, vol. 38, no. 16, pp. 17754-17762, 2024. doi: 10.1609/aaai.v38i16.29728
[31] C. Niu, Y. Wu, J. Zhu, S. Xu, K. Shum, R. Zhong, J. Song, and T. Zhang, “RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models,” in Proc. 62nd Annual Meeting of the Association for Computational Linguistics, vol. 1, 2024, pp. 10862-10878. doi: 10.18653/v1/2024.acl-long.585
[32] L. Zheng et al., “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena,” in Advances in Neural Information Processing Systems 36, 2023, pp. 46595-46623. Proceedings
[33] P. Liang et al., “Holistic Evaluation of Language Models,” Transactions on Machine Learning Research, 2023. OpenReview
[34] W. Zou, R. Geng, B. Wang, and J. Jia, “PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models,” in Proc. 34th USENIX Security Symp., 2025, pp. 3827-3844. USENIX
[35] A. Shafran, R. Schuster, and V. Shmatikov, “Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documents,” in Proc. 34th USENIX Security Symp., 2025, pp. 3787-3806. USENIX
[36] S. Abdelnabi, K. Greshake, S. Mishra, C. Endres, T. Holz, and M. Fritz, “Not What You’ve Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection,” in Proc. 16th ACM Workshop on Artificial Intelligence and Security, 2023, pp. 79-90. doi: 10.1145/3605764.3623985
[37] E. Tabassi, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1. Gaithersburg, MD, USA: National Institute of Standards and Technology, 2023. doi: 10.6028/NIST.AI.100-1
[38] C. Autio et al., Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1. Gaithersburg, MD, USA: National Institute of Standards and Technology, 2024. doi: 10.6028/NIST.AI.600-1
[39] H. Booth, M. Souppaya, A. Vassilev, M. Ogata, M. Stanley, and K. Scarfone, Secure Software Development Practices for Generative AI and Dual-Use Foundation Models: An SSDF Community Profile, NIST SP 800-218A. Gaithersburg, MD, USA: National Institute of Standards and Technology, 2024. doi: 10.6028/NIST.SP.800-218A
[40] European Parliament and Council of the European Union, “Regulation (EU) 2024/1689 of 13 June 2024 Laying Down Harmonised Rules on Artificial Intelligence,” Official Journal of the European Union, L 2024/1689, 12 Jul. 2024. Official text
[41] European Commission, “Guidelines on the Scope of the Obligations for General-Purpose AI Models Established by Regulation (EU) 2024/1689 (Artificial Intelligence Act),” draft approved by the Commission, Brussels, Belgium, 18 Jul. 2025. Official July 2025 document
[42] European Commission, “General-Purpose AI Code of Practice,” 10 Jul. 2025. Official code
[43] S. Amershi et al., “Software Engineering for Machine Learning: A Case Study,” in Proc. 41st IEEE/ACM Int. Conf. Software Engineering: Software Engineering in Practice, 2019, pp. 291-300. doi: 10.1109/ICSE-SEIP.2019.00042
[44] D. Kreuzberger, N. Kühl, and S. Hirschl, “Machine Learning Operations (MLOps): Overview, Definition, and Architecture,” IEEE Access, vol. 11, pp. 31866-31879, 2023. doi: 10.1109/ACCESS.2023.3262138
[45] M. M. John, H. H. Olsson, and J. Bosch, “An Empirical Guide to MLOps Adoption: Framework, Maturity Model and Taxonomy,” Information and Software Technology, vol. 183, art. 107725, 2025. doi: 10.1016/j.infsof.2025.107725
[46] M. Zarour, H. Alzabut, and K. T. Al-Sarayreh, “MLOps Best Practices, Challenges and Maturity Models: A Systematic Literature Review,” Information and Software Technology, vol. 183, art. 107733, 2025. doi: 10.1016/j.infsof.2025.107733
[47] A. Paleyes, R.-G. Urma, and N. D. Lawrence, “Challenges in Deploying Machine Learning: A Survey of Case Studies,” ACM Computing Surveys, vol. 55, no. 6, art. 114, pp. 1-29, 2023. doi: 10.1145/3533378
[48] K. Nisar, “Bridging Prototyping and Production: A Systems-Level Study of ML Workflow Standardization in Enterprise AI Adoption,” International Journal of Computational and Experimental Science and Engineering, vol. 11, no. 4, pp. 4741-4757, 2025. doi: 10.22399/ijcesen.5364
[49] A. R. Hevner, S. T. March, J. Park, and S. Ram, “Design Science in Information Systems Research,” MIS Quarterly, vol. 28, no. 1, pp. 75-105, 2004. doi: 10.2307/25148625
[50] K. Peffers, T. Tuunanen, M. A. Rothenberger, and S. Chatterjee, “A Design Science Research Methodology for Information Systems Research,” Journal of Management Information Systems, vol. 24, no. 3, pp. 45-77, 2007. doi: 10.2753/MIS0742-1222240302
[51] T. L. Saaty, “How to Make a Decision: The Analytic Hierarchy Process,” European Journal of Operational Research, vol. 48, no. 1, pp. 9-26, 1990. doi: 10.1016/0377-2217(90)90057-I
[52] V. Belton and T. J. Stewart, Multiple Criteria Decision Analysis: An Integrated Approach. Boston, MA, USA: Kluwer Academic Publishers, 2002. doi: 10.1007/978-1-4615-1495-4
[53] E. Triantaphyllou and A. Sánchez, “A Sensitivity Analysis Approach for Some Deterministic Multi-Criteria Decision-Making Methods,” Decision Sciences, vol. 28, no. 1, pp. 151-194, 1997. doi: 10.1111/j.1540-5915.1997.tb01306.x