From Central BI to Federated Self-Service: A Platform-as-a-Product Reference Architecture for Data Mesh for a Multinational Manufacturer
Abstract
Many large enterprises still route analytics demand through a centralized business intelligence competency center (BICC) that owns ingestion, modeling and reporting. In multinational manufacturers with several ERP systems and plant-level databases, such centers have become delivery bottlenecks, with request backlogs measured in quarters. Data mesh proposes domain ownership, data as a product, a self-serve data infrastructure platform and federated computational governance, but it offers little implementation guidance for hybrid estates that combine public cloud with on-premise systems. This paper presents a reference architecture for a self-service data platform delivered as an internal platform-as-a-service and managed as a product. It is built on Microsoft Azure (ADLS Gen2, Azure Data Factory, Azure Synapse Analytics, Azure Purview) together with on-premise SQL Server and Power BI, and is drawn from a ten-quarter rollout across seven business domains of a global Tier-1 industrial manufacturer. We describe the layered architecture, a tiered use-case onboarding model with an explicit readiness score, workspace and capacity isolation with usage-based chargeback, a RACI split of governance duties between the central platform team and domain teams, and the product-management practices used to scale adoption. Using representative data from 64 onboarded use cases and two historical baselines, we compare the approach with a centralized warehouse and a pure data-lake model. Median time-to-onboard fell from 12.6 weeks under the BICC to 3.9 weeks at steady state, and the dataset reuse ratio rose to 0.42, at a moderate governance cost. We close with lessons for adopting data mesh incrementally rather than as a reorganization
Article Information
Journal |
International Journal of Science, Research and Technology |
|---|---|
Volume (Issue) |
Vol. 5 No. 3 (2022): International Journal of Science, Research and Technology (IJSRAT) |
DOI |
|
Pages |
7784-7794 |
Published |
May 15, 2022 |
| Copyright |
All rights reserved |
Open Access |
This work is licensed under a Creative Commons Attribution 4.0 International License. |
How to Cite |
Umashanker Bandari, Sundara Vadivel Thilagaraj (%2022). From Central BI to Federated Self-Service: A Platform-as-a-Product Reference Architecture for Data Mesh for a Multinational Manufacturer. International Journal of Science, Research and Technology , Vol. 5 No. 3 (2022): International Journal of Science, Research and Technology (IJSRAT) , pp. 7784-7794. https://doi.org/10.15662/IJSRAT.2022.0503002 |
References
[2] H. Chen, R. H. L. Chiang, and V. C. Storey, "Business intelligence and analytics: From big data to big impact," MIS Quarterly, vol. 36, no. 4, pp. 1165–1188, 2012.
[3] Z. Dehghani, "How to move beyond a monolithic data lake to a distributed data mesh," martinfowler.com, May 2019.
[4] Z. Dehghani, "Data mesh principles and logical architecture," martinfowler.com, Dec. 2020.
[5] C. Imhoff and C. White, "Self-service business intelligence: Empowering users to generate insights," TDWI Best Practices Report, The Data Warehousing Institute, 2011.
[6] P. Alpar and M. Schulz, "Self-service business intelligence," Business & Information Systems Engineering, vol. 58, no. 2, pp. 151–155, 2016.
[7] M. Skelton and M. Pais, Team Topologies: Organizing Business and Technology Teams for Fast Flow. Portland, OR, USA: IT Revolution Press, 2019.
[8] W. H. Inmon, Building the Data Warehouse, 4th ed. Indianapolis, IN, USA: Wiley, 2005.
[9] R. Kimball and M. Ross, The Data Warehouse Toolkit: The Definitive Guide to Dimensional Modeling, 3rd ed. Indianapolis, IN, USA: Wiley, 2013.
[10] D. Linstedt and M. Olschimke, Building a Scalable Data Warehouse with Data Vault 2.0. Waltham, MA, USA: Morgan Kaufmann, 2015.
[11] J. Dixon, "Pentaho, Hadoop, and data lakes," James Dixon's Blog, Oct. 2010.
[12] H. Fang, "Managing data lakes in big data era: What's a data lake and why has it became popular in data management ecosystem," in Proc. IEEE Int. Conf. Cyber Technology in Automation, Control, and Intelligent Systems (CYBER), 2015, pp. 820–824.
[13] C. Giebler, C. Gröger, E. Hoos, H. Schwarz, and B. Mitschang, "Leveraging the data lake: Current state and challenges," in Proc. Int. Conf. Big Data Analytics and Knowledge Discovery (DaWaK), LNCS vol. 11708, Springer, 2019, pp. 179–188.
[14] M. Armbrust et al., "Delta Lake: High-performance ACID table storage over cloud object stores," Proc. VLDB Endowment, vol. 13, no. 12, pp. 3411–3424, 2020.
[15] M. Armbrust, A. Ghodsi, R. Xin, and M. Zaharia, "Lakehouse: A new generation of open platforms that unify data warehousing and advanced analytics," in Proc. Conf. Innovative Data Systems Research (CIDR), 2021.
[16] A. Halevy, F. Korn, N. F. Noy, C. Olston, N. Polyzotis, S. Roy, and S. E. Whang, "Goods: Organizing Google's datasets," in Proc. ACM SIGMOD Int. Conf. Management of Data, 2016, pp. 795–806.
[17] Microsoft, "Power BI adoption roadmap," Microsoft Power BI documentation, 2021.
[18] E. Evans, Domain-Driven Design: Tackling Complexity in the Heart of Software. Boston, MA, USA: Addison-Wesley, 2003.
[19] M. E. Conway, "How do committees invent?" Datamation, vol. 14, no. 4, pp. 28–31, 1968.
[20] K. Weber, B. Otto, and H. Österle, "One size does not fit all—A contingency approach to data governance," ACM Journal of Data and Information Quality, vol. 1, no. 1, Art. 4, 2009.
[21] V. Khatri and C. V. Brown, "Designing data governance," Communications of the ACM, vol. 53, no. 1, pp. 148–152, 2010.
[22] DAMA International, DAMA-DMBOK: Data Management Body of Knowledge, 2nd ed. Basking Ridge, NJ, USA: Technics Publications, 2017.
[23] Microsoft, "Enterprise-scale for analytics and AI," Microsoft Cloud Adoption Framework for Azure documentation, 2021.
[24] M. D. Wilkinson et al., "The FAIR guiding principles for scientific data management and stewardship," Scientific Data, vol. 3, Art. 160018, 2016.
[25] B. Beyer, C. Jones, J. Petoff, and N. R. Murphy, Eds., Site Reliability Engineering: How Google Runs Production Systems. Sebastopol, CA, USA: O'Reilly Media, 2016.
[26] M. Cagan, Inspired: How to Create Tech Products Customers Love, 2nd ed. Hoboken, NJ, USA: Wiley, 2017.
[27] N. Forsgren, J. Humble, and G. Kim, Accelerate: The Science of Lean Software and DevOps. Portland, OR, USA: IT Revolution Press, 2018.