An Ingestion-Gated Lakehouse Architecture for Multi-CRO SDTM Harmonization in Clinical Trial Data Management
Abstract
Sponsors of global clinical programs typically receive Study Data Tabulation Model (SDTM) datasets from several Contract Research Organizations (CROs), each with its own Electronic Data Capture configuration, mapping conventions, and delivery cadence, and nonconformance is commonly found late, through downstream review. This paper proposes an ingestion-gated lakehouse architecture for multi-CRO SDTM harmonization. Data custody stays in CRO-local environments; a conformance gate at each CRO boundary evaluates every delivery against rules, schemas, and controlled-terminology versions pinned for the study and served by a sponsor metadata control plane; and only accepted or flagged domains enter a bronze, silver, and gold lakehouse, where subject identity and terminology are harmonized. The design separates storage services from the conformance engine, classifies validation failures into classes with defined dispositions, treats ADaM derivation and traceability as a distinct layer, and specifies rule governance, subject-identity resolution, failure handling, regulatory and validation controls, and non-functional requirements. The central mechanism is earlier detection: moving the discovery of conformance issues from downstream review to the point of delivery, where correction is cheaper and remains with the originating CRO. To make the proposal evaluable, the paper defines seven architecture metrics and a reproducible, seeded simulation of the detection mechanism (14 synthetic studies, 5 synthetic CROs, 1,247 domain deliveries), extended by a parameter sensitivity analysis and a break-even structure. The simulation demonstrates the mechanism under disclosed assumptions and is not a prediction of performance. Its outcome depends jointly on the gate's catch rate and on how much resolution cost an early catch avoids; only the unit resolution cost range (USD 28-225) is drawn from the literature, and the break-even analysis uses placeholder costs and an avoided-downstream-cost term that the paper cannot quantify. No implementation in a production clinical environment is reported, and most of the proposed metrics, the identity-resolution design, and the regulatory validation controls are specified but neither implemented nor measured. The contribution is the architecture, its mechanism, and a framework for evaluating it in practice
Article Information
Journal |
International Journal of Science, Research and Technology |
|---|---|
Volume (Issue) |
Vol. 8 No. 3 (2025): International Journal of Science, Research and Technology (IJSRAT) |
DOI |
|
Pages |
14273-14301 |
Published |
June 23, 2025 |
| Copyright |
All rights reserved |
Open Access |
This work is licensed under a Creative Commons Attribution 4.0 International License. |
How to Cite |
Sai Krishna Puli (%2025). An Ingestion-Gated Lakehouse Architecture for Multi-CRO SDTM Harmonization in Clinical Trial Data Management. International Journal of Science, Research and Technology , Vol. 8 No. 3 (2025): International Journal of Science, Research and Technology (IJSRAT) , pp. 14273-14301. https://doi.org/10.15662/IJSRAT.2025.0803007 |
References
2. Belligoli, P., Weberpals, C., & Flores Lopez, Y. (2026). Closing the loop: Validating AI-generated SDTM mappings using CDISC CORE and synthetic data [Conference paper]. PharmaSUG 2026. https://pharmasug.org/proceedings/2026/SS/PharmaSUG-2026-SS-261.pdf
3. Bitol. (n.d.). Open Data Contract Standard [Specification]. GitHub. Retrieved October 6, 2026, from https://github.com/bitol-io/open-data-contract-standard
4. Clinical Data Interchange Standards Consortium. (n.d.-a). Analysis Data Model (ADaM). Retrieved October 6, 2026, from https://www.cdisc.org/standards/foundational/adam
5. Clinical Data Interchange Standards Consortium. (n.d.-b). CDISC Controlled Terminology. Retrieved October 6, 2026, from https://www.cdisc.org/standards/terminology/controlled-terminology
6. Clinical Data Interchange Standards Consortium. (n.d.-c). CDISC Rules Engine (CORE) [Computer software]. GitHub. Retrieved October 6, 2026, from https://github.com/cdisc-org/cdisc-rules-engine
7. Clinical Data Interchange Standards Consortium. (n.d.-d). Dataset-JSON. Retrieved October 6, 2026, from https://www.cdisc.org/standards/data-exchange/dataset-json
8. Clinical Data Interchange Standards Consortium. (n.d.-e). Define-XML. Retrieved August 16, 2026, from https://www.cdisc.org/standards/data-exchange/define-xml
9. Clinical Data Interchange Standards Consortium. (n.d.-f). Study Data Tabulation Model (SDTM). Retrieved October 6, 2026, from https://www.cdisc.org/standards/foundational/sdtm
10. Confluent. (n.d.). Schema evolution and compatibility for Schema Registry on Confluent Platform. Retrieved October 6, 2026, from https://docs.confluent.io/platform/current/schema-registry/fundamentals/schema-evolution.html
11. Databricks. (n.d.). What is the medallion lakehouse architecture? Retrieved October 6, 2026, from https://docs.databricks.com/lakehouse/medallion.html
12. Dehghani, Z. (2020, December 3). Data mesh principles and logical architecture. MartinFowler.com. https://martinfowler.com/articles/data-mesh-principles.html
13. Electronic records; electronic signatures, 21 C.F.R. pt. 11 (2026). https://www.ecfr.gov/current/title-21/chapter-I/subchapter-A/part-11
14. Food and Drug Administration. (2025, April 9). Electronic study data submission and data standards for Clinical Data Interchange Standards Consortium Dataset-JavaScript Object Notation (Docket No. FDA-2025-N-0129). Federal Register, 90(67). https://www.govinfo.gov/content/pkg/FR-2025-04-09/html/2025-06051.htm
15. International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. (2025). ICH harmonised guideline: Guideline for good clinical practice E6(R3) (Final version, adopted 6 January 2025). https://database.ich.org/sites/default/files/ICH_E6%28R3%29_Step4_FinalGuideline_2025_0106.pdf
16. Pinnacle 21. (n.d.). OpenCDISC validation framework. Retrieved October 6, 2026, from https://pinnacle21.com/projects/validator/opencdisc-validation-framework
17. Zhang, L. X., Lanoue, J., & Nielsen, U. (2026). From specification to SDTM at speed: Deploying the SDTM engine in production [Conference paper]. PharmaSUG 2026. https://pharmasug.org/proceedings/2026/DS/PharmaSUG-2026-DS-440.pdf