Standards and Conventions

Return to home

These standards are opinionated and designed to ensure consistency, governance, and automation across an organisation. They largely reflect an Azure Databricks environment, however can be adapted to other platforms. Organisations should adapt these standards to fit their existing internal conventions.

Mesh

Domain Names

All lower case: {optional: organisation_}{functional area/domain}_{subdomain}

e.g: intuitas_corporate e.g: central_hub - the reserved domain for the central platform capability (ingestion and cleansing)

Note: domain names may legitimately contain single underscores (e.g. intuitas_corporate, ambo_sim). This is why the separator discipline reserves the double underscore (__) for structural boundaries: a parser splitting on __ always recovers the domain token intact.

Platform

Environments

This is the single canonical environment taxonomy. Every environment token used anywhere in this document (catalogs, containers, jobs, groups, bundles, Key Vaults) is drawn from this table. Sections do not define their own environment sets.

Token	Environment	Class	Notes
`dev`	Development	Promoted	First environment on the promotion path
`test`	Test	Promoted	System and integration testing
`pat`	Production acceptance testing	Promoted	Known elsewhere as preprod or uat — use `pat`
`prod`	Production	Promoted	May be implied where a pattern marks the environment optional
`sandbox`	Sandbox	Non-promoted	Exploratory work; no promotion path
`poc`	Proof of concept	Non-promoted	Time-boxed; decommission on completion

Promoted environments participate in the CI/CD promotion path (dev → test → pat → prod).
Non-promoted environments never feed the promotion path and should be isolated accordingly.
preprod is not a sanctioned token — use pat. Where existing estates used preprod storage containers, new containers should be created as pat; existing containers may be grandfathered under the adaptability principle.

Environment and region topology for Azure resources:

Most Azure resource patterns below (VNET, resource group, ADF, SQL Server) intentionally omit an environment token, on the assumption that environments are segregated at subscription or resource-group level. Where multiple environments share a subscription, append -{env} to the resource name; multi-region estates may additionally append a region token (e.g. -aue for Australia East). Key Vault is the exception: its pattern always carries the environment (see Key Vault), because secrets differ materially between environments.

VNET

Name: vn-{organisation_name}-{domain_name}
e.g: vn-intuitas-corporate

Organisations aligned to the Cloud Adoption Framework abbreviations may prefer vnet- over vn-; either is acceptable if applied consistently.

Resource Groups

Name: rg-{organisation_name}-{domain_name}
e.g: rg-intuitas-corporate

Databricks Workspace

Name: ws-{organisation_name}-{domain_name}
e.g: ws-intuitas-corporate

This is the single definition of the workspace name pattern; the Databricks section references it rather than restating it.

Key Vault

Name: kv-{org_abbrev}-{domain_abbrev}-{env}
e.g: kv-intu-corp-prod

Key Vault names are limited to 3–24 characters (alphanumerics and hyphens, must start with a letter, no consecutive hyphens) and are globally unique (Azure resource name rules). The full-length form kv-intuitas-corporate-prod is 26 characters and therefore invalid — Key Vault names must use the registered organisation and domain abbreviations (see Appendix B for the abbreviation-registry rule).

The environment token is mandatory in Key Vault names, so that dev and prod vaults for the same domain can both comply with the standard.

Secrets and Secret Scopes

Key Vault secret name: {system}--{purpose}--{credential_type}
e.g: financedb—etl—password
e.g: fhirhouse—ingest—client-secret
Databricks secret scope name: {domain}-{source_system}-{env}
e.g: clinical-fhirhouse-prod
e.g: corporate-workday-dev

Secret scopes should be Key Vault-backed where the platform supports it, so that rotation is managed in one place.

Entra Group Names

The single standard for Entra group names is defined in Security → Entra:

grp-{org}-{domain}-{plat}-{scope}-{role}-{env}{optional: -{region}}{optional: -ext-{partner}}{optional: -{idx}}

The grp- pattern is the only standard; shorter prefixes such as eg-{organisation_name}-{domain_name} are not sanctioned, as they capture neither platform, scope, role nor environment.

Service Principals and Managed Identities

Name: sp-{org}-{domain}-{purpose}-{env}
e.g: sp-intuitas-engineering-bundle-deploy-prod
e.g: sp-intuitas-clinical-adf-ingest-dev

Notes:

One service principal per purpose per environment; do not reuse a dev principal in prod.
Production Declarative Automation Bundles run as a service principal named under this standard (see Declarative Automation Bundles).
Prefer workload identity federation over client secrets for CI/CD authentication, so that no long-lived credential exists to rotate or leak.
User-assigned managed identities follow the same pattern with an mi- prefix: e.g: mi-intuitas-corporate-adf-ingest-prod.

Azure Data Factory (ADF)

Name: adf-{organisation_name}-{domain_name}
e.g: adf-intuitas-corporate

ADF names are globally unique across Azure. Note that under this blueprint ADF is an exception pattern for hybrid connectivity (self-hosted integration runtime reach), not the default ingestion or orchestration tool — see the Azure Data Factory section.

SQL Server

Name: sql-{organisation_name}-{domain_name}
e.g: sql-intuitas-corporate

Azure SQL logical server names are globally unique and lowercase.

SQL Database

Name: sqldb-{purpose}-{organisation_name}-{domain_name}-{optional: environment}
e.g: sqldb-metadata-intuitas-corporate
e.g: sqldb-metadata-intuitas-corporate-dev

Storage

The section describes naming standards and conventions for cloud storage resources.

Storage account names are the tightest namespace in the estate: 3–24 characters, lowercase letters and digits only, globally unique. Three rules apply to every storage account pattern below:

Strip all separators. Hyphens and underscores are illegal in storage account names; domain names containing _ (e.g. intuitas_corporate) must have the underscore removed or be replaced by their registered abbreviation.
Use the abbreviation registry. Maintain a registry of 3–5 character abbreviations for the organisation and each domain (e.g. intuitas → intu, corporate → corp). Apply abbreviations whenever the full form would exceed the 24-character budget; when abbreviating, abbreviate the domain first, then the organisation.
Validate in IaC. Include a name-availability and length check in infrastructure-as-code validation, and permit a single trailing digit for global-uniqueness collisions (e.g: dlintucorp2).

Lakehouse Storage

Lakehouse storage account name: dl{organisation_name}{domain_name} (separators stripped; abbreviate per the rules above where over budget)
e.g: dlintuitascorporate (19 characters — within budget)
e.g: dlintucorp (abbreviated form)

Lakehouse Storage Containers

Name: {environment} using the canonical tokens from Environments: dev/test/pat/prod (and sandbox/poc where provisioned)
e.g: dev
e.g: pat

Use pat (not preprod) so that container names, catalog names and job names all share one environment vocabulary.

Lakehouse Storage Folders

Managed tables are the default under this blueprint: for the EDW and Infomart zones, Unity Catalog manages the physical layout beneath the catalog storage root, and no folder convention applies. The explicit folder mapping below applies to the Raw zone (whose landing/checkpoint/schema paths are addressed directly) and to organisations that choose external or managed-location storage at schema level — in which case folder names must match the schema names they serve.

Level 1 Name: {zone} (raw/edw/infomart) — (bronze/silver/gold) if using medallion terminology
Level 2 Name: {layer_name}, matching the schema name where the folder backs a schema
e.g:
- raw/landing
- raw/ods
- raw/pds
- raw/schema (for Autoloader metadata)
- raw/checkpoints (for Autoloader metadata)
- edw/stg, edw/ref, edw/mart (external/managed-location option only)
- infomart/stg, infomart/mart (external/managed-location option only)
- edw and infomart under managed tables: layout automatically determined by Unity Catalog

Generic Blob Storage

Generic Blob storage can be used for all non-lakehouse data; or alternatively within the lakehouse storage account in the appropriate container and folder.

Resource: ADLS Gen2
Generic storage account name: sa{organisation_name}{domain_name}{functional_description} (separators stripped; abbreviate per the storage rules above — the full form frequently exceeds 24 characters)
e.g: saintucorpmeta (14 characters; saintuitascorporatemetadata at 27 characters would be invalid)
Tier: Standard/Premium (depends on workload)
Redundancy:
- Minimum ZRS or GRS for prod
- Minimum LRS for sandbox, poc, dev, test and pat

Generic Blob Files and Folders

No standard naming conventions for files and folders.

Microsoft Fabric and Power BI

Naming for the Fabric/Power BI consumption layer, which serves data from the lakehouse (see the serving patterns in the architecture levels).

Fabric workspace: {org}-{domain}-{purpose}-{env}
e.g: intuitas-corporate-analytics-prod
e.g: intuitas-clinical-reporting-dev
Mirrored Azure Databricks Unity Catalog items: named to match the source catalog, so lineage is self-evident.
e.g: the mirrored item for catalog corporate__prod is named corporate__prod
Reference: Mirrored Azure Databricks Unity Catalog (GA).
Semantic model: {domain} {product} Semantic Model
e.g: Corporate Late Payments Semantic Model
e.g: Clinical Patient Outcomes Semantic Model

Semantic models are end-user-facing and therefore use title case with spaces, unlike engineering assets. Reports and Power BI apps take the product name without the “Semantic Model” suffix.

Databricks

This section provides naming standards and conventions for Databricks. Product names reflect the platform as at July 2026 — the consolidated old-to-new mapping is in Appendix A.

Workspace, Cluster and Workflow Names

Workspace name: defined once under Platform → Databricks Workspace: ws-{organisation_name}-{domain_name}
e.g: ws-intuitas-corporate
Shared cluster name: {domain}__{purpose}__{env} — all lowercase, no spaces, environment as a __{env} suffix
e.g: corporate__engineering_shared__dev
e.g: clinical__streaming_ingest__prod
Personal (single-user) cluster name: {first_name}_{last_name}__{purpose}
e.g: jane_citizen__sandbox
Workflow names follow the Job Names grammar below. Workflows are authored in Lakeflow Jobs (formerly Databricks Workflows).

Cluster and workflow names are lowercase and space-free, with the environment as a __{env} suffix, consistent with the job grammar beside them. The separate cluster convention in the Streaming section applies to Kafka/event-broker clusters only — it does not name Databricks compute.

Jobs and Pipelines

Job Names

Job names: {domain}__{zone}__{purpose}__{source}{optional: __target}{optional: __schedule}{optional: __version}__{env}
e.g: clinical__raw__ingest__fhircdr__dev
e.g: corporate__edw__transform__workday__daily__prod

Jobs run on Lakeflow Jobs (formerly Databricks Workflows), the primary orchestrator under this blueprint — including dbt tasks and Power BI publishing tasks. See Level 2 for the orchestration patterns.

Lakeflow Declarative Pipelines (formerly Delta Live Tables)

Lakeflow Declarative Pipelines (formerly Delta Live Tables (DLT); GA) use the __pipeline__ name token, which distinguishes declarative pipelines from ad hoc jobs.

Pipeline names: {domain}__{zone}__pipeline__{dataset}{optional: __schedule}{optional: __version}__{env}
e.g: clinical__raw__pipeline__fhircdr__dev
e.g: supplychain__infomart__pipeline__inventorymart__prod

Note on zone terminology:

This framework uses raw/edw/infomart terminology (aligning with enterprise data warehousing concepts)
Databricks documentation uses bronze/silver/gold terminology (medallion architecture)
Both are valid; choose one and use it consistently:
- raw = bronze (raw ingested data)
- edw = silver (cleaned, conformed data)
- infomart = gold (business-ready, requirement-aligned data)

Notes:

On {zone}: if the business outcome is Infomart, call it Infomart, even if the pipeline produces Raw and EDW along the way. i.e. “This is the production pipeline in the supply chain domain, which builds and maintains the curated Infomart zone dataset called Inventory Mart.”
Include pipeline so the asset is distinguishable from ad hoc jobs.
{dataset} can be a logical grouping (e.g. patient, encounter, claims).
Streaming tables and materialised views created by a pipeline are Unity Catalog objects and follow Schema and Object Conventions, including the streaming table and materialised view guidance.

Lakeflow Connect Ingestion Assets

Lakeflow Connect managed connectors (GA for SQL Server, Salesforce, Workday, ServiceNow, SharePoint, Google Analytics 4 and others, as at July 2026) are the first-choice ingestion pattern under this blueprint (see Level 2). They introduce two named asset types:

Ingestion pipeline name: {domain}__raw__ingest__{source_system}{optional: __schedule}__{env} (the job grammar, with ingest as the purpose)
e.g: corporate__raw__ingest__workday__hourly__prod
Ingestion gateway name (database connectors): {domain}__gateway__{source_system}__{env}
e.g: clinical__gateway__patientflowmanager01__prod

Tables landed by Lakeflow Connect follow the ODS conventions in Schema and Object Conventions, with __lakeflow as the channel token — e.g: central_hub__dev.ods__fhirhouse__dbo__lakeflow.encounter. Targets of Zerobus Ingest (GA over gRPC; the REST interface is Beta) follow the same conventions with __zerobus as the channel token.

Orchestration Job Names

Orchestration job names: {domain}__orchestrate__{workflow_name}{optional: __schedule}{optional: __version}__{env}
e.g: clinical__orchestrate__fhirworkflow__daily__dev

which then orchestrates:

clinical__raw__pipeline__fhircdr__prod (or clinical__bronze__pipeline__fhircdr__prod if using medallion terminology)
clinical__edw__pipeline__fhirclean__prod (or clinical__silver__pipeline__fhirclean__prod if using medallion terminology)
clinical__infomart__pipeline__clinicalmart__prod (or clinical__gold__pipeline__clinicalmart__prod if using medallion terminology)

The structural token is orchestrate (not orchestration).

Job Logging

Default: job and pipeline event logs are written to the meta schema of the domain catalog.
e.g: clinical__prod.meta.pipeline_event_log
Stricter isolation (sanctioned option): a dedicated audit catalog {domain}__audit__{env}, with event log schema audit__event_log.
e.g: clinical__audit__prod.audit__event_log.pipeline_event_log

The default is the meta schema of the domain catalog; the audit catalog is an explicit granularity option for stricter isolation (see Catalog Naming and Conventions).

Optional Name Components

Versioning: add __v1, __v2 if a job is redesigned but the old one remains in service.
e.g: corporate__edw__transform__workday__v2__prod
Scheduling frequency: suffix with __hourly, __daily or __weekly where relevant.
e.g: clinical__orchestrate__fhirworkflow__weekly__prod

Schedule suffixes are always double-underscored (__daily, not _daily), consistent with the separator discipline.

SQL Warehouses

Name: {domain}__wh__{purpose}__{env}
e.g: corporate__wh__analytics__prod
e.g: clinical__wh__genie__dev

Notes:

Prefer serverless warehouses. Cost attribution for serverless compute is via serverless usage policies (below) and the system billing tables — cluster tags do not apply to serverless compute.
Warehouse size is deliberately not encoded in the name: sizes change, purposes endure.

Compute Policies

Cluster policy name: cp-{domain}-{workload}
e.g: cp-corporate-etl
e.g: cp-clinical-streaming

Cluster policies enforce the mandatory tag set from Tags and Classification on classic compute; they are the primary cost-attribution control for non-serverless workloads.

Serverless usage policy name: up-{domain}-{env}
e.g: up-corporate-prod

Serverless usage policies (formerly serverless budget policies) are in Public Preview as at July 2026. They are the attribution mechanism for serverless compute, where cluster policies and cluster tags do not apply.

Catalog Naming and Conventions

Refer to Data zones and layers for further context and definitions applicable to this section.

The choice of granularity depends on domain topology, zone and layer conventions, and the desired level of segregation for access and sharing controls (i.e. catalog or layer level). All structural components are separated by double underscores:

Granularity	Pattern	Example
Domain (minimum)	`{domain}__{env}`	`corporate__dev`
Domain-zone	`{domain}__{zone}__{env}`	`corporate__raw__dev`
Domain-zone-layer	`{domain}__{zone}__{layer}__{env}`	`corporate__raw__ods__dev`
Subdomain-zone	`{domain}__{subdomain}__{zone}__{env}`	`corporate__finance__raw__dev`
Audit (purpose-scoped)	`{domain}__audit__{env}`	`clinical__audit__prod`

Notes:

{zone}: raw/edw/infomart; {layer}: ods/pds/stg/ref/mart; {env} from the canonical Environments taxonomy. prod may be implied where the organisation prefers unsuffixed production catalogs — if so, state the decision once and apply it everywhere.
Projects are temporary constructs, and hence are not recommended for catalog naming.
In the examples throughout this document we have opted for domain-level catalogs, with schema separation for the lower levels of grain via prefixes — i.e. central_hub__dev.ods__fhirhouse__dbo__lakeflow.encounter.
The audit catalog is the sanctioned stricter-isolation option for event and audit logs; the default remains the meta schema of the domain catalog (see Job Logging).

Deliberate design point: because domain names legitimately contain single underscores (ambo_sim, intuitas_corporate), names that mix single and double underscores as separators (e.g. corporate_raw__dev) cannot be machine-parsed into domain, subdomain and zone. The double underscore therefore separates every structural component: corporate__raw__dev, corporate__finance__raw__dev.

Catalog storage root: abfss://{env}@dl{organisation_name}{domain_name}.dfs.core.windows.net/{domain_name}__{env}_catalog
e.g: abfss://dev@dlintuitascentralhub.dfs.core.windows.net/central_hub__dev_catalog

Unity Catalog names are stored lowercase, must not contain ., / or spaces, and are limited to 255 characters; hyphens require backtick-quoting in SQL. See Appendix B for the consolidated constraints.

Externally Mounted (Lakehouse Federation) Catalog Names

Foreign catalog name: {domain_name (owner)}__fc__{source_system}{optional: __{descriptor}}{optional: __{env}}
e.g: corporate__fc__sqlonpremsource
e.g: corporate__fc__sqlonpremsource__prod

Each foreign catalog hangs off a named connection object — see Unity Catalog Infrastructure.

Unity Catalog Infrastructure

Naming for the metastore-level objects that catalogs depend on. These are administered via infrastructure-as-code rather than referenced in day-to-day SQL, so hyphenated names — which would need backtick-quoting in SQL — are acceptable, and deliberately distinguish infrastructure objects from data objects.

External location: exloc-{storage_account}-{container}
e.g: exloc-dlintuitascorporate-dev
Storage credential: stcred-{storage_account}
e.g: stcred-dlintuitascorporate
Connection (Lakehouse Federation): conn-{source_system}-{env}
e.g: conn-financedb-prod

Related identities and secrets are named under the Platform standards: Service Principals and Managed Identities and Secrets and Secret Scopes.

Tags and Classification

Tag keys are snake_case, drawn from a single registry shared by Unity Catalog objects, Declarative Automation Bundles and Azure resources. Controlled vocabularies apply to the values (domain names from the mesh register; environment tokens from Environments).

Tag key	Value	Applied to
`owning_domain`	`{domain_name}`	catalogs, schemas, bundles, Azure resources
`managing_domain`	`{domain_name}` — e.g. if delegating to the central_hub domain	catalogs, bundles
`environment`	canonical environment token	catalogs, bundles, Azure resources
`owner`	owning group or email address	catalogs, bundles
`manager`	managing team or email address	bundles
`project`	`{project_name}`	bundles
`bundle`	`{bundle_name}`	bundle-deployed resources

Tag keys must be automation-safe: no spaces or parentheses (use owning_domain, not domain (owner)).

For classification, use governed tags and ABAC policies (GA May 2026) together with data classification for PII discovery. Row-filter and column-mask functions are named mask_{data_class}:

e.g: mask_pii_email
e.g: mask_medicare_number

Schema and Object Conventions

Refer to Data zones and layers for further context and definitions applicable to this section.

Refer to Physical Models for the standard relating to column naming, types, and conventions.

Terminology hierarchy:

Data is organised in a three-level hierarchy:

Zone — top-level organisational boundary (Raw, EDW, Infomart)
Layer — processing layers within a zone (landing, ods, pds, cleansed, standardised, integrated, mart)
Schema — the Unity Catalog schema that stores the objects (ods, pds, edw__cleansed, edw__standardised, edw__integrated, edw__dq, im__stg, im, meta)

Example: the Raw zone contains the ODS layer, which maps to the ods schema.

EDW schema names are structural pairs ({zone}__{layer}, e.g. edw__cleansed) under the separator discipline; compact single-token schemas (ods, pds, im, meta) carry an unambiguous zone and need no prefix.

Schema-Level External Storage Locations

Recommendations:

For managed tables (the default): do nothing. Let dbt create schemas without additional configuration; Databricks manages storage and metadata beneath the catalog storage root. e.g: abfss://dev@dlintuitascentralhub.dfs.core.windows.net/central_hub__dev_catalog/__unitystorage/catalogs/{catalog_guid}/tables/{table_guid}
For granular control over schema-level storage: pre-create schemas with LOCATION mapped to external paths, with folder names matching the schema names they serve (see Lakehouse Storage Folders).
Ensure dbt’s dbt_project.yml and environment variables align with storage locations.

Metadata Schemas and Objects

Contains metadata that supports engineering and governance; contents vary with the engineering and governance toolsets.

Schema naming convention: meta{optional: __{function}}
Object naming convention: {function/descriptor}
e.g: corporate__dev.meta.ingestion_control
e.g: corporate__dev.meta__ingestion.ingestion_control

Raw (Bronze) (Data According to Systems)

The Raw zone stores raw, immutable data as it is ingested from source systems, in two primary schemas:

ods — Operational Data Store (current state)
pds — Persistent Data Store (historical snapshots)

The objective of the Raw zone conventions is clarity over which zone, layer and schema an object belongs to, what the data relates to, where it was sourced from, and via what channel it arrived (there may be nuances in data depending on its channel).

Note: consistent double-underscore (__) delimiters ensure machine-parseability. Parentheses notation (e.g. dbo(adf)) must be avoided in schema and object names.

Persistent Landing:

Persistent Landing uses Unity Catalog Volumes for raw files and unstructured data as they arrive.

Volume naming: {domain}__{env}.landing__{source_system}{optional: __{source_schema}}{optional: __{channel}}.{volume_name}
- {domain}__{env} — domain and environment (e.g. corporate__dev, central_hub__prod)
- {source_system} — source system identifier (e.g. workdayapi, saphr)
- {optional: __{source_schema}} — source schema or subsystem identifier
- {optional: __{channel}} — ingestion channel/method (e.g. __adf, __fivetran, __api)
- {volume_name} — descriptive volume name or source object identifier
e.g: corporate__dev.landing__workdayapi.schedule_volume
e.g: central_hub__prod.landing__fhirapi__patients.raw_data

Operational Data Store (ODS):

ODS can be replicated from source systems, or prepared for use from semi/unstructured data via hard transformation.

Database-replicated ODS (structured sources such as SQL Server):

Format: {domain}__{env}.ods__{source_system}{optional: __{source_schema}}{optional: __{channel}}.{source_object}
- {optional: __{source_schema}} — source schema, if applicable (e.g. __dbo, __reporting)
- {optional: __{channel}} — ingestion channel (e.g. __lakeflow, __zerobus, __adf, __fivetran)
- {source_object} — table name as per source
e.g: clinical__dev.ods__patientflowmanager01__reporting.patients
e.g: clinical__dev.ods__fhirhouse__dbo__adf.encounter
e.g: central_hub__dev.ods__fhirhouse__dbo__lakeflow.encounter

Prepped semi/unstructured ODS data:

Format: {domain}__{env}.ods__{source_system}{optional: __{source_descriptor}}{optional: __{channel}}.{source_object}
- {optional: __{source_descriptor}} — source descriptor or subsystem identifier
- {optional: __{channel}} — ingestion channel (e.g. __kafka, __databricks, __api)
- {source_object} — named as per source, or a unique assigned name (e.g. topic/folder name)
e.g: clinical__dev.ods__ambo_sim__confluent__kafka.encounter
e.g: corporate__dev.ods__workdayapi__employees.raw_data

Persistent Data Store (PDS):

PDS conventions mirror ODS conventions, with pds as the layer:

e.g: clinical__dev.pds__patientflowmanager01__reporting.patients
e.g: clinical__dev.pds__fhirhouse__dbo__adf.encounter
e.g: clinical__dev.pds__ambo_sim__confluent__kafka.encounter

PDS historisation is implemented with dbt snapshots emitting the standard SCD columns (effective_from_datetime, effective_to_datetime, updated_datetime) — see Snapshots and PDS Historisation.

Two PDS grammars coexist by design: ingestion-produced PDS (Autoloader, Lakeflow) uses source-scoped schemas per the platform grammar above (pds__{source_system}__{source_schema}), while dbt-snapshot-based historisation of seeds, reference and internal objects targets the flat pds schema. The _snapshot suffix on snapshot-produced objects is a dbt-mechanics concession and may be removed via alias where object-name purity matters.

EDW (Silver) (Data According to Business Entities)

The EDW (Enterprise Data Warehouse) zone transforms raw data into the enterprise’s business entities through three sub-layers, each representing a distinct stage of transformation. Each stage is a schema (see Level 2 - EDW (Silver) zone for the rationale):

edw__cleansed — 1:1 with source; typed, deduplicated, technically clean. Grain unchanged
edw__standardised — still source-separated; column names and code values mapped to the central glossary vocabulary
edw__integrated — the definitive business entities: unified keys resolved through keysets, survivorship applied, business history materialised
edw__dq — data quality test results and shaped data quality reports

Notes:

All schemas may optionally carry additional descriptors if not already decomposed at domain level, i.e. edw__cleansed__, edw__integrated__.
Entity names that align to transactional events/facts are plural; all other entity names are singular.
Optional warehousing constructs retain their own schemas: Raw Vault edw__rv and Business Vault edw__bv, with objects named per Data Vault standards (e.g: corporate__dev.edw__rv.hs_payments__finance_system__adf).

Sub-layer flow:

Raw Sources → Cleansed → Standardised → Integrated (entities + keysets) → Infomart views

The workbench:

Each sub-layer has a workbench of stg__ models whose prefix aligns to the target layer (e.g: stg__cln__sap__facilities__01_renamed_and_typed feeds the materialised sap__facilities). Workbench models are ephemeral by default — they exist to keep the logic readable, not to store data. A step is materialised only when the object itself needs to be inspectable: 07_desensitised (an audit boundary), 06_business_validated (quality reporting compares before and after), and keysets, which are always tables.

Canonical transformation steps (single vocabulary), applied within workbenches:

Step	Purpose
`00_keyed`	Key assignment/generation (including unified keys resolved via keysets)
`01_renamed_and_typed`	Column renaming and type casting
`02_deduped`	Deduplication (within or across sources)
`03_cleaned`	Data cleaning
`04_filtered` / `04_split`	Filtering or splitting data
`05_column_selected`	Column selection
`06_business_validated`	Business rule validation (including cross-source conflict resolution)
`07_desensitised`	Sensitive data masking/removal

Ordinals are fixed identifiers of the canonical steps, not sequence numbers: a pipeline applies whichever steps it needs, in ascending ordinal order.
Cross-source unification carries no step token of its own — the union of standardised models is inherent in the first integrated workbench step (typically 00_keyed).
Where a pipeline requires both filtering and splitting, 04_filtered applies first and the split takes 05_split — a descriptor variant occupying 05_column_selected’s ordinal position — so the two operations never contend for the same ordinal.
Consecutive steps MAY be combined into a single workbench model named by the highest step applied (e.g: stg__cln__sap__accounts__03_cleaned contains renaming, typing, deduplication and cleaning). The materialisation exceptions above are the only step boundaries that must remain distinct objects.

Minimum-viable ladder:

Scenario	Minimum required
Single source, glossary-aligned schema	Cleansed only, then straight to the integrated entity (standardised may be skipped)
Multiple sources sharing an entity	Standardised objects per source, a keyset, and an integrated entity
Regulated/PII data	An explicit `07_desensitised` workbench model is mandatory

Cleansed (edw__cleansed):

Casting, typing, deduplication and flattening only — no renaming to business vocabulary, no joins. One object per source object, at source grain, so downstream pipelines re-run from here rather than re-parsing Raw.

{domain}__{env}.edw__cleansed.{source_system}{optional: __{source_channel}}__{source_object}

Materialised cleansed and standardised objects carry no layer prefix — the schema states the stage. The cln__/std__ prefixes exist only in dbt model names (a global namespace) and are stripped via the alias override.

e.g: central_hub__prod.edw__cleansed.sap__facilities
e.g: central_hub__prod.edw__cleansed.workday__locations
e.g: central_hub__prod.edw__cleansed.crm__adf__customers

Standardised (edw__standardised):

Column names and code values mapped to the central glossary, one object per source system — the physical home of semantic mapping. Integration becomes a trivial union because all standardised objects for an entity share the same glossary-aligned schema; lineage remains attributable to a single source; and each source’s steward owns their own mapping. Skip this sub-layer only where a single, well-designed source already speaks the glossary vocabulary.

{domain}__{env}.edw__standardised.{source_system}__{entity}

e.g: corporate__prod.edw__standardised.sap__employee
e.g: corporate__prod.edw__standardised.workday__employee
e.g: corporate__prod.edw__standardised.crm__customer

Integrated (edw__integrated):

The enterprise single source of truth. Sources collide here for the first time: keys are resolved through keysets, survivorship rules pick winning attributes, and business history is materialised. Entities carry no object-type prefix — dim_/fact_ are consumption terms and belong to the Infomart.

{domain}__{env}.edw__integrated.{entity}

e.g: corporate__prod.edw__integrated.employee (unions SAP and Workday via keys__employee)
e.g: corporate__prod.edw__integrated.customer
e.g: corporate__prod.edw__integrated.payments (transactional — plural per the plurality rule)
e.g: health__prod.edw__integrated.encounters
e.g: corporate__prod.edw__integrated.date (generated calendar entity — no source, no keyset, no history; surfaced for consumption as im.dim_date. Name it calendar where the SQL reserved word grates)

Integrated workbench models take entity-centric names: e.g: stg__employee__00_keyed, stg__employee__06_business_validated.

History is materialised once, here:

Raw (Bronze) remains the immutable arrival history (landing/PDS); cleansed and standardised are current-state or simple append.
SCD2 is default-on for core entities (customers, assets, employees), using the standard SCD columns (effective_from_datetime, effective_to_datetime, updated_datetime).
Not every column deserves history. Track (Type 2) only the attributes whose changes matter to reporting — the ones that change how metrics group — and simply overwrite (Type 1) operational metadata. The business decides which is which; it is not an engineering guess.
Detect changes with a hash over the Type 2 columns, not column-by-column comparison.
Transactional events are immutable and never carry SCD tracking.
History can in principle be rebuilt from Raw, but retroactive rebuilds contend with schema drift, survivorship replay and heavy compute — treat Raw as insurance, not the plan. Where SCD2 is genuinely not yet required, delta time travel and Change Data Feed on integrated tables are the pragmatic fallback.

Keysets:

A keyset is a simple key mapping table: it resolves each system’s keys to the standardised key for the universal entity, so that sources conform to one identity in the integrated layer. One keyset per entity; downstream models join through the keyset rather than mapping keys themselves.

Name: keys__{entity}
e.g: corporate__prod.edw__integrated.keys__employee

One row per (source_system, source_key), pointing at the entity key (hash-valued, per Surrogate Keys in dbt) and the business key where one exists, with the standard SCD columns — so if a mapping needs to change, the old row is closed and a new one opened. Where matching is more than a shared natural key, record how the match was made (method, rule, confidence) so stewards can inspect and correct it as data rather than code.

Keysets are materialised tables, built in isolation — remapping never ripples through transformation logic.
Keysets are identity linkage; grant them more strictly than the schemas they serve.
Keysets resolve which records are the same entity; survivorship rules — defined in the glossary — decide whose attributes win.

The pattern is the Data Vault Same-As Link, adopted on its own; the rest of the Vault is not carried over.

Reference data:

Reference data are integrated entities: curated, conformed reference entities and code sets intended for broad reuse, keyed by natural/business keys, often current-state with limited effective dating. Built from standardised sources like any other entity (with a keyset where multiple systems provide the same code set). The ref_ prefix is mandatory and reference objects take singular nouns:

e.g: corporate__prod.edw__integrated.ref_facility (facility_id, facility_name, facility_code, location_key)
e.g: corporate__prod.edw__integrated.ref_account_code (account_code, account_name, hierarchy_level)
e.g: corporate__prod.edw__integrated.ref_country_code (country_code, country_name)

Reference objects are always singular (ref_country_code, not ref_country_codes), consistent with the entity-singularity rule. Reference entities are shaped into consumption dimensions in the Infomart (e.g: ref_facility → im.dim_facility) — see Reference Data Standards and Conventions.

Infomart (Gold) (Data According to Requirements)

The Infomart zone is the optional, views-by-default consumption interface. Because integrated entities already carry unified keys, conformed vocabulary and materialised history, information marts are expressed as views over edw__integrated unless a serving pattern demands materialisation: Fabric mirroring excludes views; view shares to open recipients are materialised at the provider’s cost; and performance-critical serving may warrant tables or materialised views (see Level 2).

Products are requirement-aligned and predominantly source-agnostic, with optionality for source-specific views where needed. Two primary schemas:

im__stg — the product workbench (stg__ shaping models; ephemeral by default, as in the EDW workbenches)
im — information marts (final business-aligned products)

Naming convention:

{domain}__{env}.{schema}.{object_type}{product_name}{optional: __{source_system}}{optional: __{transformation_step}}

{object_type} — object-type prefix, mandatory: stg__ for staging; fact_, dim_ or obt_ for final products
{product_name} — product or business-aligned name. Entity-aligned names follow the plurality rule (facts plural, dimensions singular); descriptive product names (e.g. downtime_by_region) are exempt. Product marts are requirement-aligned, not technical constructs.
{optional: __{source_system}} — only when a source-specific view is needed
{optional: __{transformation_step}} — staging only

The staging prefix is stg__ (double underscore) in both the EDW and Infomart zones. Final Infomart objects always carry their object-type prefix (fact_, dim_, obt_); unprefixed products such as im.orders are non-compliant.

The product workbench (im__stg) applies product-shaping transformations: pivoting, aggregation, joining across integrated entities, business rule application, desensitisation, complex calculations. Workbench models are ephemeral by default, per the EDW rule; materialise only where an inspectable object is required — 07_desensitised remains an audit boundary, and a shaped intermediate reused by several products may warrant materialisation. Product-shaping steps without a canonical token (pivoted, aggregated, joined) take pipeline-sequence ordinals and must not reuse canonical descriptor words; canonical steps such as 07_desensitised keep their canonical ordinal.

e.g: corporate__prod.im__stg.stg__downtime_by_region__01_pivoted
e.g: corporate__prod.im__stg.stg__downtime_by_region__07_desensitised
e.g: corporate__prod.im__stg.stg__late_payments__01_aggregated
e.g: corporate__prod.im__stg.stg__late_payments__02_joined
e.g: health__prod.im__stg.stg__patient_outcomes__01_joined
e.g: health__prod.im__stg.stg__patient_outcomes__07_desensitised

Information marts (final products) in the im schema:

Fact tables (aggregated/business-focused):

e.g: corporate__prod.im.fact_downtime_by_region
e.g: corporate__prod.im.fact_late_payments
e.g: corporate__prod.im.fact_regional_account_payments
e.g: health__prod.im.fact_patient_outcomes

Dimension tables (product-specific):

e.g: corporate__prod.im.dim_region
e.g: corporate__prod.im.dim_payment_category

One Big Tables (OBT) — denormalised aggregates:

e.g: corporate__prod.im.obt_fte_calculations
e.g: corporate__prod.im.obt_financial_summary
e.g: corporate__prod.im.obt_executive_dashboard

Source-specific products (when needed):

e.g: corporate__prod.im.fact_account_payments__old_finance_system
e.g: corporate__prod.im.fact_account_payments__new_finance_system
e.g: corporate__prod.im.obt_employee_metrics__sap
e.g: corporate__prod.im.obt_employee_metrics__workday

Streaming Tables, Materialised Views and Metric Views

Storage format and materialisation are never encoded in object names. No _delta or _iceberg suffixes — format (Delta, managed Iceberg, UniForm) is metadata; names describe function. Likewise, a materialised view serving a fact is simply fact_{...} — the materialisation is a refresh strategy, not an identity.
Streaming tables created by ingestion into the Raw zone follow the ODS conventions unchanged. Elsewhere (e.g. intermediate streaming transformations), prefix with st_:
e.g: clinical__prod.edw__cleansed.st_encounter_events
Materialised views are named by their functional role (dim_, fact_, obt_, stg__ patterns above).
Metric views (Unity Catalog metric views, GA): mv_{subject}, in the mart schema of the relevant zone.
e.g: corporate__prod.im.mv_late_payments
Other Unity Catalog securables — registered models, feature tables, vector search indexes, Lakebase synced tables — take three-level names and follow the catalog and schema conventions above; no special grammar applies.

Databricks Apps and Genie Agents

Databricks Apps (GA): {domain}__app__{descriptor}__{env}
e.g: corporate__app__payments__prod
e.g: clinical__app__triage__dev
Genie Agents (formerly Genie Spaces) (GA July 2026): {domain}__genie__{descriptor}
e.g: clinical__genie__patient_flow
e.g: corporate__genie__late_payments

Genie Agents are end-user-facing: the display title may be a friendly name (“Patient Flow”), but the underlying asset name follows the standard so that ownership and domain remain machine-readable. Genie Agents surfaced through Genie One (formerly Databricks One) inherit these names.

Naming for OpenSharing (formerly Delta Sharing) shares and recipients. See Level 2 for the sharing architecture and decision tree.

Share names: {domain}{optional: __{subdomain}}{optional: __{purpose}}__{schema_name or description}__{object_name or description}__{share_purpose and/or target_audience}
e.g: intuitas_corporate__finance__reporting__im__fact_late_payments__acme_bank
e.g: clinical__im__fact_patient_outcomes__research_partners
Recipient names: {counterparty_org}__{domain} — the counterparty organisation, then the local domain whose data they receive.
e.g: acme_bank__corporate
e.g: uni_research__clinical

Notes:

Shareable assets extend beyond tables to views, dynamic views, materialised views, streaming tables, volumes, models, notebooks and managed Iceberg tables.
Tables carrying row filters or column masks cannot be shared — share dynamic views instead, named per their functional role.
Prefer OIDC token federation over long-lived bearer tokens for open recipients.

Azure Data Factory

Under this blueprint ADF is an exception pattern, retained for hybrid connectivity where self-hosted integration runtime reach is required — see Level 2.

Linked service names: ls_{database_name}{optional (if org/domain not evident in database_name): _{organisation_name}_{domain_name}}
e.g: ls_financedb_intuitas_corporate
Dataset names: ds_{database_name}_{object_name}
e.g: ds_financedb_payments
Pipeline names: pl_{description} — e.g. copy_{source_name}_to_{destination_name}
e.g: pl_copy_financedb_to_landing
Trigger names: tr_{pipeline_name}{optional: _{start_time or frequency}}
e.g: tr_pl_copy_financedb_to_landing_daily

Streaming

These conventions apply to Kafka/event-broker estates (e.g. Confluent, Azure Event Hubs’ Kafka surface). They do not name Databricks compute — see Workspace, Cluster and Workflow Names.

Cluster name (Kafka/event-broker cluster): {domain}__cluster{optional: __{env}}
e.g: clinical__cluster__prod
Topic names: {domain}__{entity}{optional: __{source_system}}{optional: __{source_channel}}{optional: __{env}}
e.g: clinical__encounter__ambo_sim__prod
e.g: corporate__payment__financedb__cdc__prod
Consumer group names: {domain}__{unique_group_name}{optional: __{env}}
e.g: clinical__encounter_ingest__prod

The cluster convention is explicitly scoped to event brokers; all separators between structural components are double underscores.

dbt

The following standards and conventions relate to dbt projects. dbt is the lead SQL transformation framework under this blueprint, typically executed as dbt tasks in Lakeflow Jobs — see Level 2.

Documentation and Model Metadata

Within each respective model folder (as needed):

md: _{path to model folder, using __ between path segments}__docs.md
e.g: models/edw/ambo_sim__kafka__local/_edw__ambo_sim__kafka__local__docs.md
model yml: _{path to model folder, using __ between path segments}__models.yml
e.g: models/edw/ambo_sim__kafka__local/_edw__ambo_sim__kafka__local__models.yml

The separator between path segments is the double underscore — necessarily, since folder names themselves may contain single underscores (ambo_sim).

Sources

Folder: models/sources/{raw|edw|infomart} — a top-level folder, not nested within a zone folder
yml: {schema}__sources.yml (one for each source schema)
e.g: models/sources/raw/ods__ambo_sim__kafka__sources.yml

Sources live at the top level (models/sources/), never nested within a zone folder.

Model and Folder Names

dbt model names are verbose (inclusive of schema and domain) to ensure global uniqueness and better traceability to folders. Actual object names should be aliased to match the object naming standards.

The optional {domain__} / {edw__} / {im__} prefixes apply only in multi-domain monorepos that need the extra model-name uniqueness; single-domain projects omit them and require no alias. Where the prefixes are used, the alias is implemented by overriding generate_alias_name to strip the configured prefix, so physical object names match the object naming standards:

-- macros/generate_alias_name.sql
{% macro generate_alias_name(custom_alias_name=none, node=none) -%}
    {%- set prefix = var('alias_strip_prefix', '') -%}
    {%- if custom_alias_name -%}
        {{ custom_alias_name | trim }}
    {%- elif prefix and node.name.startswith(prefix) -%}
        {{ node.name[(prefix | length):] }}
    {%- else -%}
        {{ node.name }}
    {%- endif -%}
{%- endmacro %}

Raw

The Raw zone contains the operational data store (ODS) for current state and the persistent data store (PDS) for historical tracking.

ODS — current state:

Folder: seeds/{optional: domain}{optional: __subdomain(s)}/ → materialises to {domain}__{env}.ods
Folder: models/sources/raw/{optional: domain}{optional: __subdomain(s)}/ → references external sources in {domain}__{env}.ods

Seed naming convention (CSV files, materialising to the ODS schema):

ref_{entity}.csv for reference/lookup data — singular entity nouns
config_{descriptor}.csv for configuration data
e.g: ref_country_code.csv → corporate__dev.ods.ref_country_code
e.g: ref_currency_rate.csv → corporate__dev.ods.ref_currency_rate
e.g: config_holidays.csv → corporate__dev.ods.config_holidays

A reference set may legitimately exist in both ods (the seed as it arrived) and edw__integrated (the curated, conformed model) — e.g. ods.ref_country_code → edw__integrated.ref_country_code; this is by design, not duplication.

PDS — historical tracking:

PDS historisation uses dbt’s native snapshot materialisation, not ordinary models — see Snapshots and PDS Historisation.

Folder: snapshots/{optional: domain}/pds/, targeting schema pds
Snapshot names: {source_object}_snapshot
e.g: snapshots/pds/ref_country_code_snapshot.yml → corporate__dev.pds.ref_country_code_snapshot
e.g: snapshots/pds/config_holidays_snapshot.yml → corporate__dev.pds.config_holidays_snapshot

EDW

EDW models follow the sub-layer conventions defined in Schema and Object Conventions. Each sub-layer folder carries its own stg/ workbench (ephemeral by default). Model names carry the cln__/std__ layer prefix for dbt’s global namespace; the generate_alias_name override strips it, so materialised objects carry no prefix.

Cleansed:

Folder: models/edw/cleansed/{source_system}{optional: __{source_channel}}/
Models: cln__{source_system}{optional: __{source_channel}}__{source_object}
Workbench: models/edw/cleansed/{source_system}/stg/ — stg__cln__{source_system}{optional: __{source_channel}}__{source_object}__{transformation_step}
e.g: edw/cleansed/sap/cln__sap__facilities.sql → central_hub__dev.edw__cleansed.sap__facilities
e.g: edw/cleansed/sap/stg/stg__cln__sap__facilities__01_renamed_and_typed.sql (ephemeral)

Standardised:

Folder: models/edw/standardised/{source_system}/
Models: std__{source_system}__{entity}
Workbench: models/edw/standardised/{source_system}/stg/ — stg__std__{source_system}__{entity}__{transformation_step}
e.g: edw/standardised/sap/std__sap__employee.sql → corporate__dev.edw__standardised.sap__employee

Integrated (entities and keysets):

Folder: models/edw/integrated/{entity}/
Models: {entity}.sql (unprefixed) and keys__{entity}.sql (materialised table)
Workbench: models/edw/integrated/{entity}/stg/ — stg__{entity}__{transformation_step} (ephemeral, except the materialised boundary models)
Reference entities: models/edw/integrated/ref/{reference_entity}/ — ref_{reference_entity}.sql, singular nouns
e.g: edw/integrated/employee/employee.sql → corporate__dev.edw__integrated.employee
e.g: edw/integrated/employee/keys__employee.sql → corporate__dev.edw__integrated.keys__employee
e.g: edw/integrated/ref/facility/ref_facility.sql → corporate__dev.edw__integrated.ref_facility

Raw Vault:

Folder: models/edw/rv
Models: edw__rv__{vault object named as per data vault standards}

Business Vault:

Folder: models/edw/bv
Models: edw__bv__{vault object named as per data vault standards}

Infomart

Staging:

Folder: models/infomart/{optional: domain}{optional: __subdomain(s)}/mart/{product}/stg
Models: {optional: domain__}{optional: im__}stg__{product}__{transformation_step}{optional: __{source_system}}

Dimensions, facts and OBTs (consumption views over integrated entities):

Folder: models/infomart/{optional: domain}{optional: __subdomain(s)}/mart/{product}
Models: {optional: domain__}{optional: im__}{dim_|fact_|obt_}{product}{optional: __{source_system}} — the object-type prefix is mandatory; views by default, materialised only where a serving pattern demands it

Tests and exposures:

Singular tests: assert_{expectation} — e.g: tests/assert_positive_value_for_total_amount.sql
Exposures: exp_{consumer}_{product} — e.g: exp_powerbi_late_payments

Example dbt Model Structure

The structure below reflects a single catalog for domain+environment, with schema separation for zones and layers:

{{domain/enterprise}_project_name}
├── README.md
├── analyses
├── seeds  <ods>
│   ├── ref_country_code.csv  <e.g: corporate__dev.ods.ref_country_code>
│   └── config_holidays.csv  <e.g: corporate__dev.ods.config_holidays>
├── dbt_project.yml
├── macros
│   ├── generate_schema_name.sql  <required override — see dbt_project.yml and Schema Configuration>
│   └── utilities
│       └── all_dates.sql
├── models/sources
│   └── raw
│       └── ods__source_system_1__sources.yml
├── models/raw
│   ├── _raw.md
│   └── ods
│       └── (typically empty — ODS populated by seeds/sources)
├── models/edw
│   ├── _edw.md
│   ├── cleansed  (1:1 with source, typed and deduplicated) <edw__cleansed>
│   │   ├── source_system_1
│   │   │   ├── _source_system_1__docs.md
│   │   │   ├── _source_system_1__models.yml
│   │   │   ├── cln__source_system_1__object.sql  <e.g: central_hub__dev.edw__cleansed.source_system_1__object (alias strips cln__)>
│   │   │   └── stg  (ephemeral workbench)
│   │   │       └── stg__cln__source_system_1__object__03_cleaned.sql
│   │   └── source_system_2__adf
│   │       ├── cln__source_system_2__adf__accounts.sql  <e.g: central_hub__dev.edw__cleansed.source_system_2__adf__accounts>
│   │       └── stg
│   │           └── stg__cln__source_system_2__adf__accounts__01_renamed_and_typed.sql
│   ├── standardised  (glossary-mapped, still source-separated) <edw__standardised>
│   │   ├── source_system_1
│   │   │   ├── _source_system_1__models.yml
│   │   │   ├── std__source_system_1__employee.sql  <e.g: corporate__dev.edw__standardised.source_system_1__employee>
│   │   │   └── stg
│   │   │       └── stg__std__source_system_1__employee__06_business_validated.sql  <materialised — audit boundary>
│   │   └── source_system_2
│   │       └── std__source_system_2__employee.sql
│   ├── integrated  (entities, keysets, materialised history) <edw__integrated>
│   │   ├── employee
│   │   │   ├── _employee__docs.md
│   │   │   ├── _employee__models.yml
│   │   │   ├── keys__employee.sql  <e.g: corporate__dev.edw__integrated.keys__employee (materialised keyset)>
│   │   │   ├── employee.sql  <e.g: corporate__dev.edw__integrated.employee (SCD2; unions std__ sources via keyset)>
│   │   │   └── stg  (ephemeral workbench)
│   │   │       ├── stg__employee__00_keyed.sql
│   │   │       └── stg__employee__02_deduped.sql
│   │   ├── payments
│   │   │   └── payments.sql  <e.g: corporate__dev.edw__integrated.payments (transactional — plural)>
│   │   └── ref  (reference entities)
│   │       ├── facility
│   │       │   ├── ref_facility.sql  <e.g: corporate__dev.edw__integrated.ref_facility>
│   │       │   └── stg
│   │       │       └── stg__facility__00_keyed.sql
│   │       └── country_code
│   │           └── ref_country_code.sql  <e.g: corporate__dev.edw__integrated.ref_country_code (single source, no workbench)>
├── models/infomart  <im>
│   ├── _infomart.md
│   └── finance
│       └── mart
│           ├── _finance__models.yml
│           ├── fact_payments.sql  <e.g: corporate__dev.im.fact_payments (view over edw__integrated.payments)>
│           ├── dim_employee.sql  <e.g: corporate__dev.im.dim_employee (view over edw__integrated.employee)>
│           └── stg  <im__stg>
│               └── stg__payments__01_pivoted_to_orders.sql
├── packages.yml
├── snapshots  <pds>
│   └── pds
│       ├── ref_country_code_snapshot.yml  <e.g: corporate__dev.pds.ref_country_code_snapshot>
│       └── config_holidays_snapshot.yml  <e.g: corporate__dev.pds.config_holidays_snapshot>
└── tests
    └── assert_positive_value_for_total_amount.sql

dbt_project.yml and Schema Configuration

The yml structure below reflects a single catalog for domain+environment, with schema separation for zones and layers:

models:
  health_lakehouse__engineering__dbt:
    +persist_docs: # enables injection of metadata into unity catalog
        relation: true
        columns: true

    raw:
      ods:
        +description: "Operational data store - current state"
        +schema: ods
        +materialized: table


    edw:
      cleansed:
        +description: "Cleansed - typed, deduplicated, 1:1 with source"
        +schema: edw__cleansed
        +materialized: table
        stg:
          +description: "Cleansed workbench"
          +materialized: ephemeral
      standardised:
        +description: "Glossary-mapped, source-separated"
        +schema: edw__standardised
        +materialized: table
        stg:
          +description: "Standardised workbench (audit-boundary models override to table in-model)"
          +materialized: ephemeral
      integrated:
        +description: "Business entities, keysets, materialised history"
        +schema: edw__integrated
        +materialized: table
        stg:
          +description: "Integrated workbench (keysets and audit boundaries override to table in-model)"
          +materialized: ephemeral
      rv:
        +description: "Raw vault (optional; only where Data Vault modelling is adopted)"
        +schema: edw__rv
        +materialized: table
      bv:
        +description: "Business vault (optional; only where Data Vault modelling is adopted)"
        +schema: edw__bv
        +materialized: table

    infomart:
      +materialized: view # consumption interface - views by default
      mart:
        +description: "Information marts (business-aligned products; materialise only where serving demands)"
        +schema: im
        +materialized: view
        stg:
          +description: "Product workbench (audit-boundary models override to table in-model)"
          +schema: im__stg
          +materialized: ephemeral

seeds:
  health_lakehouse__engineering__dbt:
    +schema: ods
    +description: "Static reference and configuration data loaded from CSV files to ODS"

snapshots:
  health_lakehouse__engineering__dbt:
    +schema: pds

Required schema-name override:

Out of the box, dbt renders custom schemas as {target_schema}_{custom_schema} (e.g. dbt_ods, not ods) — following the yml above verbatim will not produce the schema names this standard mandates unless generate_schema_name is overridden to return the custom schema as-is:

-- macros/generate_schema_name.sql
{% macro generate_schema_name(custom_schema_name, node) -%}
    {%- if custom_schema_name is none -%}
        {{ target.schema }}
    {%- else -%}
        {{ custom_schema_name | trim }}
    {%- endif -%}
{%- endmacro %}

Surrogate Keys in dbt

Surrogate hash keys are generated via the dbt_utils package (declared in packages.yml — this is not dbt core functionality): dbt_utils.generate_surrogate_key(['column_a', 'column_b']).

Do not use the older dbt_utils.surrogate_key(), which was removed in dbt_utils v1.0 (December 2022) and errors outright. Note that generate_surrogate_key treats nulls differently — nulls do not collide with empty strings — so hash values differ for rows with null components; the surrogate_key_treat_nulls_as_empty_strings variable restores legacy behaviour where backwards compatibility of key values is required.

Databricks identity columns remain a valid alternative for surrogate keys outside dbt-managed tables, with known limitations: concurrent writes are not supported, identity columns cannot be partition columns, and they cannot be added to an existing table via ALTER TABLE. See Physical Models.

Snapshots and PDS Historisation

PDS historisation uses dbt’s native snapshot materialisation (dbt 1.9+ YAML-defined snapshots), configured with snapshot_meta_column_names so that snapshots emit the blueprint’s standard SCD columns directly:

dbt default column	Blueprint standard column
`dbt_valid_from`	`effective_from_datetime`
`dbt_valid_to`	`effective_to_datetime`
`dbt_updated_at`	`updated_datetime`

snapshots:
  - name: ref_country_code_snapshot
    relation: ref('ref_country_code')
    config:
      schema: pds
      unique_key: country_code
      strategy: check
      check_cols: all
      hard_deletes: new_record
      snapshot_meta_column_names:
        dbt_valid_from: effective_from_datetime
        dbt_valid_to: effective_to_datetime
        dbt_updated_at: updated_datetime

Notes:

The single physical SCD column standard is effective_from_datetime / effective_to_datetime / updated_datetime. Date-grain columns (effective_from_date) are permitted only for slowly-moving reference data, and that choice must be stated explicitly in the model documentation.
Deleted-row tracking uses the hard_deletes: new_record config, which adds a dbt_is_deleted flag column.
Two PDS grammars coexist by design: ingestion-produced PDS (Autoloader, Lakeflow) uses source-scoped schemas per the platform grammar (pds__{source_system}__{source_schema}), while the dbt-snapshot-based historisation above — for seeds, reference and internal objects — targets the flat pds schema. The _snapshot suffix is a dbt-mechanics concession and may be removed via alias where object-name purity matters.

Deviations from the dbt Style Guide

This blueprint deviates from the dbt style guide in three deliberate respects:

Deviation	dbt style guide	This blueprint	Why
Staging prefix	`stg_{source}__{entity}`	`stg__{source}__{entity}`	`__` separates all structural components under the separator discipline; the prefix is a structural component
Staging granularity	Fewer staging models plus intermediates	Numbered micro-steps (`__01_renamed_and_typed` … `__07_desensitised`) as separate models	Auditability and step isolation; DAG and object-count inflation is mitigated by the prescribed view materialisation
Fact prefix	`fct_`	`fact_`	Readability for business-facing consumers; applied consistently across all zones

CI/CD

The following standards and conventions relate to Continuous Integration and Continuous Delivery constructs.

Repository Naming

All lowercase with hyphens as separators
Format: {org}-{domain}-{purpose}{optional: -{descriptor}}

Examples:
- intuitas-corporate-dbt
- intuitas-corporate-ingestion-framework
- intuitas-corporate-cicd-templates
- intuitas-corporate-infrastructure

Branch Naming

All lowercase with hyphens as separators
Naming convention: {type}{optional: -{ticket_id}}-{description}

Types:
- feature: new functionality
- bugfix: bug fixes
- hotfix: critical fixes for production
- release: release branches
- docs: documentation updates only
- refactor: code improvements with no functional changes
- test: test-related changes
Examples:
- feature-eng123-add-new-data-source
- bugfix-eng456-fix-null-values
- hotfix-prod-outage-fix
- release-v2.1.0
- docs-update-readme
- refactor-optimize-transforms
- test-add-integration-tests

Branch Lifecycle

Simple branch lifecycle:

main/master: primary branch
branch: short-lived development branches, merged or rebased to main/master

Comprehensive team branch lifecycle:

Main/master: primary branch
Development: active development branch
Feature/bugfix: short-lived branches merged to development
Release: created from development, merged to main/master
Hotfix: created from main/master for urgent fixes

Declarative Automation Bundles (formerly Databricks Asset Bundles)

Declarative Automation Bundles (formerly Databricks Asset Bundles) are encouraged for all Databricks projects.

Project/bundle name:
- {domain_name}__databricks (for general Databricks projects)
- {domain_name}__dbt (for dbt Databricks bundles)

Bundle tags — snake_case keys from the shared registry (see Tags and Classification):

tags:
   environment: ${bundle.target}
   project: health_lakehouse
   bundle: health_lakehouse__engineering__databricks
   owning_domain: intuitas_engineering
   managing_domain: intuitas_engineering
   owner: engineering-admin@intuitas.com
   manager: engineering-engineer@intuitas.com

Resources:
- Folder level 1: {meaningful sub-project name}
- Folder level 2:
  - notebooks
  - workflows
databricks.yml:
- Dev target: deploys under the developer’s user folder (mode: development isolates and prefixes assets per developer).
- Prod target: deploys to a shared workspace path — /Workspace/Shared/.bundle/${bundle.name}/${bundle.target} — and runs as a service principal, never a personal user.

Example databricks.yml:

# Declarative Automation Bundle definition for health_lakehouse__engineering.
# See https://docs.databricks.com/aws/en/dev-tools/bundles/ for documentation.
bundle:
  name: health_lakehouse__engineering__databricks

include:
  - resources/*.yml
  - resources/**/*.yml

targets:
  dev:
    mode: development
    default: true
    workspace:
      host: https://adb-0000000000000000.15.azuredatabricks.net
      root_path: /Workspace/Users/${workspace.current_user.userName}/.bundle/${bundle.name}/${bundle.target}
  prod:
    mode: production
    workspace:
      host: https://adb-0000000000000000.15.azuredatabricks.net
      root_path: /Workspace/Shared/.bundle/${bundle.name}/${bundle.target}
    permissions:
      - group_name: grp-intuitas-engineering-dbx-bundle-health-lakehouse-operator-prod
        level: CAN_MANAGE
    run_as:
      service_principal_name: "00000000-0000-0000-0000-000000000000"

Notes:

The run_as service principal is named per the Platform standard (e.g: sp-intuitas-engineering-bundle-deploy-prod); service_principal_name takes its application ID.
CI/CD pipelines authenticate as the service principal via workload identity federation, so no long-lived client secret exists to rotate or leak.

Production targets must deploy to a shared workspace path and run as a service principal. Personal-user production deployment breaks when the user leaves and entangles production assets with a personal workspace lifecycle.

Security

Security standards and conventions provided here are a starter set; existing organisational and applicable industry standards should take precedence. Consult with your cybersecurity advisor.

Entra

Most organisations will already have an established set of groups and conventions. Where there are gaps, the following standard applies.

Recommended areas to align to organisational governance and cyber requirements:

Naming conventions for admin, service and user groups
Role-based access alignment (least privilege, separation of duties)
Alignment to domains — cross-domain vs domain-specific group patterns

Entra group names:

Pattern: grp-{org}-{domain}-{plat}-{scope}-{role}-{env}{optional: -{region}}{optional: -ext-{partner}}{optional: -{idx}}

Rules:

All lowercase, hyphen-separated; no spaces. Keep to ≤ 120 characters in total, and prefer well under 40 — display names truncate at around 40 characters in most picker UIs. Use registered abbreviations where needed.
Environment tokens are drawn from the canonical Environments taxonomy (dev/test/pat/prod) — tokens such as PRD and UAT are not sanctioned.
No PII in names.
Use Security groups (not Microsoft 365 groups) for RBAC.
Scope tokens must not contain . or / — encode hierarchy with hyphens (e.g. uc-sch-claims-curated, not uc-sch-Claims.Curated). Dots and slashes break the hyphen-delimited parse and are hostile to automation.
Enforcement is via the provisioning pipeline (groups are created by IaC only): Entra group naming policy applies to Microsoft 365 groups, not the security groups this standard mandates, so a naming policy cannot enforce it.
Groups with role of admin or owner are created as role-assignable groups with PIM-eligible (not permanent) assignment. Note the tenant limit of 500 role-assignable groups — reserve them for genuinely privileged scopes.
External partner access (-ext-{partner}) is governed through Entra ID Governance access packages; the group is the resource the package assigns, not a directly-managed membership list.
{idx} is a two-digit disambiguator (01–99) for the rare case where otherwise identical names are required.

role:

owner — full control of the named scope
admin — administrative (non-ownership) rights
contrib — create/modify within scope
editor — modify data/artifacts, not permissions
reader — read-only
steward — governance/metadata rights
custodian — key/secret/storage control
operator — run/ops rights (pipelines, jobs, bundles)
viewer — read dashboards/reports

plat:

dbx (Databricks), uc (Unity Catalog), pbi (Power BI/Fabric), adf (Data Factory),
dlk (Data Lake), sql (Azure SQL), kva (Key Vault)

scope (or object):

Databricks workspace: ws-{workspace}
Databricks bundle: bundle-{bundle_descriptor}
Unity Catalog: uc-meta (metastore), uc-cat-{catalog}, uc-sch-{catalog}-{schema}, uc-obj-{catalog}-{schema}-{object}
Power BI / Fabric: pbi-ws-{workspace}
Data Lake: dlk-path-{container}-{path segments, hyphen-separated}
Key Vault: kva-{purpose}

Scope tokens reference the logical name without its environment suffix — the group’s own -{env} component carries the environment (e.g. catalog claims__prod → scope uc-cat-claims). Single underscores within domain or object tokens are retained (e.g. uc-cat-ambo_sim); they do not conflict with the hyphen-delimited grammar.

Examples:

grp-intuitas-clinical-dbx-ws-analytics-admin-prod
grp-intuitas-clinical-uc-uc-cat-claims-owner-prod
grp-intuitas-clinical-uc-uc-sch-claims-curated-reader-pat
grp-intuitas-finance-pbi-pbi-ws-execdash-viewer-prod
grp-intuitas-corporate-kva-kva-keys-custodian-prod
grp-intuitas-clinical-dlk-dlk-path-curated-claims-reader-prod-aue
grp-intuitas-clinical-dbx-ws-partnerlake-reader-prod-ext-acme
grp-intuitas-clinical-dbx-ws-analytics-reader-prod-01

Policies

Under development. (Contact us to know more).

Recommended areas to align to non-functional requirements:

Data retention (duration, archival, legal hold)
Key retention and rotation cycles
Backup and recovery standards
Incident response and escalation procedures
Access review and recertification

Frameworks

Under development. (Contact us to know more).

Recommended areas to align to industry and cyber compliance:

Engineering standards (e.g., code repositories, CI/CD security, IaC policies)
Security frameworks (e.g., NIST, ISO 27001, CIS Benchmarks, Zero Trust)
Compliance mappings (HIPAA, GDPR, SOC2, local regulatory obligations)

Appendix A: Product Name Mapping (2024-2026)

Use the current name; on first use in a document, add “(formerly X)”. Other blueprint documents link to this table rather than repeating it. Statuses are as at July 2026.

Use this	Formerly	Status	Reference
OpenSharing	Delta Sharing	GA	Microsoft Learn
Lakeflow Declarative Pipelines	Delta Live Tables (DLT)	GA	Databricks docs
Lakeflow Jobs	Databricks Workflows	GA	Databricks docs
Lakeflow Connect	— (new)	GA (SQL Server, ServiceNow, GA4, SharePoint, Salesforce, Workday et al)	Databricks docs
Zerobus Ingest	— (new)	GA (gRPC); REST Beta	Databricks docs
Declarative Automation Bundles	Databricks Asset Bundles (DABs)	GA	Databricks docs
Serverless usage policies	Serverless budget policies	Public Preview	Microsoft Learn
Data Quality Monitoring	Lakehouse Monitoring	Profiling GA; anomaly detection Public Preview	Databricks docs
Genie One	Databricks One	GA (June 2026)	Microsoft Learn
Genie Agents	Genie Spaces	GA (July 2026)	Databricks blog
ABAC policies + governed tags	Tag policies (preview naming)	GA (May 2026)	Databricks docs
Unity Catalog metric views	— (new)	GA	Databricks docs

Appendix B: Platform Naming Constraints

The limits every pattern in this document is designed to. Review annually.

Resource	Length	Allowed characters	Uniqueness scope
Unity Catalog objects (catalog, schema, table, view, volume)	255	No `.`, `/` or spaces; stored lowercase; hyphens require backtick-quoting in SQL	Within parent object
Azure storage account	3–24	Lowercase letters and digits only	Global
Azure Key Vault	3–24	Alphanumerics and hyphens; must start with a letter; no consecutive hyphens	Global
Blob storage container	3–63	Lowercase letters, digits and hyphens	Storage account
Entra group display name	256 (practical ~40 — picker UIs truncate)	Our grammar: lowercase, hyphens; no `.` or `/`	Tenant (uniqueness not enforced — enforce in the provisioning pipeline)
Azure Data Factory	3–63	Alphanumerics and hyphens	Global
Azure SQL logical server	1–63	Lowercase letters, digits and hyphens	Global

References: Azure resource name rules, Databricks SQL names reference.

Length budgets and the abbreviation registry:

Length budget. Every naming pattern must be checked against its worst-case realistic tokens at design time. A pattern that fits corporate must also fit supplychainlogistics — if it cannot, the pattern must mandate abbreviations, not hope.
Abbreviation registry. Maintain a registry of 3–5 character abbreviations for the organisation and each domain (e.g. intuitas → intu, corporate → corp). Abbreviations are drawn from the registry only — never improvised — so that the same domain abbreviates identically everywhere. When over budget, abbreviate the domain first, then the organisation.
Separator stripping. Where a target namespace forbids separators (storage accounts), strip them; where it forbids underscores but allows hyphens (Key Vault), the registered abbreviations must not rely on them.
Validate in IaC. Include name-availability, length and character-set checks in infrastructure-as-code validation for all globally-unique resource types, and permit a single trailing digit for collisions.

Return to home

Standards and Conventions

Table of Contents

Mesh

Domain Names

Platform

Environments

VNET

Resource Groups

Databricks Workspace

Key Vault

Secrets and Secret Scopes

Entra Group Names

Service Principals and Managed Identities

Azure Data Factory (ADF)

SQL Server

SQL Database

Storage

Lakehouse Storage

Lakehouse Storage Containers

Lakehouse Storage Folders

Generic Blob Storage

Generic Blob Files and Folders

Microsoft Fabric and Power BI

Databricks

Workspace, Cluster and Workflow Names

Jobs and Pipelines

Job Names

Lakeflow Declarative Pipelines (formerly Delta Live Tables)

Lakeflow Connect Ingestion Assets

Orchestration Job Names

Job Logging

Optional Name Components

SQL Warehouses

Compute Policies

Catalog Naming and Conventions

Externally Mounted (Lakehouse Federation) Catalog Names

Unity Catalog Infrastructure

Tags and Classification

Schema and Object Conventions

Schema-Level External Storage Locations

Metadata Schemas and Objects

Raw (Bronze) (Data According to Systems)

EDW (Silver) (Data According to Business Entities)

Infomart (Gold) (Data According to Requirements)

Streaming Tables, Materialised Views and Metric Views

Databricks Apps and Genie Agents

OpenSharing (formerly Delta Sharing)

Azure Data Factory

Streaming

dbt

Documentation and Model Metadata

Sources

Model and Folder Names

Raw

EDW

Infomart

Example dbt Model Structure

dbt_project.yml and Schema Configuration

Surrogate Keys in dbt

Snapshots and PDS Historisation

Deviations from the dbt Style Guide

CI/CD

Repository Naming

Branch Naming

Branch Lifecycle

Simple branch lifecycle:

Comprehensive team branch lifecycle:

Declarative Automation Bundles (formerly Databricks Asset Bundles)

Security

Entra

Policies

Frameworks

Appendix A: Product Name Mapping (2024-2026)

Appendix B: Platform Naming Constraints