Stage C

Conform

SP_CORE_CONFORM

Conform means: land each source’s evidence into the programme’s locked canonical model — one shared entity shape, not each system’s native schema. That is the first moment enterprise meaning is persisted. Native keys remain; lane-unique payload sits in extensions. Every assertion carries three control axes — valid, system, and warehouse — not one “load timestamp” blob.

Choosing the canonical model

You cannot conform without a target. The canonical model is that target: the overlap you deliberately lock when multiple vendors / systems / sources of the same product type describe the same subjects — typed columns in SP_CORE_CONFORM — while lane-only fields stay in extensions. It is not “copy one source’s schema.” Which entities belong is up to the implementor. Best practice: forensic lane inventories, AI promote / map / extensions drafts, then human lock. Industry standards are optional bias — not a wholesale copy into CORE.

Roles, links, and optional industry-model lenses (ACORD, FHIR, …): ER modelling. Shape gates above are non-negotiable.

Adaptors — landed source → canonical Conform

An adaptor is how a source conforms: the deterministic map from a landed datasource (adaptor input / SRC — still native-shaped) onto the locked canonical entity contract in Conform. It keeps native keys inspectable, lifts temporal axes, and parks lane-unique fields in extensions. Cleaning a lake is not adapting; rewriting meaning in BI is not adapting.

Fields that are not in the locked canon are not dropped. The adaptor parks that lane variance under extensions (namespaced per source) so SME evidence stays on the Conform assertion. Consumers can reach it through Span / Publish lineage without jumping back to the operational systems — and a later programme decision can promote a field into typed CORE without rewriting history.

CRISP is tool-agnostic: the adaptor contract is fixed; delivery tooling is a team choice. Common helpers:

Landing / ELT products (Fivetran, Airbyte, warehouse COPY, …) typically feed adaptor input. Adaptors then shape that land into Conform — do not conflate ingest with conformity. If a source stores a date range and you need a row per month, or the same person is stored twice in one feed and the birthdays must match, do that in silver when you can. If you cannot, the adaptor can consult the Rules catalogue first. Detail: Adaptor input.

Three axes — valid, system, warehouse

Split by what the interval means, not by entity. Restatement, source-assertion history, and platform ingest are different questions — they must not share one clock.

Adaptor input (SRC / L2) keeps per-source shapes until adaptors run — cleaning a lake is not conforming. Promotion from extensions to typed columns is a governed decision. Conform rows stay stable after load; Identity rebuilds separately so lag is visible.

Pseudocode

function conform_load(lane, batch):
  src = read_adaptor_input(lane, batch)     # native keys intact

  for entity in programme_entities:         # locked canonical model
    mapped = VW_CORE_{entity}(src)          # adaptor view
    # Stamp axes — adaptors supply valid/system; trusted load stamps warehouse_*
    mapped.valid_from, mapped.valid_to = lift_business_time(mapped)
    mapped.system_from, mapped.system_to = lift_assertion_time(mapped)
    mapped.warehouse_from = load_now()
    mapped.warehouse_to   = NULL

    MERGE INTO SP_CORE_CONFORM.{entity}
      ON natural_or_surrogate_key
      WHEN new_assertion THEN INSERT
      WHEN correction THEN close prior (warehouse_to) + INSERT successor

  # Programme canon columns may still be unset — Rules apply next
  return conform_batch_id

Shape Conform as an assertion model (headers, roles/links — not FACT_*): ER modelling. Lane payload and promotion: Extensions.