Stage R

Rules

SP_CORE_CONFORM_RULE

The usual story: Conform has the rows, then ordered passes stamp programme values onto those same facts. The schema holds three apply surfaces — two kinds of SPEC, and one kind of specialised Rule — not a single “map + rules” blob. Letter R is that catalogue. Some of it can also be used on source tables before the write — splitting a range, checking a birthday the feed captured twice — so you do not persist facts you already know are broken.

Where Rules sits in CRISP

Rendering diagram…

Highlighted: Rules hold the programme catalogue — usually stamped onto Conform rows; some checks and splits can run on source tables first.

What surfaces does Rules use?

Three surfaces — two SPEC, one Rule

Split by shape of the join, not by business domain. Catalogue and Map share one vocabulary table but answer different questions. Specialised Rules leave that table when the outcome is not a simple core_code.

SPEC · Catalogue

Catalogue

CONFORM_RULE_SOURCE_VALUE_MAP · row_kind=CATALOGUE

Which programme codes are approved?

The closed list of words the programme is allowed to stamp — not the spelling the source used.

CLAIM_PAYMENT

An approved kind of money movement — not the claim itself.

SPEC · Map

Map

CONFORM_RULE_SOURCE_VALUE_MAP · row_kind=MAP

Which catalogue entry does this native literal belong to?

This source’s spelling of that word, usually per system. Same letters in another feed can mean something else.

“PAID” on the claims PAS → CLAIM_PAYMENT

Same word in another system might mean something else — the map is per source.

RULE · Specialised families

Specialised families

CONFORM_RULE_FINANCIAL_SIGN_RULE (flagship)

How should this amount be interpreted for the enterprise?

Leave the vocabulary table when the outcome is not a simple approved code — sign, stage, a Publish flag, a split, or a check across rows.

Claim payment of 500 → enterprise −500

Native 500 stays on the row; reports add the signed column.

Orchestration only: CONFORM_RULE_APPLY_ORDER sequences which surface runs when — not domain content. Rejected patterns: one physical SPEC per domain, or one giant table for Catalogue + Map + sign + lifecycle together.

How are catalogues and maps stored?

Vocabulary table — row kinds (SPEC)

Catalogue and Map are row kinds in CONFORM_RULE_SOURCE_VALUE_MAP — same table, different jobs.

row_kind	What it does	For example
CATALOGUE	The approved programme tokens for a scope — not the spelling the source used.	PRODUCING_BROKER is allowed; “PBR” is not the programme word.
MAP	This source’s spelling → that catalogue token, usually per source system.	Primary PAS sends “PBR”; the map stamps PRODUCING_BROKER on the role.

What does a vocabulary scope mean?

Vocabulary scopes — type vs entity

Types ask “which approved programme code does this literal belong in?” — not “are these two natives the same subject?” (that is Identity). spec_entity names a vocabulary scope (e.g. PARTY_ROLE_TYPE), not “the party domain’s SPEC table.”

Kind	What it does	For example
spec_type	Reusable on many Conform columns.	United Kingdom, GBR, and UK all become GB on every location.
spec_entity	One attribute family; still rows in the same map.	“Named insured” and “NINS” both become NAMED_INSURED on the role.

When is a specialised rule needed?

Specialised rule families (RULE)

Add a specialised family only when the join is not a simple native spelling → approved code. Signage is the teaching example: times plus or minus one. Lifecycle needs several facts on the row at once — the same source word on two different events. Measure eligibility stamps a boolean or enum on the row; Publish later filters on it. Splitting source rows and comparing across them share this catalogue but often run before the write. Status codes, aliases, and timezones stay in Catalogue/Map — they are not extra families.

Family	What it does	For example
FINANCIAL_SIGN CONFORM_RULE_FINANCIAL_SIGN_RULE	Turn a native amount into enterprise-centre money (times +1 or −1).	A claim payment of 500 is an outflow — stamp −500 so every report adds the same way.
LIFECYCLE CONFORM_RULE_LIFECYCLE_RULE	Programme stage when the same source word appears on two different events — you match more than that word.	Two money rows both say PREMIUM. One is the bind (written) — technical debt. One is the later bank receipt — cash. Mapping PREMIUM alone cannot choose.
MEASURE_ELIGIBILITY CONFORM_RULE_MEASURE_ELIGIBILITY_RULE	Stamp a boolean or enum on the Conform row so Publish can filter. The row does not move.	Both technical-debt rows stay in Conform. Written premium is flagged TRUE for GWP; the other is FALSE. Publish only sums the TRUE rows.
GRAIN_EXPANSION CONFORM_RULE_GRAIN_EXPANSION	One source row → several (prefer silver). Keep a pointer to the original.	Cover from January through March arrives as one row; the programme needs a row per month.
INTEGRITY_CHECK CONFORM_RULE_INTEGRITY_CHECK	Compare across rows — on source tables first, or on Conform if the check needs programme codes.	The same person is stored twice in one feed — birthday on the customer, again on the policy. Do the dates match?

Where do those last two families run? Learn more about Adaptor input

Grain expansion and integrity — where they run

The last two families in the table above — grain expansion (one cover row → a row per month) and integrity (birthday on the customer vs on the policy) — are still Rules. Prefer to run them in silver, on the source tables adaptors already read. Keep those tables as flat as the landing zone: the source’s keys, plus a note if you split a row.

If silver cannot change, the same instructions sit in the Rules catalogue. The adaptor reads them and fans out (or holds bad rows aside) just before Conform. Conform is then where the split lands as enterprise facts — January, February, March — not Identity, not Span. A check that needs programme codes or signed amounts waits until after that stamp, still on Conform. Detail: Adaptor input.

Where grain expansion and integrity run

Preferred — silver (source tables)

Cover January–March is still one source row. Split it here. Compare the two birthdays here. Conform then just receives what is already safe.

One cover rowJanuaryFebruaryMarch

Adaptor writes Conform — three enterprise rows, already checked

If silver cannot — Rules, then Conform

Silver stays one row per source row. The Rules catalogue still says “split this range” and “these dates must match.” The adaptor reads that and fans out just before the write.

Silver

Still one cover row

Rules

Split · check — adaptor reads this

Conform

January · February · March land here

Failures stay with that source. A check that needs signed amounts waits until after that stamp — still on Conform, still Rules. Not Identity, not Span.

Prefer to split and check in silver. If silver cannot change, the same Rules drive the adaptor: one cover row fans out into three Conform facts. Identity and Span do not do this split.

What else can Rules stamp?

Programme scopes (still Catalogue/Map)

Timezone lift and optional Identity eligibility are extra scopes in the vocabulary table — not extra families.

Scope	What it does	For example
source_temporal_timezone	How adaptors lift a source datetime that has no zone.	A London PAS stores noon with no offset — lift it as Europe/London, not as UTC noon.
cluster_eligibility	Optional funnel — which natives may enter name-match Identity.	Skip person rows from a mixed feed that should not enter organisation name-match.

What must stay true for Rules?

Theory

Catalogues and maps are release-controlled and inspectable. Pipeline code must not silently invent enterprise tokens.
Pass order matters: pass-2 MAP rows often scope by canonical source_system stamped in pass 1.
Delivery tooling may inline joins into INSERT … SELECT or dbt intermediates. Stamping catalogues onto Conform rows still follows the write; shaping or checking source tables can consult the same store first.
When catalogues or rule families change on existing rows, production needs MERGE or governed re-apply — not silent drift on historical assertions.

What does apply look like in code? Learn more about Governance

Pseudocode

Ordered apply — Catalogue / Map SPEC, then specialised Rules

# SP_CORE_CONFORM_RULE — three surfaces
order = SELECT * FROM CONFORM_RULE_APPLY_ORDER
        WHERE enabled ORDER BY apply_sequence

for pass in order:
  match pass.spec_kind:
    FEED_TO_CANON:                    # Map SPEC (source_system)
      # MAP: extensions.source_feed_code → fact.source_system
      UPDATE conform SET source_system = map_feed(extensions)

    SOURCE_VALUE_MAP:                 # Catalogue + Map SPEC
      # CATALOGUE = approved core_code set
      # MAP = native_token → core_code (scoped by source_system)
      UPDATE conform
      SET typed_col = MAP.core_code
      FROM CONFORM_RULE_SOURCE_VALUE_MAP MAP
      WHERE MAP.row_kind = 'MAP'
        AND MAP.scope_source_system = conform.source_system
        AND MAP.native_value = conform.native_token

    FINANCIAL_SIGN | LIFECYCLE | MEASURE_ELIGIBILITY | …:
      # Specialised RULE families — not the source-value map
      apply_family_join(pass.spec_rule_family)

mark_load_ready_for_identity()

Next: Identity