Conforming data from multiple systems...where does it happen?

Data Vault 2.0
, ,
1

Hi, I have a scenario where we have some claims data from 2 source systems representing 2 different areas of the Business. Part of the requirement is to have one table with the claims from both systems. Whilst the 2 source tables have some columns in common both have their own unique columns as well. I’ve modelled and ingested the sources in to the Raw Vault but we’ve been having a debate about where the ‘conforming’ should take place.

Personally, I feel it should be the Information Delivery layer as part of the views but Business Vault has also been mentioned. Business Vault doesn’t feel appropriate since there is no actual business logic being applied and it’s just an amalgamation of source columns from 2 table in to one.

2

Hey there Martin,

I’ve also heard talk about people making ‘mega sats’ in the business vault. aiming to conform all data into one easily retrievable space, I assume the motivation behind this is to reduce the number of joins in your delivery layer. I personally disagree with the design philosophy as I think it reduces the scalability of the vault.

But I think I’d probably suggest it’s narrowly within the standards to do something like this. It might just be a worse solution than the one that you have rightly proposed.

Frankie

1 Like
3

BV is not a copy of RV, create an info-mart with the columns you need. Using a PIT to support this and make it efficient is probably the right way to go about it

1 Like
4

That’s true, Putting the same data in RV and BV puts your single source of truth into question. More points in your favour Martin :joy:

5

Hey guys,
Where is your consultant’s attitude?
“It depends!” :joy:
If it’s a stupid UNION, I agree with @patrickcuba.
And here, some columns are in synch, some not. Is there any “decode” to apply, any “case … when”, then it’s worth bringing a BV View.
(And if you’re dealing with corporate fixes reporting like P&L or risk, anyway, materialized)

6

It does sound more like a stupid UNION, there may be some decoding in some of the other domains that we’re covering as part of the project but those alone could go in the BV.