How does one go about implementing a Business Vault using dbtvault ? I have some business (soft) rules that can be nicely implemented as dbt models but I am wondering how they should be configured and orchestrated.
We create a model holding a particular business vault rule’s SQL code. This should generate a staging layer table. Add to the SQL additional fields to make up the metadata needed for the staging layer - record source (the business rule), load datetime, etc.
That staging layer table is now ready to load straight back into the Data Vault by first priming the stage (adding any hashdiffs or other metadata columns not yet calculated) and mapping using hub, link, satellite or other models as needed.
With dbt dependencies ({{reference}}) we now have a chain - Raw Vault load, followed by the business rules and further Data Vault loads. If business rules depend on the output of other business rules add further layers of dependencies as needed.
Further, it’s logical to equate 1 Business Rule to 1 dbt Macro.
If put your logic for a business rule into a macro, your team can re-use the macro to assign the same business rule. If needed, you could put checks in to throw exceptions if they’re using it incorrectly, and more. Having the business rule in a macro also makes it far more test-able.