vault_insert_by_rank to load various raw_vault tables. We have created our DBTVAULT_RANK column in our staging layers. During the base load, only RANK 1 is being inserted into the raw_table. In order to load the other ranked rows,
dbt run needs to be called again, indicating that subsequent loads are working ok. Is this expected?
Right now, we loose records with RANK > 1 when there is a base load.
I expect all ranks to be loaded during the base load. This is important so that we don’t loose records from our up-stream processes that incrementally stage data for loading.
dbtvault’s documentation explains the vault_insert_by_rank materialization to working like this:
vault_insert_by_rankcustom materialisation provides the means
to iteratively load raw vault structures from an arbitrary rank
column, created in the staging layer.
Why would the base load only load the 1st rank?
dbt version: 1.4.1
dbtvault Snowflake: 1.4.0