vault_insert_by_rank base load only inserts rank 1 on sat

AutomateDV (formerly known as dbtvault)
1

What’s Happening
We’re using vault_insert_by_rank to load various raw_vault tables. We have created our DBTVAULT_RANK column in our staging layers. During the base load, only RANK 1 is being inserted into the raw_table. In order to load the other ranked rows, dbt run needs to be called again, indicating that subsequent loads are working ok. Is this expected?

Right now, we loose records with RANK > 1 when there is a base load.

Expected Behavior
I expect all ranks to be loaded during the base load. This is important so that we don’t loose records from our up-stream processes that incrementally stage data for loading.

dbtvault’s documentation explains the vault_insert_by_rank materialization to working like this:

The vault_insert_by_rank custom materialisation provides the means
to iteratively load raw vault structures from an arbitrary rank
column, created in the staging layer.

Why would the base load only load the 1st rank?

Environment
dbt version: 1.4.1
dbtvault Snowflake: 1.4.0

2

Hi. Thanks for your report and a big welcome to our forum :slight_smile: This is a technical bug report for dbtvault so would be better submitted on our GitHub in future, however on this occasion I can tell you this is not a bug, but a well-documented feature of how materialisations work in dbt.

This is also described in the dbtvault documentation here: Materialisations - dbtvault

EDIT: This has brought to our attention that this specific detail is in the vault_insert_by_period materialisation section of the documentation, however it applies to the vault_insert_by_rank materialisation as well. We will update the docs to clarify this!