We have a reliable date received in one of our data feeds. To initially load the data, I assume it is fair game to use the date received as the load date time in Dbt Vault, and then change back to the run date.
The initial load should have the following dates:
- applied date (the date the record was captured - the date the record state is applicable* at a point in time*) - i.e. the extract date (you need to figure out a reliable date for this)
- load date (the date the record was loaded)
Data Vault should always be bi-temporal.
- This means the initialised DV will have applied date per record but the same load date for all records — because you have used either a load date parameter - or a current_timestamp() context function.
- For the regular feed, yes, load date is still that parameter / context function.
You should also append something in the record source column to show that it was a part of an initial load, that way you can easily segregate the two if needed, ex. “SAP-load_file.parquet” should become “INITIAL-LOAD-(2025-03-28)-SAP-load_file_.parquet”