Does it makes sense to use a Vault with an Iceberg, Hudi or Snowflake lake or warehouse?

With the advent of Delta Lakes and other similar structures, does it still make sense to have a date metadata information in all the tables? I can now do all the As-Of queries etc using built in capabilities.

As that’s the case, why would I need those date columns in a new / greenfield Vault?

Depends on what that date column is… remember, we have two metadata date columns in DV

  • Applied Date - as the date the data was captured from the source, the “applied” date of the state of the business rule from the source app/system

  • Load Date – when data was loaded to a DV

Which date mostly resembles the date you’re looking at? Is it the part_date you are referring to?