Clarification on src_eff: "EFFECTIVE_FROM" in Staging Files

Is the effective from the updated/modified date in a data source, assuming that it is reliable, or is it the created date, meaning that it will always be the same regardless of the load?

If the record is updated, is it the updated/modified date that is the new effective from, or have I misunderstood?

Thanks.

Hey Mark,

They have a slack channel with more product focussed support, click the slack channel invite here: https://automate-dv.com/contact/

1 Like

Hi @MarkyMark,
I assume you’re talking about the pattern “Effectivity Satellite”.
The Effectivity Satellite is about the start and end of relationships, technically and functionally. Use the ldts to determine your DWH’s knowledge beginning and end, and add functional dates delivered by the source. It’s important to separate the technical timeline from the functional/source-driven one.
Caring about this allows you to deliver data in different ways (requested) and helps you with full audibility.
I hope that helps.
For further questions, please correct my assumptions.
Warm regards
Volker

:heart::family_man_woman_girl_boy: loving people, :heart:⌗ loving data
Disclaimer: I’m not a native English speaker. If I got you wrong or my answer feels confusing, please apologize. I’m focused and kind by nature. :wink:

Hi Mark,

Yes, I think you’re correct.

The LoadDate is the audit date for the RAW DV. This records when your ETL has loaded the change row and provides auditability for the DV.

However, the Effective From Date or the Applied Date is the date at which the change row you are processing was generated in the source. So, this would be the updated/modified date.

For example, lets say the row was generated on 15/03/2025 (showing in updated/modified date) but your ETL is broken and takes 5 days to fix. The Effective Date would be 15/03/2025 but the LoadDate would be 20/03/2025.

You use your Effective From / Applied Date to identify current row and for Point-inTime requirements.

Thanks,

Carl