all, currently we are using Azure data factory for data ingestion/orchestration and facing an issue, where ADF is not able to lookup on Binary data (may be due to limitation) for comparing to determine whether the hash key exist. As an interim solution, we have decided to change the datatype of HK column to Varchar instead of Blob/Binary. Any other suggestions/recommendations.
binary data type for hash keys enforces good data architecture practices… your hash-keys should never leave your chosen data platform.
How can be resolve ADF??
That’s your job mate — i’m just telling you that you’re obviously pulling data into another platform (ADF) to do joins; that’s an anti-pattern even outside of a data vault.
We’ve seen customers complain about the same thing when using PowerBI or Talend, my response was: “why the hell are you pulling hash keys into those platforms?”.
Hash-key and hashDiffs are nothing more than tools to help you bring your data together in-platform (in-db, or aka predicate pushdown).
If you’re pulling DV tables into ADF to do joins you’re doing DV wrong.
1 Like
What are the implications or issues of choosing between binary hash vs Char/Varchar??
