we are to source data from a SAP ERP system. The source system just extracts the data in the raw format as it is. meaning, 100s of columns in one interface file which has technical columns inside. the column names are ZZABC (very technical as described in SAP).
we are spending a lot of time to just understand the interface and the meaning of each column as of now as there is no clear definition about the columns in the files.
How to decide which columns to source and which ones to skip from the interface file?
Has anyone faced such issue before?
How are we supposed to name the columns and data type in the raw vault satellites? the SAP names just don’t make any sense. should I understand each column name in SAP and then give some user friendly name to it?
While SAP’s data model is hard to understand (even for German speakers) it does have good metadata and descriptions built in. Mustafa from Snowflake did a piece on this on medium the other week which is a good starting place. Please don’t start giving names yourself…