I am working on an increasingly complicated Data Vault and docmentation is starting to become more and more clumbersome.
I was wondering how other people are doing it.
I have been using Figma to create a visual representation of how things (sats, hubs and links etc) connect.
if you are using tool such as Erwin, you can document in the canvas and in the objects.
Or if you are using lucid or draw then manual entry
Tools like dbt have documentation capabilities built-in which allow you to document columns, tables/views, tests and more. Of course not everyone uses dbt, but it’s a good approach/model for documentation.
If you’re looking specifically at diagramming, dbdiagram which uses their own DSL called DBML is a great tool that allows you to create diagrams as code, which makes it easier to version control them and share them with your team. Python libraries are available to handle DBML as well, which provides automation to an extent.
Other diagram-as-code approaches like Mermaid are available, which supports ERDs as well.
Enterprise tools like sqlDBM are also fantastic offerings.
Of course both of these approaches rely on you and your developers maintaining the diagrams, so often at scale the way to go is generating and automating the process. For example, generating your diagrams from your implementation and also generating your implementation from your diagrams.
As mentioned, tools like Erwin, WhereScape, VaultSpeed etc all have capabilities to do this (and specialist features for doing this in a Data Vault context), however these are enterprise tools and may not suitable for your specific circumstances.
Some excellent points above but one technique I’ve used in the past is creating concept models and initial diagrams within Draw.io (diagrams.net) and then storing that within a git repository alongside the rest of the project to create a version control almost since it is XML underneath.
You can also create ERDs with the SQL plugin as well which is documented here. This is useful when you do actually have Data Vault structures in place and can get some DDL to document.
It is not quite as sophisticated as some mentioned above but it really depends on your circumstances and how comfortable you are with the tool.