Best practices for modeling of ETL chain?

[vladap]

Hi Guys,

what modelling language / best practices would you suggest me to use in EA for modelling whole chain of DB ETL? I am thinking about a combination of Archimate + Database Modeling (UML)

- Business layer (the context of the various areas and business architecture & processes)
- Application layer (architecture of applications, databases, servers, environments, how they are connected, etc)
- Data layer - here it is split in ETL and ELT sub-layers
    - Source layer- files from various subsystems, direct database connections
    - Staging layer - database with tables in production environment where source data is ingested
    - Core layer - database with tables where data is processed and accumulated 
    - Mart layer - database with tables where data sub-sets and KPIs are processed for reports
- Reporting layer (output reports related to functional areas defined in the business context, resulting from the data layer processes)

[Glassboy]

I don't think I'd use ArchiMate for that.  Probably just UML.

[adepreter]

Indeed you cannot do this with AchiMate. You need a language that can manage levels of detail.

In Labnaf, you can model your ETL chains of DBs like this: "Application Interaction Details" viewpoint.

In order to manage complexity and to enable zoom in/out, it is important to keep two levels of details.

• Level 1: Application to Application, ignoring the integration system in the middle. These applications are used as APM entries and for dashboards. They carry application attributes.
• Level 2: Component/Data Store to Component/Data Store. Here you do model the integration system

Cheers,
Alain