data Lineage and Impact Analysis

[grousseaux]

I'm wondering if it's possible to create any kind of data lineage in the tool? Here's an example of what I'd like to accomplish: Let's say we have an API that exposes data from a number of tables. Let's assume one of those values is first name. That API is connected to another API that has an ESB (Enterprise Service Bus) as the transport. Within the ESB, transformations occur on that data. Finally the data arrives at the target API, it's then passed to a Stored Procedure that does more manipulation.

The desired outcome is two part:
1. Let's say the source system First name came from is the golden record or system of record. And lets' say first name is used in many interfaces doing the same type of thing as described above across many applications. If we want to change the source system and first name will be affected, I'd like visibility to see where all First name ended up, every table and every column. Now I can easily see the impact to the change on first name.

2. To have a one stop view that for a particular interface, allows me to see the journey of the data from source to target and everything that was done to it along the way. Similar to a data mapping but more from start (app1.table.column) to end(app2.table.column) as opposed to the start to the end of the just the API path.

Currently we use the mapping component supplied by Geert and are using to map the API(ESB) source to target. I could use the same method but expand the end points... but I thought I'd see if anyone has found another way to accomplish this.

[Geert Bellekens]

Hi, that's how I've done it at other clients.

We mapped data from technical message models (XSD, JSON Schema) to the Logical Data Model (LDM), and then the columns from the database to the LDM.
We also model what we call "Logical Messages" that are linked to BPMN Messages or data objects.
These logical messages are a subset of the LDM. This is done so we can track exactly what data is used by which process and process step.

Then we also link the process steps with a RACI Matrix to Roles and with another matrix to application components.

Lastly we made an export export that combines most of this information in a handy format that can be filtered.

Geert