Master data models from information flows

[Mats Gejnevall]

In our architecture we model information flows. Now we want to create models and analyze where information is created (mastered) and where it is used.
That can be done by hand by looking at our information flow models and create new models depicting this (or creating them in the relationship matrix).
That is a lot of work!
Is there a way to “automatically” create those models/matrices from the information flows? In most cases the direction of an information flows indicates the creator (master) and the user of information.

Thanks for any suggestion
Mats

[Uffe]

Hej!

I'm not quite sure what you're asking here. You say you've got the information flows modelled, but you want them in a model?

As for relationships matrices, they're on-the-fly views, not something that gets stored in the project. You can create a matrix profile, but that's it. There's no API to manage matrix profiles, so building something that'll create profiles based on a model would require a bit of digging in the data model.

Also, since a relationship matrix only ever shows one link in a chain they're not a good tool for information flow analysis. Unless all your information flows only ever stretch from one source to one sink, in which case the analysis is trivial anyway.


/Uffe

[Glassboy]

In our architecture we model information flows. Now we want to create models and analyze where information is created (mastered) and where it is used.
That can be done by hand by looking at our information flow models and create new models depicting this (or creating them in the relationship matrix).
That is a lot of work!
Is there a way to “automatically” create those models/matrices from the information flows? In most cases the direction of an information flows indicates the creator (master) and the user of information.

Interesting idea, but what comes immediately to mind is that the data that makes up your information is likely to be captured or created in a multitude of locations that may not match the system that is authoritative for the information. 

[Nizam]

You can generate a report based on the connector direction, it will be a simple script to iterate through Data Entities and print the Source and Target.

But if you want to be assertive of which is your authoritative master / copy etc you might need to consider a slightly more detailed modeling option.

We use an extended information flow with a tagged value indicating if the Data Entity is a Master,Store,Source Of Truth , etc. if this information is captured at the information flow level, you can easily extract it as a matrix.

[Paolo F Cantoni]

Hi Mats,

If I understand you correctly, there are a couple of problems with what you are attempting to do.  The first is conceptual.  I hope you agree that the real Master data is the attribute or property, not the object or entity.  "Objects' are "Master Data Objects" because they include one or more Master Data Items (attributes/properties/features).  If you don't agree, good luck  .

Each Master Data Item has a lifetime.  Different sources can create/modify the value of the item at different points during its lifetime.  Indeed, at the same point, multiple sources may change the value.

Normally, information flows document flows between objects/entities not between the attributes.  There are ways around it, but you will need to decide what you are tracking.

Normally, you need to track which systems/business processes access the item and in what manner (CRUD).  This suggests that an ArchiMate "Accesses" relationship may be more appropriate.

We have implemented a "source of Truth" relationship which is (effectively) derived from the accesses and indicates that the supplier can be considered a SoT for the related attribute.

HTH,
Paolo

[Richard Freggi]

Sorry Mats I don't think you can be successful in this one (I'd be interested to know how it works out for you...)
Some considerations:
1. Data does not 'flow', it is queried.  Data flows are a hang-on from 1970s mainframe philosophy that stuck around because people liked the idea although it only has very limited usefulness (a bit like flowcharts).  I don's know if an architecture based on information flows can be efficient/effective/flexible (I'd be interested to learn more about this)
2. Since data is queried / provided by each table/class, each query typically contains a mix of different attributes from different classes / tables / whathaveyou + the query logic.  These are called messages (hello sequence diagram!!!)
3. Can we reconstruct a data model from a sequence diagram?  Yes with some effort and some modeler judgement / experience / assumptions, as long as the messages are between participants (I think a Data Flow Diagram maps poorly on to an interaction/collaboration diagram)

TL;DR:  There be dragons where you are going, methinks.  DAMA website has some good resources about data architecture.

[Glassboy]

1. Data does not 'flow', it is queried.  Data flows are a hang-on from 1970s mainframe philosophy that stuck around because people liked the idea although it only has very limited usefulness (a bit like flowcharts).  I don's know if an architecture based on information flows can be efficient/effective/flexible (I'd be

Rubbish.  I've documented thousands of flat files transferred between systems in one organisation alone.  There was nothing that remotely resembled a structured query anywhere.  It was a wholesale flow of data between systems that was then used to create information in those systems.

It's a very common occurrence.

[KP]

Sorry Paolo I don't think you can be successful in this one

Did you mean to address Mats, not Paolo?

[Paolo F Cantoni]

Sorry Paolo I don't think you can be successful in this one

Did you mean to address Mats, not Paolo?

I hope so because Richard seems to be saying the same as me (AFAICT).  Although I agree with Glassboy that messages are flows of data between entities.

I kept going round and round in circles (especially with my business users) until I realised that it was the attributes and not the entities that were "Master Data".  After that, we were even able to resolve the "that's MY data" problem.

I personally think that the concept of "Source of Truth" is misleading⁽¹⁾.  The implication is that there is one source of absolute truth.  Neither of which is correct.

Paolo
⁽¹⁾ And if you try to find a consistent definition on the interweb, you're unlucky.

[Richard Freggi]

Yes Paolo I got the names mixed up.  Gotta stop posting before my 2nd cup of coffee in the morning.  Original post edited with correct name (Mats).

[Glassboy]

I personally think that the concept of "Source of Truth" is misleading⁽¹⁾.  The implication is that there is one source of absolute truth.  Neither of which is correct.

It's just a symptom of people mixing data and information and not understanding the context of either.  There generally is only one source for each context.  There's a reason that Zachman had a list of "what" at the contextual level.  You need to create the list before you start looking at the lower levels; but people never do.

[Paolo F Cantoni]

I personally think that the concept of "Source of Truth" is misleading⁽¹⁾.  The implication is that there is one source of absolute truth.  Neither of which is correct.

It's just a symptom of people mixing data and information and not understanding the context of either.  There generally is only one source for each context.  There's a reason that Zachman had a list of "what" at the contextual level.  You need to create the list before you start looking at the lower levels; but people never do.

"System of Record" is another misunderstood term.  People often conflate it with "source of truth" and even "system of authorship", which may or may not (and often aren't) Systems of Record.

Paolo

[Glassboy]

"System of Record" is another misunderstood term.  People often conflate it with "source of truth" and even "system of authorship", which may or may not (and often aren't) Systems of Record.

A system of record is there to meet the requirements of a "why".  If you don't know the "why" - the contractual or legal obligation - there is no "record".

[Paolo F Cantoni]

"System of Record" is another misunderstood term.  People often conflate it with "source of truth" and even "system of authorship", which may or may not (and often aren't) Systems of Record.

A system of record is there to meet the requirements of a "why".  If you don't know the "why" - the contractual or legal obligation - there is no "record".

Can you expand on that?  I've not heard it that way before.

Paolo

[Glassboy]

A system of record is there to meet the requirements of a "why".  If you don't know the "why" - the contractual or legal obligation - there is no "record".

Can you expand on that?  I've not heard it that way before.

There are two sorts of things organisations do.  Things they want to do and things they have to do.  When you have a good look at what a system of record it is doing (in my experience) it is capturing data about entities or events that relate to something the organisation has to do.

For example (if you are using ArchiMate) at the motivation level you should have a Stakeholder and a Driver for example "NZ Police" and "Comply with suspicious transaction reporting requirements of the Anti-Money Laundering and Countering Financing of Terrorism Act 2009".  There should also be a Goal along the lines of not trigger the punitive damages associated with not meeting the obligations.  These motivational elements will all connect some how to a system of record for transactions.  In an industry like banking this system probably predates business analysts and architects fucking things up.  At some stage someone probably trained in systems analysis laid all the ground work for a mature and robust data model for transactions.

Where we run in to trouble is when legislation changes or a new concept is introduced and the design doesn't start at the conceptual layer.  You don't know why you are making a change beyond what is in the project scope document.  You end up with a system that records things, but not the record you need to meet the obligation.  Or a very fragile record.