Formalism to model data integration across systems

[Daniel Bouchard]

Hi,

To my knowledge, there is no inter-system formalisms to model data integration across system who enable a full traceability of information exchanged between systems.

Do any of you know such a formalism?

If there is no such formalism, do any of you has to map relationship between system and the information exchanged between them? If yes, how did you do it?

Thank you,
Regards,
Daniel

[qwerty]

Could you be a bit more specific? What do you want to integrate between which systems?

q.

[Daniel Bouchard]

Let say you work for a University and you have a student management system, an HR system, a finance system and many more custom system. Many of these system live in silo with their dedicated database and application server and some of them live in a shared database and application server. There is an operational data store in the middle of them in order to support interoperability between them with master data in it. Some system have the responsibility to feed the operational data store and/or to provide data via web API, ETL or service bus.

Imagine that there is an information governance in place and that the owner of the data related to HR want to know exactly which system use his confidential data in order to be able to comply to the controller requirement.

What tool/formalism can help us to

• get full traceability of information flow
• know where is the reliable source of the information
• etc.

[philchudley]

Would NIEM work in this case?

[Daniel Bouchard]

No because I live in Canada and to my understanding, it is the US government who is using NIEM.

[qwerty]

I guess, this is simply a modeling issue. I would not expect a pattern to exist for it. Somehow we modelers/analysts need a reason for our existence. On the other hand, it does not sound like something which could not be solve in a reasonable time. The problem with this is, that you have too many constraints that differ from stakeholder to stakeholder to be formalized.

q.

[Glassboy]

I'm unsure exactly what you mean by formalism in this context as it is a philosophy term meaning  an emphasis on form over content, but I am going to take it to mean a standard, standard notation, or convention.

What you ask can be modelled in standard UML and is not hard to do in EA nor is outside standard UML modelling

Basically you create the following

• a high-level domain model or ontology that describes what everything is, i.e a user is a generalisation of natural person.  If this is a baffling concept I suggest you just check out schema.org and see if the semantic models there work for you.
• a collection of packages that contain the schemas for your applications.  Things like Active Directory are well documented, but for your other systems you may end up having to use EA to extract the table structures from the application databases.
• a mapping of your entities to their representation in the application schemas
• a series of class diagrams that use associations between attributes to show how the attributes are related.
  

I generally also create a component diagram where every application\system is a component and every sub-set of entity data is an information item (e.g. user/phone data).  I create an information flow between the two components and convey the appropriate information item on it.  I then create an asembly between the two components and describe the integration mechanism (web service, etl, batch file et al).  Then to tidy up the diagram I realize the information flow on the assembly.

NB.  Realizing an information flow on an assembly doesn't currently display, so you first draw the assembly as an association, realize the information flow, and then in the advanced menu convert the association to an assembly).

[Eve]

No because I live in Canada and to my understanding, it is the US government who is using NIEM.

It may have been developed for the US government, but it's now used much more widely than that. If a predefined shared vocabulary and standardized way of mapping it to XML is going to help you, then the only other issue would be the time taken to understand how it works.

You could also define a class model to define the shared vocabulary and use a different schema composer profile to describe and generate schemas for the messages passed between systems. The complicated part of that plan is getting everyone to agree on the base vocabulary.

[Daniel Bouchard]

Thank you all for your replies.

I will do some exploration with the strategy suggested by Glassboy.