Multiple projects sharing/working on a common baseline

[Modesto Vega]

Indeed, it all about communication and what in a post catches the attention of people. In hindsight, I have also been guilty of "misspeaking", baseline is much quicker to write than our “(current knowledge of) the present state” (substitute present with As Is if you wish). My apologies inciting an almost ideological discussion when I am after a practical discussion.

By the way the qualification “current knowledge” before "present state" is quite important as we don’t have the present state fully codified in our repository and some projects, if they are transformational in nature, have a component (please don’t think, in UML terms, UML) aimed at discovering the "present state". Architecture can be a bit like map mapping, surveying existing landscapes and not just building new landscapes.

Since my misspeaking seems to have sent the thread into an unintended direction, I am happy to settle with the "Current knowledge of the present state" (or with any abbreviation that anybody reading this may come up with).

In my opinion, there are really 3 sets of questions at hand here:

1) When does a target state becomes the present/As Is state? How long does it take to become the present/As Is state? How do you bring the target state into the "Current knowledge of the present state"?

2)There are plenty of complex of organisations and products out there running concurrent projects with significant architectural dependencies that will all result on a significant change to the present state, I am sure that I am not the only one in this forum that faces or has faced this situation. The specific set of questions, in addition to those above, are:

How to handle projects aimed at introducing something that is going to be used/consumed by multiple other projects? How to handle projects aimed at replacing something that is used/consumed by existing systems (including when the existing systems are not part of the "current knowledge of the present state"?

3) If a project does not become part of the present state, what do be do with it? [Note: Please keep in mind that the project may have built nothing that gets deployed but may have improve our current knowledge of the present state.]

I think everybody agrees that you can't have a "free for all", where anybody can change anything at any time.  We also agree that we wish to have a single "model".

Agreed, although reading this thread I have doubted it sometimes. Some of the replies could be construed as a "free for all", no single "model" needed endorsement. This is because the lack of engagement about the practicalities about how do this with Sparx EA.

Of course, Sparx EA does not have magic bullets but there are many Sparxian technical devices that can be used to create a repository and those "doppelgangers" and reconcile them with their ancestors. Firstly, as the "baseline" misspeaking above clearly indicates the structure of the repository, in terms of "folders", is likely not to be irrelevant.

Secondly, Sparx EA seems to offer 3 technical devices to create those "doppelgangers" and trace them back to their "ancestors":

• Transforms, which we use to move from logical to physical, physical to logical and, partially, to move from conceptual to logical
• Dropping an elements as an Instance of an Object
• Just let everybody create their own elements an carry out a periodical reconciliation

Thirdly, some of the new elements are not "doppelgangers". They are actually new and need to be brought into the present state.

I don't think anybody has commented on repository structures, technical Sparxian devices to improve efficiency and traceability, or any of the 3 sets of questions above.

And, finally, many thanks to Paolo for his assistance of trying to refocus the thread.

[Glassboy]

Yeah, I was replying to Modesto.  I do object to your use of "future state" tho.  Projects work on a target state.  They may be lucky and model one of the possible futures, but generally they don't.

Yes, my bad, so used to misspeaking in this area.  Just to make sure I understand your point, while each project may be working on states which might exist in the future, they are not describing the future start since we are always in the present.  They are only describing the target the project is aiming for.   Have I understood correctly?  If so, I'll amend the original text appropriately (I hope).

In terms of the models we create the "baseline" is an attempt to capture what exists now, for some value of now and some value of exists.  Most baselines suffer from epistemological issue, for example what is an application or or a system.

Projects generally have business cases and requirements and all sorts of other malarkey.  At some point someone - usually a solution architect - will decide that if one set of blocky shapes are replace with another set of blocky shapes, the promises inherent in the malarkey will be fulfilled.  This dream gets sold to a whole bunch of people and off everyone goes.  They take their target architecture and start building.  And time passes.

At some stage the project will end.  Either they create the dream, or they run out of money, or the powers that be stop loving the dream.

If the solution goes live, and you create another "baseline" you'll find that it does not match the original baseline plus the projects future state.  Not only has the present been advancing, but people who have been doing the building have been dealing with the real world practicalities of creating something.  There's a whole lot of blockey shapes changed, missing, or not recorded after the project finishes.

My preferred use of Sparx EA is that a project gets a copy of the the baseline, and the copies have a derived (or other easily ignorable) relationship between the original and the copy.  Once the the project has been completed, someone not involved in the project, creates a new baseline using the project material only as a reference.  The project material is made read only (as are all the other project records in your EDRMS et al).

There are a few reasons I like this approach.  For example if you are involved in a project that has staff impact implications and you don't want possible targets known until one has been selected and consultations blah blah have happened, you can work in an entirely separate repository.

If a project is stopped or failed, no damage has been done to your baseline (you can even severe all of the relationships).

If there is an IQA process run on the project it is very easy to show what the project did, and what the real impact on the enterprise was.

[Glassboy]

1) When does a target state becomes the present/As Is state? How long does it take to become the present/As Is state? How do you bring the target state into the "Current knowledge of the present state"?

See my answer to Paolo, but target states never become present states.  For two reasons.  Your "as is" is an incomplete snap shot of the present, and the present always changes.  And target states are aspirations.  The future state is the current state plus what was actually done, and all of the background change that impacted on trying to get to the target.

[Paolo F Cantoni]

Perhaps, some (hopefully) clarity might be in order...

Each item in the repository has a set of properties which represent its "state".  How we interpret its state depends on its purpose.  If it is supposed to represent the physical item as it presently is, then it is a present state item.  If it represents how we intend the item to be at the completion of some work package then it is a target state item.  Note: the target state has an implicit timepoint for the completion of the work package.

Now, from Glassboy's input, there is a utility in "freezing" that target state so that progress towards the target state can be measured.  The Frozen Target state is handled by locking it.  We, therefore, need an "in-flight" item that can be changed as it evolves toward the Target.  Getting complicated, isn't it?

Unlike Glassboy, I favour having every project in the same repository; with the exception highlighted (that of a project requiring privacy).  In the case of a project that has to remain private, they work off a snapshot clone as recommended by Glassboy.  We can keep the Enterprise present state in the clones updated through queries.

With the effluxion of time (such a lovely phrase), as we have all said, the enterprise present state item, and the in-flight items evolve.  If the in-flight item is a doppelganger of the enterprise items, they may be compared and contrasted.  The in-flight items can also be compared and contrasted with the target item.

When the project completes, and there was a production outcome, the in-flight item can be compared and contrasted with the enterprise item and the two consolidated (we have a process that does a significant portion of this automagically).  This is known as "promoting" the project item to the enterprise.  If there is no pre-existing enterprise item, we create an enterprise master and convert the in-flight to a doppelganger (which is also linked to the target item).

So, at the end of this (successful completion) process, each item in a project has a total of three copies; a master in Enterprise and a target and in-flight in the project.  The project is then locked down and eventually archived.

Other project completions can be locked down (if needed) and either archived or deleted.

A key concept in this scenario is that there is NO future state, there are a present state and target states.  We can create and maintain previous (present) state histories through creating snapshots at points in time.  For example at the calendar or financial year-end.

Thoughts?
Paolo

[Geert Bellekens]

Perhaps, some (hopefully) clarity might be in order...
<snip>
Thoughts?
Paolo

I think the idea might be somewhat OK in theory but it still doesn't address the problem of two project modifying the same artifact.
They both clone the artifact from the "current state", but one project will go into production before the other, modifying the "current state".
When the time comes the other project is trying to go into production, they will notice that their assumption of the current state has been made invalid.

You don't have this problem when you work on a single artifact. Both projects modify the exact same element and will notice that another project is doing work on that same element as well. At that moment they will need to get together and communicate with each other to find a solution. That is already hard when all are working on the same project, with the same elements,  but it is almost impossible if each project is working on it's own island, with clones of the actual element.

Another issue I have with this approach, the "when the project completes" stage.
I've seen this way of working in a number of organisations, and in each and every one of them this "when the project completes we'll merge the changes to the baseline" stage is almost never executed.
When a project is complete, the people that should do this are simply already doing other stuff and can't be bothered with something like this.

I know, it's an organization and processes issue, but that is what I have observed.

Look good in theory, never seen it work in practice.

Geert

[Modesto Vega]

I'll try to comment later on other aspects of the latest posts but, in terms of how to organise a repository, there are 2 options clearly emerging here:

• Let everybody work on the same items in a common repository (and dare to say monolithic structure) - This appears to be the approach championed by Geert
• Create a copy (partial or full) of the present state for each project and promote to the present state when needed - This appears to be approach championed by Paolo

We trialled option 1 and found that, given the complexity of the organisation and the resulting large size of the repository,

• most people using the repository, often including myself, get lost and have too deal with too much unnecessary noise.
  This appears to be because Sparx EA does not allow us, AFIK, to create a "dynamic" project view of the model - with this I mean a view the only displays those elements in the repository relevant to a project. Please note that somehow I was expecting a Model View to this but it does not. Please feel to improve my knowledge if I got this wrong.
  
• We cannot use group security/locks. This means that I have to rely on the assumption that all users are top Sparx EA players (I know they are not playing at the same level, and I also know that, in practical terms, education and awareness has its limits).

We are looking at the 2nd approach but of course this presents a number of challenges:

• First and foremost, we do not want to give each project a full copy of the present state. They don't need it, the repository contains a lot of information that is not relevant to most specific projects - e.g.,somebody working on a Customer Loyalty project does not have to know about the intricacies of HR, Accounting or Supply Chain Management systems.
  The challenge with this approach is therefore how to give each project a snapshot they can work on.
• Secondly, at some point we need to promote the outputs of projects to the present state, this can happen is projects do not go live because some projects add to our knowledge of the present state of architectural affairs. The challenge here is to do this in as much of automagical way, to quote Paolo, as possible, this means that a)we need to have the right relationships between elements and b) the absence of relationship means an architectural gap.

[Geert Bellekens]

Yes, I think you summarized it very well.

The "getting" lost can usually be solved by structuring the repository.
In general our structure is something like:

- Business Model
  - Business Processes
    - Domain A
    - Domain B
    - Domain C
- Functional Model
 - Use Cases
    - Domain A
    - Domain B
    - Domain C
- Technical Model
 - Technical Data Model
   - Database X
   - Database Y

Each user has a particular role, and knows which part of the model is relevant for this role. A Business Analyst uses the Business Model where a Functional Analyst uses the Functional Model.

We use locking with the option "require user lock to edit", so no group locks. In theory every user could lock a any part of the model and start changing stuff, but I'v never seen an issue where a Functional Analyst started changing business models, or a Business Analyst started updating use cases.
They all stay in their own corner, and the "Require user lock to edit" option raises awareness that they are about to change something, so there are no accidental mistakes.

I've used this way of working at multiple clients that have lots of users (50 and more)

One key component that we added ourselves is some kind of change management, where we mark the changes we make with a change request, user, date and comment. This allows other users to see who last changed an item, and when or if this change has made it into production by looking up the change request in the ALM system.

Geert

[Modesto Vega]

Yes, I think you summarized it very well.

The "getting" lost can usually be solved by structuring the repository.
In general our structure is something like:

- Business Model
  - Business Processes
    - Domain A
    - Domain B
    - Domain C
- Functional Model
 - Use Cases
    - Domain A
    - Domain B
    - Domain C
- Technical Model
 - Technical Data Model
   - Database X
   - Database Y

This is a structure I am familiar with and have used in other organisations. However, it does not seem to work here.

2 questions:

• Aren't you relying on each element belonging to one domain and one domain only?
• In this approach, who is responsible for the domain architecture?

Also please define a domain for me?

[Geert Bellekens]

Yes, an item always belongs to a single domain. (we do have domains such as "cross-domain" however  )

A domain is a "business" grouping of related things (often related processes)
Think things like Accounting, Inventory, Rolling Material,...
For example our Business Analysts are divided over the different domains. They are considered to be the SME (Subject Matter Expert) on anything in that domain.
A BA for the domain Accounting will only change business processes in the domain Accounting, although he might be familiar with business processes of other domains (because he needs to call them)

Most models have between 4 and 12 domains.

Above that we have the Enterprise Architect who decides which domains exist, and guards what belongs in which domain.

Geert

[Modesto Vega]

Yes, an item always belongs to a single domain. (we do have domains such as "cross-domain" however  )

A domain is a "business" grouping of related things (often related processes)
Think things like Accounting, Inventory, Rolling Material,...
For example our Business Analysts are divided over the different domains. They are considered to be the SME (Subject Matter Expert) on anything in that domain.
A BA for the domain Accounting will only change business processes in the domain Accounting, although he might be familiar with business processes of other domains (because he needs to call them)

Most models have between 4 and 12 domains.

Above that we have the Enterprise Architect who decides which domains exist, and guards what belongs in which domain.

Geert

When I read the above for the 1st time, I though you were arguing that non cross-domains are similar as the 4 columns in the Zachman Framework (why, how, what, who, where and when) and non-cross domains are similar to subject areas (typically based on Porter's value chain).

Having re-read your post, I don't think you did not mean this. Furthermore, looking at your examples I am having trouble picturing how you can have each element is a single domain. Let's look at a practical example, if I were going to apply your approach to a big e-commerce site I cannot quite picture how you are going to have the Customer in just one domain. The Customer, as an actor, spans multiple domains: Registration and Identity Management, Ordering (aka Placing and Order), Billing, Order Fulfilment, Returns, and Customer Relationship Management. Furthermore, I would also see an Order spanning multiple domains.

As a result, I have trouble visualising how you can achieve the one-element-per-domain principle with the following structure:

- Business Model
  - Business Processes
    - Registration and Identity Management
    - Ordering
    - Billing
    - Order Fulfilment
    - Returns
    - Customer Relationship Management
- Functional Model
 - Use Cases
    - Registration and Identity Management
    - Ordering
    - Billing
    - Order Fulfilment
    - Returns
    - Customer Relationship Management
- Data Models
    - Registration and Identity Management
    - Ordering
    - Billing
    - Order Fulfilment
    - Returns
    - Customer Relationship Management
- Technical Model
 - Technical Data Model
   - Database X
   - Database Y

P.S.: This is not a very complicated architecture.
P.S2: You are a very lucky professional if all your projects are so neatly partitioned, most of mine at least 2 domains.

[Glassboy]

So the last few posts in this thread show why people's modelling (and projects) fail.  You've taken a bailiwick and called it a domain and started to believe you can achieve the dream that way.  Once again you've build epistemological issues into the core of your modelling.  And with BAs, architects, and business SMEs all looking at your "domain" through different lenses, your vision will resemble a child's kaleidoscope, all set within a scope of an EA who knows very little about most things (even if (s)he is T shaped).

A domain needs to be specific and defined, as it is in DDD, problem domain analysis, or domain theory.  Not marketing or organisational jargon.

[Geert Bellekens]

Yes indeed, Actors are often not divided into those same domains (although some actors are only used in a single domain)

The same can be said for partner roles in the business models.

Geert

[Geert Bellekens]

So the last few posts in this thread show why people's modelling (and projects) fail.  You've taken a bailiwick and called it a domain and started to believe you can achieve the dream that way.  Once again you've build epistemological issues into the core of your modelling.  And with BAs, architects, and business SMEs all looking at your "domain" through different lenses, your vision will resemble a child's kaleidoscope, all set within a scope of an EA who knows very little about most things (even if (s)he is T shaped).

A domain needs to be specific and defined, as it is in DDD, problem domain analysis, or domain theory.  Not marketing or organisational jargon.

I don't care if you call it a domain, or a business grouping, or a kangaroo.
It is essentially nothing more than a way to group the model into logical parts. I think about it as more of a practical nature (we can't just chuck everything in a single bin, that would just be too much).
If there are too many processes, usecases, or "things" in a domain, we divide it up further into subdomains, and if needed sub-sub-domains.

What we don't do is attach special meaning, or theories to it.

The only thing we care about is that our model helps achieve our goals, which are usually about developing and maintaining working software.

Geert

[Paolo F Cantoni]

[SNIP]

The only thing we care about is that our model helps achieve our goals, which are usually about developing and maintaining working software.

Geert

(my emphasis)

Well, in the last few organisations I've been in, we didn't develop much software, but we were very interested in managing how the many systems (sometimes many hundreds) (most often COTS) interacted and how to make sure the business got the right information at the right time for the right agent.

As I've said before, Sparx EA isn't just a product, it is a platform and can be used for a variety of scenarios.  Modesto needs to say what his context is.  I've been describing the usage I indicated.  If you are mainly software development based, your needs are different and a different structure and behaviour will apply.  Sparx is flexible enough as a platform to allow both modes to operate.

Paolo

[Glassboy]

Well, in the last few organisations I've been in, we didn't develop much software, but we were very interested in managing how the many systems (sometimes many hundreds) (most often COTS) interacted and how to make sure the business got the right information at the right time for the right agent.

or even Infrastructure as Code