!!PLEASE HELP!! BAD problem with replicas

[Aaron B]

No, I have not heard anything from Support on this topic or on its predecessor topic of some time ago

We have been responding to your emails, but have not received any further correspondence from your end, nor have we received any delivery failure emails.  Please check your inbox (and perhaps your junk filters) to confirm whether you have received our emails.

Yes, I know, I let things get a little off-topic, sorry about that.  The one thing that I really need to know is, when doing a replica resync to the design master, and I get the inevitable conflicts (and I know and accept that I will get them from time to time), how can I go about fixing or even finding the conflicts?  I get this cryptic list of database tables and record and who-knows-what, and an invitation to select the original or overwrite with the conflict, and I have absolutely no idea what any of it means or how to select the right one.  There really has to be a way to relate the conflict to a point on the diagram so that I can compare the original vs the conflict and know what I'm doing when I select the one I want to keep.  If I can solve that problem then things should be OK.

Replication is performed at the database level (it is actually a function of the Microsoft JET database engine that is used by .EAP file repositories).  EA cannot provide any visual representation of the conflicting changes.  It's possible that some of these 'cryptic' changes you are seeing are changes to the diagram layout, which can be difficult to interpret at the database level.  In these cases you may just need to pick which model you believe has the most reliable copy of the diagrams (the replica, or the master).  Having the owners of these replica models involved in this process may help if they can comment if they made any meaningful diagram changes.

As specified in the documentation, it is advised that potential conflicts should be avoided by having your users work in different areas of the model.  Also when merging replicas and encountering conflicts, the respective developers who own the replicas should be present during the conflict resolution process to help advise about the changes.
http://www.sparxsystems.com/uml_tool_guide/uml_model_management/replication.html

As suggested earlier by Geert, using the XMI-based Baselining, Differencing and Merging capabilities provided by EA might work a lot better for you than Replication and we would advise trying this after resolving your current replication conflicts.  XMI can be exported from each model, which can then be directly compared to your 'master' copy of the model and selectively merged in.
http://www.sparxsystems.com/uml_tool_guide/uml_model_management/baselinesanddifferences.html

If problems persist, it's possible that we may need to arrange for copies of your master and replicas to be sent to us for further analysis, but we would not be able to identify specific changes that you may wish to keep or discard and would likely just elect to overwrite the changes to the master with the conflict found in the replica.

[EricP]

Good evening, Aaron.

Sorry, but I'm going to have to split this response into two parts to conform to the forum limit on message sizes.

We have been responding to your emails, but have not received any further correspondence from your end, nor have we received any delivery failure emails.  Please check your inbox (and perhaps your junk filters) to confirm whether you have received our emails.

I have gotten exactly zero responses from Sparx on this topic or its predecessor.  None.  I have gotten responses from you on other topics.  When I post something to the registered users support request form, I always get back the email from [email protected] that says "Your Registered Support Request has been sent to Sparx Systems.  We will endeavor to investigate the problem and get back to you as soon as possible."  I always get those, without fail.  And I always get the emails from this form that tell me that a reply has been posted.  So since I'm getting all of those emails, like clockwork, I absolutely can't imagine a reason why I'm not getting your responses to my emails.

I went through my entire email database including the spam box and there was nothing from Sparx that addressed this issue, other than the auto-generated responses indicated above..

Replication is performed at the database level (it is actually a function of the Microsoft JET database engine that is used by .EAP file repositories).  EA cannot provide any visual representation of the conflicting changes.

Tell me this... EA draws diagrams based on what is in the database.  Why, then, can't EA draw a diagram showing the nonconflicted parts in black, and the conflicted parts in blue (for the design master) and red (for the replica that's causing the conflicts)?  I could then just breeze through the diagram and delete the blue and red parts that I don't want, and keep the rest.

It's possible that some of these 'cryptic' changes you are seeing are changes to the diagram layout, which can be difficult to interpret at the database level.  In these cases you may just need to pick which model you believe has the most reliable copy of the diagrams (the replica, or the master).  Having the owners of these replica models involved in this process may help if they can comment if they made any meaningful diagram changes.

Any diagram of any complexity at all is subject to conflicts at the re-sync operation.  Those conflicts may occur in two, three, or more separate parts of the database.  I can have an army of software engineers sitting with me to do the conflict resolution and there is no way to know what specific conflict goes with what part of the diagrams.  Without that knowledge, or at least a hint (like the name of the drawing where the conflict is found), any attempt to "believe (which model) has the most reliable copy of the diagram" is a coin toss at best.  Anyway, they may both be equally reliable... they just may reflect different designers' ideas of "reliable".

As specified in the documentation, it is advised that potential conflicts should be avoided by having your users work in different areas or the model.  Also when merging replicas and encountering conflicts, the respective developers who own the replicas should be present during the conflict resolution process to help advise about the changes.

We did that.  Each of us worked on a different part of the model.  We got conflicts (lots and lots of them) anyway.  As for having the owner of the replica here during the conflict resolution process, how can that help if neither of us has any idea what conflict, as listed in the conflicts table at Tools->Manage .EAP files->Conflict Resolution, goes with what part of the diagram?  Like I said, a coin toss or Jan-ken-pon session, at best.

<END OF PART 1>

[EricP]

<PART 2>

As suggested earlier by Geert, using the XMI-based Baselining, Differencing and Merging capabilities provided by EA might work a lot better for you than Replication and we would advise trying this after resolving your current replication conflicts.

If I interpreted Geert correctly, XMI-based differencing and merging requires that the model be arranged in packages, right?  We didn't do that, largely because I never was able to get a resolution to an earlier problem, on another project for another client, where I was reverse engineering a complex and poorly written directory full of files, ended up with the expected huge mess when I imported all of that into EA, tried to figure out a way to package things while maintaining connections, and could not do it (and couldn't get a workable solution from Sparx).  On this project, I am using what someone else on this forum called the "kitchen sink" diagram that contains everything, and then other diagrams that break things out and implement classes as links to the kitchen sink diagram.  I suppose those other diagrams could be packages but I have no idea how to maintain the necessary connections between the packages.  Anyway, it's too late to redesign the whole thing now.

If problems persist, it's possible that we may need to arrange for copies of your master and replicas to be sent to us for further analysis, but we would not be able to identify specific changes that you may wish to keep or discard and would likely just elect to overwrite the changes to the master with the conflict found in the replica.

I can do that myself, but I can't just "elect to overwrite... with the conflict..." because just as often, we'll find that the original is the correct one... which we could determine if there was a way to tell which conflict went with which part of the diagram.  Without that capability it's hopeless as near as I can tell.

I think that what I have to do now is remove the replication capability and go back to a "standalone" (non-replicate) database.  I tried that earlier today and what I ended up with, while it looks reasonsble, is something in which I have absolutely zero confidence that it is correct.  I went through the "remove replication" wizard until I got to the part that said "Enter the full path and filename of the base Enterprise Architect project".  What is that?? The only thing I could find that it would accept is the last version of the database before I turned it into a design master and started generating replicas.  That version of the database is almost three weeks old and there has been a LOT of work done on it since then.  So is that what I'm supposed to use for a base, or is there something else.

Sorry, I'm getting off-topic again, but this has turned into a real mess and I have to find a way to fix it.

<END OF PART 2>

[Aaron B]

It sounds like your model is unfortunately too intertwined to be usefully managed by replication, or possibly even for the other compare and merge functions that were mentioned.  For what particular reasons was replication decided upon originally for this project?  Are the software engineers working geographically dispersed?

IMO It's sounding like the best solution for you would be to use a single shared repository and have all users making live changes to this central database.  If they are working geographically dispersed, a WAN Optimizer service can be configured to help improve performance while accessing the model over slower connections.

In addition to having all users access the shared repository, you might also consider configuring user security to allow users to "lock" individual components temporarily to avoid concurrent edits that might result in data loss (i.e. like the replication conflicts you are currently getting).

[RoyC]

As I said earlier, there are much broader and far-reaching issues in your problem, and Geert and Aaron have talked about them. So, to be very specific about one point in your posts....

Interpreting the Resolve Conflicts dialog:

Ignore the Tables with conflicts panel. The Conflicting Records panel lists the GUIDs of elements that have conflicting changes.  If you highlight a GUID, the Conflict Details panel shows which characteristics, properties, parameters etc are in conflict for that element. So, the key is: find out which element has the GUID.  Unfortunately, there is no simple way to do that.  You could:

• Design a SQL search to match the GUID to an element name

• Use the API functions to do the same.

That is probably not the best answer for you, sorry. If you cannot create a SQL search that will identify the element for a GUID, ask on the forum and perhaps someone can help you. You can then use the Model Search to locate the element in the Project Browser or diagram.

[EricP]

For what particular reasons was replication decided upon originally for this project?  Are the software engineers working geographically dispersed?

No, there are only two of us and so far we have been working in the same room, but the other one is going to be working at home starting next week so he'll be 50 km or so away... still not what one would call geographically dispersed.

The reasons we decided on replication was it seemed like a good way for both of us to be working (on different parts of the model) at the same time.  We have to be able to do that.  We can't just have one person locking the database at a time so that the other person can't do any work.

IMO It's sounding like the best solution for you would be to use a single shared repository and have all users making live changes to this central database.  If they are working geographically dispersed, a WAN Optimizer service can be configured to help improve performance while accessing the model over slower connections.  In addition to having all users access the shared repository, you might also consider configuring user security to allow users to "lock" individual components temporarily to avoid concurrent edits that might result in data loss (i.e. like the replication conflicts you are currently getting).

We're using EA Professional, which I'm pretty sure doesn't support all of that (right?).

I don't mind upgrading to whatever version of EA I need to make this work *_IF_* I can be SURE that it's going to work.  At this point in the project it would be a disaster if I put aside all my other work and set up an MySQL server and go through whatever I need to go through to convert my .EAP database to an SQL database (is that even possible?) and figure out a way to make it visible to the outside on a DSL connection, only to discover at the end of all of that that it still doesn't work the way we need it to work.

So, OK, I can't use replication.  I've come to accept that.  So, how do I get my database back to the point where it was before I started all this replication stuff (but with the changes made since then)?  As I said in my past email, I went through the process of removing replication, using a 3-week old database as my base project (it was the only one the replication removal wizard would accept), and what I got out of it looks reasonable but I haven't any idea if it's really correct.

[EricP]

We have been responding to your emails, but have not received any further correspondence from your end, nor have we received any delivery failure emails.

Just by way of followup to this one... I've been on the phone with Scott Hebbard off and on all evening, and we have discovered that email from Scott's personal email at Sparx comes through to here, but email from [email protected] does not.  I am keeping an eagle eye on my spam bucket and there is nothing in there from Sparx.  So, there is a problem somewhere that Scott is looking into (else, why would email from Scott's personal mailer get here and the one from support@... not?).  Anyway, that is why I haven't been getting any of the email you have sent in response to this problem.

[EricP]

That is probably not the best answer for you, sorry. If you cannot create a SQL search that will identify the element for a GUID, ask on the forum and perhaps someone can help you. You can then use the Model Search to locate the element in the Project Browser or diagram.

Good evening, Roy.

There are dozens of those GUID entries in the error tables.  Since (I'm told) it's an Access database, I guess I could see if I could find any of those GUID entries with Access.  It may take longer to do that than it would take to just manually inspect all of the diagrams and try to catch the differences manually.  (I have no idea how to write SQL searches and may have time to learn to do that in six or 8 months or so...)

I guess I'll try that, though, and see what I can figure out.  Right now, though, my main objective is to try to get the database back into the shape it was in before I started all this replication stuff.  As noted in other messages in this thread, that's not going well either.

I may have damaged our database beyond any confidence that I can get a full recovery.  For a class 3 medical device, this is a very bad situation.

[EricP]

Incidentally, on a related note... Even when I can get what looks like a successful re-sync with the design master, I have discovered, too late to do anything about it, that if I go into Tools->Manage .EAP files->Resolve Conflicts, I get a list of conflicts even though I never got that dialog box at the end of the re-sync process that said I had conflicts.  No way of knowing how long that situation has been going on, hence we have pretty much lost all confidence in the integrity of our database.

[EricP]

EA draws diagrams based on the contents of the EAP database.  So, is it not possible to add the feature to EA to read a conflicted database, draw in black those parts that are not conflicted, and draw in blue (for the original) and red (for the replica) those parts that are conflicted?  Then I could just go through and look for reds and blues on my diagram, and in each case delete the ones I don't want and save the others.  That would be a breeze, and this whole issue would go away.

[Geert Bellekens]

If he works for you, give him a raise.  If he doesn't, hire him.  :-)

Just to be clear, I don't work for SparxSystems. Hiring me would be a tad difficult too since I literally live on the other side of the globe
I'm a freelance consultant, so if you're interested in hiring me, just drop me an email  8-)

For the comparing part, I still think you are best of with the xmi compare option.
I don't know if anyone mentioned it, but you can also export the whole model to xmi, and compare those. It will give you a longer list of differences, but I think anything is better then searching elements by GUID one by one.

If still want to search by GUID, you can use the EA SQL search, that will be a lot easier then going trough MS-Access first, and then trough EA.
If you really need this I could help you with writing the SQL search definition.

On the topic of moving to a "real" database, at my current client we work with an SQL server database and the "locking" feature. (we also use VC, but only for a "common" model that is shared between different projects).
To give you an idea, we are working with about 20 analysts on a model containing about 50000 elements, so if it works for us I'm pretty sure it'll work for your situation.
The advantage of this "locking" mechanism ist that you can lock on element level as opposed to package level using VC. So you'll have a lot less conflicts that way.

It is possible to move an eap file to a DBMS database, see "Tools/Data Management/Project Transfer"

All the best in trying to clean up the mess, i feel for you  :-/

Geert

[EricP]

For the comparing part, I still think you are best of with the xmi compare option.  I don't know if anyone mentioned it, but you can also export the whole model to xmi, and compare those. It will give you a longer list of differences, but I think anything is better then searching elements by GUID one by one.

Yeah, me too.   :-/

OK, how does this sound?

1.  Merge the replica into the master, resolving any conflicts in favor of the MASTER.

2.  Export both complete models to XMI

3.  Compare the XMI's; hopefully they'll contain clues as to where the differences are on the diagram.

I did try exporting one of the models to XMI earlier and the XMI file looked almost as cryptic and meaningless as the conflicts report with all the GUID tags, but I didn't try the compare so maybe that'll give a few clues.

One thing... when I do the export and compare, I need to use the replica as it was BEFORE I did the merge, right?  The replica/merge documentation that I can find says that the merge will sync both master and replica so that they are identical, and I can only interpret that to mean that if I resolve all conflicts in favor of the master, then the replica will also be changed to match the master.  That would not be good, if I'm to try to find the differences.

[Geert Bellekens]

That sounds about right, except for the compare bit.
You don't need to export the master to xmi, just the replica (before the synch)
You then compare the xmi file (replica) to the model in EA.
(right click on the root model, "Package Control"/"Compare to XMI")
That will give you a nice overview of the differences between the two models. (in a non-cryptic way)

Geert

[EricP]

Geez, man, don't you ever sleep?  

OK, this looks like it might work.  At least it shows what elements of the model contain the differences.

Here's a related question.  I open my design master, and select Tools->Manage .EAP File->Synchronize Replicas.  I select my replica and start the merge.  I get a dialog box that says "There are synchronization conflicts with replica <path.to.replica>.  Open <path.to.replica> to resolve these conflicts".

Now, before I close the master and open the replica, I check Tools->Manage .EAP File->Resolve Replication Conflicts, just to see how bad things are.  I see NO conflicts at all in the master.

Now I close the master and open the replica, and hit Tools->Manage .EAP File->Resolve Replication Conflicts, and HERE is where the conflicts are... lots and lots and lots of conflicts  .  

Here is my question.  I am in the replica, not in the master (because that's what the error box at the end of the merge operation told me to do).  So, I can select a conflicting record and hit either "Keep Current" or "Overwrite with Conflict".  If I hit "Keep Current" am I keeping the version that's in the replica (because that's the model that's currently open) or am I keeping the version that's in the master (since that's, well, the master)?

I think that what I want to do is resolve all conflicts in favor of the replica, and then export the replica to XMI and compare with the master... except, if I resolve all conflicts in favor of the replica, won't that also change the master to match?

[EricP]

I open my design master, and select Tools->Manage .EAP File->Synchronize Replicas.  I select my replica and start the merge.  I get a dialog box that says "There are synchronization conflicts with replica <path.to.replica>.  Open <path.to.replica> to resolve these conflicts".  Now, before I close the master and open the replica, I check Tools->Manage .EAP File->Resolve Replication Conflicts, just to see how bad things are.  I see NO conflicts at all in the master.  Now I close the master and open the replica, and hit Tools->Manage .EAP File->Resolve Replication Conflicts, and HERE is where the conflicts are

OK, now this is getting weirder and weirder.

I did everything noted above, and then BEFORE I resolved any of the conflicts, I exported the replica to XMI and did a compare with the master.  Found NO differences at all.

That I don't understand.

How is it possible that...

1.  after merge, the master shows no conflicts;
2.  the replica shows lots of conflicts; and yet
3.  the master and the replica match?