Sparx Systems Forum

Enterprise Architect => Bugs and Issues => Topic started by: EricP on December 03, 2009, 08:31:25 am

Title: !!PLEASE HELP!! BAD problem with replicas
Post by: EricP on December 03, 2009, 08:31:25 am
I really, Really, REALLY need help on trying to get this design masters - replicas stuff to work.

I made replicas for myself and the other software engineer on the job, we worked on them for a few days, now when I try to sync with the design master, I can sync the first replica with the master successfully (doesn't matter which one I do first), then when I do the second one I get conflicts.  I get a Windows error box that says:

"There are synchronization conflicts with replica <path.to.replica.file>.  Open <path.to.replica.file> to resolve these conflicts."

I open up the replica file and I get ABSOLUTELY NO indication at all what I need to do to fix the conflict.  No hints, no pointers, no help of any kind, only a dialog box that says there is a conflict with no idea where the conflict is.

This is screwing up an important and expensive project like you wouldn't believe.  Earlier, before the problem got this critical, I posted a couple of requests for help on the Sparx registered users support request form, and so far no reply.  I really need to know how to resolve these kinds of sync conflicts before we can continue on the project, and every day puts us farther behind schedule.

I know that Sparx support people don't necessarily read this forum regularly (not really sure why... I would think that questions answered here in public where many people can see and find the answers would benefit both Sparx and its customers), so while I'm waiting and hoping for a workable answer from Sparx, I'm also hoping that someone here has dealt with this problem and has a solution.

Thanks...
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: Geert Bellekens on December 03, 2009, 06:53:01 pm
Eric,

Could you please exlain in a bit more detail what you mean with "design master" and "replica"?
I don't recall seeing those terms ever before, not on the forum, and not in the EA user documentation.

Geert

PS. Does it have to do with the synchronisation of the eap databases?
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: EricP on December 03, 2009, 10:26:13 pm
Quote
Does it have to do with the synchronisation of the eap databases?

Good morning, Geert.

Yes, that's it exactly.  The terms "design master" and "replica" are used within the EA documentation to describe the process.

You go to Tools->Manage .EAP file->Make Design Master to convert an EA database (which we could call a "standalone" EA database for lack of a better term) into a Design Master.  Then you can go to Tools->Manage .EAP file->Create New Replica to create a replica.  You make as many of those replicas as you need and distribute one to each member of the team who needs one, then when you get their changes back you use Tools->Manage .EAP file->Synchronize Replicas to merge them back into the design master.

The Synchronize Replicas part is what is causing all the problem.  I get conflicts, which is expected, but I get absolutely no hint as to where the conflicts are or how to fix them.  There is a Tools->Manage .EAP file->Resolve Replication Conflicts, but all it does is give you a dialog with some VERY cryptic entries that are impossible to understand for just about everybody except the people who designed and wrote the system.
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: Geert Bellekens on December 03, 2009, 11:33:11 pm
Eric,

I personally would never opt for the design-master/replica setup.
I think you fulfill the same requirements with a version control setup.
The advantages of this setup is that you don't have conflicts anymore because of the exclusive checkouts, and you can just as easily work in an environment where the EA databases are distributed across remote locations.

If I were you I would re-consider this whole setup and look into the VC alternative.

Now for you immediate issue, have you tried to export the conflicting package to xmi and use the build-in compare tool to find the conflicts?

The compare tool is far from perfect, but I'm guessing it will give you a much better indication of the differences between the different databases then the cryptic messages from the synchronize process.

With the compare tool you can even import some of the differences into the model, which might help to resolve the synchronisation issues you have.

Geert
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: EricP on December 04, 2009, 12:58:58 am
Quote
I personally would never opt for the design-master/replica setup.  I think you fulfill the same requirements with a version control setup.  The advantages of this setup is that you don't have conflicts anymore because of the exclusive checkouts, and you can just as easily work in an environment where the EA databases are distributed across remote locations.

We have the database under version control using Subversion, and while that defaults to concurrent checkouts, we could choose locking checkouts if we wanted to.  But, we can't do that.  We're both heavily into design work on this project and we can't live with having only one person at a time able to work on the design.  That's why I don't mind if I run into conflicts, as long as I get some kind of a hint as to where they are so I can fix them.

Quote
Now for you immediate issue, have you tried to export the conflicting package to xmi and use the build-in compare tool to find the conflicts?

No, haven't tried that yet.  I just generated an XMI and couldn't find anything in there that I could make any sense out of, then I tried the EA compare tool (Tools->Data Management->Project Compare) and got a cryptic and undecipherable list of elements that looked like the output of Tools->Manage .EAP File->Resolve Replication Conflicts.

What I don't understand is why it's so difficult to get any kind of an answer from Sparx support on this.  For some weeks now I have been asking the general question of how we're supposed to validate the output of the replica-synchronize process, and have gotten no answer at all.  I've been an EA user, off and on (would use it on a project and then not use it again for a year or two because it wouldn't be needed) almost since EA first came out.  It was not so long ago at all that a request for help to Sparx support resulted in an almost immediate and usually very useful answer.  Maybe they're getting too big and too busy.  I certainly hope not... I mean I certainly wish them all the success in the world but yet with growth comes inertia and it becomes harder and harder to get support.  I certainly hope that's not what's happening here.

I was going to call them this morning but I mis-calculated the time difference between here and there, so I plan to call them this evening (tomorrow morning, their time).
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: Geert Bellekens on December 04, 2009, 01:24:05 am
Eric,

Integrating EA with VC does not mean storing the database in a VC.
In EA each package is controlled separately in the VC by means of an xmi file.
If you integrate the VC with EA then EA will do the checkin/out for you, using exclusive locks.
So you can still work together with a lot of people on the design, only not with multiple people on the same package (lowest level of package)

The project compare is not what you are looking for to use the compare tool. You can export a package to xmi, and then compare that xmi file with a package in EA.
- Right click on package to compare
- Select Package Control/Compare to XMI file

That will give you an overview of the new/changed/deleted items, and it will indicate what exactly is different.

As for the Sparx support, I can't say that I have any complaints on that area. I've logged several support/bug reports recently and I always got a response within a day or two.

Maybe this is just too difficult, even for the average Sparxian :-/

Geert
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: EricP on December 04, 2009, 01:37:06 am
Quote
Now for you immediate issue, have you tried to export the conflicting package to xmi and use the build-in compare tool to find the conflicts?

I meant to ask... is there a compare tool that works on the XMI files?  Only compare tool I know of works with .EAP and DBMS (or is the XMI file considered DBMS?).  I tried that and got a output consisting of Table (t_attribute, t_attributeconstraints, stuff like that), Source Records, Target Records, and Difference.  Absolutely no way of knowing how any of that relates to the drawings.
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: EricP on December 04, 2009, 01:50:30 am
Quote
Integrating EA with VC does not mean storing the database in a VC.  In EA each package is controlled separately in the VC by means of an xmi file.

By "package" do you mean UML packages, or something else?

I haven't been using UML packages, because of an issue I had with maintaining connections across packages, about which I never could get an explanation from Sparx that I could understand.  What I am doing is to maintain a composite diagram showing all of my classes (what someone else here referred to as the "kitchen sink" diagram) and then create other diagrams each with the subsets I need, placing the classes there as links.  I really don't know (and so far can't find in the help) any way to store that in a VC system except as the master database file.


Quote
The project compare is not what you are looking for to use the compare tool. You can export a package to xmi, and then compare that xmi file with a package in EA.
- Right click on package to compare
- Select Package Control/Compare to XMI file

(Note: You'll find a question on this that appears after your latest note... I sent that before I saw this.)

I do have a few UML packages in my component diagram, and I tried right clicking on one of those and could not find anything that looked like "package control / compare to XMI".

Quote
As for the Sparx support, I can't say that I have any complaints on that area. I've logged several support/bug reports recently and I always got a response within a day or two.

Yeah, I always used to get that too, and I have spread far and wide what fast and expert support Sparx provides.  That's what makes this so mysterious and disturbing.  As you say, maybe it's just too difficult... in which case, why provide the replica / synchronize capability at all if it is so fragile and so hard to use?

Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: Geert Bellekens on December 04, 2009, 03:03:15 am
Yes, by "package" I mean UML package.
I think I know what you mean by the issue with maintaining cross-package connections, but I don't think that is necessarily an issue anymore.

About the issue with the "package control" option, I think you have to select the package in the project browser, not in a diagram.

Geert
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: EricP on December 04, 2009, 03:39:29 am
Quote
About the issue with the "package control" option, I think you have to select the package in the project browser, not in a diagram.

OK, that seems to have worked, although when I compared it with an XMI file I generated earlier, I got "An error occurred during the compare process", with no indication of what the error was.  Of course, that was comparing a small package against the whole of the project's XMI file so I wouldn't expect it to work all the way through.

No immediately clear way I can see to generate an XMI file from just a package, but I'll research that in the Help.

In any case, we're a ways along on this project without benefit of packages, and I'm not sure how to go about packing up everything into packages at this late date, something else I tried to get an answer to from Sparx support and never got an answer... this was on another project when I was reverse engineering some C++ code and used EA's Source Code Engineering to generate a class diagram from a bunch of .h files.  What I got was an unholy mess with dozens of classes, just like I expected, and during the manual cleanup I asked Sparx how to arrange things in packages and never did get it figured out.
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: Geert Bellekens on December 04, 2009, 04:50:41 am
Export to xmi: right-click on package, "import export"/"export to xmi"

Geert
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: EricP on December 04, 2009, 05:02:39 am
Quote
Export to xmi: right-click on package, "import export"/"export to xmi"

I figured it should be something like that, but I don't have any "import export" option when right clicking on a package... options I get are Properties, Open in Relationship Matrix, Documentation, Open Package, Create Linked Document, Add, Find, Feature Visibility, Lock Element, Selectable, Dockable, Appearance, Z-Order, UML Help, and Delete.
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: Paolo F Cantoni on December 04, 2009, 09:45:23 am
Quote
Quote
Export to XMI: right-click on package, "import export"/"export to XMI"

I figured it should be something like that, but I don't have any "import export" option when right clicking on a package... options I get are Properties, Open in Relationship Matrix, Documentation, Open Package, Create Linked Document, Add, Find, Feature Visibility, Lock Element, Selectable, Dockable, Appearance, Z-Order, UML Help, and Delete.
Eric,

I think you're right clicking on the package in the diagram.  You need to right click on the package in the browser.

Under EAUI (EA's Unique interface) the two actions have separate sets of options...

Paolo
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: RoyC on December 04, 2009, 10:04:53 am
To get the import/export option, right-click on the package name in the Project Browser.

That context menu also has the options for managing Baselines (to compare the current package with a snapshot of the package taken earlier), comparing the package with an XMI file (for non version controlled packages) and comparing the package with the most recent package file in the version control system (version-controlled packages).

I checked with Support, and they do have some recent bug reports from you. I think they have asked you to provide a copy of your .eap file for examination - is that the case?

You are covering a lot of ground in this thread. Without knowing the restrictions or flexibility of your work environment, it's hard to know how to help; Geert is doing a fantastic job though, and his advice is excellent. (Just don't ask him about ducks and trains - unless you have had a few beers to fortify you!).
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: EricP on December 04, 2009, 10:23:04 am
Quote
I checked with Support, and they do have some recent bug reports from you. I think they have asked you to provide a copy of your .eap file for examination - is that the case?

Good evening (morning), Roy.

No, I have not heard anything from Support on this topic or on its predecessor topic of some time ago, when I asked how I would go about validating that the replicate / synchronize worked correctly and didn't introduce any unwanted changes in the database.

I finally called Sparx on the phone and spoke with Scott who is going to look into it.

Quote
You are covering a lot of ground in this thread. Without knowing the restrictions or flexibility of your work environment, it's hard to know how to help; Geert is doing a fantastic job though,

Yes, I know, I let things get a little off-topic, sorry about that.  The one thing that I really need to know is, when doing a replica resync to the design master, and I get the inevitable conflicts (and I know and accept that I will get them from time to time), how can I go about fixing or even finding the conflicts?  I get this cryptic list of database tables and record and who-knows-what, and an invitation to select the original or overwrite with the conflict, and I have absolutely no idea what any of it means or how to select the right one.  There really has to be a way to relate the conflict to a point on the diagram so that I can compare the original vs the conflict and know what I'm doing when I select the one I want to keep.  If I can solve that problem then things should be OK.

Sending a copy of my EAP file will be difficult because it's proprietary stuff and I don't have permission from my client to send it... if you really need it I'll ask my client and see if they will allow it.  But this is really a general-type question... if one gets re-sync conflicts on ANY database, how is one supposed to know where to look and how to interpret the conflict?

And yes, Geert is great!  If he works for you, give him a raise.  If he doesn't, hire him.  :-)

Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: Aaron B on December 04, 2009, 02:52:40 pm
Quote
No, I have not heard anything from Support on this topic or on its predecessor topic of some time ago
We have been responding to your emails, but have not received any further correspondence from your end, nor have we received any delivery failure emails.  Please check your inbox (and perhaps your junk filters) to confirm whether you have received our emails.

Quote
Yes, I know, I let things get a little off-topic, sorry about that.  The one thing that I really need to know is, when doing a replica resync to the design master, and I get the inevitable conflicts (and I know and accept that I will get them from time to time), how can I go about fixing or even finding the conflicts?  I get this cryptic list of database tables and record and who-knows-what, and an invitation to select the original or overwrite with the conflict, and I have absolutely no idea what any of it means or how to select the right one.  There really has to be a way to relate the conflict to a point on the diagram so that I can compare the original vs the conflict and know what I'm doing when I select the one I want to keep.  If I can solve that problem then things should be OK.
Replication is performed at the database level (it is actually a function of the Microsoft JET database engine that is used by .EAP file repositories).  EA cannot provide any visual representation of the conflicting changes.  It's possible that some of these 'cryptic' changes you are seeing are changes to the diagram layout, which can be difficult to interpret at the database level.  In these cases you may just need to pick which model you believe has the most reliable copy of the diagrams (the replica, or the master).  Having the owners of these replica models involved in this process may help if they can comment if they made any meaningful diagram changes.

As specified in the documentation, it is advised that potential conflicts should be avoided by having your users work in different areas of the model.  Also when merging replicas and encountering conflicts, the respective developers who own the replicas should be present during the conflict resolution process to help advise about the changes.
http://www.sparxsystems.com/uml_tool_guide/uml_model_management/replication.html

As suggested earlier by Geert, using the XMI-based Baselining, Differencing and Merging capabilities provided by EA might work a lot better for you than Replication and we would advise trying this after resolving your current replication conflicts.  XMI can be exported from each model, which can then be directly compared to your 'master' copy of the model and selectively merged in.
http://www.sparxsystems.com/uml_tool_guide/uml_model_management/baselinesanddifferences.html

If problems persist, it's possible that we may need to arrange for copies of your master and replicas to be sent to us for further analysis, but we would not be able to identify specific changes that you may wish to keep or discard and would likely just elect to overwrite the changes to the master with the conflict found in the replica.
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: EricP on December 04, 2009, 04:10:01 pm
Good evening, Aaron.

Sorry, but I'm going to have to split this response into two parts to conform to the forum limit on message sizes.

Quote
We have been responding to your emails, but have not received any further correspondence from your end, nor have we received any delivery failure emails.  Please check your inbox (and perhaps your junk filters) to confirm whether you have received our emails.

I have gotten exactly zero responses from Sparx on this topic or its predecessor.  None.  I have gotten responses from you on other topics.  When I post something to the registered users support request form, I always get back the email from [email protected] that says "Your Registered Support Request has been sent to Sparx Systems.  We will endeavor to investigate the problem and get back to you as soon as possible."  I always get those, without fail.  And I always get the emails from this form that tell me that a reply has been posted.  So since I'm getting all of those emails, like clockwork, I absolutely can't imagine a reason why I'm not getting your responses to my emails.

I went through my entire email database including the spam box and there was nothing from Sparx that addressed this issue, other than the auto-generated responses indicated above..

Quote
Replication is performed at the database level (it is actually a function of the Microsoft JET database engine that is used by .EAP file repositories).  EA cannot provide any visual representation of the conflicting changes.

Tell me this... EA draws diagrams based on what is in the database.  Why, then, can't EA draw a diagram showing the nonconflicted parts in black, and the conflicted parts in blue (for the design master) and red (for the replica that's causing the conflicts)?  I could then just breeze through the diagram and delete the blue and red parts that I don't want, and keep the rest.

Quote
It's possible that some of these 'cryptic' changes you are seeing are changes to the diagram layout, which can be difficult to interpret at the database level.  In these cases you may just need to pick which model you believe has the most reliable copy of the diagrams (the replica, or the master).  Having the owners of these replica models involved in this process may help if they can comment if they made any meaningful diagram changes.

Any diagram of any complexity at all is subject to conflicts at the re-sync operation.  Those conflicts may occur in two, three, or more separate parts of the database.  I can have an army of software engineers sitting with me to do the conflict resolution and there is no way to know what specific conflict goes with what part of the diagrams.  Without that knowledge, or at least a hint (like the name of the drawing where the conflict is found), any attempt to "believe (which model) has the most reliable copy of the diagram" is a coin toss at best.  Anyway, they may both be equally reliable... they just may reflect different designers' ideas of "reliable".

Quote
As specified in the documentation, it is advised that potential conflicts should be avoided by having your users work in different areas or the model.  Also when merging replicas and encountering conflicts, the respective developers who own the replicas should be present during the conflict resolution process to help advise about the changes.

We did that.  Each of us worked on a different part of the model.  We got conflicts (lots and lots of them) anyway.  As for having the owner of the replica here during the conflict resolution process, how can that help if neither of us has any idea what conflict, as listed in the conflicts table at Tools->Manage .EAP files->Conflict Resolution, goes with what part of the diagram?  Like I said, a coin toss or Jan-ken-pon session, at best.

<END OF PART 1>
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: EricP on December 04, 2009, 04:10:39 pm
<PART 2>

Quote
As suggested earlier by Geert, using the XMI-based Baselining, Differencing and Merging capabilities provided by EA might work a lot better for you than Replication and we would advise trying this after resolving your current replication conflicts.

If I interpreted Geert correctly, XMI-based differencing and merging requires that the model be arranged in packages, right?  We didn't do that, largely because I never was able to get a resolution to an earlier problem, on another project for another client, where I was reverse engineering a complex and poorly written directory full of files, ended up with the expected huge mess when I imported all of that into EA, tried to figure out a way to package things while maintaining connections, and could not do it (and couldn't get a workable solution from Sparx).  On this project, I am using what someone else on this forum called the "kitchen sink" diagram that contains everything, and then other diagrams that break things out and implement classes as links to the kitchen sink diagram.  I suppose those other diagrams could be packages but I have no idea how to maintain the necessary connections between the packages.  Anyway, it's too late to redesign the whole thing now.

Quote
If problems persist, it's possible that we may need to arrange for copies of your master and replicas to be sent to us for further analysis, but we would not be able to identify specific changes that you may wish to keep or discard and would likely just elect to overwrite the changes to the master with the conflict found in the replica.

I can do that myself, but I can't just "elect to overwrite... with the conflict..." because just as often, we'll find that the original is the correct one... which we could determine if there was a way to tell which conflict went with which part of the diagram.  Without that capability it's hopeless as near as I can tell.

I think that what I have to do now is remove the replication capability and go back to a "standalone" (non-replicate) database.  I tried that earlier today and what I ended up with, while it looks reasonsble, is something in which I have absolutely zero confidence that it is correct.  I went through the "remove replication" wizard until I got to the part that said "Enter the full path and filename of the base Enterprise Architect project".  What is that?? The only thing I could find that it would accept is the last version of the database before I turned it into a design master and started generating replicas.  That version of the database is almost three weeks old and there has been a LOT of work done on it since then.  So is that what I'm supposed to use for a base, or is there something else.

Sorry, I'm getting off-topic again, but this has turned into a real mess and I have to find a way to fix it.

<END OF PART 2>
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: Aaron B on December 04, 2009, 04:54:42 pm
It sounds like your model is unfortunately too intertwined to be usefully managed by replication, or possibly even for the other compare and merge functions that were mentioned.  For what particular reasons was replication decided upon originally for this project?  Are the software engineers working geographically dispersed?

IMO It's sounding like the best solution for you would be to use a single shared repository and have all users making live changes to this central database.  If they are working geographically dispersed, a WAN Optimizer service can be configured to help improve performance while accessing the model over slower connections.

In addition to having all users access the shared repository, you might also consider configuring user security to allow users to "lock" individual components temporarily to avoid concurrent edits that might result in data loss (i.e. like the replication conflicts you are currently getting).
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: RoyC on December 04, 2009, 05:02:04 pm
As I said earlier, there are much broader and far-reaching issues in your problem, and Geert and Aaron have talked about them. So, to be very specific about one point in your posts....

Interpreting the Resolve Conflicts dialog:

Ignore the Tables with conflicts panel. The Conflicting Records panel lists the GUIDs of elements that have conflicting changes.  If you highlight a GUID, the Conflict Details panel shows which characteristics, properties, parameters etc are in conflict for that element. So, the key is: find out which element has the GUID.  Unfortunately, there is no simple way to do that.  You could:

That is probably not the best answer for you, sorry. If you cannot create a SQL search that will identify the element for a GUID, ask on the forum and perhaps someone can help you. You can then use the Model Search to locate the element in the Project Browser or diagram.
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: EricP on December 04, 2009, 05:09:37 pm
Quote
For what particular reasons was replication decided upon originally for this project?  Are the software engineers working geographically dispersed?

No, there are only two of us and so far we have been working in the same room, but the other one is going to be working at home starting next week so he'll be 50 km or so away... still not what one would call geographically dispersed.

The reasons we decided on replication was it seemed like a good way for both of us to be working (on different parts of the model) at the same time.  We have to be able to do that.  We can't just have one person locking the database at a time so that the other person can't do any work.

Quote
IMO It's sounding like the best solution for you would be to use a single shared repository and have all users making live changes to this central database.  If they are working geographically dispersed, a WAN Optimizer service can be configured to help improve performance while accessing the model over slower connections.  In addition to having all users access the shared repository, you might also consider configuring user security to allow users to "lock" individual components temporarily to avoid concurrent edits that might result in data loss (i.e. like the replication conflicts you are currently getting).

We're using EA Professional, which I'm pretty sure doesn't support all of that (right?).

I don't mind upgrading to whatever version of EA I need to make this work *_IF_* I can be SURE that it's going to work.  At this point in the project it would be a disaster if I put aside all my other work and set up an MySQL server and go through whatever I need to go through to convert my .EAP database to an SQL database (is that even possible?) and figure out a way to make it visible to the outside on a DSL connection, only to discover at the end of all of that that it still doesn't work the way we need it to work.

So, OK, I can't use replication.  I've come to accept that.  So, how do I get my database back to the point where it was before I started all this replication stuff (but with the changes made since then)?  As I said in my past email, I went through the process of removing replication, using a 3-week old database as my base project (it was the only one the replication removal wizard would accept), and what I got out of it looks reasonable but I haven't any idea if it's really correct.
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: EricP on December 04, 2009, 05:13:35 pm
Quote
We have been responding to your emails, but have not received any further correspondence from your end, nor have we received any delivery failure emails.

Just by way of followup to this one... I've been on the phone with Scott Hebbard off and on all evening, and we have discovered that email from Scott's personal email at Sparx comes through to here, but email from [email protected] does not.  I am keeping an eagle eye on my spam bucket and there is nothing in there from Sparx.  So, there is a problem somewhere that Scott is looking into (else, why would email from Scott's personal mailer get here and the one from support@... not?).  Anyway, that is why I haven't been getting any of the email you have sent in response to this problem.

Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: EricP on December 04, 2009, 05:21:43 pm
Quote
That is probably not the best answer for you, sorry. If you cannot create a SQL search that will identify the element for a GUID, ask on the forum and perhaps someone can help you. You can then use the Model Search to locate the element in the Project Browser or diagram.

Good evening, Roy.

There are dozens of those GUID entries in the error tables.  Since (I'm told) it's an Access database, I guess I could see if I could find any of those GUID entries with Access.  It may take longer to do that than it would take to just manually inspect all of the diagrams and try to catch the differences manually.  (I have no idea how to write SQL searches and may have time to learn to do that in six or 8 months or so...)

I guess I'll try that, though, and see what I can figure out.  Right now, though, my main objective is to try to get the database back into the shape it was in before I started all this replication stuff.  As noted in other messages in this thread, that's not going well either.

I may have damaged our database beyond any confidence that I can get a full recovery.  For a class 3 medical device, this is a very bad situation.
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: EricP on December 04, 2009, 05:25:03 pm
Incidentally, on a related note... Even when I can get what looks like a successful re-sync with the design master, I have discovered, too late to do anything about it, that if I go into Tools->Manage .EAP files->Resolve Conflicts, I get a list of conflicts even though I never got that dialog box at the end of the re-sync process that said I had conflicts.  No way of knowing how long that situation has been going on, hence we have pretty much lost all confidence in the integrity of our database.
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: EricP on December 04, 2009, 05:27:48 pm
EA draws diagrams based on the contents of the EAP database.  So, is it not possible to add the feature to EA to read a conflicted database, draw in black those parts that are not conflicted, and draw in blue (for the original) and red (for the replica) those parts that are conflicted?  Then I could just go through and look for reds and blues on my diagram, and in each case delete the ones I don't want and save the others.  That would be a breeze, and this whole issue would go away.
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: Geert Bellekens on December 04, 2009, 06:00:45 pm
Quote
If he works for you, give him a raise.  If he doesn't, hire him.  :-)
Just to be clear, I don't work for SparxSystems. Hiring me would be a tad difficult too since I literally live on the other side of the globe :)
I'm a freelance consultant, so if you're interested in hiring me, just drop me an email  8-)

For the comparing part, I still think you are best of with the xmi compare option.
I don't know if anyone mentioned it, but you can also export the whole model to xmi, and compare those. It will give you a longer list of differences, but I think anything is better then searching elements by GUID one by one.

If still want to search by GUID, you can use the EA SQL search, that will be a lot easier then going trough MS-Access first, and then trough EA.
If you really need this I could help you with writing the SQL search definition.

On the topic of moving to a "real" database, at my current client we work with an SQL server database and the "locking" feature. (we also use VC, but only for a "common" model that is shared between different projects).
To give you an idea, we are working with about 20 analysts on a model containing about 50000 elements, so if it works for us I'm pretty sure it'll work for your situation.
The advantage of this "locking" mechanism ist that you can lock on element level as opposed to package level using VC. So you'll have a lot less conflicts that way.

It is possible to move an eap file to a DBMS database, see "Tools/Data Management/Project Transfer"

All the best in trying to clean up the mess, i feel for you  :-/

Geert
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: EricP on December 05, 2009, 12:16:10 am
Quote
For the comparing part, I still think you are best of with the xmi compare option.  I don't know if anyone mentioned it, but you can also export the whole model to xmi, and compare those. It will give you a longer list of differences, but I think anything is better then searching elements by GUID one by one.

Yeah, me too.   :-/

OK, how does this sound?

1.  Merge the replica into the master, resolving any conflicts in favor of the MASTER.

2.  Export both complete models to XMI

3.  Compare the XMI's; hopefully they'll contain clues as to where the differences are on the diagram.

I did try exporting one of the models to XMI earlier and the XMI file looked almost as cryptic and meaningless as the conflicts report with all the GUID tags, but I didn't try the compare so maybe that'll give a few clues.

One thing... when I do the export and compare, I need to use the replica as it was BEFORE I did the merge, right?  The replica/merge documentation that I can find says that the merge will sync both master and replica so that they are identical, and I can only interpret that to mean that if I resolve all conflicts in favor of the master, then the replica will also be changed to match the master.  That would not be good, if I'm to try to find the differences.
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: Geert Bellekens on December 05, 2009, 12:32:18 am
That sounds about right, except for the compare bit.
You don't need to export the master to xmi, just the replica (before the synch)
You then compare the xmi file (replica) to the model in EA.
(right click on the root model, "Package Control"/"Compare to XMI")
That will give you a nice overview of the differences between the two models. (in a non-cryptic way)

Geert
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: EricP on December 05, 2009, 01:37:59 am
Geez, man, don't you ever sleep?   :)

OK, this looks like it might work.  At least it shows what elements of the model contain the differences.

Here's a related question.  I open my design master, and select Tools->Manage .EAP File->Synchronize Replicas.  I select my replica and start the merge.  I get a dialog box that says "There are synchronization conflicts with replica <path.to.replica>.  Open <path.to.replica> to resolve these conflicts".

Now, before I close the master and open the replica, I check Tools->Manage .EAP File->Resolve Replication Conflicts, just to see how bad things are.  I see NO conflicts at all in the master.

Now I close the master and open the replica, and hit Tools->Manage .EAP File->Resolve Replication Conflicts, and HERE is where the conflicts are... lots and lots and lots of conflicts  :'(.  

Here is my question.  I am in the replica, not in the master (because that's what the error box at the end of the merge operation told me to do).  So, I can select a conflicting record and hit either "Keep Current" or "Overwrite with Conflict".  If I hit "Keep Current" am I keeping the version that's in the replica (because that's the model that's currently open) or am I keeping the version that's in the master (since that's, well, the master)?

I think that what I want to do is resolve all conflicts in favor of the replica, and then export the replica to XMI and compare with the master... except, if I resolve all conflicts in favor of the replica, won't that also change the master to match?
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: EricP on December 05, 2009, 02:02:28 am
Quote
I open my design master, and select Tools->Manage .EAP File->Synchronize Replicas.  I select my replica and start the merge.  I get a dialog box that says "There are synchronization conflicts with replica <path.to.replica>.  Open <path.to.replica> to resolve these conflicts".  Now, before I close the master and open the replica, I check Tools->Manage .EAP File->Resolve Replication Conflicts, just to see how bad things are.  I see NO conflicts at all in the master.  Now I close the master and open the replica, and hit Tools->Manage .EAP File->Resolve Replication Conflicts, and HERE is where the conflicts are

OK, now this is getting weirder and weirder.

I did everything noted above, and then BEFORE I resolved any of the conflicts, I exported the replica to XMI and did a compare with the master.  Found NO differences at all.

That I don't understand.

How is it possible that...

1.  after merge, the master shows no conflicts;
2.  the replica shows lots of conflicts; and yet
3.  the master and the replica match?
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: Geert Bellekens on December 05, 2009, 06:30:53 pm
That seems very weird indeed.
I have the idea that the whole replication thing is kind of screwed up.

If the xmi compare does not show differences I guess you can safely assume that both are are in fact equal.

Geert

PS. I do sleep from time to time, but I have small children that sometimes wake me up in the middle of the night, and while waiting for them to go asleep I sometimes get bored and check my email :)
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: ebeb on January 07, 2010, 11:19:51 pm
Hi!

I just wanted to ask, if there is any progress with this issue or consensus what and how to do to prevent this mess. I'm asking because I want to use replicas in a similar use case as described except that we actually are geographically dispersed and have a high latency WAN and am kind of afraid now ;).

Quote
If he works for you, give him a raise.  If he doesn't, hire him.
[smiley=thumbsup.gif] [smiley=thumbsup.gif] [smiley=thumbsup.gif]
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: Geert Bellekens on January 07, 2010, 11:43:23 pm
Quote
what and how to do to prevent this mess.

Don't use it ;D

Geert
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: EricP on January 07, 2010, 11:58:48 pm
Quote
I just wanted to ask, if there is any progress with this issue or consensus what and how to do to prevent this mess. I'm asking because I want to use replicas in a similar use case as described except that we actually are geographically dispersed and have a high latency WAN and am kind of afraid now ;).

Good morning, ebeb.

The consensus, if there is one, seems to be to use XMI compares to resolve conflicts.  I found that that kinda, sorta worked, but not really, then I ran out of time to work with it (or to document it in a way that I could intelligently discuss the shortcomings).  Basically, I had to pick an interim solution, and get back to work on the project.

I found that after merging the replica back into the master, all of the conflicts were in the replica, and none were in the master.  So, the other software engineer and I checked the master to verify that at least the easily-visible stuff was as it should be (attributes and methods of classes, links between classes), then we discarded the replicas and generated new ones.  We have been working with those since then and so far, so good.

We have resigned ourselves to the fact that we can no longer trust the EA database as the final authority on the system.  Our ideal solution would have been to make all changes in the model and then generate new code and it would Just Work.  That's no longer possible (if in fact it ever was), and so we're considering the code as the final authority, making all necessary changes in the code, and then re-sync'ing the code back into the model.  Come design review time, we'll have to manually inspect all aspects of the model to make sure it represents the real world (code) rather than inspecting the code to make sure it accurately represents the model.  Kind of a backwards way of doing it, and not what I wanted, but it's working that way and it's not a huge amount of extra work.

What I would really, Really, REALLY like, in EA 7.6, would be that in the event of conflicts after merge, I call up the conflicted diagram and I see the master version of the conflict in one color and the replica version of the conflict in another color.  Then I could just select and delete the things that are incorrect and keep the things that are correct, then save the diagram and the problem is solved.

Maybe the Sparxians will do something like that...
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: ebeb on January 08, 2010, 12:11:31 am
Hey Eric,

thanks for the reply.

I actually don't get it why you were initially using replicas. You wrote in one post that you wanted to access different packages of one eap file from different users.

That works very well with one single shared eap file, even better with additional version control set up.

In our case, we have one eap, and most of the packages VCed using SVN. That assures that only one user is accessing a package at a time.

The only reason for us to use replicas is the high WAN latency when accessing the eap file when doing home office.
Title: Re: !!PLEASE HELP!! BAD problem with replicas
Post by: EricP on January 08, 2010, 12:31:21 am
Quote
I actually don't get it why you were initially using replicas. You wrote in one post that you wanted to access different packages of one eap file from different users.

We aren't using "packages" in the sense of UML "packages", though I've found that some of the EA features that work on "packages" also work on individual diagrams that EA considers "packages", like XMI export.

Instead, what I did was implement what someone else here characterized as the "kitchen sink" diagram, that contains everything in the model, then sub-diagrams are implemented where each element (class, struct, etc.) is a link to the corresponding element on the kitchen sink diagram.  That works well, except that any change that's made to a sub-diagram also shows up on the kitchen sink diagram, so all those changes have to be merged in from different users.  I don't know of a way to do that other than with replicas.

Perhaps that wasn't the right way to do it but I ran into problems with UML packages on another project, mainly regarding how to maintain connections between packages and account for the fact that certain classes, structs, and enum lists are used in many (most) packages.  I haunted this forum and Sparx support for a while trying to figure that out and never did come up with a good solution, so for this project I decided to use replicas, trusting that they would work better than they apparently do.

If we had a better way to resolve conflicts, like what I suggested earlier about showing conflicts in different colors on the diagram, replicas would work just fine.