Author Topic: Bug in GetAllLatest (Read 13703 times)

dschmid2 · « **on:** July 07, 2010, 10:57:15 pm »

We discovered a serious bug concerning the retrieval of packages from SourceControl.
When doing a GetAllLatest in a model, I want that all package XML files are retrieved from SourceControl and (forced) imported into my model. This should replace all model elements with the ones that are read from the files.
When I randomly choose a package afterwards and do a "Compare with Controlled Version", there are differences showing up. I can do as many GetAllLatest as I want, it doesn't change.
Only when I do a GetLatest of the particular package, the package is finally imported and the content replaced. An subsequent "Compare" shows that no more differences are detected.

This bug must have been introduced in one of the latest releases...

The GetAllLatest functionality is not very stable anyway. We used to call it twice always to make sure that all packages and dependencies have been resolved.

Geert Bellekens · « **Reply #1 on:** July 07, 2010, 11:12:17 pm »

Oh, serious indeed

Did you test on the latest version (8.0.858)?
Did you report it to sparx using the link on the bottom of the page?

Geert

dschmid2 · « **Reply #2 on:** July 07, 2010, 11:33:49 pm »

Yes, we have 858 installed and I reported it to Sparx.

HowardB · « **Reply #3 on:** July 13, 2010, 10:51:39 am »

Daniel,

there has been no bug introduced into the Get All Latest functionality as you are suggesting.

Indeed, the basic principle of the Get All Latest functionality - that it recursively calls Get Latest, for each version controlled package in the model - has not changed since its inception.

That's right, Get All Latest simply calls Get Latest for each package, one after the other.

Now, when EA imports a package from XMI, it must first delete the existing package and its content from the model, before it can re-import it from the XMI.

When the package and its contents are deleted, that can have an effect on other packages in the model.

All associations using elements that are deleted will also be deleted.
The elements that are deleted will also be removed from all diagrams on which they were previously shown.

EA simply cannot use elements that do not exist in the model.

When the elements are re-imported into the model, there are certain situations in which the previously existing associations can not be restored. Similarly there are some situations in which elements will not be replaced on the diagrams on which they previosuly appeared.

The order in which packages are re-imported into the model can have an effect on whether cross-package dependencies are correctly restored.

The Sparx Systems white paper on "Version Control Best Practices" details situations in which Cross Package Dependencies can be affected by XMI export/import, and gives some guidance on how to avoid and minimize the impact of these effects.

Daniel, you apparently have some understanding of the need to have all elements present in the model in order to resolve dependencies, but perhaps you didn't realise that forcing EA to re-import packages actually causes them to be deleted from the model before being re-imported.

Please DO NOT select the option to "force reload from XMI".

Those packages where new content has been committed to version control will be re-imported as necessary.

I believe this on its own will virtually eliminate the problems you are seeing.

We are currently working on some new functionality that will scan and reconcile XMI file content with the model DB, to "put-back" connectors and elements that are contained in the XMI, but might have been erroneously removed from the model, due to the order of re-importing packages.

Until then, please follow the guidelines in the Best Practices document.

I hope this helps,

Howard Britten.

Frank Horn · « **Reply #4 on:** July 14, 2010, 06:22:54 am »

Quote

When the elements are re-imported into the model, there are certain situations in which the previously existing associations can not be restored. Similarly there are some situations in which elements will not be replaced on the diagrams on which they previosuly appeared.

The order in which packages are re-imported into the model can have an effect on whether cross-package dependencies are correctly restored.

So never trust EA to restore your data!

I wouldn't touch the "Get All Latest" command with a ten foot pole, except in one special situation:

When you build up a new project of controlled packages which contain nested controlled packages, import the root packages via the "Get Package" command, and then perform "Get All Latest" to fill the stubs.

In my experience, EA's version control is only safe if you stick to some very restrictive rules:

1. No cross references
Organize your controlled packages with acyclic dependencies (i.e if something in package1 has a depency on anything in package2, nothing in package2 must have a dependency on anything in package1, or anything depending on anything in package1).

2. One or all checkout
Only check out one package at a time unless you are aware of the dependency graph. If you are aware: when changing packages on which other packages depend, check out ALL dependent packages as well.

3. Communicate
After checking in, inform all users of the changed packages and urge them to update their projects.

4. Update before check out
Before you check out a package depending on other packages, perform "Get Latest" on all packages on which it depends. Don't believe EA when it tells you a package is up to date, but ENFORCE reloading from XMI!

5. Update depth first
When performing "Get Latest", do it individually on each package, and do it in order of dependency (i.e first the packages which depend on nothing else, then those which only depend on the packages previously updated, and so on).

HowardB · « **Reply #5 on:** July 14, 2010, 09:52:55 am »

Whilst I agree with some of the points Frank makes, (they are essentially what we recommend in the "VC Best Practices" white paper), I must disagree with Frank's insistence that you always choose "Force Reload from XMI".

Forcing a reload from XMI causes packages and elements to be unnecessarily deleted from the model before re-importing them.

...and deleting the elements from the model, is precisely what is causing the problems with the cross-package dependencies that you are complaining about!

regards,
Howard.

dschmid2 · « **Reply #6 on:** July 14, 2010, 03:26:00 pm »

Actually we were having troubles when NOT doing a force Reload from XMI, we used to have missing dependencies and even missing classes. After starting to use the forced reload (twice in a row), things went somehow better. But as a matter of facts, with both methods we end up from time to time with models that were corrupt.

Since we encountered this issue with both methods, there this fear of never knowing when its going to happen next.

Worst of all, there is no evidence or message alerting that something could be missing, not logfile, just nothing. It can happen that only some time later (and after lots of modification and check-ins/outs) we get aware that something must have been lost on the way. And from that point on, its a real pain to reconstruct things...

Frank Horn · « **Reply #7 on:** July 14, 2010, 05:37:50 pm »

Howard,

I agree that forcing reload from xmi is unnecessary when all users know what they're doing and everyone is duly updating packages in the right order.

But the tiniest flaw in this workflow can lead to loss of data.

Imagine we are both working on a project with two controlled packages A and B, where B depends on A, and we both have a local eap file containing A and B.

Now you check out both A and B, add some classes to A, and associations from B-classes to A-classes. Then you check in both packages.

Then I open my project to work on package B. Here's what can happen:

Case 1: Everything allright
I've received your notification and I know that B depends on A. So I call "Get Latest" first on A and then on B. EA will know that the packages are out of date and reload from xmi anyway, so I don't have to force it. Everything will be fine.

Case 2: Just made it
I know that you have worked on B, but I'm not aware of the changes you made to A, or I don't know that I have to take care of this. I check out B and don't see your changes. Now I wonder what's going on and call "Get Latest" for A as well, just to be on the safe side. Still no changes in B. Then I call "Get Latest" for B again, but EA tells me it's up to date. Only "Undo Checkout" or "Get Latest" for B with forced reload will get your changes into my model.

By the way, what if I call "Get All Latest" on the whole project? Can I trust EA to import first A and then B? Or do I have to call "Get All Latest" twice, the second time forcing reload?

Case 3: Messed up
I don't know anything, check out B, and spend the day working on B. Then I check in B. Next time you checkt out B, your changes will have gone down the drain.

I think EA should at least give a warning when things are being deleted on import. Better still, it should keep stubs for all such things, in all cases.

dschmid2 · « **Reply #8 on:** July 15, 2010, 03:33:28 pm »

Dear guys from EA

During the GetAllLatest operation, I noticed several times, that in the progress window, it shows a line telling "Error ...." but I never managed to read the full content, because the operation went on. So I know that somewhere something is wrong, but I have no clue what EA is complaining about...

PLEASE! Add a simple logging functionality, that prints in the System Message window all the packages that have been imported during a GetAllLatest operation. Please dump the filename, package name the repository version and eventual problems. This can't be such a big thing. And it would help us to at least verify which packages were updated, determine which classes were replaced etc.

Thanks a lot!!!

Geert Bellekens · « **Reply #9 on:** July 15, 2010, 04:03:13 pm »

Have you oficially requested this feature?
Although posting on this forum is very interesting, it won't put the feature on Sparx TODO list.
Only an official feature request using the link on the bottom of the page will.

Geert

suki · « **Reply #10 on:** April 18, 2011, 11:01:16 pm »

Hello,
we use EA 864 and in this version is still the problem decribed above.

We work with local EAPs (many users) a have one central storage in subversion.
For us is not possible to keep those difficult rules described by Frank because we are more than 50 users who works with it.

Has anyone functioning solution how to update local EAPs from XMI files from Subversion without loosing relation between objects and damage of diagrams?

Some bugs can be repared running GetAllLatest twice but not all of them except the fact that run of GetAllLatest on our great database takes more than one hour. It is unusable :-(.

Thanks in advance for all comments.

HowardB · « **Reply #11 on:** April 19, 2011, 09:46:37 am »

Hello Suki,

New functionality has been introduced into the Get All Latest command in EA version 9.0, that addresses the issues of elements and relationships being removed from diagrams during XMI imports.

EA 9.0 (beta 2) is available now, for download by registered users.

If your project is so large that it takes around an hour to run Get All Latest, maybe you should consider hosting the project in a DBMS, such as MySQL or SQL Server. In that case, you would have no need to run Get All Latest, as all users would be connecting to the same model database, and always accessing the latest model information.

best regards,
Howard Britten.

KP · « **Reply #12 on:** April 19, 2011, 10:28:59 am »

Quote

EA 9.0 (beta 2) is available now, for download by registered users.

Beta 2 is actually a public beta - there is a 30 day time-limited trial version for unregistered users. Follow the links on the front page...

lubos · « **Reply #13 on:** April 19, 2011, 05:47:14 pm »

Hello,

could you describe what are the changes in version 9 in more detail, please? This info is necessary for us to decide if the new version is suitable in our environment and way of work.
I try to avoid situation when we will trust this new version and a new problems with data/work lost will arrise...

Thank you

suki · « **Reply #14 on:** April 19, 2011, 06:28:21 pm »

Thank you for your comment.
We know that Beta 9.0 is available but they would be useful more details in change log (see lubos comment).
From my point of view is better to wait for official release than work with beta on production environment, do you agree?

Concerning your DBMS recomendation. We worked with DBMS in past so I know advantages and disadvatages DBMS.

Our users work on different places often using internet connection only. So work with database was very slow therefore we moved to local EAPs with VC.
Work with EAPs is much more faster for routine work and it allows to manage exclusive access to the package. This is what we exactly need. Unfortunatelly synchronization of versioned packages is nearly unusable. With each update we are losing some work documented in past.

suki

Sparx Systems Forum

News: