XMI 2.1 Inefficient and Out of memory

[logo]

I want to import a moderate size (200 Classes/Tables) data model into EA 6.5.799 using XMI. To do this I created an XSLT generating the XMI file. The original XML file is 2.5 MB (approx 90 classes) and the generated XMI is 215,3 MB! Importing the XMI gives "Out of memory" but  the file still imports (don't know if all data made it). The file has lots of tag values. Removing classes or tag values make it import without "Out of memory" message, e.g. removing approx 10 tag values per attribute shrink the original XML to 1.5 MB and the XMI to 115.7 MB. I haven't yet tried to import all 200 classes.
I have tried this on two PCs (2GB and 4GB primary memory), get the "Out of memory" on both.
QUESTION: Is there a way to make EA import large XMI files without "Out of memory"?
COMPLAINT: Why is XMI 2.1 and the EA implementation that inefficient? Isn't it intended for more than toy size models?

[Paolo F Cantoni]

Hi logo,

Where did the original XML datamodel come from?

How confident are you the XSLT creates the right output?

Can you round-trip a very small model (say 5 classes).
Import to EA, Export back out, compare XMIs?

HTH,
Paolo

[logo]

Hi
I can import just a few classes into EA without problems. Then I reexport and compare the two XMIs. I think my original looks OK compared to the reexported.
I have managed populate up to 90 classes and they looks fine in EA.
Looking at the Commit Charge in the task mgr the "Out of memory" problem seems to happen when more than 1 GB memory is consumed.  EA still manage to import the classes. When the Commit Charge exceeds 2,25 GB the import fail compleatly with "Out of memory". EA may not manage virtual memory correctly.
The data model is from a Data Engineering tool for a SCADA system. I haven't yet implemented population of the DE tool data dictionary from XMI and will not do as long as the EA import doesn't work.
Thanks for responding
Lars-Ola

[logo]

Hi again Paolo
A 505 MB XMI import comes out as a 12 GB XMI export. Obviously I can improve my XMI generator.
Thanks
Lars-Ola

[Paolo F Cantoni]

Hi again Paolo
A 505 MB XMI import comes out as a 12 GB XMI export. Obviously I can improve my XMI generator.
Thanks
Lars-Ola

Hi Lars,

So this isn't a Database data model?  I just ask, because I had similar high memory uses when I accidentally tried to import (via ODBC) an entire multi-schema DB (I'd forgotten to set the owner properly).

You can try significantly increasing your swap file, just to get the stuff in if you still have problems after refactoring your generator...

HTH,
Paolo

[logo]

Hi Paolo
It is a database model but it has a lot of additional meta data so I can't just use the standard database connect. Instead I create the XMI file with lots of tag values.
The problem with the XSLT was it created <attributes> for all columns from all tables fore every <element> instead of just the columns that belong to the table (simple programming error, Sigh!). Fixing this solved the "Out of memory problem" but it still takes time (10 minutes). So I am happy now and will move from Rose to EA.
Thanks
Lars-Ola

[Paolo F Cantoni]

Hi Paolo
It is a database model but it has a lot of additional meta data so I can't just use the standard database connect. Instead I create the XMI file with lots of tag values.
[size=13][SNIP][/size]
Thanks
Lars-Ola

Hi Lars-Ola,

Now that you have the model "in play", you should be able to save a baseline and then reverse engineer the ODBC database and see what (if anything) changes (via Compare Baselines).

If you have the time and inclination, I'd appreciate the feedback.

Paolo

[logo]

Paolo
I can't do a full roundtrip until I can populate our data dictionary. This will happen later. However, I have re-exported the XMI from EA and some information seems to get lost in that step. I have described the roundtrip with some detail in a word document. Here is a summary from it
1) export of XMI from our SCADA
2) import to EA (version 6.5.799)
3) re-export from EA
In the third step some of the <thecustomprofile:XXX> elements disappear. I made a statistics program to count them. The original file (step 1) had 16903 <thecustomprofile:XXX> while the re-export (step 3) has 16634. In particular the <thecustomprofile:IDES_REF_INT_TYPE> had 120 in the original file and 114 in the re-exported. The 6 lost ones from the original file are
<thecustomprofile:IDES_REF_INT_TYPE base_Attribute="PROCESS_VALUE(LIMIT_VALUES_IRN)" IDES_REF_INT_TYPE="Z"/>
<thecustomprofile:IDES_REF_INT_TYPE base_Attribute="THERMAL_UNIT(IO_CURVE1_IRN)" IDES_REF_INT_TYPE="Z"/>
<thecustomprofile:IDES_REF_INT_TYPE base_Attribute="THERMAL_UNIT(IO_CURVE2_IRN)" IDES_REF_INT_TYPE="Z"/>
<thecustomprofile:IDES_REF_INT_TYPE base_Attribute="THERMAL_UNIT(IO_CURVE3_IRN)" IDES_REF_INT_TYPE="Z"/>
<thecustomprofile:IDES_REF_INT_TYPE base_Attribute="THERMAL_UNIT(IO_CURVE4_IRN)" IDES_REF_INT_TYPE="Z"/>
<thecustomprofile:IDES_REF_INT_TYPE base_Attribute="THERMAL_UNIT(IO_CURVE5_IRN)" IDES_REF_INT_TYPE="Z"/>
They are visible in EA so they made the import.
Do you think I shall send an error report on this?

[Paolo F Cantoni]

Yes.  Make a formal report.  Give Sparx as much information as you can to help them diagnose the problem...

Paolo