v15.2 - Self-inconsisent Lilliputian behaviour of EA scripting

[Paolo F Cantoni]

We have a VBS script to import shapescripts into MDG stereotypes defined in a specific MDG definition repository.  From this repository, we then emit the MDG.
The script works well and allows us to make global changes with an editor and then import those changes into the MDG.  As you'd expect, the shapescript text is added to a zip file and then that file is Base64 encoded and then the script puts the base64 encoding into its wrapper and updates the value of stereotype _image.Default.  We can then open the _image attribute and voilà, we can see the shapescript correctly unpacked by EA and correctly emitted by the MDG.

We have determined a need to go in the other direction, that is, export the shapescripts. So we thought we'd go in the opposite direction, get the value of the .Default property, unpack the wrapper, output the file as a Base64 file, extract the str.dat from the zip file and voilà we have the shapescript back.  Because we do extensive testing, we discovered that the original files were UTF-8 and EA was generating a form of UTF-16 (known as UCS-2 BOM).   This generated a file twice as large and our file comparator tests would fail.  We then found the iconv program to allow us to convert the extracted file back to UTF-8 and as a result we were able to get binary identity for the files!   

EXCEPT this didn't happen for all the shapescripts!  Further testing revealed that some (only a small number) were coming out as UCS-2 LE (little-endian) and the iconv program (expecting UCS-2 BOM converts the output to "Chinese characters".

All the extractions use this piece of code (NOTE: we always delete the files before each iteration, so we aren't accidentally bringing forward bad file specifications):

Code: [Select]

dim sImageAttributeDefault, sBase64Text, sImageText, iStartPoint, iLength
dim sEncodedShapescript
sBase64Text = "base64"&Chr(34)&">"
sImageText = "</Image>"
sImageAttributeDefault = oShapescriptImage.Default
iStartPoint = InStr(sImageAttributeDefault,sBase64Text)+8
iLength = InStr(sImageAttributeDefault,sImageText)-InStr(sImageAttributeDefault,sBase64Text)-8
sEncodedShapescript = Mid(sImageAttributeDefault,iStartPoint,iLength)
dim oStrB64File
Set oStrB64File = oFileSystemObject.CreateTextFile(sSTR_B64_FILE_REFERENCE, True)
oStrB64File.write(sEncodedShapescript)
oStrB64File.close
Consequently, can anyone (preferably a Sparxian) explain why sometimes EA decides to emit the sSTR_B64_FILE_REFERENCE file in different encodings?

TIA,
Reported,
Paolo

For those unfamiliar with the "Lilliputian" reference, see: https://en.wikipedia.org/wiki/Lilliput_and_Blefuscu

[qwerty]

Maybe it's connected with the zip code you're using? Anyhow, I never understood why the shape scripts needed to be packed at all. usuallly they are really short (under 1k) and in average the saving would be near to zero if not negative for not uncommon 3-liners.

Intersting observation though.

q.

[Geert Bellekens]

I also wrote a script to export shapescripts, but I don't remember having any encoding issues (not saying there aren't any, just that I never noticed any)

Here's the script I use: https://github.com/GeertBellekens/Enterprise-Architect-VBScript-Library/blob/master/Framework/Tools/Script%20Management/ExportAllShapeScripts.vbs

Geert

[Paolo F Cantoni]

I also wrote a script to export shapescripts, but I don't remember having any encoding issues (not saying there aren't any, just that I never noticed any)

Here's the script I use: https://github.com/GeertBellekens/Enterprise-Architect-VBScript-Library/blob/master/Framework/Tools/Script%20Management/ExportAllShapeScripts.vbs

Geert

Thanks, Geert,
Are you able to supply the most important bit...  The  decodeBase64zippedXML() function?

Paolo

[Geert Bellekens]

Yeah, that's in one of the !INCluded scripts:

https://github.com/GeertBellekens/Enterprise-Architect-VBScript-Library/blob/master/Framework/Utils/XML.vbs

Geert

[Paolo F Cantoni]

Yeah, that's in one of the !INCluded scripts:

https://github.com/GeertBellekens/Enterprise-Architect-VBScript-Library/blob/master/Framework/Utils/XML.vbs

Geert

Thanks, Geert,
However, I'm not sure it helps me much as I'm not using XML documents.  Nevertheless, it's given me some ideas.

Paolo

[Paolo F Cantoni]

Just to round this thread off...  I've managed to get binary identity on all the round-tripped files by investigating the iconv functionality and using a two-step process.  UCS-2LE BOM -> UTF-16 -> UTF-8.  It turned out the BOM wasn't being removed in the original process and the file size (once I'd stopped the corruption) would grow incrementally each round trip.  The file system equivalent to a memory leak, I guess.  Anyway, the two-step process strips the BOM information and generates a true UTF-8 file.

Paolo