Problem to check-in to PTC

[ngong]

For some weeks now, I am using "Package Control" together with PTC, prior to roll it out in the team. Once you get used to it, it seems to be quite productive.
However, today I tried to check-in a package and got the message MKS125628: Could not compute text delta ... (see attached).
I repaired the data base, compacted it. But nothing helped.

Any suggestion?

(how to attach a screenshot?)
The whole failure message from PTC is: MKS125628: Could not compute text delta. Likely cause is that you are attempting to check in binary data into a text archive.
I am using EA13 producing into an .eap file, PTC/Integrity 10.7.0.7925 with APIU 4.15.7925.

[qwerty]

I assume you found https://support.ptc.com/apps/solution_preview/solution/lang/presolution?lang=en&n=CS115699 (I don't know PTC).

To attach a picture you need to place it no a public server and use the link with the picture icon.

q.

[ngong]

Thank you, qwerty.

the failure screenshot can be looked at here: http://picpaste.de/2017-05-10-NoCheckIn-aNLWHHyE.PNG

Yes, the failure seems to be identified: PTC is justifying the new package xml as binary and refuses to check it in, over a text file. (Strange!)

As my current user account at PTC is not strong enough to view the link you, qwerty, supplied (I love open source) I found this link:
https://www.ptcusercommunity.com/thread/40277

They propose - as a workaround - to format the XML. However, it was formatted and I formatted it newly using Eclipse XML plugin. Nothing solved the issue.
Could it be a characterset issue?

[qwerty]

I would guess so. You could scan the file with a little <any scripting language here>-script. Non-ASCII should be re-coded using &-notation. I guess that might help.

q.

[ngong]

I intensified my analysis and found out, that the characterset idea was not the solution. It was a lengthy line that did not go away when formatting the XMI file using Eclipse formatter. For some reason, EA writes the UML:TaggedValue tag="$ea_attsclassified" and the UML:TaggedValue tag="$ea_oprsclassified". For one UML:Class in the XMI holds line 21222 and line 21223 each with 16695 characters.

I substituted that class by a new one and got rid of the problem.
What are these tags for?

[KP]

What are these tags for?

$ea_attsclassified holds the GUID of every attribute typed by the current class. $ea_oprsclassified holds the GUID of every operation whose return value is typed by the current class.

[ngong]

Thank you KP,

i understand: if the current class is int or uint8_t or the like, which is heavily used throughout the model, it es very likely, that $ea_attsclassified gets very long.
If it exceeds 16k, I got trouble using the version control with PTC/Integrity-Source.

Is there any chance to get line breaks in the value of $ea_attsclassified?

[qwerty]

Since it's XML you can likely convert it yourself with a little (XSLT) script.

q.

[ngong]

Has not been that easy, qwerty:

• The long line can grow very long: currently I am at 150k and the model is less than 20% through.
  
• [The long lines in the XMI file are values of just two Tagged values $ea_attclassified and $ea_oprsclassified, that is: breaking the lines has to be done on String level, not XML.
  

They look like
UML:TaggedValue tag="$ea_attsclassified" value="{B4A1C8DF-1D75-4226-A88B-258C986E2C37},{6E172079-E70F-408F-9630-915719CD029D},{B9C03329-21FF-4120-A27D-F27A7A6A0399}, ... (>150k characters, maybe grows to 1M)

I could have used tokenize function of xslt and substitue every },{ by },\n{.
However, it seems to me likely that I would see heap overflow, if the xslt processing copies attribute values for some reason.
And also it sounds sane to me to put the changes to the EA-XMI file to a minimum, leaving the lines just below 16k.

My solution was a transformation utility (~200 lines POJO Java) that does just that, while paying heed to buffer sizes. I was lucky that the inserted newlines did not harm the EA-import of the XMI file or violate some checksum or the like.
My procedure is as follows:

• try to check-in the package
• accept the failure
• insert as few as possible newlines
• check in using PTC
• synchronize version control state in EA

Do you think, I could find a better solution? This one would be hard to sell to my users.

Kind Regards


Rolf

p.s. BTW, I was told by the PTC maintenance that there is an unwritten rule, that a printable String in Java should not contain more than 16k consecutive characters without a newline. I did not know that one. However, if there is some truth in it, it might be an argument for Sparx not to have such long attribute values in their XMI. Even though the better solution would be, that I can define to PTC how to distinguish a character from a binary file.

[qwerty]

If context permits, a single regex can probably do the job. Not much effort and easy to test. Probably worth a try.

q.

[skiwi]

Why click when you can embed using 'Insert Image' in the BB toolbar

[Eve]

p.s. BTW, I was told by the PTC maintenance that there is an unwritten rule, that a printable String in Java should not contain more than 16k consecutive characters without a newline. I did not know that one. However, if there is some truth in it, it might be an argument for Sparx not to have such long attribute values in their XMI. Even though the better solution would be, that I can define to PTC how to distinguish a character from a binary file.

That "rule" doesn't make any sense at all when handling a defined file format (xml) which doesn't have any such restriction.

Interestingly, XML exported by EA uses normalized newline characters inside attribute values. So a value containing a newline will still be on a single line.

[ngong]

Hi qwerty,
looking forward to your regex, finding long lines and inserting a newline at the nearest comma just less than 16536, say POS, and nearest but less than 16536+POS, say POS1 and nearest 16536+POS+POS1...

But the Java code is not the real question. The real question would be: How to automate the procedure to break the attribute string and check the package in.

As it looks for me, EA+PTC seems not to be a very good combination.

Kind Regards

Rolf

[qwerty]

Probably this Python script will work without any regex:

Code: [Select]

out = open("some.xml", "w") # create test data
out.write(",".join(["some text "*1630]*3))
out.close()

breakPos = 16000
out = open("clean.xml", "w")
for line in open("some.xml"):
    if len(line) < breakPos:
        out.write(line)
        continue
    p = 0
    while True:
        pNext = line.find(",", p+breakPos)
        if pNext < 0:
            out.write(line[p:])
            out.write("\n")
            break
        pNext += 1
        out.write(line[p:pNext])
        out.write("\n")
        p = pNext
out.close()

It will split lines longer than 16000 chars after the next comma following multiples of 16k. So what I'd do (besides voting against PTC as versioning system) it to use a script for the check-in that will post-process EA's XML with the above.

q

q.

[ngong]

super qwerty,

that is similar to what I did in Java You did it in python, not a regex, and if the "," is not in the next 384 chars, the line gets too long. To achieve searching backwards, you got to introduce some  buffer handling. Add also a good response for any failure that may happen and also looking for non-printable characters, you will get to ~100 lines, what is ~200 in Java. Think to deliver your tool to some other team members abroad.

However, the question remains: If I open in EA the Package Control dialog and choose check-in, how to pipeline the xml through your tool prior to hand it over to PTC? Avoiding the failure message that comes with check-in or put.

Kind Regards


Rolf