Where is diagram layer data stored?

[Ian Mitchell]

I'm trying to find out which diagram objects are in which layers on a specific diagram.
As expected, the layers are saved in t_diagram.StyleEx, with an ID given to each.
I was expecting to find that layer ID in t_diagramobjects.Objectstyle, but nothing looks useful.
I even looked in t_xref, but nothing their either.
Anyone any ideas? I'm sure it's somewhere obvious...and I've lost my most recent copy of 'Inside EA' :-(

[MichaelJ]

Inside the StyleEx column of the t_diagram table there's a property pair called "Lyrs=<Layer Name>=<Layer Number>:....;". For example, "Lyrs=Layer1=1:Layer2=2:Layer3=3"

The same StyleEx column of t_diagram, ALSO contains an entry for each and every element "linked" to a specific layer on a diagram (though not visible in any way shape or form? Really!? Is is so hard to show this to a user!!?), using the following pattern

     "OPTIONS_<DUID>=Lyr=<Layer Number>:"

Please note, the DUID is the diagram object's ID in the context of a diagram and is not the object's ea_guid value.

[qwerty]

Ian, send me a mail and I send you a copy. But Layers are new stuff. I didn't feel like adding all that "crazy new stuff" added from V14 on.

q.

[Ian Mitchell]

Ah - thanks @MichaelJ - I didn't notice all the other stuff in the diagram.styleEx. So much magic in there...
A strange place to put this, when the obvious place is t_diagramObjects for information about which diagram element is in which layer.
And thanks Q - I just found a link to a 'read' version of 'InsideEA' to get it direct from LeanPub which I already bought.
@Q - I wonder if, given that you're not going to update it, whether InsideEA might move over to being a more collaborative effort, so you don't have to do all the updates. A wiki? This forum is great, but it's not structured in the same way as InsideEA, which can sometimes make things hard to find.

[qwerty]

That wiki had been discussed ages ago. If you feel the urge, go ahead. My liking in EA has ever been cooled down soo much since, say, V6.

q.

[MichaelJ]

Ah - thanks @MichaelJ - ... So much magic in there...

You're welcome.

There's so much "magic" because the team clearly don't follow sound data design practice, for example, as you mentioned, moving layer information into t_diagramobjects or let's go one step further and say that

• layers ought to be contained in their own table "t_layers" or even "t_diagramlayers"
• layer elements should be visible in the Layers toolwindow -- it's just common UX sense, this is a rare commodity in EA!
• should remove ALL element layer information from the DB when removing the layer from the front-end -- this does not happen currently, and is clearly a data BUG which is a sure sign the implemention was not tested before release

...My liking in EA has ever been cooled down soo much since, say, V6.

I'm sure you're not the only one ;-).

[Geert Bellekens]

The thing is, that the database structure hasn't changed in many years, and now apparently this has become a religion at Sparx HQ

"Thou shalt not change the Database Structure"!

To be nitpicky, I think the better name for this table would t_DiagramLayer without the "s"
Tablenames are typically singular

Geert

[qwerty]

Wait until you discover t_xref.

q.

[qwerty]

The thing is, that the database structure hasn't changed in many years, and now apparently this has become a religion at Sparx HQ

"Thou shalt not change the Database Structure"!

To be nitpicky, I think the better name for this table would t_DiagramLayer without the "s"
Tablenames are typically singular

Geert

Well, at least one new table > V13.5 (can't recall which and what for but Uffe had asked for it).

q.

[MichaelJ]

..."Thou shalt not change the Database Structure"!...

Lol! That's funny in the context of it being a religion at Sparx HQ.

The uneasy thought about this "religion" is that they are going down a world of hurt. It is not good data design to simply add key/value pairs ("metadata") to an object where no logical relationship exists between the object and the metadata. For example, in this post raised by Ian, they use "StyleEx" column to hold data about "layers" and "layer elements". As experienced modellers, we understand it is better to model this data inside one or more separate tables. These tables then hold data about the "layers" and their "layer elements", since there is a known relations to each other. (for example introducing the new "t_DiagramLayer" as you mentioned)

The fact they cram this "metadata" into a column that logically handles "styles", is evidence this was a conscious design decision by some (inexperienced) individual/developer. Sure, introduce new features with the least amount of change possible. No problem. However, this “Layers” feature is poorly implemented (as usual). It begs the question: was the implementation approved by actual modellers? Clearly, no. A product must be adapatable to future changes and to the needs of its customers. If DB structures must be changed to liberate new features, then surely, so-be-it. And the changes are handled carefully with  Development, Product, Test/QA before release.

What do you think, in your opinion, is the reason the EA team are not able to realise DB changes and resort to weird hacking of the data (which then introduces bugs into the product and just frustrates users)?

[qwerty]

I once had to coach a retrained welder (for the IT department). Judging from that guy I'd guess that Sparx employed retrained farmers.

q.

[Sunshine]

I once had to coach a retrained welder (for the IT department). Judging from that guy I'd guess that Sparx employed retrained farmers.

Bit harsh on Sparx there mate. Get out the wrong side the of the bed today?

I suspect back in the day like many vendors when they change the database schema it caused a whole world of pain for customers to upgrade so they tried not to change the schema since. To be fair they done a pretty good job of fitting new features in to the existing schema over the last decade or so. But I guess there is a balancing act. At some point there comes a time when they can't squeeze any more into the existing schema without making it real ugly and just have to bite the bullet. Maybe its close to that point now.
I'm sure most customers wouldn't mind running a DB script over their DB to upgrade the schema. Its just got to work without any issues.

[qwerty]

Bluntly, no. I would not mind if the DB design were not optimal. But that database has so many flaws that it can be called broken. And no attempt to fix it. It's just getting worse over time. I mean, they change the UI completely each version without having any pain (those customers are just a PITA, right?). But that DB is nothing more than a patchwork. With more patches coming each release.

q.

[Uffe]

"Thou shalt not change the Database Structure"!

Well, at least one new table > V13.5 (can't recall which and what for but Uffe had asked for it).

Well I've asked for a lot of things over the years, but never (explicitly) a new table. Did they name it after me?

Anyway, much as I too would love to see a cleanup of the schema (and there's a free band name if anyone wants it) I can't see it happening.

The fundamental problem is the poor design of the extensibility feature: the fact that users and third parties can add their own functionality through an API which exposes the DB schema. If it didn't, the schema could change as necessary and only the inside of the API, which is under Sparx' control, would need to be modified. The API would then hide all the magic tables and columns behind attributes in the various classes.

A new feature could then be implemented as a bit of a hack and cleaned up in a later release. But with all that third-party functionality out there using all those StyleExs and PDATAs, any hack gets frozen in carbonite the second the beta is released. And that in turn breeds a culture not of good design but of cobbling things together.

So that's what's broken, I think, but it was broken a long time ago.

I mean, they change the UI completely each version without having any pain (those customers are just a PITA, right?). But that DB is nothing more than a patchwork.

I would suggest that two core tenets of Sparxianism are clearly

• Thou shalt maintain backwards compatibility, yea, even unto the tenth major version, and
• Users can always be retrained

Or it might be that tinkering with the UI is easily done and easily tested, but getting into core functionality, especially if it's poorly structured and poorly documented which I suspect it probably is, is a different proposition entirely.

With more patches coming each release.

And there speaks a man who has taken it upon himself to document someone else's schema. I feel your pain, brother.


/Uffe

[qwerty]

Well I've asked for a lot of things over the years, but never (explicitly) a new table. Did they name it after me?

No, no. IIRC it was you asking for something and I discovered that new table where the requested was stored. I might as well have just dreamed of that?

Finding something in this "forum" is almost impossible. But I'll try to get back to that post...

q.

P.S.: https://www.sparxsystems.com/forums/smf/index.php/topic,44748.msg263574.html#msg263574