Book a Demo

Author Topic: UML driven testing  (Read 8853 times)

bjones

  • EA Novice
  • *
  • Posts: 12
  • Karma: +0/-0
  • If i Post mY passworD herE i won't fOrget iT  ;)
    • View Profile
UML driven testing
« on: January 31, 2007, 10:21:03 am »
Hi,

I wrote a tool that uses UML scenarios to drive tests.
It discovers interfaces using introspection for COM objects, Web Interfaces, and .Net assemblies.

I am using exports of EA scenarios to generate the test templates, then override one or more actors with the TestTool to generate the tests.  

When I run the tests, initially they return "not implemented" and "not verified" because they contain pseudo code names for everything based on the object model names in EA.

I then manually map the pseudocode names to the discovered methods on the binary I wish to test. And manually add parameterization with a matrix override.  Hence a={1,2,3} and b={4,5,6} will generate nine test cases.  My paramiterization can use xpath assignments, and can apply to any node, entity, or defined variable.

Based on this work flow, it would be helpful if a requirement object had a schema that would accept a structured language definition:
<Requirement>
 <Capability></Capability>
 <Conditions>
   <Condition></Condition>
 </Conditions>
 <Constraints>
   <Constraint></Constraint>
 </Constraints>
 <Verifications>
   <Verify></Verify>
 </Verifications>
 <ModelLink></ModelLink>
</Requirement>

I could then autogenerate the condition validation steps and the verification steps directly from EA as well.

We express Conditions as pseudo states (i.e. Material.Present)  and Verifications as either conditions or assertions.

I haven't yet schematized the matrix paramiterization description.

«Midnight»

  • EA Guru
  • *****
  • Posts: 5651
  • Karma: +0/-0
  • That nice Mister Grey
    • View Profile
Re: UML driven testing
« Reply #1 on: January 31, 2007, 10:30:22 am »
Take a look here:http://www.sparxsystems.com.au/cgi-bin/yabb/YaBB.pl?board=general;action=display;num=1170174449;start=1#1Perhaps you'd like to get involved when (if) this gets rolling.

In the meantime, this would likely fit well with formatted Notes in elements (of all kinds, not just requirements). Lately we're mostly talking about RTF, but perhaps XML is the way to go.

Could you just include the XML in the Requirement Details field, or provide a reference to external XML via the Files tab? Like you I'd like to keep this inside the EA model (via the schema) but perhaps an external file would be enough for now.
No, you can't have it!

bjones

  • EA Novice
  • *
  • Posts: 12
  • Karma: +0/-0
  • If i Post mY passworD herE i won't fOrget iT  ;)
    • View Profile
Re: UML driven testing
« Reply #2 on: January 31, 2007, 10:41:13 am »
I would like to do a lot of data mining for "completeness" and "comprehensiveness" type reviews.  

Hence I have another whole level of structure I would like to innclude as well.

For instance, I classify each requirement into one of these categories:

Commanded Behavior
Required Behavior
Information Display
Transformation
Work Piece
SYnchronization

Each type has a schema that requires more information.

For instance, a Synchronization requirement has a master, one or more slaves, a data set, and quality of service information.

A query based on a slave actor name would give me a full set of requirements for that actor.

If I add the Structured language to the Notes, then I must export everything before I can do queries.

It would be must nicer if I could attach an xsd to the note field so that the type definition would determine the structure of the field.

bjones

  • EA Novice
  • *
  • Posts: 12
  • Karma: +0/-0
  • If i Post mY passworD herE i won't fOrget iT  ;)
    • View Profile
Re: UML driven testing
« Reply #3 on: January 31, 2007, 10:48:03 am »
Sorry,

The direct answer is that I would much prefer to have it in the model.

«Midnight»

  • EA Guru
  • *****
  • Posts: 5651
  • Karma: +0/-0
  • That nice Mister Grey
    • View Profile
Re: UML driven testing
« Reply #4 on: January 31, 2007, 02:31:45 pm »
What about creating a metamodel with the elements you describe? Then you could associate these with a requirement, and provide the structure through relationships.

Each individual element could have a Notes compartment that applies only to that element.
No, you can't have it!

bjones

  • EA Novice
  • *
  • Posts: 12
  • Karma: +0/-0
  • If i Post mY passworD herE i won't fOrget iT  ;)
    • View Profile
Re: UML driven testing
« Reply #5 on: January 31, 2007, 04:31:36 pm »
Hmm,

Ouch, That is really painful for data entry. If I am doing it right.  I am having to open a dialog for every element of the schema?  

I'll try some more tomorrow.  Thanks.
« Last Edit: January 31, 2007, 04:39:50 pm by bjones »

Paolo F Cantoni

  • EA Guru
  • *****
  • Posts: 8626
  • Karma: +259/-129
  • Inconsistently correct systems DON'T EXIST!
    • View Profile
Re: UML driven testing
« Reply #6 on: January 31, 2007, 06:53:31 pm »
Quote
[size=13][SNIP][/size]
For instance, I classify each requirement into one of these categories:

Commanded Behavior
Required Behavior
Information Display
Transformation
Work Piece
Synchronization
[size=13][SNIP][/size]
We haven't got to adding the testing to our model, but we are interested in getting in to this in the near future.

Would you be able to provide a more normative definition of each of these categorizations?  Also, do you apply them as stereotypes?

TIA,
Paolo
Inconsistently correct systems DON'T EXIST!
... Therefore, aim for consistency; in the expectation of achieving correctness....
-Semantica-
Helsinki Principle Rules!

«Midnight»

  • EA Guru
  • *****
  • Posts: 5651
  • Karma: +0/-0
  • That nice Mister Grey
    • View Profile
Re: UML driven testing
« Reply #7 on: February 01, 2007, 03:12:55 am »
Also, what about patterns? Would EA's support in this area be able to lessen the work of using a metamodel?
No, you can't have it!

bjones

  • EA Novice
  • *
  • Posts: 12
  • Karma: +0/-0
  • If i Post mY passworD herE i won't fOrget iT  ;)
    • View Profile
Re: UML driven testing
« Reply #8 on: February 01, 2007, 08:11:08 am »
Quote
<SNIP>

Would you be able to provide a more normative definition of each of these categorizations?  Also, do you apply them as stereotypes?

TIA,
Paolo


Currently I am just starting with EA so the only thing I currently do with it is draw UML diagrams and export them. But I am hoping to make it central to our whole Requirements Engineering and Testing processes.

I will take the discussion of our processes to another discussion so as not to clutter the new features requests.
See General Board:Requirements Engineering:Capability Types
« Last Edit: February 01, 2007, 08:57:21 am by bjones »

bjones

  • EA Novice
  • *
  • Posts: 12
  • Karma: +0/-0
  • If i Post mY passworD herE i won't fOrget iT  ;)
    • View Profile
Re: UML driven testing
« Reply #9 on: February 05, 2007, 09:50:02 am »
In order to give proper credit I have gone back and found where I got the classification system.

Michael Jackson published some works through IEEE years ago and now has a book on it.

http://www.amazon.com/exec/obidos/ASIN/020159627X/qid=967405083/103-5241225-1144601

I will purchase the book just to refresh my memory.

bjones

  • EA Novice
  • *
  • Posts: 12
  • Karma: +0/-0
  • If i Post mY passworD herE i won't fOrget iT  ;)
    • View Profile
Re: UML driven testing
« Reply #10 on: February 28, 2007, 11:22:07 am »
Quote
...

In the meantime, this would likely fit well with formatted Notes in elements (of all kinds, not just requirements). Lately we're mostly talking about RTF, but perhaps XML is the way to go.


Well, I've played a bit more...
There are two levels of formatted text that I would use.
The first helps in parsing the elements of a well-formed requirement. (Capability, Constraints, Conditions). For conditions, I am viewing them as states and I express them as StateMachineName.State (it should have the option for a natural language description for documentation purposes.  This can be used to stub out two test methods: Verify_StateMachine.State and Set_StateMachine.State.

For constraints, I am adopting a set description. I haven't flushed out all the details. But I see constraints as limits on parameterization, hence the set description paradigm.

Constraints would set the boundaries for good values, the tester can select values outside the boundaries to force failures.

So where they are kept in the model must permit "mining" the information for test generation.

Quote
Could you just include the XML in the Requirement Details field, or provide a reference to external XML via the Files tab? Like you I'd like to keep this inside the EA model (via the schema) but perhaps an external file would be enough for now.


For now I am just exporting the sequence diagrams and manually adding conditions and constraints.  I am looking for a long term, more user friendly approach so I can get our processes more widely adopted.

I have been playing with tagged values. Perhaps these can meet the need.