Book a Demo

Author Topic: Modelling testcases in UML  (Read 29750 times)

Geert Bellekens

  • EA Guru
  • *****
  • Posts: 13523
  • Karma: +574/-33
  • Make EA work for YOU!
    • View Profile
    • Enterprise Architect Consultant and Value Added Reseller
Modelling testcases in UML
« on: September 28, 2009, 06:14:50 pm »
Hi,

We are planning to model our testcases on a functional level.
A TestCase for us is related to one specific usecase scenario, and it describes what needs to be executed, and what the desired results are for this testcase.
We are thinking of modelling this using usecase instances. Each testcase is an instance of the usecase from which it tests a scenario. In the scenario of the TestCase we will then describe the flow and desired results.

What do you think of this approach, and how do you model testcases?

Any input is greatly appreciated.

Geert

Oliver F.

  • EA User
  • **
  • Posts: 573
  • Karma: +2/-1
  • Aren´t we all in the model business ?
    • View Profile
    • Karl Storz homepage
Re: Modelling testcases in UML
« Reply #1 on: September 28, 2009, 09:16:08 pm »
Personally I have not done any test modelling but the guys downstairs use quality centre to maintain test scenarios from use cases.
To integrate with EA they use an internal toolset from Siemens (which is not publically accessible as far as I know) but Canoic has the following solution:

http://www.canoniccorp.com/products-cadeo-eaqc.aspx

Maybe this helps.

Oliver

NeilS

  • EA User
  • **
  • Posts: 27
  • Karma: +0/-0
    • View Profile
Re: Modelling testcases in UML
« Reply #2 on: September 28, 2009, 11:35:12 pm »
Hi Geert

Your approach makes sense to me but a test case is different to a use case - and is not an instance of one.

Your typical QA engineer would be interested in maximum field lengths, illegal characters etc and the usual things that excite QA engineers but not the rest of us!

I haven't thought this through but couldn't you stereotype a use case with "test case."  You could then add tag values to add specific values if appropriate.

Neil

Geert Bellekens

  • EA Guru
  • *****
  • Posts: 13523
  • Karma: +574/-33
  • Make EA work for YOU!
    • View Profile
    • Enterprise Architect Consultant and Value Added Reseller
Re: Modelling testcases in UML
« Reply #3 on: September 29, 2009, 03:46:35 pm »
Neil, Oliver, thanks for your input.

The reasoning behind the choice to use an instance of a usecase to represent a TestCase is that one testcase actually is a "run"  of (a single scenario of) the usecase with a specific dataset.
So there might be several different testcases for one scenario that test the various conditions and cases that could occur in real life.
We are also not modelling the testcases for the test engineers. They are only there to provide the functional analysts with an overview of the automated tests.
This allows us to see which cases are covered by automated testing, and which cases aren't. In the latter case that means that we have to be extra carefull when changing something to the implementation of the usecase, or that we might consider creating a new testcase for this usecase.

Geert

Paolo F Cantoni

  • EA Guru
  • *****
  • Posts: 8626
  • Karma: +259/-129
  • Inconsistently correct systems DON'T EXIST!
    • View Profile
Re: Modelling testcases in UML
« Reply #4 on: September 29, 2009, 04:03:19 pm »
Quote
Neil, Oliver, thanks for your input.

The reasoning behind the choice to use an instance of a usecase to represent a TestCase is that one testcase actually is a "run"  of (a single scenario of) the usecase with a specific dataset.
[size=18]...[/size]
Geert
Geert,

You've touched on a "bete noir" of mine... You're quite correct in saying that: a test case (execution) is a "run"  of (a single scenario of) the usecase with a specific dataset.

However, there's also the problem of the specification of the TestCase.  It seems to me you need both - the instance of the TestCase and its specification (as a Classifier).  In a sense, it seems to me that the TestCase Instance is an intersector between the UseCase Specification and the Test Case Specification.

I do keep vacillating on whether what I've just said is the right approach or not, so I thought I'd take the opportunity to get others' feedback.

Paolo
Inconsistently correct systems DON'T EXIST!
... Therefore, aim for consistency; in the expectation of achieving correctness....
-Semantica-
Helsinki Principle Rules!

Geert Bellekens

  • EA Guru
  • *****
  • Posts: 13523
  • Karma: +574/-33
  • Make EA work for YOU!
    • View Profile
    • Enterprise Architect Consultant and Value Added Reseller
Re: Modelling testcases in UML
« Reply #5 on: September 29, 2009, 04:16:59 pm »
Paolo,

I thought about creating a classifier for the TestCase as well, and I think you are correct in that we should have one, but...
The TestCase classifier does not offer us added value at the time, only added work. For now (this could change in the future, but it doesn't seem likely) this TestCase classifier would not contain any information different from that of the usecase.
I therefore opted not to create the TestCase classifier and consider the UseCase instances to be syntactic sugar for expressing the fact that there should be a TestCase classifier with some sort of relation to the usecase, and instances of this TestCase classifier to represent the actual tests.

Makes sense?

Geert

Paolo F Cantoni

  • EA Guru
  • *****
  • Posts: 8626
  • Karma: +259/-129
  • Inconsistently correct systems DON'T EXIST!
    • View Profile
Re: Modelling testcases in UML
« Reply #6 on: September 29, 2009, 06:40:15 pm »
Yes Geert, it does make sense - that's why I keep swapping my viewpoint.

I agree that you should only create the classifier if it is of value.  My thinking is that the TestCase Classifier is a transform of the UseCase to which is added additional metadata that is common to the different TestCase instances (new for each run).

If you like, the classifier is the TestCase as it SHOULD be, the instance is as it was actually run.  The trick is to keep the amount of management to a minimum and automate as much as possible.

Paolo
Inconsistently correct systems DON'T EXIST!
... Therefore, aim for consistency; in the expectation of achieving correctness....
-Semantica-
Helsinki Principle Rules!

son-of-sargasso

  • EA User
  • **
  • Posts: 122
  • Karma: +0/-0
    • View Profile
Re: Modelling testcases in UML
« Reply #7 on: September 29, 2009, 09:38:18 pm »
Woohoo! My favourite topic.  I'll see if I can condense 15 years into a single post.  Paolo, you may have to translate this into UML.

You're both right.  And on the first steps of a fairly long journey.  So let's make a couple of truism statements.

1) A use case is a model of an outcome of an interaction between a system user and the system.  The outcome can be achieved or not acheived.
So, we have some scenarios, basic, alt and etc that describe for a particular set of conditions, whether the specified outcome will or will not be achieved.  In sophisticated models, the designer can even specify what should happen if the outcome is and is not achieved.

2) From the testing pov, we are interested in the scenarios of each use case.  Because that lets us classify our outcomes as "wins" or "losses".  This is important because all wins must win and all losses must lose.  If not then the system does not operate as expected/desired.  

3) Stakeholders cannot understand 2), [size=9](which has made me fairly comfortably well off and generally despised by a small group of designers and developers BTW)[/size]  but they do more or less understand a well presented use case.

Proposal: A test case is a model of system usage by one or more actors with a specified set of conditions.  

It is convenient, for the purpose of reporting to stakeholders as to whether a specific release of the system is of sufficient quality to warrant implementation, to talk about whether a given set of use cases, targeted for that release, do or do not provide outcomes aligned with the agreed scenarios.  Further, it is important to re-assure the stakeholders that existing user facility will not be degraded by the implementation.

So, for a given release of the system, we want to establish a set of test cases whose results, taken as a whole can be communicated to the stakeholders in a manner that lets them make an educated guess as to whether it will be beneficial, within the scope and mandate of the project (expressed or unexpressed), to allow the release to be implemented.

At this point I'm going to raise that most dreaded of all use cases, "Log In".  It has an outcome that I will specify as, "The user, attempting to access the system at that node, is granted a set of privileges within the system according to their registered profile for a certain time period given that they are known to the system, have presented valid credentials and are not currently accessing the system from the same or any other node currently accessing the system."  I realize that this is probably the best description of said supposed use case that you have seen  ;)  but it's for illustrative purposes.
 
There are but two scenarios (so far), a) the user is granted said privileges and b) the user is denied said privileges.  Now lets add a couple of commonly seen system functionalities i) you only get three tries and if you fail then 30000V is delivered through the keyboard, ii) you are informed that the system has never heard of you, but because you look like a nice guy then we'll let you register as a new user, and c) you are told that you are obviously a moron but if you calm down and count to 10 slowly, we'll send you an email with your password (but only if you know your mother's milkman's next door neighbour's dog's collar colour).  Hee hee hee, so now we have use cases with extensions based on a negative outcome.

So where do test cases fit in to all this?  Well, they're obviously not use case instances and given my last bit of ratbaggery, they are not scenarios either.  Further, they are not a single classifier as they have temporal features i.e. a result per release (In fact they can have a result per test cycle, but I've only got 1200 characters left, so I'm going for some brevity here).

In fact, I reckon that use cases are from earth, EA is from the moon, UML is from Mars and users are from a small planet near Betelgeuse (and Dog knows where the heck stakeholders are from).   Test cases apparently have to tie these all together.

But after all, they are just a model. ;)  A model expressing how an (giggle) impartial observer might view the system in (giggle) question.

(Not giggling now).  Neither UML nor EA has an adequate way of expressing this model (in my experience).  The best you can achieve, within these two constraints is to get the use case/scenario model right. Then and only then have you basis for creating a related test model.

I have spent over ten years doing this, exactly.  When I first met Paolo, I was still trying to merge testing into EA models.  Since then I have developed a system that utilizes use cases and scenarios from an EA model as the basis for a test model.  They do not belong in the same model, I think.  

I've only got a few chars left in this post so, just let me know whether you want to hear more or not.  Because I have sure got a lot more to say on this matter (as well as others in case you're not worried yet)  
br
bruce

NeilS

  • EA User
  • **
  • Posts: 27
  • Karma: +0/-0
    • View Profile
Re: Modelling testcases in UML
« Reply #8 on: September 30, 2009, 12:38:32 am »
Feel free to correct me but I don't think that the UML Gods ever sought to solve this problem!  Whatever you do you're off the road as far as UML goes..

As you say I think you either
1) do not represent this in the model at all OR
2) bastardise UML to get where you want to go (and hope that v2.3 of the Superstructure spec does not render your work obsolete!).

Certainly if option 2) is taken I'd be definitely trying to keep the use case and test case distinct - the audience is different.

Neil



Eve

  • EA Administrator
  • EA Guru
  • *****
  • Posts: 8110
  • Karma: +119/-20
    • View Profile
Re: Modelling testcases in UML
« Reply #9 on: September 30, 2009, 08:35:37 am »
Bruce, I at least get where you are coming from.  It's good stuff, although usually my alternate scenarios involve 30000V being delivered through the mouse.  Maybe that's where I'm going wrong. ;)

Geert Bellekens

  • EA Guru
  • *****
  • Posts: 13523
  • Karma: +574/-33
  • Make EA work for YOU!
    • View Profile
    • Enterprise Architect Consultant and Value Added Reseller
Re: Modelling testcases in UML
« Reply #10 on: September 30, 2009, 03:52:21 pm »
Thank you all for your suggestions. I'm certainly going to take them into consideration. (especially the ones that involve sending large amounts of electricity though parts of the human body ;D)

So to summarize, UML does not provide us with something specifically aimed for testcases.
There is no standard way of modelling testcases in UML (or EA).

On the other hand, UML is designed as a language to express software intensive systems, and my testcases will be developed as software. So I should be able to use UML to model these testcases.

For the moment I'm going to stay with my initial approach and see where that gets me.
For the current purpose of providing an inventory of testcases I think it will be sufficient.

Geert

son-of-sargasso

  • EA User
  • **
  • Posts: 122
  • Karma: +0/-0
    • View Profile
Re: Modelling testcases in UML
« Reply #11 on: October 09, 2009, 10:02:36 am »
[OT] Sorry it took so long to get back here.  I have just done a wonderful tour of Coober Pedy, William Creek, Lake Eyre, Wilpena Pound, Clare Valley and the Murraylands with my daughter.  2300km over some of the remotest roads in Aus in a 2WD Lancer, and only one flat tyre![/OT]

I just wanted to clarify something.  If you accept my premise that test cases fit somewhere near/beneath scenarios, then as a scenario is not a UML element, per se, just an attribute of a use case, you cannot express a test case in UML.  (To my way of thinking anyhow.)

I'll try and find time to knock up a diagram to show what I've learned.


bruce
« Last Edit: October 09, 2009, 11:01:38 am by barrydrive »

Paolo F Cantoni

  • EA Guru
  • *****
  • Posts: 8626
  • Karma: +259/-129
  • Inconsistently correct systems DON'T EXIST!
    • View Profile
Re: Modelling testcases in UML
« Reply #12 on: October 09, 2009, 10:53:45 am »
Quote
[size=18]...[/size]
If you accept my premise that use cases fit somewhere near/beneath scenarios
[size=18]...[/size]
Hi bruce!
Welcome back (again!), sounds a great trip!

Did you mean Test Case (not Use Case) in the clause above?

Paolo
Inconsistently correct systems DON'T EXIST!
... Therefore, aim for consistency; in the expectation of achieving correctness....
-Semantica-
Helsinki Principle Rules!

son-of-sargasso

  • EA User
  • **
  • Posts: 122
  • Karma: +0/-0
    • View Profile
Re: Modelling testcases in UML
« Reply #13 on: October 09, 2009, 11:01:04 am »
Yep, will mod.

son-of-sargasso

  • EA User
  • **
  • Posts: 122
  • Karma: +0/-0
    • View Profile
Re: Modelling testcases in UML
« Reply #14 on: October 09, 2009, 01:09:53 pm »
A couple of "by the ways".

  • The EA testcase (Alt+3) does provide a rudimentary facility to record testcases against usecases.  I did not find this powerful enough, nor did it allow alignment with scenarios as previously explained.  YMMV.
  • By allowing for test cases to exist without a related scenario, a means to manage non-functional testcases is opened.  But ...
  • I have tried to relate TCs to EA requirement elements a couple of times.  However, in reality the usecase model tends to relate more to the delivered product rather than the requirement spec.  (reality vs. expectation?  :) )  Also, the use case model often provides more functionality than the requirement spec, i.e. it gets ahead of the requirement spec.
  • In my experience, the only stakeholders who are interested in seeing that the test plan coverage vs the requirement spec are auditors.  Most other stakeholders readily adapt to (and prefer) the test plan coverage vs usecase model.  But, YMMV again.
bruce