Book a Demo

Author Topic: How should we model test plans?  (Read 16076 times)

Fredrik Israelsson

  • EA Novice
  • *
  • Posts: 4
  • Karma: +0/-0
    • View Profile
How should we model test plans?
« on: March 19, 2009, 03:12:08 am »
Hi!
I am working in a group of system developers and system testers that have just started to try to use an analysis diagram as a communication link between a developer and a tester. I would like to have comments and suggestions regarding this, because we are not at all sure that this is the best way to do it.

We connect an analysis diagram to the use case that is to be implemented, which means that this diagram resides in the Primary Use Cases package.
When drawing the diagram, we are trying to capture the developer's idea of how the use case is going to be implemented. We want to visualize required input, expected output and also all possible error states. Actions that may trigger the error states should also be included somehow.
Our intention is to make the diagram possible to read directly as a test plan for the tester, which means that the tester should be comfortable with looking at it as a replacement for a document saying "do this, then do this, then the system should respond like this". The testers should somehow be able to mark the diagram as "tested", just like in a test report written in words.
Unfortunately, our testers are not at all comfortable with this replacement yet, and we feel that we have a long way to go before the diagram could take the form we want it to have.

Do you have any comments on this? Is this possible at all? Is it good or bad to try to use an analysis diagram? I know that sequence diagrams should be used to describe details of a use case, but how could you read a sequence diagram as a test plan? How do you capture error states, for instance.

You may have noticed that I am not an UML or a process expert. I guess that we are quite a few in this world that try to use EA without learning the process properly first. I just hope we will get better along the way...

Thank you very much in advance, everyone!

Best regards,
Fredrik

jeshaw2

  • EA User
  • **
  • Posts: 701
  • Karma: +0/-0
  • I'm a Singleton, what pattern are you?
    • View Profile
Re: How should we model test plans?
« Reply #1 on: March 19, 2009, 04:30:35 am »
I'm not sure just what you mean my the term Analysis Diagram, but...
I define use case Scenarios for both requirements gathering and test specifications.  I let the stakeholders define the scenarios and provide certification test data.  Then, before delivery, each scenario is run to successful completion.

Does this help?
Jim
Verbal Use Cases aren't worth the paper they are written upon.

Fredrik Israelsson

  • EA Novice
  • *
  • Posts: 4
  • Karma: +0/-0
    • View Profile
Re: How should we model test plans?
« Reply #2 on: March 19, 2009, 05:07:38 am »
Ok, I understand. Do you write the scenarios in the place holders provided for each use case definition in EA? When the testers have run a scenario, and the test have passed, where do you mark that the scenario has passed the test? Are you able to search the model in order to find use cases passed, failed or missing scenario tests?

jeshaw2

  • EA User
  • **
  • Posts: 701
  • Karma: +0/-0
  • I'm a Singleton, what pattern are you?
    • View Profile
Re: How should we model test plans?
« Reply #3 on: March 19, 2009, 06:54:34 am »
Naw...

We write Use Cases in Textual form and document the scenarios therein.  Then we keep a Master Use Case & Scenario List in MS Excel.  We get a larger, more easily shared, view of the project that way.  

Our Use Case Diagrams are rather simple, more like icons than fully descriptive diagrams.  Only the most important scenarios are added to the diagram.  We do this more for the stakeholders benefit than for project management issues.  

We do, however, develop simple Activity Diagrams for each Scenario for communication with stakeholders.  We find that stakeholders understand Activity Diagrams better than they understand Sequence Diagrams while developers might make good use of the Sequence diagram.

HTH
Jim
« Last Edit: March 19, 2009, 06:58:24 am by jeshaw2 »
Verbal Use Cases aren't worth the paper they are written upon.

KP

  • EA Administrator
  • EA Expert
  • *****
  • Posts: 2919
  • Karma: +55/-3
    • View Profile
Re: How should we model test plans?
« Reply #4 on: March 19, 2009, 09:23:40 am »
Have a look at the Testing window (Alt+3 to open it). If you select your use case and right-click in the testing window, there is a command "Import element scenario(s)". Use that to create a test case for each scenario. Also consider setting your use case to display in rectangle notation, and then switch on display of testing information on the diagram (Diagram > Properties > Elements > Testing). Experiment a bit and check out the help - there may be something there that you can use...
The Sparx Team
[email protected]

Dermot

  • EA Administrator
  • EA User
  • *****
  • Posts: 591
  • Karma: +7/-0
    • View Profile
Re: How should we model test plans?
« Reply #5 on: March 19, 2009, 12:01:24 pm »
There is more detail available on this in the Testing Whitepaper - see:
http://www.sparxsystems.com.au/resources/whitepapers/index.html

bioform

  • EA User
  • **
  • Posts: 230
  • Karma: +0/-0
  • Forty-Two?
    • View Profile
Re: How should we model test plans?
« Reply #6 on: March 20, 2009, 03:30:26 am »
Exactly... You really should look closer at use case and business/analysis domain modeling and how you can leverage that with testing!

Use Cases should be the area where the requirements and scenarios exist. Thus you are able to exploit the scenarios with test as the EA rep indicated.

My approach is to:
  • Identify the business objects (things the business sees as "things" that are within the scope of the project), document glossary definitions,
  • Concurrently identify the use cases (conceptual level) that will implement the features of the system, that the customer needs to be able to fulfill their business use cases (these I model very lightly as I am using them as a stand-in for the lack of a BPM) and get agreement.
  • Identify the "basic" business level scenarios that the system will participate in - these can be used as the starting point of a system walk-through (thus following a specific end-to-end pathway to the outcome)

     Note: From these business scenarios, you can begin to understand the context of what the system use cases need to accomplish (requirements) and you can start adding detail to each of the system use cases encountered along the business scenario walkthrough. During this you should be able to: identify scenarios for the SysUC (basic 1st) and then describe the flow of the Actor/System interaction within the scope of the SysUC - I use activity diagrams for this... The activities within those diagrams serve as anchor points for requirements specific to an activity, further linked to UI fragments, domain objects, etc. where additional requirements are defined.
  • Update the SysUCs by adding the additional information described in the note above and the business domain objects that are involved.


     Note: IT is important to understand that the "Activity" diagrams are NOT design constraints, but an analysis flow of an interaction with the system. It is so easy to step over the line and describe the solution as the requirements, which it is NOT. The solution is just that, one solution that is intended to meet the requirements.)
  • Review the activities (steps in the flow) and their requirements with test and define "how" these will be validated at this high level (e.g., demonstration, inspection) and add some high level details. In EA I capture these as a type of user requirement called "Acceptance Criteria" and these must be reviewed with the customer. Once this is done, you can use EA's "testing" features that are available for ANY EA element, and document your initial UA tests there,

Wash and repeat this process, and you will end up with a set of SysUCs that accurately describe the functional (and some of the non-functional requirements), have domain element requirements, link to UI requirements (which can then be used as a source of input for the UI wireframes), and have a set of robust scenarios and scripts (which can be derived from the activity diagrams by following a specific scenario's path through that use case, using a set of data corresponding to the linked domain objects...)

My point being that you can develop a process that will provide everyone with what they want:
  • Analyst - Source of Requirements, and Scenarios that describe a set of interactions with the system
  • Test Teams - Source of Scenarios to be used to develop the test cases for UA, End-to-End, System/Integration, etc. and a set of data that was used to validate the UA "tests" during analysis and can be used to further explore fragments of scenarios using variations on that data (explore alternate paths, exception paths)
  • Developers - Functional Requirements for specific system capability and the ability to understand them in the business context, and see what the customers expected outcomes will be.
  • - Managers - A model to be mined for use with scenario driven releases, iterative development, or classic water fall process. You can extract metrics to show progress, risk, difficulty, etc.


It is possible to have one location be the source of "truth" which starts out vague and evolves into an accurate representation of the system under development's capabilities and requirements. By extending the traceability to realizations, then accurate/repeatable impact analysis is possible for change control, defect analysis, etc.

The model then will be something of value to maintain, exploit, and explore what-if scenarios.

MAN did I get on a soapbox or what?!!

Just a great tool that needs to be used with a thought out process.

David - BTW doing/expanding this on real-life projects!
[/list]
Time is what keeps everything from happening at once, Space is what keeps it all from happening to you. <unknown>

bioform

  • EA User
  • **
  • Posts: 230
  • Karma: +0/-0
  • Forty-Two?
    • View Profile
Re: How should we model test plans?
« Reply #7 on: March 25, 2009, 10:08:56 am »
Hmmm... Come on tear of what I suggested apart!

What are other people doing?

A good book that I am just starting to read through is:
Essential Software Testing, A use Case Approach. By Greg Fournier, CRC Press, 2009, ISBN:978-1-4200-8981-3

bioform
Time is what keeps everything from happening at once, Space is what keeps it all from happening to you. <unknown>

jeshaw2

  • EA User
  • **
  • Posts: 701
  • Karma: +0/-0
  • I'm a Singleton, what pattern are you?
    • View Profile
Re: How should we model test plans?
« Reply #8 on: March 25, 2009, 11:28:51 am »
Bravo Bioform!  Well said!  I'm  continually amazed at how many Business Analysts don't understand this.  It is what I preach from my pulpit and I've seen many development projects go down in flames when they don't understand the true value of Use Cases and how their development should be approached.

Jim
Verbal Use Cases aren't worth the paper they are written upon.

bioform

  • EA User
  • **
  • Posts: 230
  • Karma: +0/-0
  • Forty-Two?
    • View Profile
Re: How should we model test plans?
« Reply #9 on: March 27, 2009, 12:51:50 am »
Thanks Jim.

I would like to organize an effort to work the classic ATM example from business concept to Deployment using EA.

Discussion and selection of a methodology (maybe OpenUP), discussion about how we might configure EA to support the method, and then talk through the individual processes we would follow.

My first step is to start identifying web tools that we can use to manage the effort (probably limited to number of users that does not require purchase, <10?) and then identify those individuals that could be the project/effort gurus.

Then use the forum to post and work the project, e-mail to distribute the EA project file, etc. ... If anyone would be interested, just send me a message.

Bioform
Time is what keeps everything from happening at once, Space is what keeps it all from happening to you. <unknown>

Robert Sheridan

  • EA User
  • **
  • Posts: 24
  • Karma: +0/-0
  • I love YaBB 1G - SP1!
    • View Profile
Re: How should we model test plans?
« Reply #10 on: March 28, 2009, 01:45:08 am »
Very useful post.  I am not as advanced as use but have been exploring the business - system use case relationship and how we can reuse them both in future iterations and current testing.

Going to share this link with our testers and see where we go from there.

Robert