Sparx Systems Forum
Enterprise Architect => Automation Interface, Add-Ins and Tools => Topic started by: McMannus on April 03, 2015, 11:33:39 pm
-
Hi guys,
since our add-ins have gotten quite large and change impact cannot be always foreseen anymore, I'd like to introduce automated testing for the add-ins. I gave already some thoughts on that, but there are several things that I find complicated to solve.
Example 1: Let's say for example that there is functionality which simply creates a port on a component programmaticaly. The typical thing I'd do manually is creating a model containing the component and afterwards in the test case select that component, let the functionality run and afterwards check with API methods 1) if the port was created at all 2) and that it has the right properties.
Example 2: A port can also be created for the component by the user from within the toolbox. The difference is that the add-in reacts to the events and it should be checked if the reaction is correct, i.e. that the creation was not blocked by another add-in and that it has the right properties again afterwards.
Problems:
1. In both cases I have to create the right basic model manually, which is in my opinion too much work for all test cases.
2. Rolling back to the initial state of a test model is hard, because the functionality modifies the model, but EA has no means (at least what I know) for undoing modeling actions.
3. Occuring errors cannot be certainly classified to be caused by the add-in itself, since I also use the API for checking the results.
After finding those problems, I thought about creating a mock implementation for the repository that fits the needs for example 1, but triggered EA events cannot be simulated in that way. In addition, if you have multiple add-ins reacting to the same events, testing is even harder, because the order of execution is not always the same.
Question: Do you guys have any experience or even used automated tests already in your add-in projects? I'd really appreciate some other opinions on this topic.
Jan
-
Hi Jan,
The furthest I got to "simulated" testing is this little tool that simulates the EA extensions menu.
I've written a small article about it as well: Testing and debugging your Enterprise Architect C# Add-In (http://bellekens.com/2011/02/08/testing-and-debugging-your-enterprise-architect-csharp-add-in/)
Geert
-
I think this is a really interesting issue.
The main problem you have is that the AddIn events are only fired by EA UI actions. Perhaps Sparx can have a setting that can be enabled for AddIn testers to fire these events when the creating/deletion/etc... is created pro grammatically?
So I guess you will need to get one of the "GUI application" testing tools that operate as the "human" using your recorded action which can then subsequently be executed automatically. Using these with a combination of EA scripts, that you may need to write to do some further detailed checks to output results to an additional test log file during your test runs, could well address most of your requirements.
It may not work exactly as you would like, but at least it would save you the manual effort to perform the bulk of your testing.
BTW: It's been decades since I've used these testing tools but I assume they still exist, and I would imagine they are more powerful than I remember so could do even more.
Hope this helps, and if you do look at these tools I would be interested in feedback.
Adrian
-
Curious to check whether the approach I have suggested worked, I performed an experiment using a test AddIn. My tests automatically:
- Added content to an EA model
- Ran a check to verify the correct results - i.e. the AddIn worked
The experiment was successful :) and documented in my latest post Automatic testing of EA AddIns which can be found at ExploringEA.com.
Direct link to post - http://wp.me/p2RcMp-eW
-
Wow Adrian,
nice article. I will definitely also go into that direction for performing my integration tests.
-
Glad that I could provide a little help.
Also I'd be interested any feedback you have in applying to your testing which could be a lot different from my proof of concept. Any problems I'm always game for a challenge 8-)
-
We had some quite good experience with Ranorex. We use the software to validate our EA Extension.
You can record your mouse and key activities while interacting with EA and your extension. Ranorex will create test scripts which then can be replayed. They have some videos on their website which show the concepts.
You can add validation steps, for example checking if a specific element with defined stereotype appears in the Project brower. You can also modify your recordings to support data driven testing.
Sven