Setting up your CI Environment
To reap the maximum benefit from your test assets, you need to run your tests as part of a continuous integration environment.
In this scenario, the following automated steps take place on a regular, periodic basis (typically nightly or with each new checkin):
- source code check out from source control system
- build of the target executable code (including STRIDE runtime and tests)
- downloaded of executable image to the target hardware
- start of target code execution on the hardware
- running of all STRIDE tests and results publication to Test Space
The first four steps above are very build/platform specific and require specific domain knowledge to implement. There are many CI servers that help make the scheduling and execution of automated build/test cycles easier to manage. In particular, we've had good success with Hudson as it's both easy to set-up and flexible in its configuration. CruiseControl is another popular CI server and is worth serious consideration. For any particular build/test environment, there might be other tools that are more appropriate - so we recommend that you spend some time investigating the existing CI tools that are available. The WikiPedia article cited above is a good place to start as well as this matrix which compares some of the features or many popular CI servers.
The final step - executing and publishing - is easily accomplished with one or more calls to the stride runner. This tool is command -line based and should mesh well with any of the standard CI servers since most of these tools have the ability to execute command line processes.
Once these automated steps are running regularly, we recommend that the testspace to which the results are published be configured to notify on setbacks and errors. This gives the team maximum visibility of the automated tests. Any user that does not wish to be notified for the space can, of course, disable his notifications.
