Test Integration

In this article, we discuss establishment of continuous integration testing using STRIDE.

The starting point of this article assumes that your have completed the steps described in the article Test Integration.

Creating TestSpace Containers

As a first step, we must create a Project and Test Space into which our test results will be uploaded.

File:TestIntro Testspace.jpg

Newly created TestIntro Test Space

1. Using your web browser, navigate to your company's testspace site and login
2. Click on the All Projects tab, then click the Create New Test Space button.
3. If your company's site doesn't have any Projects defined you will be prompted to create a new project. If that is the case, follow these sub-steps:
   1. Fill in the Name as: Stride Deployment
   2. Fill in the Description as: sandbox for deployment tests
   3. Ensure that the Active box is checked
   4. Click the Create button
4. At the top of the New s2_test_space page, enter text as follows:
   • Name: TestIntro
   • Description: testintro sample tests
   • Max results to keep: 10
5. If you would like to grant access to other users, check the appropriate boxes in the additional users section
6. For now, skip the auto compare options, you can change these (and other options) later if desired
7. In the Project list box, choose Stride Deployment. If this isn't available, click the New Project... link and enter the project name and description as described earlier
8. Finally, click the create button
9. On the subsequent page that is displayed, you will be able to manually upload results data, but leave the browser as it is, we will be automatically uploading results in a later step.

Adding the TestIntro Tests to the Target Build

Here we will add the [Test Intro Sample|TestIntro sample tests]] to the target build. We will use the same technique used earlier in the Test Integration to incorporate these tests.

The SCL pragmas that declare the TestIntro test units are in the file s2_testintro_test.h, so this file needs to be supplied as input to the STRIDE build tools. If you are using the Makefile technique described in TestIntegration, you simply need to copy s2_testintro_test.h to a designated SCL_DIRS directory so that it will be picked up as input to s2scompile.

In addition, you need to include the TestIntro source files in the target compile/link.

Once the build is complete, download the binary to the target in preparation of running the tests.

Running and Publishing the TestIntro Tests

To execute the TestIntro tests, use a Windows or Linux host computer that has connectivity with the target system via your configured STRIDE transport (TCP/IP or seral). Additionally, make the generated STRIDE database (.sidb) visible to the host computer via a shared filesystem or a file copy to the host system.

If not already present, install the appropriate Host Tools package on the host computer.

We will now run stride with options instructing it to upload results to your company's TestSpace site. The command line gets rather long when all the needed options are specified, so it's recommended that you create an options file to submit the command line options.

Create a text file named TestIntroOpts.txt and add the following text. You will need to fill in your particular argument values where brackets [ ] are shown. Note that the --project and --space argument values must match pre-existing containers on your TestSpace site.

-r /(*) 
--database [path]/stride.sidb 
--device [arg] 
-u
--testspace [https://mycompany.stridetestspace.com@user:pwd]
--project "Stride Deployment"
--space TestIntro

To run and upload the tests, ensure that your target system is running and that your host computer is connected to the Internet, then enter the following in a host console:

stride -O TestIntroOpts.txt

The text below should be output to your host console as the test progresses.

Loading database...
Connecting to device...
  runtime version: 4.1.01
Executing test units...
  s2_testintro_cclass
    > 1 passed, 1 failed, 0 in progress, 0 not in use.
  s2_testintro_flist
    > 2 passed, 1 failed, 0 in progress, 0 not in use.
  s2_testintro_testdoubles
    > 3 passed, 0 failed, 0 in progress, 0 not in use.
  s2_testintro_testpoints
    > 3 passed, 0 failed, 0 in progress, 0 not in use.
Summary: 9 passed, 2 failed, 0 in progress, 0 not in use.
Disconnecting from device...
Saving result file...
Uploading to test space...

Notice that the last line indicates that the results are being uploaded to test space.

Viewing Results in TestSpace

Newly created TestIntro Test Space

Returning to your web browser, login to your company's test space site if needed and navigate to the All Projects page.

You should now see the TestIntro test space listed under the Stride Deployment project, and see that TestIntro's Result Set Count is now 1.

Click on the TestIntro link to be presented with the TestIntro space page. At the bottom of the page you will see a table displaying the uploaded result sets for this test space. Click on the Sequence_1 link to show detailed results for this test run.

Automated Regression Testing

best to set things up so that you automatically

• build,
• load binaries on the hardware,
• start target running,
• run Stride tests
• publish results to test space

Organizing Tests

put test results in suites

develop tests with namespace conventions

Publication of Test Results

Upload to test space

comparison with previously uploaded data

immediately see trends in

• number of tests in each suite and overall

Best Practices

Use C++ Test Classes if Possible

If possible, use C++ Test Classes to write your test units. Test Classes offer extra features and are the easiest to use for programmers.

Use Namespacing to Organize Tests

Over time, you will end up with lots of test units. Grouping these tests for display or running will become important as the count of test units grows. (You can see an example of C namespacing in the TestIntro tests.)

---