Running Tests (old)

Target-based Test Units are controlled by a host computer, physically connected to the target via a configurable communication channel (TCP/IP or serial port). This article describes some common usage patterns and presents several examples.

Prerequisites

In order to run target-based Test Units, the following prerequisites are required.

Host

• Stride Host Tools installed on the host computer; specifically, the Stride Runner application.

Target

• Target transport configured
  • TCP/IP over port 8000 is the default target transport. To apply other configurations, see Host Transport
• Target built with STRIDE library built in and one or more test cases
• Target app running

Preparing to Run Your Tests

Tests are executed on the target by running the program stride on the host computer. See the linked article for reference information.

In order for stride to run your tests, the STRIDE database (.sidb) file that was produced as part of the target build must be accessible to the program via the host filesystem. This is trivial to accomplish when the target is built on the host computer (as when building and running tests off-target) or when the host and target computers are running the same operating system and are connected via LAN. But for dissimilar operating systems, you will need to mount the target filesystem on the host computer. (Many of our customers use Samba to accomplish this.)

By default, all tests present in the database are executed in ascending alpha-numeric order (by name) (This can be overridden; see below.)

Running STRIDE Diagnostics

S2 Technologies supplies a set of diagnostic tests that can be used to verify correct operation.

By building in the files srdiag.h and srdiag.c (located in the SDK/Runtime directory of each SDK) to your target application, these tests become available.

Assuming that:

• your STRIDE database is named TestApp.sidb and resides in the current directory, and
• the corresponding target application is running on the host computer

the command line to run diagnostic tests is:

stride --database=TestApp.sidb --device=TCP:localhost:8000 --diagnostics

Console Output

Information is written to the console while the test progresses (note that the link tests can take ~30 seconds), and a results summary is written at the conclusion of the test.

Loading database...
Connecting to device...
  runtime version: 4.1.01
Diagnosing target application...
  link
  payload fragmentation
  runtime configuration
  stub-proxy deadlock
  target characteristics
  ---------------------------------------------------------------------
  Summary: 64 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

Detailed Output

Detailed results output are also created and written to the local disk.

Since we did not specify an output file to stride above, the xml output is written by default to the directory containing the designated STRIDE database. The name of the file is taken from the database file name with the .sidb replaced by .xml.

Additionally, a companion .xsl file is downloaded and written alongside the xml file if you are connected to the Internet. If you open the xml file in a browser, the xsl stylesheet will be automatically applied to render html to the browser.

Desktop Test Development

Useful for 'pre-flight' desktop testing as well as target debugging, the following command line runs the target tests without publishing results to STRIDE Test Space.

We assume the following:

• We are testing against a target build that contains Test Units that I don't want to run. (Perhaps they test code that I am not responsible for.)
• The target device is running, and is available on the LAN via TCP/IP at 192.168.0.53
• I want to run only Test Units named myTest1, myTest2, and myTest3

The command line to run the desired tests would be:

stride -r "myTest1,myTest2,myTest3" --database="/my/path/targetApp.sidb" --device=TCP:192.168.0.53:8000

A couple of things to note:

• If others may be running tests using referenced database, you should specify a unique output file location (via the -o option) to prevent overwriting the output file at the default location.
• If you will be running many tests against a specific database and/or device, consider setting environment variables to the corresponding values, or put command line options into an options file (via the -O option).
• You can refer to the device address using a DNS name if desired (e.g. TCP:testcomputer:8000)

Publishing Test Results

Saving Results to the Local Disk

When stride is executed to run tests, the following files are written to the local disk. By default, they are written to the directory in which the database is specified. You can specify a different location by using the -o option when you run stride. If these files already exist, they are overwritten.

[database_name].xml

Raw xml test output.

[database_name].xsl

Stylesheet to format the xml file for viewing in a web browser. (This file is automatically downloaded only if you are connected to the Internet.)

Displaying Local Results Data

If you open the xml file in a browser, the xsl stylesheet (if present) will be automatically applied to the xml so that the data is rendered in html within the browser.

Publishing Results to STRIDE Test Space

stride can also be instructed to upload test results to STRIDE Test Space.

Example: Desktop Testing with Publication to Test Space

This example demonstrates the running of specified target tests followed by publication of the results to STRIDE Test Space.

Here, we create an options file since the command line is getting very long. (We could put several of these option values in environment variables instead if desired.)

The option file is named myOptions.txt, is located in the current directory and contains the following text:

--database "/my/path/targetApp.sidb"
--device TCP:192.168.0.53:8000
-r /(myTest1,myTest2, myTest3)
-u
--testspace https://username:password@mycompany.stridetestspace.com
--project "Super Repeater"
--space "Input Processor"
-o myoutputfile.xml

The command line is then:

stride -O "myOptions.txt"

Note: In order for publishing to succeed, the username/password must be correct and the project and space must already exist. If you are using TestSpace for the first time, you should check with your administrator about how to create projects and spaces.

Continuous Integration Testing

TBD

---