Training Test Macros

Objectives

This Training Module focuses on the basics of writing and executing tests. It covers the following topics:

• The C++ class Test Unit packaging option
• How to leverage Test Macros
• Using Notes and Test Logs, what is the difference?
• Creating Test Documentation via the build process
• Executing Tests using the Runner

The test unit is implemented in two source files: TestBasic.cpp and TestBasic.h. The comments and descriptions are contained in TestBasic.h.

The Test Unit has test cases already implemented (used for reference) and has a test method that you are required to implement (called Exercise). Currently this method is empty and returns a NOT IN USE status.

Instructions

Build and Run TestApp

• Build TestApp using the SDK makefile
• Invoke TestApp found in the /out/bin directory
• Create an option file (myoptions.txt) that includes the following content in either (SDK\Windows or SDK/Posix).

Windows

 ##### Command Line Options ######
 --device "TCP:localhost:8000"
 --database "%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb"
 --output "%STRIDE_DIR%\SDK\Windows\sample_src\TestApp.xml"
 --log_level all

Linux

 ##### Command Line Options #####
 --device "TCP:localhost:8000"
 --database "$STRIDE_DIR/SDK/Posix/out/TestApp.sidb"
 --output "$STRIDE_DIR/SDK/Posix/sample_src/TestApp.xml"
 --log_level all 

• Execute the Test Basic Test Units

 > stride --options_file myoptions.txt --run TestBasic
 Loading database...
 Connecting to device...
 Executing...
   test unit "TestBasic"
     > 3 passed, 1 failed, 0 in progress, 1 not in use.
   --------------------------------------------------------------
   Summary: 3 passed, 1 failed, 0 in progress, 1 not in use.

 Disconnecting from device...
 Saving result file...

You can also review the details of the test results using a Browser. Open TestApp.xml which can be found in the sample_src directory (based on the output option). By opening the xml file in a web browser the xsl is automatically applied to create html.

Implement Exercises

Now edit the training source code to complete the following exercises:

Exercise()

• Assignment 1: Add an srNOTE to Exercise() that will add a simple message to the test report (e.g. "Exercise ...")
  • Hint: You will probably want to read about Test Macros

• Assignment 2: Within Exercise() validate that sut_mult(1,1) does NOT equal sut_add(1,1)
  • Hint: You will use a Test Macro.

Check Results

• Before you rebuild TestApp.exe, you will need to shut it down by entering 'q' in its console window or closing the window directly.

• After rebuilding, invoke TestApp.exe once again.

• Execute only TestBasic using the stride runner:

 > stride --options_file myoptions.txt --run TestBasic

 Loading database...
 Connecting to device...
 Executing...
   test unit "TestBasic"
     > 4 passed, 1 failed, 0 in progress, 0 not in use.
   --------------------------------------------------------------
   Summary: 4 passed, 1 failed, 0 in progress, 0 not in use.

 Disconnecting from device...
 Saving result file...

• If you have TestApp.xml already open in your browser, you can simply refresh (F5), to view the latest test results.

Run and Publish Results

When you have completed the Exercise(s) publish your results to Test Space. To make it easier for now we recommend that you update your existing option file (myoptions.txt) with the following:

 #### Test Space options (partial) #####
 #### Note - make sure to change username, etc. ####
 --testspace https://username:password@yourcompany.stridetestspace.com
 --project Training
 --name YOURNAME

  > stride --options_file myoptions.txt --run TestBasic --space TestBasic --upload

 Loading database...
 Connecting to device...
 Executing...
   test unit "TestBasic"
     > 4 passed, 1 failed, 0 in progress, 0 not in use.
   ------------------------------------------------------------
   Summary: 4 passed, 1 failed, 0 in progress, 0 not in use.

 Disconnecting from device...
 Saving result file...
 Uploading to test space...

Validate Uploaded Results

Navigate to Test Space using your browser, then validate your results against the pre-configured baseline. For details, please see Confirming Training Exercise Results

Note: This space has been set up with a Baseline of expected test results that you can use to validate your results.

You can view the results on test space by pointing your browser at https://yourcompany.stridetestspace.com. Log in using the credentials you supplied earlier in the options file.

Reference

The following reference information is related to Test Unit basics.

Wiki

• Test Units Overview - Provides a general overview Test Units (i.e. writing tests in C and C++)
• Test Unit Packaging - Discusses the three types of packaging that can be used for Test Units (we prefer Test Classes even for c programmers)
• Test Macros - Optional macros that provide shortcuts for testing assertions and automatic report annotation (you will want to use these).
• Notes - Used to add logging information to your test logic (automatically added to test reports)
• Test Logs - Used to add logging information to your source code (added to test reports if enabled)
• Tracing - The Runner allows tracing on logs and test points.

Samples

• Test Class Sample - Specifically the Basic Simple Test Unit
• Test Macro Sample - This sample covers simple uses of each of the Test Macros.

Note - Other Test Unit Samples related to packaging that are useful are the following:

• Test C Class Sample
• Test Function List Sample