Training Test Macros

Objectives

This Training Module focuses on the basics of writing and executing tests. It covers the following topics:

• The different Test Unit packaging options
• How to leverage Test Macros
• Using Notes and Test Logs, what is the difference?
• Creating Test Documentation via the build process
• Executing Tests using the Runner

There are two test files used -- TestBasic.cpp & TestBasic.h. They implement three Test Units, each of which uses a different packaging option:

• TestBasic_FList
• TestBasic_CClass
• TestBasic_Class

All of the test methods in each test unit have already been implemented (used for reference) except for the tests called TestExercise. Currently these methods are empty and return a NOT IN USE status.

IMPORTANT NOTE - The test methods (cases) within each Test Unit are exactly the same, regardless of packaging. (In fact the CClass and Class Test Units just call the C-based test logic used by the FList Test Unit.) The only difference is in the packaging of the test methods into each Test Unit.

Instructions

Build and Run TestApp

• Build TestApp using SDK makefile
• Startup TestApp
• Create an option file (myoptions.txt) using the following content (Windows example)

 ##### Command Line Options ######
 --device "TCP:localhost:8000"
 --database %STRIDE_DIR%\SDK\Windows\out\TestApp.sidb
 --output %STRIDE_DIR%\SDK\Windows\sample_src\TestApp.xml
 --log_level all

• Execute Test Basic Test Units only

 > stride -O myoptions.txt --run TestBasic_FList TestBasic_CClass TestBasic_Class

 Loading database...
 Connecting to device...
 Executing...
   test unit "TestBasic_FList"
     > 3 passed, 2 failed, 0 in progress, 1 not in use.
   test unit "TestBasic_CClass"
     > 3 passed, 2 failed, 0 in progress, 1 not in use.
   test unit "TestBasic_Class"
     > 3 passed, 2 failed, 0 in progress, 1 not in use.
   --------------------------------------------------------------
   Summary: 9 passed, 6 failed, 0 in progress, 3 not in use.

 Disconnecting from device...
 Saving result file...

• You can also review the details of the test results using a Browser. Open TestApp.xml which can be found in the sample_src directory (based on the output option). By opening the xml file in a web browser the xsl is automatically applied to create html.

Implement Exercise

• TestBadFailure()
  • Change the Test Macro type used here such that subsequent test methods will execute
• TestExercise()
  • Add a NOTE indicating that TestExercise is executing (i.e. "Test Exercise ...");
  • Validate that sut_mult(1,1) does NOT equal sut_add(1,1)

• Execute only TestBasic_FList

 > stride -O myoptions.txt --run TestBasic_FList

 Loading database...
 Connecting to device...
 Executing...
   test unit "TestBasic_FList"
     > 4 passed, 2 failed, 0 in progress, 0 not in use.
   --------------------------------------------------------------
   Summary: 4 passed, 2 failed, 0 in progress, 0 not in use.

 Disconnecting from device...
 Saving result file...

Run and Publish Results

When you have completed the Exercise(s) publish your results to Test Space. To make it easier for now we recommend that you update your existing option file (myoptions.txt) with the following:

 #### Test Space options (partial) #####
 #### Note - make sure to change username, etc. ####
 --testspace https://username:password@yourcompany.stridetestspace.com
 --project Training
 --name YOURNAME

  > stride -O myoptions.txt --run TestBasic_FList TestBasic_CClass TestBasic_Class --space TestBasic --upload

 Loading database...
 Connecting to device...
 Executing...
   test unit "TestBasic_FList"
     > 4 passed, 2 failed, 0 in progress, 0 not in use.
   test unit "TestBasic_CClass"
     > 4 passed, 2 failed, 0 in progress, 0 not in use.
   test unit "TestBasic_Class"
     > 4 passed, 2 failed, 0 in progress, 0 not in use.
   ------------------------------------------------------------
   Summary: 12 passed, 6 failed, 0 in progress, 0 not in use.

 Disconnecting from device...
 Saving result file...
 Uploading to test space...

Note: This space has been set up with a Baseline of expected test results that you can use to validate your results.

Reference

The following reference information is related to Test Unit basics.

Wiki

• Test Units Overview - Provides a general overview Test Units (i.e. writing tests in C and C++)
• Test Unit Packaging - Discusses the three types of packaging that can be used for Test Units (we prefer Test Classes even for c programmers)
• Test Macros - Optional macros that provide shortcuts for testing assertions and automatic report annotation (you will want to use these).
• Notes - Used to add logging information to your test logic (automatically added to test reports)
• Test Logs - Used to add logging information to your source code (added to test reports if enabled)
• Tracing - The Runner allows tracing on logs and test points.

Samples

• Test Function List Sample - Specifically the Basic Simple Test Unit
• Test Macro Sample - This sample covers simple uses of each of the Test Macros.

Note - Other Test Unit Samples related to packaging that are useful are the following:

• Test Class Sample
• Test C Class Sample