Difference between revisions of "Test Units Overview"

From STRIDE Wiki
Jump to: navigation, search
(→‎Unique STRIDE Test Unit Features)
 
(341 intermediate revisions by 11 users not shown)
Line 1: Line 1:
−
==Introduction==
+
== What are STRIDE Test Units? ==
  
−
STRIDE enables testing of C++ code through the use of xUnit style test classes. Test classes can be written by engineers, captured using an SCL pragma, and executed from the host. STRIDE facilitates the execution of some or all of the test classes by automatically creating entry points for the execution of test classes on the target.
+
'''STRIDE Test Units''' is a general term for [http://en.wikipedia.org/wiki/XUnit xUnit-style] test modules running within the STRIDE runtime framework. These tests--written in C and C++--are compiled and linked with your embedded software and run in-place on your target hardware. They are suitable for both developer unit testing as well as end-to-end integration testing.
  
−
==Using C++ test classes==
+
An external [[Stride Runner|Test Runner]] is provided which controls the execution of the tests and publishes test results to the local file system and optionally to S2's Internet [[STRIDE Test Space]].
  
−
===Prerequisites===
+
== Test Unit Features ==
 +
In all cases, STRIDE Test Units provide the following capabilities typical of all [http://en.wikipedia.org/wiki/XUnit xUnit-style] testing frameworks:
 +
* Specification of a test as a test method
 +
* Aggregation of individual tests into test suites which form execution and reporting units
 +
* Specification of expected results within test methods (typically by using one or more [[Test Code Macros]])
 +
* Test fixturing (optional setup and teardown)
 +
* Test parametrization (optional constructor/initialization parameters)
 +
* Automated execution
 +
* Automated results report generation
  
−
The CPP Test class support only works in the 2.0.0802 version of STRIDE or later.  The host PC must also have a recent distribution of ActiveState perl installed.
+
=== Unique STRIDE Test Unit Features ===
 +
In addition, STRIDE Test Units offer these unique features:
 +
; On-Target Execution
 +
: Tests execute on the target hardware in a true operational environment. Execution and reporting is controlled from a remote desktop (Windows, Linux or FreeBSD) host
 +
; Dynamic Test and Suite Generation
 +
: Test cases and suites can be created and manipulated at runtime
 +
; [[Using Test Doubles|Test Doubles]]
 +
: Dynamic runtime function substitution to implement on-the-fly mocks, stubs, and doubles
 +
; [[Test Point Testing in C/C++|Behavior Testing]] (Test Points)
 +
: Support for testing of asynchronous activities occurring in multiple threads
 +
; Multiprocess Testing Framework
 +
: Support for testing across multiple processes running simultaneously on the target
 +
; Automatic Timing Data Collection
 +
: Duration are automatically measured for each test case.
 +
; Automatic Results Publishing to Local Disk and Internet
 +
: Automatic publishing of test results to [[STRIDE Test Space]]
  
−
===How to get started===
+
[[Category:Test Units]]
−
 
 
−
The following is an outline of the required steps to get started with writing C++ test classes.
 
−
 
 
−
* Create a new Studio workspace (or open an existing one).
 
−
* Set the workspace to cpp mode (for compilation).
 
−
* Add '''%STRIDE_DIR%'''\inc\srtest.h to the Source Files folder of your workspace.
 
−
* Add '''%STRIDE_DIR%'''\scripts\TestClasses\TestClass_Preprocess.pl (or .js) to your Script Files folder.  This script is a preprocessor and code generator.  It searches all source files in the current workspace and looks for test classes that have been captured via the ''scl_test_class'' pragma. It then generates c-linkage wrapper functions that instantiate and execute each test class.  By default, this generated source file has a name of the form {WORKSPACE_NAME}TCR.cpp and will be located in the workspace directory. If you need to change the name and/or location of this generated file, simply set the '''outFile''' property of the '''''STRIDE.testclass_codegen''''' object prior to calling the generate method.  If you will be adding a script to compile the workspace (see next step), this preprocessing script must always '''preceed''' the compilation step.
 
−
* Add a script to compile the workspace.  In it's simplest form, this script can simple be something like:
 
−
  #perl syntax
 
−
  Win32::OLE->Option(Warn => 3);
 
−
  $main::studio->Workspace->Compile();
 
−
 
 
−
  // jscript syntax
 
−
  studio.Workspace.Compile();
 
−
* Add a script to generate the Intercept Module(IM).  For the simple STUB generation required for C++ test class execution, you can use '''%STRIDE_DIR%'''\scripts\TestClasses\TestClass_Preprocess.pl (or .js) for this purpose.
 
−
* Add scripts to build and execute your application.  If you are using a host based simulator, we examples available of both. If you are using actual devices, the steps required for building and starting the application are specific to the target environment.
 
−
* Create one or more test classes to implement your C++ test logic.  See below for more information on creating test classes.
 
−
* Ensure that the Studio workspace include path has the location to all of your test class declaration (header) files.
 
−
* Once you have created one or more test classes, save the workspace and run the scripts, starting with the TestClass_Preprocess script, followed by compile, IM generation, and application building.
 
−
* [''Optional''] If your application is running, you can test execute individual test classes interactively using the Studio interface view.  Open the user interface view corresponding to the test class you would like to execute and call it.  The return values will indicate how many tests produced each of 4 result types.  Furthermore, the input to the entry point will allow you to select all methods or execution (the default) or individual methods -- this is done via a dropdown list of enumerated values.
 
−
* Once you are confident that the test classes are behaving as expected, you can generate one or more execution scripts using the Script Wizard.  Sample templates for executing test class entry points are provided in the '''%STRIDE_DIR%'''\templates\Script Wizard directory.
 
−
* for integration with larger regression test workspaces, we recommend that engineers check-in their test class code and, optionally, the template generated scripts that can be used to execute their test classes.
 
−
 
 
−
==CPP test class requirements==
 
−
 
 
−
The STRIDE test class framework requires the following of each test class:
 
−
* The test class must have a suitable default (no-argument) constructor.
 
−
* The test class must have one or more public methods suitable as test methods.  Allowable test methods always take no arguments (void) and return either void or simple integer types (int, short, long, char or bool).  At this time, we do not allow typedef types or macros for the return values.
 
−
* the scl_test_class pragma must be applied to the class.
 
−
 
 
−
===Simple example using return values for status===
 
−
 
 
−
  #include <srtest.h>
 
−
 
 
−
  class Simple {
 
−
  public:
 
−
    int tc_Int_ExpectPass(void) {return 0;}
 
−
    int tc_Int_ExpectFail(void) {return -1;}
 
−
    bool tc_Bool_ExpectPass(void) {return true;}
 
−
    bool tc_Bool_ExpectFail(void) {return false;}
 
−
  };
 
−
  #ifdef _SCL
 
−
  #pragma scl_test_class(Simple)
 
−
  #endif
 
−
 
 
−
===Simple example using runtime test service APIs===
 
−
 
 
−
  #include <srtest.h>
 
−
 
 
−
  class RuntimeServices_basic {
 
−
  public:
 
−
    void tc_ExpectPass(void)
 
−
    {
 
−
        srTestCaseAddComment(srTEST_CASE_DEFAULT, "this test should pass");
 
−
        srTestCaseSetStatus(srTEST_CASE_DEFAULT, srTEST_PASS, 0);
 
−
    }
 
−
    void tc_ExpectFail(void)
 
−
    {
 
−
        srTestCaseAddComment(srTEST_CASE_DEFAULT, "this test should fail");
 
−
        srTestCaseSetStatus(srTEST_CASE_DEFAULT, srTEST_FAIL, 0);
 
−
    }
 
−
    void tc_ExpectInProgress(void)
 
−
    {
 
−
        srTestCaseAddComment(srTEST_CASE_DEFAULT, "this test should be in progress");
 
−
    }
 
−
  };
 
−
  #ifdef _SCL
 
−
  #pragma scl_test_class(RuntimeServices_basic)
 
−
  #endif
 
−
 
 
−
 
 
−
===Simple example using srTest base class=== 
 
−
 
 
−
  #include <srtest.h>
 
−
 
 
−
  class MyTest : public stride::srTest {
 
−
  public:
 
−
    void tc_ExpectPass(void)
 
−
    {
 
−
        testCase.AddComment("this test should pass");
 
−
        testCase.SetStatus(srTEST_PASS, 0);
 
−
    }
 
−
    void tc_ExpectFail(void)
 
−
    {
 
−
        testCase.AddComment("this test should fail");
 
−
        testCase.SetStatus(srTEST_FAIL, 0);
 
−
    }
 
−
    void tc_ExpectInProgress(void)
 
−
    {
 
−
        testCase.AddComment("this test should be in progress");
 
−
    }
 
−
    int tc_ChangeMyName(void)
 
−
    {
 
−
        testCase.AddComment("this test should have name = MyChangedName");
 
−
        testCase.SetName("MyChangedName");
 
−
        return 0;
 
−
    }
 
−
    int tc_ChangeMyDescription(void)
 
−
    {
 
−
        testCase.AddComment("this test should have a description set");
 
−
        testCase.SetDescription("this is my new description");
 
−
        return 0;
 
−
    }
 
−
 
 
−
  };
 
−
  #ifdef _SCL
 
−
  #pragma scl_test_class(MyClass)
 
−
  #endif
 

Latest revision as of 17:09, 25 September 2013

What are STRIDE Test Units?

STRIDE Test Units is a general term for xUnit-style test modules running within the STRIDE runtime framework. These tests--written in C and C++--are compiled and linked with your embedded software and run in-place on your target hardware. They are suitable for both developer unit testing as well as end-to-end integration testing.

An external Test Runner is provided which controls the execution of the tests and publishes test results to the local file system and optionally to S2's Internet STRIDE Test Space.

Test Unit Features

In all cases, STRIDE Test Units provide the following capabilities typical of all xUnit-style testing frameworks:

  • Specification of a test as a test method
  • Aggregation of individual tests into test suites which form execution and reporting units
  • Specification of expected results within test methods (typically by using one or more Test Code Macros)
  • Test fixturing (optional setup and teardown)
  • Test parametrization (optional constructor/initialization parameters)
  • Automated execution
  • Automated results report generation

Unique STRIDE Test Unit Features

In addition, STRIDE Test Units offer these unique features:

On-Target Execution
Tests execute on the target hardware in a true operational environment. Execution and reporting is controlled from a remote desktop (Windows, Linux or FreeBSD) host
Dynamic Test and Suite Generation
Test cases and suites can be created and manipulated at runtime
Test Doubles
Dynamic runtime function substitution to implement on-the-fly mocks, stubs, and doubles
Behavior Testing (Test Points)
Support for testing of asynchronous activities occurring in multiple threads
Multiprocess Testing Framework
Support for testing across multiple processes running simultaneously on the target
Automatic Timing Data Collection
Duration are automatically measured for each test case.
Automatic Results Publishing to Local Disk and Internet
Automatic publishing of test results to STRIDE Test Space
Retrieved from "https://www.stridewiki.com/index.php?title=Test_Units_Overview&oldid=13977"