Training Tests in C/C++
Contents
Background
The STRIDE Framework provides support for implementation of tests in the native C/C++ of the device under test. Once written, these tests are compiled using the device toolchain and are harnessed, via the STRIDE Intercept Module into one or more applications under test on the device. These tests have the unique advantage of executing in real-time on the device itself, allowing the tests to validate actual device conditions during test.
Please review the following reference articles before proceeding:
Why would I want to write tests in native code ?
Here are some of the scenarios for which on-target test harnessing is particularly advantageous:
- direct API testing. If you want to validate native APIs by driving the APIs directly, native code is the simplest way to do so. STRIDE provides convenient assertion macros to validate your variable states. API testing can also be combined with native expectation tests (using Test Point instrumentation) to provide deeper validation of expected behavior of the units under test.
- unit testing of C objects or C++ classes. The STRIDE native tests execute in the same context as the rest of your code, so it's possible to fully unit test any objects that can be created in your actual application code.
- validation logic that requires sensitive timing thresholds. Sometimes it's only possible to validate tight timing scenarios on target.
- high-volume data processing scenarios. In some cases, the volume of data being processed and validated for a particular test scenario is to large to be easily handled by an off-target harness. In that case, native test units provide a convenient way to write tests that validate that data without shipping the data off-target during testing.
What's more, you might simply prefer to write your test logic in C or C++ (as opposed to perl on the host). If that's the case, we don't discourage you from using a toolset that your more comfortable with - particularly if it enables you to start writing tests without a new language learning curve.
Are there any disadvantages ?
Sure. Most of the disadvantages of native on-target tests concern the device build process. If your device build is particularly slow (on the order of hours or days), then adding and running new tests can become a tedious waiting game. Testing is always well served by shorter build cycles, and on-target tests are particularly sensitive to this.
In some cases, the additional code space requirements of native tests is a concern, but this concern is also mitigated by ever-increasing device storage capacities. On platforms that support multiple processes (e.g. embedded linux or WinMobile), it's possible to bundle tests into one or more separate test process, which further mitigates the code space concern by isolating the test code in its own application.
Samples
For this training, we will be using some of the samples provided in the C/C++_Samples. For any sample that we don't cover here explicitly, feel free to explore the sample yourself. All of the samples can be easily built and executed using the STRIDE off-target framework.
The first three samples that we cover are introductions to the three different test unit packaging mechanism that we support in STRIDE. A good overview of the pros and cons for each type is presented here. The last sample we discuss is the TestPoint sample, which demonstrates expectation testing in native code on target (i.e. both the generation and validation of the test points are done on target).
TestClass
This sample shows a variety of techniques available for packaging and writing test units using classes. If you have a C++ capable compiler, we recommend that you use test classes to package your unit tests, even if your APIs under test are C only. Review the source code in the directory and follow the sample description here.
observations:
- all of these example classes have been put into one or more namespaces. This is just for organization purposes, mainly to avoid name collisions when built along with lots of other test classes. Your test classes are not required to be in namespaces, but it can be helpful in avoiding collisions as the number of tests in your system grows.
- we've documented our test classes and methods using doxygen style comments. This documentation is automatically extracted by our tools and added to the results report - more information about this feature is here
- you can optionally write test classes that inherit from a base class that we've defined (stride::srTest). We recommend you start by writing your classes this way so that your classes will inherit a few methods and members that make some custom reporting tasks simpler.
- exceptions are generally handled by the STRIDE unit test harness, but can be disabled if your compiler does not support them (see s2_testclass_basic_exceptions_tests.h/cpp).
- parameterized tests are supported by test classes as well. In these tests, simple constructor arguments can be passed during execution and are available at runtime to the test unit. The STRIDE infrastructure handles the passing of the arguments to the device and the construction of the test class with these arguments. Parameterization of test classes can be a powerful way to expand your test coverage with data driven test scenarios (varying the input to a single test class).
TestFList
This sample demonstrates a simpler packaging technique that is appropriate for systems that support C compilation only (no C++). FLists are simple a collection of functions that are called in sequence. There is no shared state or data, unless you arrange to use global data for this purpose. Review the source code in the directory and follow the sample description here.
observations:
- flist tests support setup/teardown fixturing, but not parameterization or exception handling.
- we've again provided documentation using doxygen formatting for these samples. However, because there is no storage-class entity with which the docs are associated in an FList, there are some restrictions to the documentation, which you can read about here.
- notice how the scl_test_flist pragma requires you to both create a name for the test unit (first argument) and explicitly list each test method that is part of the unit. This is one disadvantage of flist over a test class (the latter does not require explicit listing of each test since all conorming public methods are assumed to be test methods).
TestCClass
This sample demonstrates a more sophisticated (and complicated) packaging technique for systems that support C compilation only. Test C Classes are defined by a structure of function pointers (which may also include data) and an initialization function. Review the source code in the directory and follow the sample description here.
observations:
- the scl_test_cclass pragma requires a structure of function pointers as well as an initialization function that is called prior to running the tests. The initialization function must take the c class structure pointer as it's first argument and it will assign values to all the function pointer elements as well as perform any other initialization tasks. The pragma also accepts an optional deinitialization function that will be called after test execution (if provided).
- we've provided documentation using doxygen formatting for these samples. Because the test functions themselves are bound at runtime, the test documentation must be associated with the function pointer elements in the structure - read more here.
- parameterized tests are also supported by test c classes. Arguments to the initialization function that follow the structure pointer argument are considered constructor arguments and can be passed when running the test.
- because the test methods are assigned to the structure members at runtime, it's possible (and recommended) to use statically scoped functions, so as not to pollute the global function space with test functions. That said, you are free to use any functions with matching signatures and linkage, regardless of scope.
TestPoint
The last sample we'll consider demonstrates how to do expectation testing - that is, validation of STRIDE Test Points - with native test code. Review the source code in the directory and follow the sample description here.
