Training Instrumentation

Background

Source instrumentation is the means by which application domain experts strategically instrument the source under test so as to enable themselves or others to write expectation tests against the running code. Source instrumentation is one of the first steps toward enabling expectation testing of your application.

Source instrumentation is accomplished by a set of simple yet powerful macros provided by the STRIDE Runtime library. The macros are easily activated/deactivated for any build through the use of a single preprocessor macro value. When activated, they provide a means to validate the behavior of your application as it is running.

Please review the following reference articles before proceeding:

• Instrumentation Overview
• Test Points
• Test Logs

How Do I Instrument ?

As described in the overview, you can begin instrumenting your source code by including the srtest.h header file and then adding srTEST_POINT* and srTEST_LOG* in source locations of interest. These macros will be inactive (no-op) unless STRIDE_ENABLED is defined during compilation.

Where Should I Instrument ?

When thinking about where to instrument your source under test, consider these high-value locations:

• Function entry and exit points. Include call parameters as data if appropriate.
• State transitions
• Data transitions (i.e. any point where important data changes value)
• Error conditions

What's The Difference between a Test Point and a Test Log ?

Test Points can be used for validation since they are what's checked when you run a STRIDE expectation test. Test Logs, on the other hand, are purely informational and will be included in the report, according to the log level indicated when the STRIDE Runner was executed. Refer to the background links above for more information on each of these instrumentation types.

What About Data ?

Including data with your test points adds another level of power to the validation of your source. Here are some general recommendations for using data effectively:

• Try to use string data payloads wherever possible. String payloads are considerably more human-friendly when viewing test results and they allow for relatively simple string comparison validation.
• If you need to do complex validation of multi-field data in a test point, consider using an object serialization format such as JSON. Standard formats like this are readily parsable in host scripting languages. If, however, you will only be writing expectation tests in native code for execution on the target, then string serialization formats might be too cumbersome for validation. In that case, using binary payloads (structures, typically) is reasonable.
• TBD
• TBD

Sample Code: s2_expectations_source

For this training, we will simply review some existing sample source code and explain the motivation behind some of the instrumentation. In particular, we will peruse the source code associated with Test Point Samples (test in native code) and the Expectations Sample (test in script).

Samples/test_in_c_cpp/TestPoint/s2_testpoint_source.c

TBD

Samples/test_in_script/Expectations/s2_expectations_source.c

TBD