Building an Off-Target Test App

From STRIDE Wiki
Revision as of 13:00, 8 September 2015 by Ivailop (talk | contribs) (→‎Interpreting Results)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Prerequisite

This article guides you through building a test application (TestApp) for the purpose of running sample code using the STRIDE Off-Target Environment.

It requires an installation of the STRIDE Framework desktop package. If not installed, please see Desktop Installation for more information.

It also requires that your desktop contains one of the following compilers:

Building a TestApp

SDK Makefile

The SDK Makefile is set up so that all .c .cpp and .h files found in the directory SDK\Windows\sample_src (or SDK/Posix/sample_src for Linux) are included in the compile and link of the testapp target.

Further--as a pre-compilation step--any .h files found in sample_src are submitted to the STRIDE Build Tools. This will result in

  • the detection of test pragmas used to declare Test Units in these .h files
  • the detection of function pragmas used to declare remoting of functions also found in .h files
  • the inclusion of metadata into the sidb file created
  • the generation of an Intercept Module required for executing tests

Build Steps

To begin, be sure that TestApp is not running then perform the following steps:

NOTE: If you experience any build problem please make sure to read Troubleshooting Build Problems for possible resolution.

Linux/FreeBSD

  1. Build the test app 
    cd $STRIDE_DIR/SDK/Posix/src
make testapp
  2. Note that the following artifacts are produced by the build:
    $STRIDE_DIR/SDK/Posix/out/bin/TestApp
    the test application
    $STRIDE_DIR/SDK/Posix/out/TestApp.sidb
    the STRIDE interface database file which contains metadata describing the interfaces remoted by the test app (along with other data)

Windows

NOTE: In case you have Cygwin and GNU Compiler Collection installed and prefer to use it, please follow the build steps for Linux (see previous section) and ignore the one in here.

  1. If using Microsoft Visual Studio, open a Visual Studio Command Prompt to ensure that the compiler and linker are on your PATH.
  2. Build the test app using the supplied GNU make. (You will get Makefile errors if you use the default make.) 
    cd %STRIDE_DIR%\SDK\Windows\src
..\bin\make testapp
  3. Note that the following artifacts are produced by the build:
    %STRIDE_DIR%\SDK\Windows\out\bin\TestApp.exe
    the test application
    %STRIDE_DIR%\SDK\Windows\out\TestApp.sidb
    the STRIDE interface database file which contains metadata describing the interfaces remoted by the test app (along with other data)

NOTE: If you prefer to use Visual Studio to build/debug/run your testapp, we provide instructions here about how to accomplish this.

Diagnostics

The test app we just built does not have any user tests in it. At this point it provides a starting point for test that we will subsequently add.

However, a set of diagnostic tests that verify operation of the STRIDE runtime itself are always built into the generated TestApp executable. If desired (we recommend you to do so) you could run them by doing the following:

  1. Invoke the TestApp. In order to see TestApp's output, we recommend that you manually run in a console window (or Windows equivalent):
    Linux
    $STRIDE_DIR/SDK/Posix/out/bin/TestApp
    Windows
    %STRIDE_DIR%\SDK\Windows\out\bin\TestApp

    (...or launch from the file explorer)

  2. Note TestApp's output upon startup. 
    --------------------------------------------------
STRIDE Test Console Application.
Enter 'Ctrl+C' to Quit.
--------------------------------------------------
Listening on TCP port 8000
starting up...
"_srThread" thread started.
"stride" thread started.
  3. From a second console window, invoke stride as follows, to verify connectivity with the test app and STRIDE runtime operation:
    Linux
    stride --diagnostics --database="$STRIDE_DIR/SDK/Posix/out/TestApp.sidb" --device=TCP:localhost:8000 --run="*"
    Windows
    stride --diagnostics --database="%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb" --device=TCP:localhost:8000 --run="*"

    As the tests run you will see output in both the TestApp (target) and stride (host) console windows.

    The host console window output is shown here: 

    Loading database...
Connecting to device...
  runtime version: 5.x.yy 
Executing diagnostics...
  test unit "Link"
    Loopback ............
    Payload Fragmentation
    Stub-Proxy Deadlock
    Target Characteristics
    > 4 passed, 0 failed, 0 in progress, 0 not in use.
  test unit "Stat"
    > 2 passed, 0 failed, 0 in progress, 0 not in use.
  test unit "Time"
    > 2 passed, 0 failed, 0 in progress, 0 not in use.
  --------------------------------------------------------------------- 
  Summary: 8 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...
  4. Note the Summary results shown in the host output; all in use tests should pass.
  5. To exit TestApp, give the target window focus and enter Ctrl-C (or 'q' under Windows).

Samples

The initial desktop installation of STRIDE does not set up any source code (with the exception of a set of system diagnostic tests) for automatic inclusion in a test application. The desktop framework distribution, however, comes with a set of Samples.

To demonstrate how to build a sample, will will add the Test Intro Sample that provide an overview of STRIDE testing techniques. For an overview from a C++ perspective, please see Test Intro Cpp Sample.

The following steps are applicable for all Samples.

To begin, be sure that TestApp is not running, then copy the .c and .h files found in Samples/test_in_c_cpp/TestIntro to SDK/Windows/sample_src (or SDK/Posix/sample_src for Linux).

Note: Only files in the sample_src directory will be picked up by the makefile. Files in any subdirectories will be igored.

Once the files have been copied to sample_src, simply build TestApp as described above. Note if you had previously copied other sample source to this directory, you should decide whether to remove those files first. When you complete the test app build, any source that is in this directory at the time of build will be included in the test app.

Running the Tests

Here we will run all tests in the TestApp.sidb database.[1]

  1. Run the build above TestApp in a console window.
  2. Invoke stride in a separate console window (different from the running TestApp) -- as shown below and verify Summary results.

Here are the command line parameters that we will submit to stride 

--database ./out/TestApp.sidb 
--device TCP:localhost:8000

A couple of things to note:

  • If you setup an environment variable for the device option than it is not required in the option file. Note: Command line options override environment variables.
  • You may want to create a text file named RunTestIntro.txt in the SDK\Windows (or SDK/Posix for Linux) directory as an option file to submit to stride.
  • stride --help provides options information

If you haven't done so already, start TestApp running in a separate console window.

Now run stride as follows (starting from the SDK\Windows or SDK/Posix directory): 

stride --options_file RunTestIntro.txt --run="*"

The output should look like this: 

Loading database...
Connecting to device...
Executing...
  test unit "s2_testintro_flist"
    > 2 passed, 1 failed, 0 in progress, 0 not in use.
  test unit "s2_testintro_cclass"
    > 1 passed, 1 failed, 0 in progress, 0 not in use.
  test unit "s2_testintro_testdoubles"
    > 3 passed, 0 failed, 0 in progress, 0 not in use.
  test unit "s2_testintro_testpoints"
    > 3 passed, 0 failed, 0 in progress, 0 not in use.
  test unit "s2_testintro_parameters"
    > 2 passed, 0 failed, 0 in progress, 0 not in use.
  --------------------------------------------------------------------- 
  Summary: 11 passed, 2 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

Interpreting Results

Open TestApp.html in your browser; this file is created in the directory from which you ran stride (or the directory via the --output command line option).

If you're interested in the details of the tests, please see to the test documentation contained in the test report.

Notes

  1. ↑ Note that the S2 diagnostic tests are treated separately, and are not run unless the --diagnostics option is specified to stride.
Retrieved from "https://www.stridewiki.com/index.php?title=Building_an_Off-Target_Test_App&oldid=14655"