Difference between revisions of "Types of Testing Supported by STRIDE"

From STRIDE Wiki
Jump to: navigation, search
 
(14 intermediate revisions by 2 users not shown)
Line 1: Line 1:
−
The STRIDE test system supports three types of testing:
+
== Overview ==
 +
 
 +
One of the questions that should be asked is ''what is the '''value''' of the test?'' If the test does not discover any defects or does not provide ongoing regression, then the value is questionable. Also ''what is the '''effort''' in implementing the test?'' STRIDE has been uniquely designed to support maximizing the '' '''value''' '' of the test while minimizing the '' '''effort''' '' to implement it.
 +
 
 +
The STRIDE test system supports three general types of testing:
 
* '''Unit Testing'''
 
* '''Unit Testing'''
 
* '''API Testing'''
 
* '''API Testing'''
−
* '''Behavior Testing'''
+
* '''Integration Testing'''
 +
 
 +
== Unit Testing ==
 +
'''Unit Testing''' is supported following the model found in typical [http://en.wikipedia.org/wiki/XUnit xUnit-style] testing frameworks. STRIDE also offers a number of [[Test_Units_Overview | additional features]] optimized for testing embedded software.
 +
 
 +
Traditional '''Unit Testing''' presents a number of challenges in testing embedded software:
 +
 
 +
* Testing functions/classes in '''isolation''' requires a lot of extra work, especially if your software was not designed upfront for testability
 +
* The software is often not well suited for '''others''' to participate in the test implementation, since there is too much internal knowledge required to be productive
 +
* It can be difficult to automate execution of the full set of tests on the real target device
 +
 
 +
'''Unit Testing''' legacy software may have limited value, particularly if the software is stable with respect to defects. The best ''return-of-effort'' is often experienced when focused on '''brand new''' software components.
 +
 
 +
== API Testing ==
 +
STRIDE supports '''API Testing''' by leveraging the same [[Test Units Overview | techniques]] available for Unit Testing.
  
−
One of the questions that should be asked is ''what is the '''value''' of the test?''. If the test presents no defects or does not provide ongoing regression than the value is questionable. Also ''what is the '''effort''' in implementing the test?''. There is always time involved with implementing a test. STRIDE has been uniquely designed to support maximizing the '' '''value''' '' of the test while minimizing the '' '''effort''' '' to implement it.
+
'''API Testing''' differs from unit testing in that the tests focus on direct testing (calling) of a well-defined interface.  
 
   
 
   
−
= Unit Testing =
+
* The design of ''public interfaces'' often lends itself to testing in isolation ''without'' implementing special test logic (i.e. no stubbing required), which make the test implementation simpler.  
−
[[Test Units Overview | STRIDE Unit Testing]] supports the general model found in typical [http://en.wikipedia.org/wiki/XUnit xUnit-style] testing frameworks. STRIDE also offers a number of [[Test API | additional features]] optimized for testing embedded. But traditional '''Unit Testing''' does present a number of challenges for testing embedded software:
+
* Public APIs are most likely documented and as a result, ''non domain experts'' can more easily participate in the test implementation
 +
 
 +
Although '''API Testing''' often represents a smaller percentage of the software being exercised, this kind of testing is typically well understood, easy to scope, and often has a better ''return-on-effort''.
  
−
* Testing functions/class in '''isolation''' requires a lot of extra work, especially if not designed upfront
+
== Integration Testing ==
−
* Testing legacy software might have very limited value (i.e. can't find any defects)
+
STRIDE also supports '''Integration Testing''', which is different than Unit Testing or API Testing in that it does not focus simply on calling functions and validating return values. To learn more about some of the unique testing techniques well suited for this type of testing [[What_is_Unique_About_STRIDE#Expectations | '''read here''']].
−
* Typically not well suited for '''others''' to participate in the test implementation (too much internal knowledge required)
 
−
* Can be difficult to automate execution of the full set of tests on the real target device
 
  
−
= API Testing =
+
'''Integration Testing''' focuses on validating a larger scope of the software while executing under normal operating conditions.  
−
STRIDE supports '''API Testing''' leveraging the same [[Test Units Overview | techniques]] available for Unit Testing. The difference is the focus of API Testing, which often has a better ''return-on-effort''. API Testing traditionally is about driving well-defined public interfaces. The design of ''public interfaces'' often lends itself to testing in isolation ''without'' implementing special test logic (i.e. no stubbing required). Also public API are typically documented. As a result, ''others'' can more easily participate in the test implementation. Although API Testing often represents a smaller percentage of the software being exercised, the '''value''' and '''effort''' required is well understood.
 
  
−
= Behavior Testing =
+
* Tests are performed typically with fully functional software build
−
STRIDE support '''Behavior Testing''' which is different than Unit Testing or API Testing in that it does not focus on calling functions and validating return values. Behavior Testing, rather, validates the ''expected sequencing of the software'' executing under normal operating conditions. To learn more about the uniqueness of Behavior Testing click [[What is Unique About STRIDE | '''here''']]. We believe the Behavior Testing has a very high ''return-on-effort'' and can be easily deployed into legacy software systems.
+
* There are minimal code isolation challenges
 +
* Test results provide a sanity check on the health the software  
  
−
Some of the advantages of '''Behavior Testing''':
+
We believe that '''Integration Testing''' has a very high ''return-on-effort'' and is more applicable to legacy software systems.
−
* Test with fully functional software builds (no isolation issues)
 
−
* Applies to a large percentage of the software
 
−
* Can execute with functional and black-box testing
 
−
* Easy to add [[Test Point | Test Points]] to provide coverage
 
−
* Others can easily participate in test implementation
 
−
* Test can be written using [[Perl_Script_APIs#STRIDE::Test | scripting on the host]] and [[Expectation_Tests_in_C/C%2B%2B | native target code]]
 
−
* Validation is fully automated (not done manually by a tribal expert)
 
−
* Very useful for continuous integration
 
−
* Design knowledge is extracted and made available other team members via [[Source Instrumentation Overview | instrumentation]]
 
  
  
 
[[Category: Overview]]
 
[[Category: Overview]]

Latest revision as of 10:13, 4 November 2010

Overview

One of the questions that should be asked is what is the value of the test? If the test does not discover any defects or does not provide ongoing regression, then the value is questionable. Also what is the effort in implementing the test? STRIDE has been uniquely designed to support maximizing the value of the test while minimizing the effort to implement it.

The STRIDE test system supports three general types of testing:

  • Unit Testing
  • API Testing
  • Integration Testing

Unit Testing

Unit Testing is supported following the model found in typical xUnit-style testing frameworks. STRIDE also offers a number of additional features optimized for testing embedded software.

Traditional Unit Testing presents a number of challenges in testing embedded software:

  • Testing functions/classes in isolation requires a lot of extra work, especially if your software was not designed upfront for testability
  • The software is often not well suited for others to participate in the test implementation, since there is too much internal knowledge required to be productive
  • It can be difficult to automate execution of the full set of tests on the real target device

Unit Testing legacy software may have limited value, particularly if the software is stable with respect to defects. The best return-of-effort is often experienced when focused on brand new software components.

API Testing

STRIDE supports API Testing by leveraging the same techniques available for Unit Testing.

API Testing differs from unit testing in that the tests focus on direct testing (calling) of a well-defined interface.

  • The design of public interfaces often lends itself to testing in isolation without implementing special test logic (i.e. no stubbing required), which make the test implementation simpler.
  • Public APIs are most likely documented and as a result, non domain experts can more easily participate in the test implementation

Although API Testing often represents a smaller percentage of the software being exercised, this kind of testing is typically well understood, easy to scope, and often has a better return-on-effort.

Integration Testing

STRIDE also supports Integration Testing, which is different than Unit Testing or API Testing in that it does not focus simply on calling functions and validating return values. To learn more about some of the unique testing techniques well suited for this type of testing read here.

Integration Testing focuses on validating a larger scope of the software while executing under normal operating conditions.

  • Tests are performed typically with fully functional software build
  • There are minimal code isolation challenges
  • Test results provide a sanity check on the health the software

We believe that Integration Testing has a very high return-on-effort and is more applicable to legacy software systems.

Retrieved from "https://www.stridewiki.com/index.php?title=Types_of_Testing_Supported_by_STRIDE&oldid=13029"