Networked Media Open Specifications

Usage

←Installation - Docker · Index↑ · Usage - Testing Unicast Discovery→

The following guide indicates how to run the tool against an implementation of one of the NMOS APIs.

Description Screenshot
Open a browser and go to the address of the server you deployed. Testing Tool Launcher
Select a test suite from the dropdown. Test Suite Dropdown
Provide the IP or hostname and port of the relevant API under test, along with the preferred API version. Test Configuration
Optionally, select a subset of the tests to run. This should default to ‘all’. Test Selection
Once all of the field have been completed, press ‘Run’. The result of the tests will be shown after a few seconds. Example Results Window

Understanding the Results

The result of each test case will be one of the following:

Test Result Reason
Pass Successful test case.
Fail Required feature of the specification has been found to be implemented incorrectly.
Warning Not a failure, but the API being tested is responding or configured in a way which is not recommended in most cases.
Test Disabled Test is disabled due to test suite configuration; change the config or test manually.
Could Not Test Test was not run due to prior responses from the API, which may be OK, or indicate a fault.
Not Implemented Recommended/optional feature of the specifications has been found to be not implemented.
Manual Test suite does not currently test this feature, so it must be tested manually.
Not Applicable Test is not applicable, e.g. due to the version of the specification being tested.

Configuration

The testing tool has a number of general configuration settings that affect its operation, defined in nmostesting/Config.py.

To alter these, copy nmostesting/UserConfig.example.py to nmostesting/UserConfig.py and make your changes there.

MAX_TEST_ITERATIONS can be set to restrict the maximum number of resources or test points that time-consuming tests run against. The default of 0 means unlimited (all available resources or test points) for a really thorough test!

Test-specific Documentation

The following pages provide information on additional instructions for testing specific features:

Non-interactive Testing

The testing tool supports two modes of operation that support usage in larger scale or continuous integration tests.

Advanced Testing

Some of the tests contained within this tool perform a number of steps which may not be obvious without viewing the source code. To help with debugging, descriptions of such behaviour is covered by the advanced testing pages.

←Installation - Docker · Index↑ · Usage - Testing Unicast Discovery→