Testing Controllers
←Usage - Testing Controllers · Index↑ · Testing Profiles→
Fully Automated Controller Tests
The testing tool supports fully automated testing of Controllers by replacing the default Testing Façade application with your own application implementing the Question API to receive and process the requests from the NMOS Testing Tool. A complete example utilising Flask and Selenium to test the Sony nmos-js Controller can be found in the sony/nmos-js repository.
Building your fully automated test interface
The Testing Façade IP/Hostname and Port on the test suite should be set to the location of your application.
To carry out each test POST requests are sent from the Test Suite to the Question API endpoint http://<ip address or hostname>:<port>/x-nmos/testquestion/<version>.
On receiving a POST request from the Testing Tool, your application uses the question_id to instruct your chosen automated test framework (e.g. Selenium) to carry out the required tasks on your Controller and determine appropriate answers.
Your application POSTs the question_id and answer(s) to the Answer API endpoint given in answer_uri.
JSON schema and RAML files for both the Question and Answer API endpoints can be found in the testingfacade folder. An example python class for storing and retrieving the Question data can be found at testingfacade/DataStore.py
Some tests contain multiple questions, all of which must be answered correctly to pass the test.
There are three types of question that need different kinds of answer:
single_choice- report a singleanswer_idfrom theanswersobjectmulti_choice- report a list of multipleanswer_id’s from theanswersobjectaction- no answers given, simply reportnullwhen ready to move to the next question
Example JSON files for each type of Question and Answer that could be sent and received can be found in the examples folder.
On receiving a POST request to the Answer API, the test suite will process the answer and send the next question to the Question API. If the Answer API does not receive a request within the timeout period, the current test will expire and the first question of the next test will be sent.
The test suite begins with a pre_tests_message and ends with a post_tests_message. These are action questions so the answer_response is null to start and end the actual tests.