Powerful protocols like USP/TR-369, and its predecessor, TR-069, help the broadband and consumer electronics industries produce devices that can be configured, monitored, and controlled in a standardized way. That standardization means that the implementation of these protocols must be tested to ensure that they conform and will interoperate in the larger ecosystem.
In addition to the management protocol and its interface, however, is the underlying implementation and its API into the device’s drivers, applications, and other interfaces. In USP and TR-069,, these components are modeled using the TR-181 Device:2 data model, also referred to as the “data model”.
This article lays out the 3 most important kinds of testing to automate when validating your implementation to save you time while proving interoperability and increasing confidence.
About Broadband Forum Data Models
The map of a device’s capabilities and state are referred to as "data models".
The data model is used in two different ways. A supported data model describes what a device implements and what it can support. An instantiated data model describes the device's current objects and state.
The actively supported data models can be found on the Broadband Forum's website for CWMP and USP respectively. You can also learn more about how to navigate these data models in our training videos!
Topology of a USP or TR-069 Managed Device
What should I test in a device that supports TR-181?
When developing or qualifying devices that support USP or TR-069, it’s not only important to test that the protocol will operate correctly to ensure interoperability, but also to exercise your device’s implementation of the underlying data model and how it connects with the rest of the system. This type of testing takes several forms, and is extremely rigorous. Automating this testing is the only practical way of getting it done quickly and consistently.
1. Validating the supported data model
The first step is validating that the objects and parameters that a device supports can be correctly manipulated by the management protocols. Writable multi-instance objects should be able to be created and deleted; writeable parameters should allow changes. Testing like this includes:
- Verifying all required parameter names are returned using GetParameterNames
- Verifying parameter name instances can be walked using GetParameterNames
- Verifying all parameter names and values are returned using GetParameterValues
- Verifying the read and write status of each parameter
- Verifying all writable parameters can be set using SetParameterValues
- Verifying the creation and deletion of all creatable objects using AddObject and DeleteObject
CDRouter’s Device2_profiles test cases are built to test the specific parts of the data models that your implementation supports. For those building TR-369/USP solutions, CDRouter goes through a similar process with the appropriate USP Messages.
What about vendor extensions?
Developers can also test their custom objects and parameters using a custom profile. CDRouter users can specify a custom profile to tell CDRouter which parameters to test. You can also specify their syntax so you can perform data validation testing (below) on any device regardless of its custom data model.
2. TR-069 (CWMP) parameter data type validation
In the CPE WAN Management Protocol, messages are carried in XML format, and the data type of each parameter is specified and meant to be understood by the ACS as that data type.
It’s important to test that the values reported (and presumably, stored) by the device match the syntax of the parameters as they are specified in the official CWMP data models. This includes basic syntax like using the correct data type, but also abiding by the range restrictions or enumeration options defined in the data model.
Note: These types of tests are specific to CWMP, since all USP messages are transmitted as Protocol Buffer strings.
For example, let’s examine the Device.WiFi.AccessPoint.{i}.Security.] object in the Device:2 data model. This object contains the ModesSupported parameter, whose description says:
Comma-separated list of strings. Indicates which security modes this AccessPoint instance is capable of supporting. Each list item is an enumeration of:
-
None
-
WEP-64
-
EP-128
-
WPA-Personal
-
WPA2-Personal
-
WPA3-Personal
-
WPA-WPA2-Personal
-
WPA3-Personal-Transition
-
WPA-Enterprise
-
WPA2-Enterprise
-
WPA3-Enterprise
-
WPA-WPA2-Enterprise
This means that, although the XML data type is “String”, the parameter is further restricted to contain a comma separated list of values specified in this enumeration. Providing values outside of this list, or typographical errors, will not be recognized by an ACS or Controller and can cause interoperability problems.
CDRouter can validate individual parameters to ensure they use the correct syntax. It uses the official CWMP data models as reference, and tests each parameter specified in a Profile that the DUT says it supports. This behavior can be turned on and off using the cwmpDataValidation testvar.
3. Exercising real-world features and use case scenarios
Testing the software implementation of USP, TR-069, and the TR-181 data models is an important step towards guaranteeing a robust implementation. The third step is to demonstrate that a device makes changes, reports data, and activates features when told to do so by a management or control application. In CDRouter we cover this kind of real-world feature testing in several ways.
IR-181 interoperability testing
The Broadband Forum has an extensive “interoperability” test plan for devices that implement the TR-181 data model. This is defined in [IR-181], and contains test cases for Wi-Fi setup, firewall and port mapping, monitoring and diagnostics, and more. CDRouter supports all of IR-181, and we go more in-depth on the test plan in this article.
USP and TR-069 Scenarios
Validating custom features or functionality demanded by your customers can be done through easily scripted scenarios. In CDRouter, scenario test cases exist for both USP and TR-069, and allow you to send messages to manipulate objects and parameters, and then specify the expected response from your device. CDRouter also includes a start-up scenario to check a device’s initial Inform message in CWMP. Some useful ways to use scenarios include:
- Simulating ACS policy behavior when a device first connects and sends a BOOSTRAP event
- Testing byte counters and other statistics
- Testing vendor extension functionality and validating that the correct changes occur to other objects and parameters
You can read about scenario testing in our TR-069 User Guide and USP User Guide.
More USP and TR-069 feature testing in CDRouter
The QA Cafe team stays on the cutting edge of the management features released by the Broadband Forum. In addition to the data validation methods described above, devices that implement TR-143 performance diagnostics, IP diagnostics, and HTTP bulk data collection should run their devices through CDRouter’s automated tests for these features to ensure they will work, and work well, when deployed in the customer premises.
