The wireless enterprise environment operates under very different conditions than wired networks. As a result, this environment requires a fresh outlook on testing. For wireless LANs (WLANs) to succeed in the enterprise, developers must rely on system-level test solutions. Such solutions must produce repeatable outcomes, scale beyond a single device to a network, and provide systems designers with an automated solution that will decrease the cost per test.
As discussed last month in the first of this series on 802.11 testing, traditional WLAN-system testing methods are inherently at odds. Examples of such testing methods include isolated screen rooms for controlling RF interference and large open spaces for testing mobility. The fact that these methods are at odds makes it difficult to test real-world conditions, such as mobility and roaming while controlling random RF emissions. In addition, such homegrown solutions are hindered by high costs and logistical issues.
Aside from having homegrown solutions at their disposal, wireless systems designers have had a variety of off-the-shelf test equipment. This equipment focuses on point-to-point tests at the physical layer, such as spectrum analyzers, signal generators, and multipath simulators. Such equipment can't efficiently test wireless networks as systems, however. An effective system-level test solution must scale beyond single devices (clients and access points) to the network itself. It must provide the ability to test the WLAN as a system. In other words, it has to precisely analyze Layer 2 performance and the interdependence of Layer 1 and Layer 2 under various mobility and load conditions.
Thankfully, there's good news for wireless systems designers. A new generation of wireless testing solutions has emerged in response to the limitations of homegrown and point-to-point testing solutions. These solutions provide the repeatable outcomes, system scalability, and test automation that will enable the enterprise adoption of WLAN technology.
Emerging wireless test solutions control RF by individually isolating the RF devices in the test. To achieve this isolation, they use a cabled RF medium and vary the RF signal strength using attenuators. This approach provides a completely controllable test bed. It also produces a more regulated test because of the following:
- Cabling the RF increases RF accuracy.
- Enhanced isolation techniques provide interference immunity.
- RF attenuation emulates RF mobility.
The result is a test solution that produces repeatable test outcomes.
Using cable-based solutions that provide a controlled-RF environment, wireless systems designers can test real-world conditions in a controlled setting. They can then scale tests beyond single devices to entire networks. Test outcomes are repeatable and more accurate. In addition, this type of solution can be digitally controlled. A programmable software engine can thus provide automation, which enhances scalability and lowers the cost per test.
The benefits of this approach are:
Repeatability: Consistent, repeatable outcomes and measurements are nearly impossible in open-air environments. The problem is that RF interference will vary in each iteration of multiple tests and in different locations. But a cabled, controlled-RF testing environment provides the same test setting every time. It offers the highest level of repeatability and efficiency, while still allowing the simulation of real-world conditions like mobility and roaming. In addition, test setup is easier because the complexity of controlling RF interference is removed. Organizations can save money, as the time spent retesting due to inconsistent results is minimized.
Testing can be further enhanced if the system uses device isolation instead of room isolation. The isolation of individual components allows testers to control inter-device interference and create controlled channel interference in the same test setup.
Scalability: If the controlled RF environment is properly architected, system designers can scale WLAN testing from a single device to the entire network (FIG. 1). Users can configure an entire WLAN network and provide system-level testing of actual access points (APs), clients, and other wireless devices. Networks can be tested under a variety of traffic and client load conditions. Problems can then be found in the laboratory instead of at a customer site.
Client and traffic load emulation enable the development of test setups that re-create a busy network environment for the devices under test. Each emulated client can be individually addressed on Layer 2 (MAC) and Layer 3 (IP). The simultaneous active clients that provide standard Wi-Fi functionality, such as security and quality of service (QoS), should each have the ability to send and receive wireless traffic. They will then provide an extremely flexible and diverse configuration.
Because the testing is conducted in the lab, test setups needn't be complicated and expensive. The ability to control RF in the lab increases testing efficiency. It allows system designers to easily design and conduct system-level stress-test scenarios for mobility, roaming, and other conditions. Previously, testing for such conditions could only be conducted in large open areas.
Automation: Another advantage of newer test platforms is their programmable test tools. These tools provide users with an automated method of configuration and traffic monitoring. To analyze the effect of mobility on both device and network performance, for example, users can automatically configure any network device and dynamically position any network node. Combined with the ability to control RF interference, the programmable software engine allows users to analyze real-world scenarios (i.e., hidden stations, overlapping basic service sets (BSSs), roaming, and rate adaptation) while simultaneously varying settings like security, QoS, and client load.
Automation saves money and time by standardizing tools, processes, and training across organizations, geographic boundaries, and the industry. Many test platforms in today's market are standardizing on Tool Control Language (TCL)—a simple-to-use and very flexible scripting language. In addition, an automated test method supports the simple, effective setup and reuse of test configurations, allowing for repeatable test execution over time. This repeatability greatly reduces the time spent on quality assurance and benchmark test processes. As a result, the programmable platforms that provide automated testing dramatically reduce time to market and the cost of test.
In the life cycle of a wireless product, vendors must continuously test and re-test the product. Each time the hardware or software is changed to add functionality, fix an anomaly, or support a new standard, the vendor must re-test to ensure that no adverse effects were caused by the change. Functional and regression testing are only the start of the testing cycle. Interoperability testing, certification testing, conformance testing, and—most importantly—performance testing are of great significance to the success of a product that's being deployed in live networks. In addition, vendors may want to conduct more in-depth tests, such as testing coexistence with other wireless technologies.
Thorough testing of a wireless device can be long and complex. But a controlled and managed test environment, such as the one described in this article, allows accuracy and automation. It also increases the efficiency of bringing products and product versions to market.
Functional testing: This type of testing qualifies standard and vendor-specific product features and functions. Some functional tests are simply a validation of the correct device behavior, such as the configuration and operation of features. Others—like security—have significant interoperability relevance. Features that have major performance implications also are important. An example is a roaming algorithm. Often, functional testing is tedious and labor intensive. Configuration automation and test-procedure automation save considerable time, as this test is repeated for every version of a product.
Interoperability testing: Such testing ensures that diverse pieces of equipment operate together. Interoperability is a major requirement for networks in the enterprise, public hot spots, and the home. Initially, interoperability testing validates compatibility with a vendor's own equipment. Next, compatibility with other vendors' equipment is tested. Most vendors rely on Wi-Fi Alliance Interoperability Certification to ensure inter-vendor compatibility.
Certification testing: In addition to standard safety and emissions certification, reputable wireless-LAN equipment is expected to pass the Wi-Fi Alliance certification tests. Wi-Fi certification allows vendors to use the alliance's coveted logo on the product. It also provides a capabilities certificate that attests to the different band, security, and QoS tests that the product has passed.
Conformance Testing: This type of testing ensures that the device under test (DUT) adheres to the definition of the 802.11 standard. Although this was once a critical test, its relevance is diminishing as the standard has matured and the number of silicon providers has decreased due to consolidation. The University of New Hampshire (UNH) Interoperability Lab developed a suite of compliance scripts several years ago. Many vendors that believe that conformance testing is important simply send their product to the UNH-IOL lab for testing (www.iol.unh.edu).
Performance Testing is by far the most significant part of product test (FIG. 2).
It validates a product's ability to meet the stringent requirements of demanding applications. Because different types of networks have unique requirements, performance parameters vary from network to network. For example, range is an important requirement in the small-office/home-office (SOHO) wireless network. Yet security performance is more critical in the enterprise. Pure wireless throughput is a vital consideration in a public hot spot, while QoS operation is essential for WLANs that carry voice. Performance testing requires advanced tools that load and stress infrastructure and control and monitor clients.
A critical part of defining a system test is to identify the DUT and ensure that the test properly isolates and examines that DUT's specific behavior. In the type of platforms that are described in this article, the DUT may often be a wireless client. It's critical to test a wireless client in the normal usage scenario. For example, a cardbus wireless-client adapter should be tested in a PC running a Windows operating system.
Access-point testing, on the other hand, is dependent on having many clients like the cardbus client attaching to it, sending traffic, roaming, ranging, and so on. The test can then validate the AP's ability to withstand average and peak network conditions.
To create a large number of client devices, many solutions will implement emulation techniques. It's important that these softclients have the ability to operate all at once. For example, all of the clients need to be active simultaneously with bidirectional traffic capabilities. The softclients also must have features that mirror the features of a real client. In a wireless network, such features could include variable range, security configurations, QoS configuration, and traffic profiles. In both the case of the client and the AP, the device under test must be the real thing.
An off-the-shelf, chassis-and-module-based platform with a programmable software engine has existed for more than a year. The platform's programmable software engine gives users the ability to configure an entire WLAN network in a cabled environment. Many leading wireless chip-set and equipment manufacturers have adopted this platform. Using the techniques discussed, the system provides a solid test platform that has greatly expanded the test capabilities of wireless in the lab before deployment. Its basic components are:
Chassis: The chassis is the foundation of the testing system. It acts as the concentrator for RF communication between the test modules. The chassis connects DUTs via a network of RF attenuators and combiners, which eliminates RF instability and interference. In addition, the network enables "virtual positioning" that accurately positions devices in relation to the entire network. Multiple chassis and associated test modules can be cascaded to form a complex multi-AP network of test devices in a controlled environment. This environment provides repeatable test results—a situation that would be impossible to reproduce in open-air testing.
Test modules: A series of test modules with varying functionality can be inserted into the chassis, making both digital and RF connections into the backplane. Test modules are the building blocks of the system. They allow the construction of a nearly infinite variety of WLAN networks. A module-based approach is flexible. It also allows users to efficiently emulate and test a variety of real-world setups.
The testing modules use small isolation chambers to house the Wi-Fi devices. They therefore isolate them from the outside environment as well as from each other. The only path for the RF signal is channeled through attenuators and to other devices.
Test heads: These compact, device-sized isolation chambers hold APs as well as application-specific devices like handsets, laptop computers, and any other Wi-Fi devices in isolation. These chambers also provide an RF channel to connect the chassis.
To allow users to configure a variety of setups, clients, APs, and other devices in modules and tests heads are connected to the chassis in a flexible manner. The setups can scale from a single BSS with one client to multiple, overlapping BSSs with hundreds of clients.
Those clients can be operating simultaneously in any of the three 802.11 bands and on multiple channels. Once this "wireless network" is created on the benchtop, a bevy of tests are suddenly at the users' fingertips. Some of these tests were previously impossible to perform, while most of them were impossible to automate.
Software engine: The test solution includes a programmable software engine that enables users to configure and analyze the network and DUTs. It acts as a control console, providing access to the test-system functionality through a graphical user interface or a scripting language like TCL/TK. This language is used to develop a variety of applications. The software allows users to configure the wireless network under test, control test traffic, and analyze test results while simultaneously controlling the wireless environment.
Since the advent of the technology, wireless vendors and enterprise users have struggled to find the ideal environment to validate the capabilities of their solutions. As the technology has become more successful and permeated every building and public space, finding that ideal test location has become even more difficult.
Several systems, such as the Azimuth W-Series, have been introduced to solve this problem (FIG. 3). By moving devices through emulation and automation, such systems provide a platform where the test limitations are only bounded by the limitations of the user's innovation. A test solution like the one described allows roaming, rate adaptation, and many more tests to be performed in a lab without the nuisance of a mobile test setup or a shielded chamber. For example, the test setup can be programmed to emulate a fail-over roam (the result of a component failure) or a motion roam (the result of movement from AP to AP while using the client device). In fact, hundreds of performance, functional, interoperability, conformance, and certification test plans can be scripted and automated in this type of test system.
As enterprise IT managers increasingly expect WLAN networks to carry mission-critical data, the homegrown testing solutions that are unable to provide reliable test results are becoming outmoded. They will be replaced by more practical solutions that produce repeatable test results. Such solutions also will scale from single devices to entire networks while enabling automated test configuration and analysis. In the process, they'll deliver significant cost and time savings.
The test methods that incorporate RF isolation without the costly constraints and uncomfortable working conditions of a homegrown testing environment provide the highest levels of repeatability and testing efficiency. An example is the chassis-and-module-based solution that controls RF at the laboratory benchtop. It provides consistent test results while enabling repeatable emulation of stress conditions, such as mobility and roaming.
With the efficiency gained from being able to configure, operate, and analyze an entire network in one benchtop chassis, system developers can more easily analyze the functionality, performance, and interoperability of 802.11 networks under the stressful conditions of real-world networks. The flexibility and programmability of these solutions permit users to thoroughly evaluate wireless-LAN equipment and precisely analyze the results. In doing so, they simplify compliance, interoperability, functionality, and performance testing.
The adoption of such platforms will reduce the industry's reliance on homegrown, non-standard solutions. It also will facilitate industry-wide, standardized performance benchmark testing. These platforms eliminate the need for expensive, space-consuming RF screen rooms and non-standard test beds. They also allow engineers to work in the same lab without overlapping interference. These improvements of 802.11 benchmarking will drive performance levels and IT manager product-purchasing decisions.