Testing today’s advanced smartphones and mobile devices involves many phases. The final predeployment test typically is device validation, commonly performed under the field conditions where the subscribers will actually use them. However, device field testing across diverse conditions of terrain, population density, physical location, and motion is extremely time-consuming and costly and lacks repeatability.
At the same time, in-lab recreation of the dynamic real-world conditions the user and device experience throughout a wireless network also is exceptionally challenging. Field measurements may vary substantially from laboratory measurements, adding yet another layer of complexity to the testing process.
To bridge the gap between laboratory and field-measured results, field-to-lab testing has been developed. These solutions allow real-world channel conditions collected from drive testing to be replayed in a network testbed.
Field-to-Lab Testing Principles
As a user device moves throughout a network, over-the-air real-world conditions change frequently. The device may be affected by many factors including the varying speed of the device as well as the rate of change of air-effect phenomena, which are difficult conditions to replicate.
A traditional laboratory device validation methodology uses channel emulation technology with industry-approved and standardized channel models. Statistical representations of channel conditions are very useful in representing generalized RF environments needed for testing and certification; however, they do not capture the unique and specific conditions experienced by a device as it physically moves through an actual wireless network.
For device validation, operators currently perform thousands of drive tests. The operator often will choose representative drive routes to use for testing before mobile devices are approved and released for distribution to subscribers. Driving these segments is time-consuming and expensive and must be performed at specific times of the day, requiring the same level of cost of mobilization every time it’s repeated.
After this data is collected by commercial drive test tools, new field-to-lab testing methodologies allow the data to be conditioned, saved, and reused as many times as required with many different devices. As a result, R&D and QA engineers now can test products in the lab using both industry-standard models and real-world conditions from actual field data collected from locations of interest.
Cost and time savings plus the capability to test at any stage in the development cycle and at multiple locations using real-world data make the field-to-lab solution an essential tool for improving predeployment mobile infrastructure and device qualification. Field-to-lab also can help testers to optimize the network and debug post-deployment issues by collecting logs from problem areas and recreating the conditions in the lab to solve the issue at hand.
Playback of field conditions in the lab can aid device OEMs and operators to address a number of practical problems:
- Device Qualification: Numerous new handsets are qualified by major providers every month, each one demanding a rigorous set of field trials. Using a large representative set of drive logs in a field-to-lab test methodology makes it possible to perform fully automated unmanned test runs during nights and over weekends when test engineers are not present. This provides easily comparable test results to evaluate against reference handsets as well as all other devices or infrastructure from the same and other vendors.
- Network Troubleshooting: Unhappy customers often are quick to report their dissatisfaction with network performance, and providers often can quickly focus on problem areas. The solution to the problem, however, may not be so simple. Network issues such as reduced performance and dropped calls may be caused by coverage limitations, network load, configuration issues, or poor performance of specific devices under certain conditions.
Using field-to-lab testing, engineers can go directly to the scene of the problem, even if it is in another country, and replay the scenario to evaluate the issue with sophisticated analysis tools without stepping out of the lab. The same data can be used later to evaluate new devices under the same problematic conditions.
Armed with actual drive test data collected in the field, field-to-lab test methodologies allow testers to more accurately recreate additional elements important in lab-based network testing but never before factored into testing. These variables include rapid change of signal strength, multiple sectors that are visible to the user in most real deployments, and the fluctuating speed of the device or the rate of change of air-effect phenomena. Until now, lab testing typically ignored these parameters.
Field-to-lab also provides a repeatable environment for testing devices. While field testing may follow the same route each time, traffic and other conditions may significantly change from drive to drive, resulting in slower velocity and a higher number of users on the network. Using a field-to-lab test methodology that combines the important information from drive test logs with an effective and repeatable channel emulation environment creates a test solution that is closer than ever to the real world.
MIMO Systems
One of the fundamental advantages of 4G wireless technologies is enhanced throughput. Two key technology advances make this happen: multiple antenna transmission or multiple input/multiple output (MIMO) and enhanced data encoding or orthogonal frequency division multiplexing (OFDM).
MIMO provides the means by which a radio can adapt to different and varying channel conditions to deliver more or improved data transmission utilizing techniques such as diversity, spatial multiplexing, and beamforming. OFDM encoding is more resilient and delivers more reliable data transmission than legacy encoding schemes.
OFDM also divides the channel into narrowband subchannels, making it less susceptible to the frequency-selective fading that affects broadband transmission. However, while it significantly improves the reliability of transmissions in real-world propagation conditions, OFDM can only be realistically evaluated for throughput if it is tested in a fading channel.
To achieve the desired improvements, MIMO and OFDM take advantage of and depend on the propagation conditions of the real-world scenario in which they operate. As a result, recreation of such channel conditions in the lab becomes critical.
Emulating Channel Conditions and Propagation Effects
Effects such as fading, multipath, and correlation are easily recreated in the lab using a channel emulator and statistical modeling tools. For example, fading is frequency selective. As signals move from the transmitter to the receiver, the over-the-air transmission effects and obstructions in their path will cause the signal strength to vary rapidly.
As a result, while the signals may momentarily fade high at one point in the band, a nulling effect or down fade may simultaneously occur in another part of the signal bandwidth. Since the signal in broadband wireless consists of multiple subcarriers over the band, the frequency-selective fading may have a negative effect on one carrier and no effect on another. Known as fast fading, it typically is influenced by the type of environment and speed of the mobile device.
A signal may pass directly via line of sight from the transmitting antennas to the receiving antennas. However, it also will reflect off buildings, vehicles, terrain, and other objects, resulting in multiple copies of the signal with different signal strengths and delays. The varied path length of the multipath signal generates different arrival times for each copy and results in phase differences between the signals. Urban and rural environments may have multipath signals but of very different delays and strengths.
Since the antennas are relatively close to one another in a user device and relatively far apart in the base station, the proximity and orientation of antennas will define the similarity of the signals when received by one or more antennas. The more similar or diverse the signals, the greater the effect on the capability to decode, sum, or differentiate between the signals at each receive antenna.
Additional Conditions
Other varying effects that cannot be recreated in standard channel models will significantly impact the performance of 2G/3G/4G systems. For example, the signal strength of the transmission from base station to device will affect device performance, including the quality of the signal received and the capability to decode the signal. It also influences device figures of merit (FOM) such as the call quality indicator (CQI), which is reported back to the transmitter, and other parameters that will dynamically determine the devices’ transmission speed. Standard tests define the signal strength to be used and, in rare cases, the variation of the signal over time.
However, in the real world, the signal strength is highly dynamic and affected by multiple factors in the environment. For that reason, when a device in the field is subjected to rapidly changing conditions, its behavior may be very different from the behavior observed in the lab. Most tests are performed in static or simulated motion conditions with varying channel models based on the test’s defined velocity, which often does not map to real-world driving conditions such as stops, starts, acceleration, and deceleration.
Most lab tests also involve one, two, and, in rare cases, multiple base stations. However, in reality, a majority of user devices will see between three and tens of base stations or sectors at any time. The conditions and proximity of the device to the different sectors will change over time as well. Field data captured within drive test logs will detail the signal strength data seen at the device during these variable real-world conditions.
Field-to-lab testing uses the same data collected in the drive test logs to vary signal strength in real time in the laboratory network testbed exactly as it is seen in the field. Using the GPS coordinates collected as the drive test log is recorded, the field-to-lab solution can dynamically vary the propagation conditions in parallel with other effects like varying signal strength and evaluate the effects of such conditions on the device when testing in the lab.
With field-to-lab testing, users can map in some cases up to 16 sectors at a time and vary the relationship to each sector based on the data collected in the drive test. Subsequently, field-to-lab testing can evaluate how the device behaves when subjected to multiple base station signals and must make handoff and roaming decisions.
Combining the effects of straightforward fading, multipath, and antenna-correlation propagation conditions with multiple base stations, varying conditions per base station, and real-time velocity changes just as the device sees in the field significantly increases the complexity of the test environment. Field-to-lab testing methodologies are able to closely match real-world field conditions for more accurate and thorough wireless device testing.
Using Drive Test Logs
The most indispensable part of the field-to-lab test methodology involves analyzing drive test logs and conditioning the data for playback in the channel emulator. The log file allows you to easily visualize the data collected in the field and contains valuable information such as the duration of the drive test, location, and how many base stations or sectors are visible at any time.
The tools also should present the data in an easy-to-understand format such as overlaying the GPS data associated with the log file on a satellite map to show where the route is and where surrounding infrastructure may be located. The log also includes the data collected on signal strength, velocity, noise levels, and propagation conditions. Users then map the data by selecting what parameters were tested and how the data will be used when played back in the lab.
Since drive test tools were designed for different purposes, the detail and accuracy of the data may not be perfectly suited to real-time playback. Accordingly, the mapping software also may smooth data transitions or fill in blanks if data is missing. All of these conditioning steps are user-configurable.
Channel Emulator Playback
Once the data is collected, conditioned, mapped, and analyzed, the result characterizes the channel conditions between the DUT or handset and the multiple base stations or sectors in the network. This is based on network conditions seen at a specific place in a real-world deployment. The data can be streamed in real time onto the channel, which is typically a channel emulator situated between the DUT and a representative number of base stations in the lab.
Figure 1. Multiple Base Stations/Sectors Mapped Through the Channel Emulator to Recreate Real-World Conditions
Figure 1 illustrates the mapping of field-collected data into channel emulators that are part of a network testbed. In this scenario, the network testbed consists of multiple 2×2 MIMO base stations transmitting/receiving data from a single mobile device. The network testbed and the field-to-lab solution, configured for the most typical conditions, support single input/single output (SISO) and MIMO as well as 2G, 3G, and 4G radio technologies.
The Value of Automation
Automation is a necessary component in field-to-lab testing since most laboratory testbeds are complex and have many components that need to be controlled and because there are large amounts of data to crunch. Ideally, automation will reduce the need for manual intervention to a point where you could kick off a test case at the click of a button and have all the results presented when the execution is done.
In reality, there are various degrees of automation functionality that may be incorporated into a test solution. The automation capabilities of commercial field-to-lab systems range the entire spectrum from those with no automation capabilities to those that allow automation of the entire solution.
Previously, typical automation capabilities offered in this space were merely a checkbox item since everyone automated only their own equipment. As a result, discordant automation blocks had to be integrated later in the process, which at times involved so much effort that users bypassed the automation of the testbed entirely.
Today, there are field-to-lab solutions that allow you to automate the entire testbed in a modular, nonprogramming environment, enabling even novice users to write automation modules. With full end-to-end automation, the entire testing process from initializing the equipment to collecting the results is done automatically.
Full automation also facilitates the setup of test suites so a series of tests can be run one after another. The architecture of this type of automation also means that users not only can share it within their company, but also across the entire supply ecosystem, helping drive consistency in the way things are tested.
Automation also plays a significant role in improving product quality as the savings allow you to do more testing while reducing the chances of human error in the test process. It also makes testing more affordable.
Conclusion
As the wireless industry introduces products and solutions based on 4G technologies such as LTE, LTE-Advanced, and WiMAX, field-to-lab solutions—particularly those with advanced automation capabilities—expand the scope of lab testing through the addition of real-world field conditions to channel modeling. By utilizing measurements of field data and playing it back through a wireless channel emulator, R&D and QA engineers and operators can test device performance against the dynamics of the field before deploying products in the real world.
Because MIMO product performance can vary substantially depending on the RF environment, the use of field-based measurements afforded by field-to-lab testing offers a solid foundation for enabling faster time to market, improved product quality, and considerably reduced testing costs. An automated, integrated, end-to-end field-to-lab test methodology will help increase these gains substantially and bring revolutionary new mobile devices to market earlier.
About the Author
Vivek Vadakkuppattu is a product manager at Azimuth Systems. He has more than six years of global experience in the management, marketing, and development of telecommunications products. Azimuth Systems, 35 Nagog Park, Acton, MA 01720, 978-263-6610, [email protected]