[Design Application]
Automated Testing Speeds Effective Deployment Of Java Functionality
In resource-constrained embedded devices, testing has to cover dozens of operating-system/processor environments.
To fulfill its "write-once, run-anywhere" promise, Java technology must be widely deployed in resource-constrained information appliances and embedded devices. In addition to limited memory resources, developers face challenges in testing this highly heterogeneous class of products.
Dominating the desktop world is one major operating system, the Windows family, and one major processor, the x86/Pentium. But the operating-system/processor application environment in an information appliance or embedded device could be one among dozens of possible combinations. Each operating system and processor—when combined together, along with any necessary middleware, applications, and the Java virtual machine (JVM) solution—present unique testing challenges that can dramatically affect how soon a product comes to market.
This article discusses the benefits and challenges of test automation, plus its use in effectively and quickly deploying Java functionality in these highly diverse devices. It describes how engineers at Insignia Solutions developed test automation to test its Jeode virtual machine for these de-vices. Some benefits of test automation include:
Faster than manual testing
Repeatability to identify regressions immediately
Insulation from human factors, such as boredom or carelessness
Improved audit trail to identify bugs
Shorter residence time of bugs in the code, reducing the manual effort of development and test engineers
Ability to test code in live development
The main challenge of automation is the difficulty in implementing it. Some tests, particularly those involving devices that feature a graphical user interface, are hard to automate. Automated testing of embedded devices that employ real-time operating systems can also be challenging. Automation generally takes longer to set up and requires more skilled software engineers than a manual test.
Testing the Jeode platform: Insignia Solutions develops and markets Java virtual machine technologies, branded under the Jeode name. These products are targeted at information-appliance and embedded developers building devices with limited resources, like memory and battery power. Such devices can be characterized as heterogeneous because they employ a wide variety of operating-system/processor combinations. For example, to support the diversity in the embedded industry, Jeode technology is currently ported to Windows CE, NT4, VxWorks, Linux, ITRON, Nucleus, pSOS, and BSDi Unix operating systems, and ARM, MIPS, x86, SuperH-3, SuperH-4, and PowerPC processors.
Naturally, the multiplatform nature of this product complicates testing. At any one time, ports for up to six targets will be in development for an upcoming general release, perhaps six more for specific customers, and another 20 for internal development, which could be promoted to release candidates if needed.
It's important to automatically test as many platforms as possible. Some target systems, such as Windows NT and most Unix variants, are easy to automate. Others, including Windows CE and most embedded operating systems, are much more difficult. They must all be tested because the target market isn't in PC-type systems, where regular computer crashes are tolerated. Instead, it's in devices like Web terminals, set-top boxes, and networking infrastructure, where uptime and reliability are expected.
The Jeode Embedded Virtual Machine (EVM) is built from a source tree that combines platform-neutral code used in all ports, and platform-specific code, which is only used in some. For example, in Insignia's dynamic adaptive compiler, the code that decides which Java bytecodes to compile is common among all platforms. But the code generator is different, depending on which CPU the code is for. In addition, platform-specific code exists for different operating systems.
Engineers testing the Jeode EVM on a device test the amalgamation of the platform-specific and the target-neutral code. Both are equally important. Platform-neutral code may behave in unexpected and different ways on different platforms. For instance, on one real-time operating-system platform, the company discovered thread-priority problems in the platform-neutral code of the finalizer. These were due to the operating system's strict thread-scheduling behavior not seen under other less-strict schedulers.
Reprints
