A pen as an interactive device has been a second-class citizen to the keyboard and mouse. I remember using Graffiti on my Palm Pilot ages ago. MyScript’s Interactive Ink and Nebo change the pen from a basic drawing and data-entry tool to a real creation and editing tool.
So what has MyScript changed when using a stylus to enter characters or even script has been available for a while? In short, they have added a storage format that is editable and an interactive editor that handles text, equations, and diagrams in a very natural way. The system uses its own gestures for editing. As an editor I would love to have gestures that matched the usual ones I have used in the past, but they tend to be ones that only a teacher or editor love. MyScript’s choices are simple and relatively easy to learn, like a vertical bar adds a new line and space while scribbling over text or an object erases it. Now using a digital pen really feels like using a real pen, given these changes and the improvements in digital pen hardware.
Watching this video is the best way to see what these new tools do. Figure 1 shows how MyScript’s tools can change the script version of an equation into a digitized form. The same is true for text and graphics.
I had a challenge doing a hands-on test as the hardware required to use Interactive Ink is still somewhat limited. Pen-based systems like Microsoft’s Surface are needed to check out the tools, but the number of supported platforms is growing. My Windows 10 system has a Wacom tablet, but that is not compatible at this point.
On the other hand, I was able to check out MyScript’s free MyScript Calculator (Fig. 2) for Google Android and Apple platforms. This tool lets you enter numbers and equations, converts them to a more readable format, and solves the equations. The usual calculator icon on my smartphone has been replaced by MyScript’s version. It is much faster especially for more complex calculations. It is fun too.
But back to Interactive Ink and Nebo.
The MyScript Calculator highlights the handwriting recognition and basic editor support, but the effect is more apparent when the entire editing capability collection of operations comes into play. For example, the ability to create flowcharts and diagrams is very handy. Now there are other tools that do basic box conversion, but the ability to edit the text inside in the same fashion as a tool like PowerPoint is not generally available.
MyScript’s underlying data format is part of the advantage it brings to the table. This is akin to the Office file formats, but it incorporates text and graphics into one. This allows the editing capabilities and the dynamic space management necessary for convenient editing. This addresses ease of use as well as system accuracy and is similar to the problem that voice editing tools run into. Even a 99% accurate system will fall down if editing the last 1% is cumbersome and time-consuming. The format is portable, so data created on an Apple iPhone will be usable on a Windows Surface Pro.
MyScript is using the latest technology (including deep learning) to improve recognition at all levels (see “Deep Learning is Changing System Design” on electronicdesign.com). This entails a lot of up-front training, but results in a significantly smaller package with low overhead in the application itself.
Not everything uses deep learning. Recognizing and converting attributes (Fig. 3) is handled by more conventional programming. Still, it is the combination that makes the overall system fast and accurate.
Nebo is the app that most users can utilize to take advantage of MyScript’s technology. There is a cost, unlike the free MyScript Calculator, but it has significantly more functionality.
Embedded and software developers will be more interested in MyScript’s application developer toolkit (ATK). This provides access to the underlying file formats as well as the functionality found in Nebo. A host of applications can take advantage of the tool from form systems for vertical markets like medical to inventory systems.
Pen and paper are still handy and more natural for many. Digital Post-it notes have been around, as has the ability to share documents via the internet/cloud. MyScript’s technology makes it easier to use the digital form with all its advantages from sharing to searching content.
Will MyScript’s vision of pen input prevail? I hope so. Now if I can just get my hands on a compatible platform like the new Galaxy Note with the S Pen ...
