The Potential of Vision-Based Sensing Technology (VBST) Industry-Wide

What you’ll learn:

Design considerations for implementing VBST into different use cases.
Real-world examples of where VBST is being used, and how the applications enhance daily life across industries.
The future of VBST and what it might look like as more industries employ this technology.

Digital cameras have changed the way we see the world by letting us capture and share images and videos in ways that were impossible before. Artificial intelligence (AI) is helping us analyze data and identify patterns at a scale and pace unmatched in human history. Combined, the result is vision-based sensing technology. With the right considerations, VBST is taking the intelligent industry to the next level, especially in areas related to healthcare, safety, and efficiency.

Key Design Considerations for Implementing VBST Systems

When implementing vision-based sensing systems, a number of technical and design considerations should be kept in mind when planning for successful deployment:

Camera Selection and Placement

When selecting the optical sensing components, consider the image detail and accuracy requirements. Higher-resolution cameras increase the detail available in captured images. However, they often require an increase in available processing power and RAM, or accept an increase in the latency of the image-processing pipeline.

Choosing Optics and Field of View (FOV)

Ensure sensor optics are aligned with the intended use of the sensing system. Larger sensors contribute to a larger field of view, and lens focal length is inversely proportional to FOV.

When positioning cameras, aim to minimize occlusions and maximize the capture area of target data. Also consider factors such as lighting, angle, and mounting stability/vibration reduction. Think about temperature and potential weather conditions, weatherproofing, and ruggedizing where necessary.

Thermal management, humidity, dust and particulate matter, rain, snow, and wind are all environmental considerations that can be mitigated at the design stage through component selection and enclosure and mounting design. In dynamic environments, such as those found in mobile or industrial systems, shock absorption and mechanical vibration mitigation may be required. These are typically used alongside optical and digital image stabilization techniques to compensate for camera movement and ensure clear image capture.

Finally, for systems that plan to implement 3D reconstruction techniques (stereo vision, or structure from motion), ensure there are sufficient areas of image overlap and baseline physical separation. Converging camera fields of view help capture depth information accurately. In addition, ensuring that there’s a significant baseline separation improves both depth perception and reconstruction accuracy.

Controlling Lighting Conditions

When transitioning from lab-bench development systems to fielded system deployments, lighting is an often-overlooked factor. Proper lighting is crucial for clear image capture, which may require very early upfront planning in some deployments.

In-building installations, for example, may require additional lighting fixtures to fully illuminate target areas of interest. In the case of artificial lighting, it’s important to factor appropriate color temperature selection into the design of the application. Including adaptive lighting control into the system deployment allows for adjustment based on environmental conditions.

For low-light and night-time lighting conditions, high-sensitivity image sensors such as back-illuminated or stacked CMOS sensors are a proven technology. They can be combined with low-light algorithmic processing for noise reduction and image enhancement in low-light conditions. Infrared illumination combined with infrared (IR) cut filters for daytime use is an additional design option for effective image capture in low/no-light conditions.

Image Processing and Analysis Hardware Consideration

Make sure the selected processing hardware is sufficient to handle the computational load of image analysis, particularly in the case of real-time systems. Designing in computational headroom is good practice where possible. Once deployed, vision-based sensing systems often inspire and enable additional use cases, and designing in additional processing power promises a degree of future-proofing in the hardware.

In multi-camera systems, verify that cameras are synchronized to avoid discrepancies in data capture. Designing camera synchronization is usually straightforward during the implementation stage, though it can be difficult and costly to retrofit to an established or deployed system design.

VBST Data Transmission and Storage Factors

Consider the bandwidth requirements for streaming high-resolution video and consider if an edge-processing architecture is appropriate for applications where bandwidth is constrained. If video recording is also a feature of the system design, plan for an appropriate storage solution. This includes retrieval, retention, archiving, and potentially compression for efficient long-term storage.

Real-World Applications of Vision-Based Sensing Technology Across Industries

Early adopters are already benefiting from VBST. Many automakers use LiDAR and ultrasonic sensing devices in their products for safety cameras and to gather data, with the eventual goal of building autonomous vehicles. However, at least one electric-vehicle manufacturer instead opted for vision-based sensing technology to optimize performance, control costs, and streamline technology management.

In the precision agriculture space, a major farming equipment manufacturer offers computer-vision upgrades, backed by an AI-based weed identification service. This allows farmers to reduce herbicide use and waste by targeting weed-killer application in real-time to the actual weeds growing in their fields.

Retail stores that let customers skip checkout and walk out with their purchases do so using VBST systems. Such systems identify the items in their basket and use the customer’s stored payment information to run the transaction.

In the healthcare industry, small VBST systems already enable minimally invasive surgery techniques that reduce the risk of infection and the time patients need to recover. As these technologies advance, they also have the potential to reduce surgical errors through clearer visualization of fine structures and more comprehensive training on visual datasets of similar procedures.

The Future of Vision-Based Sensing Technology in Emerging Use Cases

Because of the adaptability, power, and relatively low cost of VBST systems, there are many more potential use cases to explore. In the manufacturing space, this technology has the potential to level the playing field between smaller manufacturers and enterprises, which can help bring more innovations to market faster.

Drones carrying optical cameras and supported by AI models can conduct safety inspections of manufacturing and energy equipment in places where it’s too dangerous or expensive to send technicians, such as offshore oil rigs or ships at sea. Autonomous or VBST-enhanced material-handling vehicles can scan their warehouse environments for humans to help avoid collisions and injuries while optimizing efficiency.

Smart cities could use VBST to monitor traffic conditions and redirect vehicles and pedestrians along the most efficient routes, while smart venue applications enable customized guest experiences and interactions.

VBST in the food and pharmaceutical supply chains could improve traceability and safety by monitoring temperature and time during transit and in storage. In healthcare, VBST has the potential to enhance post-operative and chronic condition monitoring for better patient outcomes while possibly reducing the need for in-person appointments.

As with any AI-driven application, use cases for VBST are often most compelling when there’s a human in the loop to monitor system functionality and act on alerts. It also needs people to imagine and implement new use cases. For designers and engineers who want to improve their organization’s safety, efficiency, and capacity to innovate, VBST technology is here. It just needs creative vision to apply it.