Machine vision solution based on STM32N6 and VD66GY
I. Solution Introduction
This solution is based on the main control platform STM32N6 paired with ST's VD66GY image sensor to monitor human posture. The system utilizes the VD66GY sensor to collect environmental image data, which is then processed through the STM32N6 processor for efficient image analysis and development.
II. Introduction to Main Components
STM32N6 A microcontroller based on Arm Cortex-M55, equipped with ST Neuro-ART accelerator, H264 encoder, Neo-Chrom 2.5D graphics processor, 4.2MB continuous static random access memory for real-time data processing and multitasking, dedicated embedded neural processing unit (600 GOPS NPU & 3 TOPS/W power consumption), image processor ISP (Parallel and MIPI CSI-2 camera module I/F) supporting 5 Mpixel camera and 30 FPS, bad pixel, downsampling, black level, exposure, de-mosaicing, column conversion, contrast, cropping, scaling, gamma value, pixel packing.
Engineers can assign machine learning inference processing tasks to the NPU, allowing the efficient Cortex-M processor, ISP, and GPU to focus on other system-level tasks or running user applications. This enables it to handle complex image recognition and processing tasks, such as object segmentation and localization, pose estimation, object classification, face/detect, anomaly detection, and more.
ST's official website provides various development kits and tools along with its own discussion community and wiki, offering users abundant resources and a free development environment.
Image Sensor - ST VD66GY
1.5M BSI sensor (1124x1364), available in RGB and RGB-IR configuration modes, with a dynamic range of 60dB, and a pixel size of 2.61μm, performing excellently in the visible light and near-infrared regions (940nm). These sensors can capture up to 88 frames per second at a resolution of 1124x1364, and offer two pixel matrix versions: RGB 2x2 core and RGB-NIR 4x4 core. The pixel construction of these devices minimizes crosstalk and maintains high MTF (modulation transfer function) in both visible light and infrared regions.
ST also provides the VD66GY S-Board, which can be directly paired with STM32N6 for development, offering support from prototype to mass production. This not only saves resources but also accelerates product time-to-market.
The VD66GY S-Board is a comprehensive, versatile hardware kit for evaluating and integrating the VD66GY image sensor. It includes all necessary hardware, from electronic components to optical devices, to perform comprehensive product evaluation or directly integrate the product into the rest of the system. It incorporates a VD66GY image sensor with an FFC output connector, an M12 lens mount, a default non-glued lens, and a ribbon cable. The non-glued lens in the kit can be flexibly focused according to your needs, or simply removed to use different lenses. The kit is equipped with an M12 lens interface, allowing for the customization of various M12 or smaller off-the-shelf lenses for your application. Its standard FFC connector enables connection to cables of various lengths or pin arrangements to suit your setup.
III. Main Functions and Applications
The VD66GY image sensor collects forward image data, and the STM32N6 performs reasoning tasks to identify and process object types within the image for corresponding functional design. Based on the high-performance characteristics of the STM32N6 controller and the VD66GY sensor, we can further expand its application scope and functions: anomaly detection in smart industry, event monitoring in smart homes, body data measurement in personal healthcare, collision detection for robots, environmental perception for automobiles, and the development of personal electronic products and sports equipment. This not only improves the efficiency and quality of existing industrial and commercial applications but also expands into broader smart technology application fields, laying a solid foundation for future technological innovation.
Additionally, leveraging the edge computing and artificial intelligence technologies of the STM32N6 (person detection, posture estimation, gesture features) can enhance the product's application service range:
High-precision person detection over long distances in various environments: smart doorbells, room occupancy detection, and alarm systems.
High-precision multi-pose estimation: behavior analysis, activity monitoring, fall detection.
Precise system control based on hand feature points: non-contact human-machine interface, intelligent electronics industry.
► Scene Application Diagram
► Product Entity Diagram
►Block diagram of the plan
► Core Technology Advantages
STM32N6:
• Microcontroller based on Arm Cortex-M55, equipped with ST neural-ART accelerator, H264 encoder, Neo-Chrom 2.5D graphics processor, 4.2MB continuous static random-access memory for real-time data processing and multitasking, dedicated embedded neural processing unit (600 GOPS NPU & 3 TOPS/W power consumption), image signal processor (ISP) supporting parallel and MIPI CSI-2 camera module interfaces for 5 Mpixel camera and 30 FPS, bad pixel correction, down-sampling, black level correction, exposure, demosaicing, column conversion, contrast, cropping, scaling, gamma value, and pixel packing.
VD66GY:
• 1.5M BSI sensor (1124x1364), configurable in RGB and RGB-IR modes, with a dynamic range of up to 60DB and a pixel size of 2.61μm, performing well in the visible and near-infrared regions (940 nanometers). These sensors can capture up to 88 frames per second at a resolution of 1124x1364 and offer two pixel matrix versions: RGB 2x2 core and RGB-NIR 4x4 core. Global Shutter CIS is suitable for machine vision, ensuring that images captured during movement are not distorted, allowing for complete image analysis.