This is a Flutter desktop application designed to efficiently detect defects in Printed Circuit Boards (PCBs) directly from their images. Leveraging a powerful combination of Flutter for a responsive graphical user interface (GUI) and Rust for high-performance machine learning inference, this tool aims to simplify and accelerate the fault detection process.
It's built to assist in quality control and inspection by identifying issues in both copper tracks and soldered components.
- Intuitive Flutter GUI
- High-performance Rust backend
- Dual-model approach for comprehensive analysis
- YOLOv11 for defect detection
- YOLOv11 with SAHI for small object detection
It uses two YOLOv11 models, each with a specific purpose:
- Copper Track Defect Detection Model: Designed to detect defects specifically within the PCB's copper tracks.
- Component Detection Model: Used to identify components soldered onto the PCB track.
The two models are mutually exclusive; that is, if an image is selected as a track image, then it cannot also be selected as a component image. This is because a PCB image containing components leads to many false positives when run on the track model.
The copper track defect model is executed directly on the selected images (after slicing).
The components are first detected on the image of a known good benchmark PCB. Then, images of other PCBs are selected and compared with the benchmark to find defects. This process also utilizes slicing.
The intuitive Flutter GUI allows users to easily upload PCB images, select the desired detection model, and visualize the identified defects.
See the demo at this YouTube link for more information.
After uploading PCB images:
Visualizing detected missing components on the PCB
Select this by checking "Is this a PCB track image?" in the UI.
See at a glance which PCB images are faulty.
See the main model training repository & report at pcb-fault-detection repo
Also, the ./test_images directory of the companion repository (Link to test_images) contains the images used in the demo.
See the Releases tab. Download the Application ZIP file, extract it and run as shown in the video.
The images are sliced into smaller square sub-images and then sent to the model for inference, as this enables it to detect small components/track defects, improves detection accuracy for small objects. The results are then stitched together to get the final result. This is called Slicing Aided Hyper Inference (SAHI).
This also helps overcome resolution limitations, as the YOLOv11 model can only take in inputs of size 640x640.
Much of the code for this has been adapted from the Python SAHI implementation, but translated to Rust.
This project utilizes the capabilities of the flutter_rust_bridge framework to allow the Flutter UI to interface with the Rust code, while also automatically generating the necessary FFI (Foreign Function Interface) bindings.
Here, we use Rust to actually execute the ML model inference using the Rust ort 2.0.0-rc.10 library. The models are stored in the ONNX format, which is compatible with ort, as it internally uses the ONNX Runtime library.
Also, we use Rust, and the image and ndarray libraries for image slicing.
Finally, we use the Rust rayon library, for easy & simple parallelism. It allows us to directly & easily run parallel operations on streams of data like (map, filter_map, flatten, various sorting functions, etc.) by using its parallel iterators.
Note: We have only compiled ort with DirectML support as the sole execution provider, along with the CPU fallback, of course. The reason for this is because the CUDA binaries significantly increased the application's size, and most devices we will demonstrate it on will not have a CUDA-compatible GPU (including the PC that this was developed on). So, for the sake of simplicity, other execution providers are not included. However, the ort library makes it very easy to add execution providers as necessary; so, if needed, they can be added. The code for using them is also included, but commented out.
Download and install Flutter & Rust. Then run:
flutter pub get
cd ./rust
cargo build
cd ..
flutter_rust_bridge_codegen generate
- Then, in the
./rust/target/debugfolder, you will find some.dlls,.libs, or.sofiles depending on your platform. - Copy the following files (or their equivalents) to a global directory (for example,
C:/local/onnxruntime-win-x64-1.22.0/lib/):DirectML.dllonnxruntime_providers_*.dll
- Then, go to
./rust_builder/windows/CMakeLists.txt(or the equivalent for your platform) and add to the last line, or replace it as shown below, to include the required binary files in the build. This step manually links the ONNX Runtime provider DLLs because they are not automatically bundled byflutter_rust_bridge's cargo-kit integration in this setup.
set(rust_lib_pcb_fault_detection_ui_bundled_libraries
"${${PROJECT_NAME}_cargokit_lib};C:/local/onnxruntime-win-x64-1.22.0/lib/DirectML.dll;C:/local/onnxruntime-win-x64-1.22.0/lib/onnxruntime_providers_shared.dll;C:/local/onnxruntime-win-x64-1.22.0/lib/onnxruntime_providers_shared.lib;C:/local/onnxruntime-win-x64-1.22.0/lib/onnxruntime.dll"
PARENT_SCOPE
)
On Linux/macOS, look for .so or .dylib files respectively, and adjust paths accordingly. Like so:
set(rust_lib_pcb_fault_detection_ui_bundled_libraries
"${${PROJECT_NAME}_cargokit_lib};~/.local/lib/onnxruntime-win-x64-1.22.0/lib/DirectML.so;~/.local/lib/onnxruntime-win-x64-1.22.0/lib/onnxruntime_providers_shared.so;~/.local/lib/onnxruntime-win-x64-1.22.0/lib/onnxruntime_providers_shared.so;~/.local/lib/onnxruntime-win-x64-1.22.0/lib/onnxruntime.so"
PARENT_SCOPE
)
Note that these lists do not contain the CUDA binaries for the reasons explained before.
This project is distributed under the MIT license. See the LICENSE.txt file in this repository for the full text.
We welcome contributions! If you have suggestions for improvements, bug reports, or would like to contribute code, open an issue/pull request.