Using a multi-step pipeline, the codebase that we developed detects target objects with a stationary camera and OpenCV processing, matrix translation developed with the Denavit Hartenburg convention, velocity-based trajectory planning, and a custom PID controller. The result is a controller for the 4 degree-of-freedom OpenManipulatorX robotic arm that allows for identification and manipulation of real world objects.

Video Demo

This is a demo of the complete pipeline. The mounted camera sends frames to the program, which uses OpenCV to detect and locate target objects within the robot's workspace. The flowchart for the system is shown here: