Project Overview
Vision-based steering represents a classical control challenge inside automated guided vehicles (AGVs) and autonomous warehouse robots. This project designed and integrated an intelligent vision node in ROS 2, using raw monocular camera feeds to perform real-time binary segmentation and center-of-mass centroid calculations.
By processing tracking contours using optimized OpenCV kernels, the node calculates the angular deflection between the robot's physical center and the center of the track's path. This tracking error is mapped in real-time to wheel drive velocity nodes (`cmd_vel`) using a precisely tuned Proportional-Integral-Derivative (PID) feedback script.
The Problem
Integrating camera sensors with motion control requires low latency and high accuracy. A basic line follower can easily lose the track during sharp turns or under changing lighting conditions, leading to control drift and path failures. The challenge was to construct a robust, real-time image processing pipeline that dynamically detects contours, filters out noise, and adjusts steering commands instantly.
What We Built
The system was implemented as a clean ROS 2 node architecture, keeping camera input, vision processing, and motor control as separate, well-defined components that communicate over ROS 2 topics. A monocular camera feed captures the track, where the perception node filters the target line color under varying lighting, computes its centroid, and determines the relative offset from the robot's physical center. Positional errors are translated into proportional steering commands. The system was validated in Gazebo simulation across different track geometries and conditions, ensuring that the same codebase executes on physical hardware with minimal changes.
Image Processing & Steering Pipeline
The vision control nodes execute frame processing sequentially to output stable steering values at 30 frames per second:
What Made It Work
Tuning the OpenCV pipeline was crucial to manage real-world variability, including HSV thresholding adjustments and contour filtering. By maintaining separate, modular ROS 2 nodes, each component could be debugged and optimized independently. This isolated development speeded up PID gain calibration and sensor latency profiling, resulting in smooth navigation and stable recovery.
What It Demonstrated
This project built practical competency in computer vision with OpenCV, ROS 2 node development, and the perception-to-action pipeline that underlies almost every real autonomous robot system - from warehouse AMRs to outdoor navigation platforms.
Technical Specifications
Perception Subsystems
- OpenCV 4 Core Libraries
- CvBridge Image Conversion
- RGB to HSV Color Masks
- Dynamic Thresholding & Filtering
- Moment Centroid Calculation
Actuation & Loop Tuning
- Proportional Gain (Kp = 0.005)
- Derivative Gain (Kd = 0.00025)
- Gazebo Camera Sensors Plugin
- /cmd_vel Navigation Messages
- Python ROS 2 Publisher Nodes
Execution & Diagnostic Logging
To launch the camera visual nodes and start the line-tracking simulation modules, run:
$ ros2 launch my_vision_follower gazebo_track.launch.py
Through OpenCV bounding-box optimization, the processing loops achieved sub-15ms execution speeds per frame, preventing tracking failures at maximum linear velocities.
Project Artifacts & Screenshots
Real-time image processing, mask tracking, and simulation dashboard screenshots from the OpenCV Line Follower ecosystem:
