Integrated Vision-Kinematics Control System for Autonomous Robotic Manipulation using CNN and IoRT

2026-05-31T20:39:18+07:00

This paper presents a novel integrated vision-kinematics control system for autonomous robotic manipulation, leveraging deep learning and the Internet of Robotic Things (IoRT). Unlike previous works that focus on isolated modules, our framework uniquely combines real-time YOLOv5-based object detection deployed on a Raspberry Pi with geometric inverse kinematics implemented on an ESP32 controller for a 6-DOF robotic arm. Under controlled conditions (fixed camera height of 40 cm, uniform workspace illumination, and high-contrast colour-coded objects), the detection module achieved a 100% detection rate on the test set (300 images containing 900 object instances) with a mean average precision ([email protected]) of 0.93 across three object classes. The system demonstrates precise end-effector positioning with an average error of 4.3 mm in open-loop control mode, a significant achievement given the absence of joint feedback sensors. All components are seamlessly integrated via an MQTT-based IoRT communication layer, ensuring reliable, low-latency data exchange for remote monitoring and control. Comprehensive experimental validation shows an 80% success rate in autonomous pick-and-place operations, with stable performance under multi-object scenarios and communication reliability exceeding 99%. This work bridges the gap between high-cost industrial systems and low-cost research platforms by demonstrating that intelligent system integration can compensate for hardware limitations, offering a scalable, modular, and cost-effective framework for intelligent robotic manipulation in Industry 4.0 applications, particularly suitable for educational robotics and light industrial automation.

International Journal of Artificial Intelligence & Robotics (IJAIR)

Integrated Vision-Kinematics Control System for Autonomous Robotic Manipulation using CNN and IoRT