object_grabber_pkg

Short Description

object_grabber_pkg is a ROS2 package for detection, localization, navigation, and grasping of everyday objects (e.g., bottles) on a mobile manipulator (e.g., Stretch 3).

The package integrates: - a YOLO segmentation model for object detection - DBSCAN-based clustering for object localization - Nav2 for navigation - an IK-based grasping sequence

---

Main Features

• Scanning with head camera to generate a PointCloud of all detected objects
• Clustering of detected points (DBSCAN) and output of object centroids
• Full workflow: Scan → Object selection → Navigation → Scan → Grasping
• Action server for scan and grasp workflows to integrate into higher-level control processes
• Configurable Nav2 parameters for different environments

---

Prerequisites

• Ubuntu 22.04
• ROS 2 Humble
• Nav2, slam_toolbox
• Drivers for used sensors (e.g., realsense2_camera, LIDAR drivers, Stretch driver)
• Python dependencies (example):
  • ultralytics (YOLO)
  • scikit-learn (DBSCAN)
  • ikpy
  • scipy

---

Installation

1. Copy the package to ~/ament_ws/src on the Stretch 3

2. System and ROS dependencies are already pre-installed on the Stretch

3. Install Python dependencies:

   pip install -U pip
   pip install ultralytics scikit-learn ikpy scipy

4. Build & source:

   cd ~/ament_ws
   colcon build --symlink-install
   source install/setup.bash

---

Start / Example

The package contains a launch file (object_grabbing_launch.py) that starts the necessary subsystems (robot driver, camera, LIDAR, Nav2 parameters). The used Maps (Map + Keepout-Map) are loaded from ~/stretch_user/maps.

Example:

source ~/ament_ws/install/setup.bash
ros2 launch object_grabber_pkg object_grabbing_launch.py

---

Important Nodes

• object_detector_node --- Scan action, object detection (YOLO), PointCloud generation and publication
• object_planner_node --- Coordination: Scan → Selection → Navigation → Grasping
• object_grabber_node --- IK calculations, grasp sequence (open/close gripper, stow, move)

---

Actions / Topics (important names)

Actions

• object_grabber/scan --- Starts a scan and provides a PointCloud with detected objects
• object_grabber/grab_object --- Executes the grasp sequence for a selected object
• object_grabber/execute --- Coordinates the full sequence: Scan → Move to target → Rescan → Grab

Topics (Visualization)

• /object_detection/image_annotated --- Annotated RGB image
• /object_detection/object_cloud --- Raw PointCloud of all detected points
• /object_detection/object_cloud_clustered --- Cluster centroids (after DBSCAN)