RADU: Realsense D435 Installation Troubles with ROS2'

Sebastian - Apr 11 '22 - - Dev Community

In the last article, I explained the various visual sensors that you can equip on your robot. After comparing the various sensors, I decided to use the Intel Realsense D435 camera. It provides an RGB camera, a depth camera, has a perfect form factor, and comes with an officially maintained ROS wrapper.

So, I started the journey to get the D435 camera working with ROS2. The journey was long, unexpected, and ultimately failed. Using Ubuntu 20.04 LTS with latest ros2-foxy, and a combination of ros2-realsense v3.2.2 && librealsense-v2.48.0 or ros2-realsense v3.2.0 && librealsense-v2.45.0, I could not get it working. Nonetheless, I decided to document my approach in this article so that others might save time when trying to get it running.

This article originally appeared at my blog admantium.com.

First Approach: Installing and Running Ubuntu Packages

The first approach uses pre-build Ubuntu Packages. You will read how to install the SDK and the library, how to start the ROS2 node and the RViz visualization.

SDK and Library Installation

The official realsense-ros documentation explains how to install the librealsense SDK and the ros2-realsense ROS2 package by using prebuild packages.

Run the following commands:

sudo apt-get install ros-$ROS_DISTRO-realsense2-camera
sudo apt-get install ros-$ROS_DISTRO-realsense2-description
Enter fullscreen mode Exit fullscreen mode

When its finished, connect your Realsense camera an execute dmesg. You should see an output similar to this:

[ 9985.717873] usb 1-1.3: new high-speed USB device number 3 using xhci_hcd
[ 9985.822961] usb 1-1.3: New USB device found, idVendor=8086, idProduct=0b07, bcdDevice=50.cd
[ 9985.822978] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 9985.822991] usb 1-1.3: Product: Intel(R) RealSense(TM) Depth Camera 435
[ 9985.823002] usb 1-1.3: Manufacturer: Intel(R) RealSense(TM) Depth Camera 435
[ 9985.826872] uvcvideo: Unknown video format 00000050-0000-0010-8000-00aa00389b71
[ 9985.827127] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) Depth Camera 435  (8086:0b07)
[ 9985.830442] input: Intel(R) RealSense(TM) Depth Ca as /devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb1/1-1/1-1.3/1-1.3:1.0/input/input1
[ 9985.832009] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) Depth Camera 435  (8086:0b07)
Enter fullscreen mode Exit fullscreen mode

This looks good. Let’s start the ROS nodes.

Starting the ROS2 node

The documentation lists several starting options. Let’s use the official one:

ros2 launch realsense2_camera rs_launch.py device_type:=d435
Enter fullscreen mode Exit fullscreen mode

However, this command does not work and creates the following error:

realsense2_camera_node-1]  03/07 11:15:46,387 ERROR [281472753719712] (handle-libusb.h:51) failed to open usb interface: 0, error: RS2_USB_STATUS_ACCESS
[realsense2_camera_node-1]  03/07 11:15:46,387 ERROR [281472994937248] (sensor.cpp:526) acquire_power failed: failed to set power state
[realsense2_camera_node-1]  03/07 11:15:46,388 WARNING [281472994937248] (rs.cpp:306) null pointer passed for argument "device"
^C[WARNING] [launch]: user interrupted with ctrl-c (SIGINT)
[realsense2_camera_node-1] [INFO] [1625310949.972289490] [rclcpp]: signal_handler(signal_value=2)
Enter fullscreen mode Exit fullscreen mode

The Github repository is well maintained and has a well-supported issue list. Searching here, I found this issue and saw the main error: The UDEV rules were missing. You can install them manually as outlined there, and then restart the ROS node.

This time, no errors popped up, and I could see see the topics.

$> ros2 topic list
/camera/color/camera_info
/camera/color/image_raw
/camera/depth/camera_info
/camera/depth/color/points
/camera/depth/image_rect_raw
/camera/extrinsics/depth_to_color
/clicked_point
/goal_pose
/initialpose
/parameter_events
/rosout
/tf
/tf_static
Enter fullscreen mode Exit fullscreen mode

Visualization with RViz

When RVIZ is started, you can click on "Add", then "By Topic" and select the topic and the plugin you want to use. I added the Camera plugin to see images from the topic /camera/color/image_raw. However, as soon as I selected this topic, I could see several errors in the terminal.

[INFO] [1625329665.271601068] [rviz]: Message Filter dropping message: frame 'camera_color_optical_frame' at time 1625329664,860 for reason 'Unknown'
control_transfer returned error, index: 768, error: No data available
Enter fullscreen mode Exit fullscreen mode

Searching for this RVIZ error, I found an related issue #1547. In this particular case, the solution involved using and explicit argument to reset the device during startup, and also to revert to an earlier version of the library.

The explicit reset is done with the following command:

ros2 launch realsense2_camera rs_launch.py device_type:=d435 initial_reset:=true --debug
....
[realsense2_camera_node-1] [INFO] [1625385699.954313972] [RealSenseCameraNode]: RealSense ROS v3.2.1
[realsense2_camera_node-1] [INFO] [1625385699.954474877] [RealSenseCameraNode]: Built with LibRealSense v2.45.0
[realsense2_camera_node-1] [INFO] [1625385699.954518413] [RealSenseCameraNode]: Running with LibRealSense v2.45.0
Enter fullscreen mode Exit fullscreen mode

However, the problem persisted with librealsense-v2.45.0. Reading more documentation and issues on Github, I learned that the ros2-realsense v3.2.2 actually requires librealsense-v2.48.0. Since this version is not available as an Ubuntu package, I would need to manually compile it, or use ros2-realsense v3.2.0 in combination with librealsense-v2.45.0. I thought "The Newer, the Better" and started the manual installation.

Second Approach: Building, Installing and Running from Source

Librealsense SDK

The librealsense installation from source is documented on Github. The essential steps are:

  • Install build essentials for your Linux system
  • Clone the GitHub repository
  • Run cmake
  • Build and install

If you are using an Ubuntu version earlier then 20.04, you need to add additional packages and patch your Kernel. The configuration with cmake is important, you need to use appropriate cmake flags when building. For example on a Raspberry Pi it is advised to use FORCE_RSUSB_BACKEND . When you want to use the native commands for streaming realsense2 pictures, use -DBUILD_NETWORK_DEVICE=ON.

The complete compilation can take a while - on my QuadCore Linux Workstation, it was about 40 minutes.

ROS2 Realsense Package

The documentation only explains the step for ROS1, but i could figure out how to get it running on ROS1 as well.

Follow these steps ...

  • install the latest Realsense SDK
  • Create a ROS2 package with ros2 pkg create --build-type ament_cmake ros2-realsense
  • Clone the desired ros2-librealsense v3.2.2 into the src folder
  • Configure and compile the sources with colcon build

... or execute these commands in your shell:

mkdir -p ~/foxy_ws/src
cd ~/foxy_ws/src/
git clone https://github.com/IntelRealSense/realsense-ros.git
cd realsense-ros/
git checkout tags/3.2.2 #Be sure to use latest tag!
cd ..
colcon build --symlink-install
Enter fullscreen mode Exit fullscreen mode

It took me about two minutes to build the package.

Starting >>> realsense2_camera_msgs
[1.382s] WARNING:colcon.colcon_core.prefix_path.colcon:The path '/home/ubuntu/realsense-ros/install' in the environment variable COLCON_PREFIX_PATH doesn't exist
[1.382s] WARNING:colcon.colcon_ros.prefix_path.ament:The path '/home/ubuntu/realsense-ros/install/realsense-ros' in the environment variable AMENT_PREFIX_PATH doesn't exist
[1.383s] WARNING:colcon.colcon_ros.prefix_path.catkin:The path '/home/ubuntu/realsense-ros/install/realsense-ros' in the environment variable CMAKE_PREFIX_PATH doesn't exist
Finished <<< realsense2_camera_msgs [25.5s]
Starting >>> realsense2_camera
Starting >>> realsense2_description
Finished <<< realsense2_description [4.27s]
[Processing: realsense2_camera]
Finished <<< realsense2_camera [2min 18s]

Summary: 3 packages finished [2min 44s]
Enter fullscreen mode Exit fullscreen mode

Starting the ROS2 node

The compilation was successful. We can now source the installation folder with . install/setup.bash and starting the librealsense-ros2 node.

ros2 launch realsense2_camera rs_launch.py enable_pointcloud:=true device_type:=d435 initial_reset:=true enable_sync:=true --debug

[realsense2_camera_node-1] [INFO] [1625394768.207055050] [RealSenseCameraNode]: RealSense ROS v3.2.2
[realsense2_camera_node-1] [INFO] [1625394768.207213882] [RealSenseCameraNode]: Built with LibRealSense v2.45.0
[realsense2_camera_node-1] [INFO] [1625394768.207256325] [RealSenseCameraNode]: Running with LibRealSense v2.48.0
[realsense2_camera_node-1] [WARN] [1625394768.207289788] [RealSenseCameraNode]: ***************************************************
[realsense2_camera_node-1] [WARN] [1625394768.207318417] [RealSenseCameraNode]:** running with a different librealsense version **
[DEBUG] [launch.launch_context]: emitting event synchronously: 'launch.events.process.ProcessStderr'
[DEBUG] [launch]: processing event: '<launch.events.process.process_stderr.ProcessStderr object at 0xffffa8320a60>'
[DEBUG] [launch]: processing event: '<launch.events.process.process_stderr.ProcessStderr object at 0xffffa8320a60>' ✓ '<launch.event_handlers.on_process_io.OnProcessIO object at 0xffffa8320e20>'
[realsense2_camera_node-1] [WARN] [1625394768.207345676] [RealSenseCameraNode]:** than the one the wrapper was compiled with!   **
[realsense2_camera_node-1] [WARN] [1625394768.207373546] [RealSenseCameraNode]: ***************************************************
Enter fullscreen mode Exit fullscreen mode

The error message confused me, but was quickly resolved: There still was ros-foxy-librealsense2 package was still installed, and during string the ros2-realsense node, it would load the older librealsense! After deinstallation and launching the node, I could continue.

ros2 launch realsense2_camera rs_launch.py device_type:=d435 initial_reset:=true --debug

[160431.153805] usb 2-1: new SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[160431.174617] usb 2-1: New USB device found, idVendor=8086, idProduct=0b07, bcdDevice=50.cd
[160431.174622] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[160431.174625] usb 2-1: Product: Intel(R) RealSense(TM) Depth Camera 435
[160431.174628] usb 2-1: Manufacturer: Intel(R) RealSense(TM) Depth Camera 435
[160431.174630] usb 2-1: SerialNumber: 020223022670
[160431.180002] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) Depth Camera 435  (8086:0b07)
[160431.181574] uvcvideo 2-1:1.0: Entity type for entity Intel(R) RealSense(TM) Depth Ca was not initialized!
[160431.181579] uvcvideo 2-1:1.0: Entity type for entity Processing 2 was not initialized!
[160431.181581] uvcvideo 2-1:1.0: Entity type for entity Camera 1 was not initialized!
[160431.181779] input: Intel(R) RealSense(TM) Depth Ca as /devices/pci0000:00/0000:00:15.0/usb2/2-1/2-1:1.0/input/input33
[160431.182552] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) Depth Camera 435  (8086:0b07)
[160431.183971] uvcvideo 2-1:1.3: Entity type for entity Processing 7 was not initialized!
[160431.183974] uvcvideo 2-1:1.3: Entity type for entity Extension 8 was not initialized!
[160431.183977] uvcvideo 2-1:1.3: Entity type for entity Camera 6 was not initialized!
Enter fullscreen mode Exit fullscreen mode

The terminal output showed new warnings, but the node was finally up.

devcon@giga:/home/work/development/core$ ros2 node info /camera/camera
/camera/camera
  Subscribers:
    /parameter_events: rcl_interfaces/msg/ParameterEvent
  Publishers:
    /camera/color/camera_info: sensor_msgs/msg/CameraInfo
    /camera/color/image_raw: sensor_msgs/msg/Image
    /camera/depth/camera_info: sensor_msgs/msg/CameraInfo
    /camera/depth/color/points: sensor_msgs/msg/PointCloud2
    /camera/depth/image_rect_raw: sensor_msgs/msg/Image
Enter fullscreen mode Exit fullscreen mode

Also, the rqt graph looked good.

Visualization with Rviz

Its time to see the results of the invested effort. Start RViz, add the camera plugin and configure it to listen to the topic /camera1/color/image_raw.

However, immediately after I could see the following message in the terminal:

Message Filter dropping message: frame 'camera_color_optical_frame'
Enter fullscreen mode Exit fullscreen mode

And no data was shown in RViz.

At this time, I created the new issue #1970 on Github. Quickly, other community members reached out to me, and I learned that others were having similar issues ROS2 and the D435. And I gave up.

Conclusion

My first attempt to get the D435 running with ROS2 failed. I invested several hours searching for issues, re-reading documentation, re-compiling, and just trying different startup parameters. I could not get it to work. But because I wanted to use this particular sensor, I tried to get it running with ROS1. After not more than 2 hours, including getting and solving other error messages, I could finally see some pictures. This particular part will be explained in a future article.

Image
10 Healthy Habits For Lost The Car Key
Image
The Greatest Sources Of Inspiration Of Key Lost
Image
Power BI Import vs. Direct Query: Understanding the Pros and Cons of Each Data Connection Mode

powerbi
Image
Bye-bye Passwords: Embracing Secure, Hassle-Free Passkeys

hacktoberfest, hanko, opensource, webdev
Image
🔐 How to Generate a GitHub Personal Access Token (PAT)
Image
Let's Talk About Open Source - FAV0 Weekly #019
Image
Setting Up a Basic Dev Environment on a Windows/Linux Machine
Image
聊聊开源 - FAV0周刊#019
Image
Crafting a Balanced Patching Strategy

devops, updates, patching, cybersecurity
Image
10 Things We Were Hate About Replacing Lost Keys To A Car
Image
Daily Quote App

hacktoberfest, opensource, hacktoberfestchallenge, githubcopilot
Image
Symfony Station Communiqué — 25 October 2024. A look at Symfony, Drupal, PHP, and programming news!

symfony, drupal, php, typo3
Image
20 Insightful Quotes On Exposure To Asbestos Lawsuit
Image
Cat Flap Installation Cost Near Me
Image
What NOT To Do With The Asbestosis Settlement Amounts Industry
Image
A Peek Inside The Secrets Of Need Spare Car Key
Image
BuzzBundle 2.65.25 Crack + License Key Latest Free Download 2022
Image
How To Get More Value From Your Get A Spare Car Key Made
Image
Transit Gateway v/s Direct Connect v/s Site-to-Site VPN

aws, cloud, vpc, networking
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Terabox Video Player