5. Stream ROS Images to web browser

5.1. Background

There are many possible packages, pipelines, protocols, and architectures for streaming images and video over a network (motion, gstreamer, janus, webrtc; to name just a few). Each option provides a different trade-off between the many aspects to be considered when streaming images and/or video, for example:

Ease of installation.
Ease and flexibility of usage.
Ability to multicast.
Automatically adapts quality to available bandwidth.
Server-side load.
… and more.

The Robot Web Tools initiative provides a number of packages that make it quick and easy to send and receive ROS messages from the web browser of any device that is on the same network as the robot that is running ROS. The key advantage of these tools is that you can send and receive ROS messages from any device without needing to install ROS on that device! Thus the Robot Web Tools provide the possibility to monitor and command you robot from the comfort of your smartphone (though you may need to do some html and JavaScript programming to customise things to your use case).

For the goal of viewing ROS Image messages from the sensor_msgs package, the Robot Web Tools initiative provides the web_video_server package as a plug-and-play solution. For completeness of referencing:

5.2. Install `web_video_server` package

The web_video_server package is installed on Ubuntu using the following command:

sudo apt install ros-noetic-web-video-server

After installation, the web_video_server package and node is available from any terminal that sources the ROS setup.bash.

5.3. Launch the `web_video_server` node

To stream images from your robot to any web browser on the network, the only thing you need to do is run the web_video_server node on your robot:

rosrun web_video_server web_video_server

This runs a HTTP server and a ROS node with the default configurations of:

address (string, default: 0.0.0.0) This is the address where the server listens for HTTP requests.
port (integer, default: 8080) This is the port number where the server listens for HTTP requests.
server_threads (integer, default: 1) The number of threads used to serve HTTP requests (these are not used for actual image streams).
ros_threads (integer, default: 2) The number of threads running ROS spinners. These threads are used to do the actual image encoding. These threads are shared between all current streams so the number does not need to match the number of connections.

You can configure these parameters of the web_video_server node by using a launch file to launch the node:

<launch>
  <!-- LAUNCH A WEB VIDEO SERVER NODE -->
  <node
    pkg    = "web_video_server"
    name   = "web_video_server"
    output = "screen"
    type   = "web_video_server"
    >
    <param name="port" type="int" value="8080"/>
    <param name="server_threads" type="int" value="1"/>
    <param name="ros_threads" type="int" value="3"/>
  </node>
</launch>

5.4. View the steam of images from a web browser

Once the web_video_server node is launched, go to the web browser on any device that is on the same network as your robot, and visit the following address:

http://<ip_address>:<port_number>/

where:

<ip_address> is replaced with the IP address or hostname of your robot, and
<port_number> is replaced with the port number that you configured when launching the web video server.

For example, if the IP address of your robot is 192.168.1.42 and you are using the default port, then the web address is:

http://192.168.1.42:8080/

This displays a webpage that lists the currently available ROS topics that have Image as their message type. Each topic is hyperlinked allowing you to open the stream or to grab a single snapshot on that topic.

As per the ROS wiki page for web_video_server, you can configure some attributes of the stream using parameters in the HTTP address. For example, to stream with a quality of 50%, use the following address:

http://<ip_address>:<port_number>/stream?topic=<topic_name>&quality=50

where <topic_name> is replaced with the name of the topic you wish to stream, which for example might be: /asc/camera_image

To further reduce bandwidth by reducing image size, specify the width and height in the stream address, for example:

http://<ip_address>:<port_number>/stream?topic=<topic_name>&quality=50&width=960&height=540

5.5. Configure your `package.xml` and `CMakeLists.txt` accordingly

If you are using OpenCV to capture and process images before publishing them as ROS messages, then you need to configure your ROS package for the libaries that convert from OpenCV format to ROS Image format.

Ensure that the package.xml of your ROS package contains the following:

<build_depend>sensor_msgs</build_depend>
<build_depend>cv_bridge</build_depend>

<build_export_depend>sensor_msgs</build_export_depend>
<build_export_depend>cv_bridge</build_export_depend>

<exec_depend>sensor_msgs</exec_depend>
<exec_depend>cv_bridge</exec_depend>

Ensure that the CMakeLists.txt of your ROS package contains the lines that are highlighted in the following:

At the top of the file:

find_package(catkin REQUIRED COMPONENTS
  message_generation
  roscpp
  rospy
  std_msgs
  geometry_msgs
  sensor_msgs
  cv_bridge
  genmsg
  roslib
)

Under the heading “Declare ROS messages, services and actions”:

generate_messages(
  DEPENDENCIES
  std_msgs
  geometry_msgs
  sensor_msgs
)

Under the heading “catkin specific configuration”:

catkin_package(
  CATKIN_DEPENDS
  roscpp
  rospy
  std_msgs
  geometry_msgs
  sensor_msgs
  cv_bridge
  roslib
)

5.6. Ensure that your Image message uses an appropriate encoding

For converting OpenCV images to ROS Images, the key lines of code in a Python node are as follows.

Import the message type, OpenCV, and the package that bridges from OpenCV to ROS:

# Import the message type
from sensor_msgs.msg import Image
# Import opencv
import cv2
# Package to convert between ROS and OpenCV Images
from cv_bridge import CvBridge, CvBridgeError

Initialise the ROS publisher for Image type messages, and the CvBridge object for converting images:

# Initialise a publisher for the images
self.image_publisher = rospy.Publisher("/asc"+"/camera_image", Image, queue_size=10)
# Initialise video capture from the camera
self.cam=cv2.VideoCapture(0)
# Initialise the OpenCV<->ROS bridge
self.cv_bridge = CvBridge()

Capture an OpenCV format of image from the camera:

# Read the camera frame
return_flag , current_frame = self.cam.read()

Convert the camera frame to ROS Image format (with bgr8 encoding) and publish it:
```
# Publish the camera frame
try:
    self.image_publisher.publish(self.cv_bridge.cv2_to_imgmsg(current_frame, "bgr8"))
except CvBridgeError as cv_bridge_err:
    print(cv_bridge_err)
```
Note that this conversion example fails if current_frame is a grayscale image because the example is requesting conversion to a colour format. The options for the encoding string is:
- mono8: CV_8UC1, grayscale image
- mono16: CV_16UC1, 16-bit grayscale image
- bgr8: CV_8UC3, color image with blue-green-red color order
- rgb8: CV_8UC3, color image with red-green-blue color order
- bgra8: CV_8UC4, BGR color image with an alpha channel
- rgba8: CV_8UC4, RGB color image with an alpha channel
Note that mono8 and bgr8 are the two image encodings expected by most OpenCV functions.

Note

For more details, explanations, and example code for converting OpenCV images to ROS image, see this cv_bridge tutorial for python

Paul N. Beuchat, 2023

This page is licensed under a Creative Commons Attribution 4.0 International License.