Vision Pipeline¶
Docker container that offers GPU support and a configured python environment for vision-pipeline and action-predictor.
Installation¶
git clone git@github.com:ReconCycle/vision-pipeline.git
git submodule update --init --recursive
Copy the directory from the Nextcloud Reconcycle repository git-data/vision-pipeline/data to the
vision-pipeline/data_limited
folder.cp config.example.yaml config.yaml
Additionally:
Setup ros-vision-pipeline.
Setup ros-basler camera.
Setup ros-realsense camera.
Camera Calibration¶
Look at the documentation in ros-basler.
Usage¶
Set the parameters you want in config.yaml
.
Run example:
python ros_pipeline.py
To enable the pipeline for realsense or basler use:
rosservice call /vision/realsense/enable True
or
rosservice call /vision/basler/enable True
Publishes:
Basler:
/vision/basler/colour
, Image/vision/basler/detections
, ROSDetections/vision/basler/markers
, MarkerArray/vision/basler/poses
, PoseArray
Realsense:
/vision/realsense/colour
, Image/vision/realsense/detections
, ROSDetections/vision/realsense/markers
, MarkerArray/vision/realsense/poses
, PoseArray/vision/realsense/gaps
, ROSGaps/vision/realsense/cluster
, Image/vision/realsense/mask
, Image/vision/realsense/depth
, Image/vision/realsense/lever
, PoseStamped
Services:
/vision/basler/enable
True/False/vision/realsense/enable
True/False/vision/vision_get_detection
VisionDetection.srv (from context_action_framework)
The /vision/vision_get_detection
service provides a single stable detection result from the requested camera.
For example, to get one Basler detection, run:
rosservice call /vision/vision_get_detection 0 False
To get a Realsense detection, run:
rosservice call /vision/vision_get_detection 1 True
where True provides the gaps as well.
** Camera Services:**
rosservice call /basler/set_sleeping
True/Falserosservice call /realsense/enable
True/False
Installation¶
git clone git@github.com:ReconCycle/ros-vision-pipeline.git
git clone git@github.com:ReconCycle/vision-pipeline.git
git clone git@github.com:ReconCycle/context_action_framework.git
cd ros-vision-pipeline
docker-compose build --build-arg 'TARGET=gpu'
cp docker-compose.example.yml docker-compose.yml
docker-compose up -d
To run the vision-pipeline follow the installation instructions there.
To run the action-predictor follow the installation instructions there.
Nvidia GPU Support¶
These steps are only necessary if the PC has an Nvidia graphics card.
You need to install the Nvidia graphics drivers and the CUDA toolkit. The Nvidia drivers are also bundled with CUDA, but I had trouble installing it this way.
Installing Nvidia graphics drivers¶
Prequisites:
sudo apt install build-essential libglvnd-dev pkg-config
Now download here the nvidia drivers and run as root to install. The Nvidia drivers require gcc-9 which is what ubuntu ships with by default.
You may need to disable the built in display driver: disable display driver
Installing CUDA toolkit¶
Install CUDA 11.3 on your host system. Go to Cuda Toolkit Archive then click on CUDA Toolkit 11.3. Select your operating system and download the runfile. Run the runfile using sudo
.
You may need gcc version 8 to run CUDA 11.3. If so, run:
sudo apt install gcc-8 g++-8
. Guide here on how to switch gcc versions.
There is an installation guide for CUDA from nvidia here. You can have a look at it for reference.
Docker running as root¶
First you need to install nvidia-container-runtime, instructions here and run:
sudo apt-get install nvidia-container-runtime
Edit /etc/docker/daemon.json
to contain:
{
"default-runtime":"nvidia",
"runtimes": {
"nvidia": {
"path": "nvidia-container-runtime",
"runtimeArgs": []
}
}
}
Then: sudo systemctl daemon-reload
Check if runtime is added sucessfully:
docker info|grep -i runtime
Docker running rootless (probably not what you want to do)¶
Do the same except put the file here: ~/.config/docker/daemon.json
.
Then: systemctl --user daemon-reload
Check if runtime is added sucessfully:
docker info|grep -i runtime
ONLY INSTALL DOCKER AFTER DOING THESE STEPS IF YOU ARE RUNNING ROOTLESS!
Running Docker container¶
Make sure you have everything set up from the previous sections. Install docker-compose.
Edit the docker-compose.yml
file and remove the ROS master and Rviz if you already have these running elsewhere. In principle the docker-compose.yml
file in its current state will provide you with a ROS master, and Rviz that can be accessed via the browser through the novnc
container.
The container is running in host
mode because this is the easiest way to give it access to the Basler camera. The ROS_IP
needs to be set correctly. Do this by running $ hostname -I
on the host and setting the ROS_IP
to this IP (take the first one if it gives multiple IP addresses).
Cloning Vision Pipeline Project¶
Clone the https://github.com/ReconCycle/vision-pipeline project. Set the path of the project in the volumes
section of the docker-compose.yml
. This is the path before the “:” sign. Do not change /root/vision-pipeline
.
volumes:
- $HOME/projects/vision-pipeline:/root/vision-pipeline
Specifying CPU or GPU target¶
If you are running a Nvidia GPU and you have configured nvidia-container-runtime then you can use the GPU target. If not, you should use CPU.
To set the target, change the following in docker-compose.yml
:
build:
context: ./build-2in1
args:
TARGET: cpu # cpu or gpu
Running The Pipeline and the Camera Publisher¶
Run:
$ cd ros-vision-pipeline
$ docker-compose up -d
or
$ docker exec -it ros-vision-pipeline bash