Vision System

TODO: Training the Vision System

Dataset Creation and Training

Labelling Images with Segmentation for use with Yolact

git clone https://github.com/wkentaro/labelme
cd labelme
conda create --name=labelme python=3.6
conda activate labelme
pip install --editable .
cd labelme
labelme

How to Generate COCO dataset from labelme labelled data

  1. Create labels.txt file in the same directory as the labelled data with contents of your labels:

__ignore__
_background_
hca_front
hca_back
hca_side1
hca_side2
battery
pcb
internals_back
internals_front
internals

  1. Run command:

cd labelme/examples/instance_segmentation
./labelme2coco.py data_annotated data_dataset_coco --labels labels.txt

For example:

./labelme2coco.py /Users/sebastian/datasets/labelme/kalo_v2_imgs_20-11-2020-selected /Users/sebastian/datasets/labelme/kalo_v2_imgs_20-11-2020-selected-coco --labels /Users/sebastian/datasets/labelme/kalo_v2_imgs_20-11-2020-selected/labels.txt

Create Train Test Split from COCO .json file

Use the script in tools/coco-train-test-split/cocosplit.py to split the COCO .json file into a train.json and test.json.

How to Train Yolact

In this project this Yolact API is used.

These instructions are no longer valid.

  1. Create dataset with NDDS. Make sure instance segmentations and class segmentations are produced.

  2. Generate COCO format using the ndds-to-coco tool. First test wether it’s producing what you want by setting TESTING_STAGE=True. To check whether it worked properly, use the coco-viewer tool. Using TESTING_STAGE=True set CATEGORIES correctly.

  3. Open yolact/data/config.py and set the following correctly: NDDS_COCO_CLASSES, NDDS_COCO_LABEL_MAP and the paths in coco_ndds_dataset.

  4. To start training, replace num_gpus and run:

$ export CUDA_VISIBLE_DEVICES=0,1,2 (or whichever GPUs to use, then)
$ python -m yolact.train --config=coco_ndds_config --save_interval=2000 --batch_size=8*num_gpus

To resume:

$ python -m yolact.train --config=coco_ndds_config --resume=weights/****_interrupt.pth --start_iter=-1 --save_interval=2000 --batch_size=8*num_gpus

For training on less data, reduce the save_interval. On few real images use --save_interval=200 instead.

  1. To view logs run: tensorboard --logdir=yolact/runs.