Vision System

This guide explains briefly how to create a dataset and train the different models.

YOLOv8 Segmentation Model for Devices and Components

1. Label a dataset using Labelme.

2. Train on the dataset using YOLOv8.

3. In vision_pipeline/config.yaml set the paths correctly:

model: yolov8

yolov8_model_file: ~/vision_pipeline/data_limited/yolov8/output_2024-07-17_20000_incl_new_jsi_imgs_p2/epoch60.pt
yolov8_score_threshold: 0.5

Classification Model

1. Train the classifier in https://github.com/ReconCycle/device_reid

2. In vision_pipeline/config.yaml set the paths correctly:

classifier_model_file: "~/vision_pipeline/data_limited/classifier/2024-07-19__14-32_classify/lightning_logs/version_0/checkpoints/epoch=339-step=339.ckpt"
classifier_threshold: 0.2
classifier_allow_list: ["kalo", "hekatron", "fumonic"]

Rotation Estimation Model (SuperGlue)

1. Train the SuperGlue model in https://github.com/ReconCycle/superglue_training.

2. In vision_pipeline/config.yaml set the paths correctly:

superglue_templates: ~/datasets2/reconcycle/2023-12-04_hcas_fire_alarms_sorted_cropped
superglue_model_file: "~/superglue_training/output/train/2024-06-26_superglue_model_evens_finished/weights/best.pt"

superglue_match_threshold: 0.5
rotation_median_filter: False
superglue_visualise_to_file: False #! saves images... don't run all the time