Detectron2 Introduction

Please Note: These operations are performed locally. If you are going to use Colab, adjustments must be made accordingly.

What is Detectron2?

• Detectron2 is a next-generation software system that implements Facebook AI Research's state-of-the-art object detection algorithms.

• Modular and Flexible Structure

  • Detectron2 is a completely modular library.

  • You can use ready-made models directly, or easily integrate your own dataset, model architecture, and loss functions.

• Model Richness and Quality

  • It offers many high-accuracy models pre-trained on large datasets like COCO.

  • Examples: Faster R-CNN, Mask R-CNN, RetinaNet, Panoptic FPN, etc.

• Performance

  • It works very efficiently on the GPU.

  • Training and inference speeds are better than many competing frameworks on the market.

How to Install

To install Detectron2 locally, the relevant packages and CUDA version must be compatible.

Installation of required packages

# It is recommended to create a new anaconda environment for local installation.
# Open Anaconda Prompt and enter the following commands.
conda create -n <env_name> python=3.9 -y
# <env_name> -> Enter your environment name.
# -----------
# Switch to the relevant environment.
conda activate <env_name>

# Check your CUDA version.
nvcc --version

Download a PyTorch version compatible with your CUDA version from Here.

Make sure it is lower than version 2.6.0.

As an example, a command for PyTorch 2.1 and CUDA 11.8 is provided below.

conda install pytorch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 pytorch-cuda=11.8 -c pytorch -c nvidia

# Detectron2 installation command.
pip install --extra-index-url [https://miropsota.github.io/torch_packages_builder](https://miropsota.github.io/torch_packages_builder) detectron2==0.6+pt<PyTorch_version><compute_platform>

# <PyTorch_version> -> PyTorch Version -> e.g.: 2.1.0
# <compute_platform> -> CUDA Version -> e.g.: cu118 // For Cuda 11.8

# example: pip install --extra-index-url [https://miropsota.github.io/torch_packages_builder](https://miropsota.github.io/torch_packages_builder) detectron2==0.6+pt2.1cu118

The installation is generally complete. You can run the following code for verification.

python -c "import detectron2; print('Detectron2 successfully imported! Version:', detectron2.__version__)"

Training Model with Detectron2 on Custom Data

Detectron2 works with the COCO dataset, so if labeling was done with YOLO, we need to convert it to COCO format.

To convert YOLO format to COCO format, you can go to the relevant cell Here.

We need to include the dataset separated as (train, valid, test).

COCO format

• dataset
  • train
    • images
    • train.json
  • valid
    • images
    • valid.json
  • test
    • images
    • test.json

It should be specified as above. If the dataset is ready, we can proceed to model training.

We can use the code below for model training.

Let's include the relevant libraries

import detectron2
from detectron2.utils.logger import setup_logger
setup_logger()

from detectron2 import model_zoo
from detectron2.engine import DefaultPredictor
from detectron2.config import get_cfg
from detectron2.utils.visualizer import Visualizer
from detectron2.data import MetadataCatalog, DatasetCatalog

from detectron2.engine import DefaultTrainer

from detectron2.engine import DefaultTrainer
from detectron2.evaluation import COCOEvaluator, inference_on_dataset
from detectron2.data import build_detection_test_loader


from detectron2.data.datasets import register_coco_instances

Let's save the Dataset as DatasetCatalog and arrange it in a way that detectron2 can accept.

# Check if my_dataset_train exists, if so, remove it
if "my_dataset_train" in DatasetCatalog.list():
    DatasetCatalog.remove("my_dataset_train")
    MetadataCatalog.remove("my_dataset_train")

# Check if my_dataset_valid exists, if so, remove it
if "my_dataset_valid" in DatasetCatalog.list():
    DatasetCatalog.remove("my_dataset_valid")
    MetadataCatalog.remove("my_dataset_valid")

# Check if my_dataset_test exists, if so, remove it
if "my_dataset_test" in DatasetCatalog.list():
    DatasetCatalog.remove("my_dataset_test")
    MetadataCatalog.remove("my_dataset_test")

# Register new datasets so detectron can process them.
register_coco_instances('my_dataset_train', {}, "./dataset/train/train.json", './dataset/train/images')
register_coco_instances('my_dataset_valid', {}, './dataset/valid/valid.json', r'./dataset/valid/images')
register_coco_instances('my_dataset_test', {}, './dataset/test/test.json', r'./dataset/test/images')

It can be visualized with the following code to check if it is recorded correctly.

import random
import cv2
from detectron2.utils.visualizer import Visualizer
from detectron2.data import MetadataCatalog, DatasetCatalog
import os

# Getting your dataset and metadata
my_dataset_train_metadata = MetadataCatalog.get('my_dataset_train')
dataset_dicts = DatasetCatalog.get('my_dataset_train')

# Choosing 3 random samples from the dataset
for d in random.sample(dataset_dicts, 3):
    print(d['file_name'])  # Printing file name

    # Checking if the image file exists
    print(f"{d['file_name']} - EXISTS? {os.path.exists(d['file_name'])}")

    # Reading the image
    img = cv2.imread(d["file_name"])

    # If the image could not be read, give a warning
    if img is None:
        print(f"Image could not be read: {d['file_name']}")
        continue

    # Starting visualization
    visualizer = Visualizer(img[:, :, ::-1], metadata=my_dataset_train_metadata, scale=0.5)
    vis = visualizer.draw_dataset_dict(d)

    # Showing the image
    cv2.imshow("Visualization", vis.get_image()[:, :, ::-1])

    # Waiting for a key press to visualize the window opened on the screen
    cv2.waitKey(0)

# Closing all windows
cv2.destroyAllWindows()

After the necessary preparations are made, the model training is provided below.

Do not forget to set the parameters below before Model Training!

Note: In Detectron2, Iteration count is used instead of Epoch count.

After determining the desired Epoch Count, calculate the Iteration count as follows:

$\displaystyle \frac{\text{EPOCH COUNT} \times \text{TRAIN IMAGE COUNT}}{\text{BATCH SIZE}} = \text{ITERATION COUNT}$

class MyTrainer(DefaultTrainer):
    @classmethod
    def build_evaluator(cls, cfg, dataset_name, output_folder=None):
        if output_folder is None:
            output_folder = os.path.join(cfg.OUTPUT_DIR, "inference")
        return COCOEvaluator(dataset_name, cfg, False, output_folder)

    @classmethod
    def test(cls, cfg, model):
        # Test test set
        results = super().test(cfg, model)

        # Test validation set if provided
        if hasattr(cfg.DATASETS, "VALID"):
            evaluator = cls.build_evaluator(cfg, cfg.DATASETS.VALID[0])
            val_loader = build_detection_test_loader(cfg, cfg.DATASETS.VALID[0])
            results["validation"] = inference_on_dataset(model, val_loader, evaluator)

        return results

# Model training parameters should be specified here.

cfg = get_cfg()
cfg.merge_from_file(model_zoo.get_config_file("COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml"))
cfg.DATASETS.TRAIN = ("my_dataset_train",)
cfg.DATASETS.TEST = ("my_dataset_test",)
cfg.DATASETS.VALID = ("my_dataset_valid",)
cfg.DATALOADER.NUM_WORKERS = 4 # Number of cores to work simultaneously, can be increased based on CPU/GPU power.
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml")
cfg.SOLVER.IMG_PER_BATCH = 32 # Batch Size
cfg.SOLVER.BASE_LR = 0.0025 # Initial LearningRate(LR)
cfg.SOLVER.MAX_ITER = 11900 # Iteration Count
cfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 840 # Image Size
cfg.MODEL.ROI_HEADS.NUM_CLASSES = 1 # Number of Classes
cfg.OUTPUT_DIR = "./output" # Location where the model will be saved.
cfg.TEST.EVAL_PERIOD = 2500 # Specifies how often evaluation (validation) will be performed in iterations.
cfg.SOLVER.CHECKPOINT_PERIOD = 1000 # Model checkpoint (saving) frequency in iterations. 

os.makedirs(cfg.OUTPUT_DIR, exist_ok = True)

trainer = MyTrainer(cfg)
trainer.resume_or_load(resume=False)
trainer.train()

If the training is finished, models (every checkpoint) will have been created in the "./output" folder as desired.

The following usage is suitable for testing. As a result of the test, we will be able to see values like AP, AP50, AP75 on the test dataset.

cfg = get_cfg()
cfg.merge_from_file(model_zoo.get_config_file("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml"))
model_path = "" # Enter the model path (.pth).
cfg.MODEL.WEIGHTS = model_path
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5   # Threshold value / can be adjusted if desired.
cfg.DATASETS.TEST = ("my_dataset_test", )
cfg.MODEL.ROI_HEADS.NUM_CLASSES = 1
cfg.MODEL.MASK_ON = False
predictor = DefaultPredictor(cfg)
evaluator = COCOEvaluator("my_dataset_test", cfg, False, output_dir="./output/eval")
val_loader = build_detection_test_loader(cfg, "my_dataset_test")
inference_on_dataset(predictor.model, val_loader, evaluator)

The Train and Test stages of the model are finished. If you are satisfied with the results, the architecture can be used after adjusting the Project/files/config/kule.yaml file.

Converting YOLO Label Format to COCO Format

We need to have our dataset in COCO format so that we can train the Detectron2 model.

For this, we will install the YOLO_2_COCO github repository.

git clone "[https://github.com/Taeyoung96/Yolo-to-COCO-format-converter.git](https://github.com/Taeyoung96/Yolo-to-COCO-format-converter.git)" && cd Yolo-to-COCO-format-converter

Now we need to prepare our YOLO label format

YOLO format

• dataset
  • train
    • images
      • labels
  • valid
    • images
      • labels
  • test
    • images
      • labels

The file path needs to be in this form.

After preparing the relevant files, you can proceed with the command below.

Enter the following command into the command prompt and replace your model's classes with the places below.

"main.py"

python main.py --yolo-subdir --path <Absolute path to dataset_root_dir> --output <Name of the json file>

# <Absolute path to dataset_root_dir> -> Main directory of the Dataset. Ex: C:\Users\...\dataset
# <Name of the json file> -> The path and json name where you want to create the JSON_annotation file. Ex: C:\Users\...\dataset\train.json 

"""
This library can be used to convert datasets in different formats to COCO format. 
To do this, you can examine the usage of the relevant library in the README.md file by typing README.md in the command prompt.
"""

If it worked correctly, the json file will have been created and can be used. To continue training, proceed from Here.