AdaptSAM: Adapt Segment Anything Model Based on Detectron2

AdaptSAM is a library built on top of Detectron2 that allows adapting the Segment-Anything Model (SAM) for custom COCO-format datasets. It supports point prompt training and leverages LORA technology for customizable adaptations of SAM.

Getting Started with SAM Detectron2

Command line-based Training & Evaluation

We provide a script tools/lazyconfig_train_net.py that trains all configurations of SAM.

To train a model with tools/lazyconfig_train_net.py, first prepare the datasets following the instructions in datasets/README.md and then run, for single-node (8-GPUs) NVIDIA-based training:

(node0)$ python ./tools/lazyconfig_train_net.py --config-file configs/finetune/finetune_lora_sam_coco.py --num-gpus 8 

To evaluate a trained SAM model's performance, run on single node

(node0)$ python ./tools/lazyconfig_train_net.py --config-file configs/finetune/finetune_lora_sam_coco.py --num-gpus 8 --eval-only --init-from /path/to/checkpoint

To evaluate a original SAM model's performance, run on single node

(node0)$ python ./tools/lazyconfig_train_net.py --config-file configs/eval/eval_sam_coco.py --num-gpus 4 --eval-only

Installation

Our environment requirements are consistent with ODISE, for installation, please refer to ODISE

Install dependencies by running:

conda create -n adaptsam python=3.9
conda activate adaptsam
conda install pytorch=1.13.1 torchvision=0.14.1 pytorch-cuda=11.6 -c pytorch -c nvidia
conda install -c "nvidia/label/cuda-11.6.1" libcusolver-dev
pip install -U opencv-python

# under your working directory
git clone https://github.com/facebookresearch/detectron2.git
cd detectron2
pip install -e .
pip install git+https://github.com/cocodataset/panopticapi.git
pip install git+https://github.com/mcordts/cityscapesScripts.git

cd ..
git clone [email protected]:YukunLi99/AdaptSAM.git
cd AdaptSAM
pip install -e .

Features

• Compatible with all features of Detectron2 framework
• Supports custom COCO-format datasets
• Supports point prompt training
• Supports LORA technology

Prepare Datasets

Dataset preparation for AdaptSAM follows Detectron2 and Mask2Former.

For Pascal VOC Instance Segmentation, run the following command to generate coco-format json file (refer to voc2coco):

$ python voc2coco.py \
    --ann_dir /path/to/annotation/dir \
    --ann_ids /path/to/annotations/ids/list.txt \
    --labels /path/to/labels.txt \
    --output /path/to/output.json \

Results

method	datasets	$AP_{100}$	$AR_{100}$	$AR_{s}$	$AR_{m}$	$AR_{l}$
SAM-Base	voc	1.3	34	33.8	28.4	37.5
AdaptSAM-Base	voc	5.8	52	22.6	38.1	65.8
SAM-Huge	coco	1.8	32.7	23.5	37.8	41.8
AdaptSAM-Huge	coco	10.2	49.7	28.4	59.5	73.2

Acknowledgement

Code is largely based on Detectron2, SAM, Mask2Former, ODISE

License

This project is licensed same as SAM model.