English | 中文

This is a MindSpore Implementation of SAM2 from Meta, FAIR.

Segment Anything Model 2 (SAM 2) is a foundation model for image/video visual segmentation task. The model architecture is a transformer with streaming memory for real-time video processing. Detailed architecture is shown as follows.

1. Install CANN 8.1.RC1 and MindSpore according to the official instructions.
2. Install requirements

   cd examples/sam2
   pip install -r requirements.txt

3. Install mindone

   cd mindone
   pip install -e .
   

   Try python -c "import mindone". If no error occurs, the installation is successful.

• 📍 SAM2 with the following features
  • ✅ Prediction of masks given a static image and a reference point.
  • ✅ Prediction of segmentation masks given a static image.

Your contributions are welcome.

Please download checkpoints using:

cd checkpoints && \
./download_ckpts.sh && \
cd ..

predict_image.py is a script to infer the mask given an input image and a reference point. Please download the images for inference first:

mkdir images
wget -P images https://raw.githubusercontent.com/facebookresearch/sam2/main/notebooks/images/truck.jpg
wget -P images https://raw.githubusercontent.com/facebookresearch/sam2/main/notebooks/images/groceries.jpg

Then run python predict_image.py --image_path images/truck.jpg and python predict_image.py --image_path images/groceries.jpg.

The results will be saved in the images/ folder as shown below:

Original (reference point marked with Star)	Prediction 1	Prediction 2	Prediction 3

You can use predict_mask.py to get the segmentation mask from a given image. Please download the images for inference first:

wget -P images https://raw.githubusercontent.com/facebookresearch/sam2/main/notebooks/images/cars.jpg

Then run python predict_mask.py.

The results will be saved in the images/ folder as shown below:

Original	Prediction