pq-yang
154 lines
4.7 KiB
Markdown
154 lines
4.7 KiB
Markdown
# Evaluation Documentation
|
|
|
|
This document shows the way to reproduce the quantitative results in our paper on two synthetic benchmarks ([YouTubeMatte](https://github.com/pq-yang/MatAnyone/tree/main?tab=readme-ov-file#youtubematte-dataset) and [VideoMatte](https://github.com/PeterL1n/RobustVideoMatting/blob/master/documentation/training.md#evaluation)) and one real benchmark ([CRGNN](https://github.com/TiantianWang/VideoMatting-CRGNN)).
|
|
|
|
## YouTubeMatte
|
|
|
|
**📦 We provide the inference results with MatAnyone 2 on the YouTubeMatte benchmark [here](https://drive.google.com/drive/folders/1fgPAx4pRGyxGIYW4NeBevDM8PpA0TG9I?usp=sharing).**
|
|
|
|
### Preparation
|
|
* [YouTubeMatte.zip (6.24G)](https://drive.google.com/file/d/1IEH0RaimT_hSp38AWF6wuwNJzzNSHpJ4/view?usp=drive_link)
|
|
* [YouTubeMatte_first_frame_seg_mask.zip (310K)](https://drive.google.com/file/d/1Zpa7SB7VZmkvRDiehVC-c_0dmFWXdfzK/view?usp=drive_linkk)
|
|
|
|
To run the inference scripts, your files should be arranged as:
|
|
```
|
|
data
|
|
|- YouTubeMatte_first_frame_seg_mask # for inference only
|
|
|- YouTubeMatte
|
|
|- youtubematte_512x288
|
|
|- youtubematte_1920x1080
|
|
```
|
|
|
|
### Batch Inference
|
|
Empirically, for low-resolution (`youtubematte_512x288`) and high-resolution (`youtubematte_1920x1080`) data, we set **different** hyperparameter values for `--warmup`, `--erode_kernel`, and `--dilate_kernel`.
|
|
|
|
```shell
|
|
# lr: youtubematte_512x288
|
|
bash evaluation/infer_batch_lr_yt.sh
|
|
|
|
# hr: youtubematte_1920x1080
|
|
bash evaluation/infer_batch_hr_yt.sh
|
|
```
|
|
|
|
### Evaluation
|
|
To run the evaluation scripts, your files should be arranged as:
|
|
|
|
```
|
|
data
|
|
|- YouTubeMatte
|
|
|- youtubematte_512x288
|
|
|- youtubematte_1920x1080
|
|
|
|
|- results
|
|
|- youtubematte_512x288
|
|
|- youtubematte_1920x1080
|
|
```
|
|
|
|
```shell
|
|
# lr: youtubematte_512x288
|
|
python evaluation/eval_lr.py \
|
|
--pred-dir ./data/results/youtubematte_512x288 \
|
|
--true-dir ./data/YouTubeMatte/youtubematte_512x288
|
|
|
|
# hr: youtubematte_1920x1080
|
|
python evaluation/eval_hr.py \
|
|
--pred-dir ./data/results/youtubematte_1920x1080 \
|
|
--true-dir ./data/YouTubeMatte/youtubematte_1920x1080
|
|
```
|
|
|
|
## VideoMatte
|
|
|
|
**📦 We provide the inference results with MatAnyone 2 on the VideoMatte benchmark [here](https://drive.google.com/drive/folders/12QM-_kyerE1tQfINoR17zYIFtrcwCK04?usp=sharing).**
|
|
|
|
### Preparation
|
|
* [videomatte_512x512.tar (1.8G)](https://robustvideomatting.blob.core.windows.net/eval/videomatte_512x288.tar)
|
|
* [videomatte_1920x1080.tar (2.2G)](https://robustvideomatting.blob.core.windows.net/eval/videomatte_1920x1080.tar)
|
|
* [VideoMatte_first_frame_seg_mask.zip (416K)](https://drive.google.com/file/d/1kN5gX4NAEa4HG-k2ir8kPcEp_18DbDHt/view?usp=drive_link)
|
|
|
|
To run the inference scripts, your files should be arranged as:
|
|
```
|
|
data
|
|
|- VideoMatte_first_frame_seg_mask # for inference only
|
|
|- VideoMatte
|
|
|- videomatte_512x288
|
|
|- videomatte_1920x1080
|
|
```
|
|
|
|
### Batch Inference
|
|
Empirically, for low-resolution (`videomatte_512x288`) and high-resolution (`videomatte_1920x1080`) data, we set **different** hyperparameter values for `--warmup`, `--erode_kernel`, and `--dilate_kernel`.
|
|
|
|
```shell
|
|
# lr: videomatte_512x288
|
|
bash evaluation/infer_batch_lr_vm.sh
|
|
|
|
# hr: videomatte_1920x1080
|
|
bash evaluation/infer_batch_hr_vm.sh
|
|
```
|
|
|
|
### Evaluation
|
|
To run the evaluation scripts, your files should be arranged as:
|
|
|
|
```
|
|
data
|
|
|- VideoMatte
|
|
|- videomatte_512x288
|
|
|- videomatte_1920x1080
|
|
|
|
|- results
|
|
|- videomatte_512x288
|
|
|- videomatte_512x288
|
|
```
|
|
|
|
```shell
|
|
# lr: videomatte_512x288
|
|
python evaluation/eval_lr.py \
|
|
--pred-dir ./data/results/videomatte_512x288 \
|
|
--true-dir ./data/VideoMatte/videomatte_512x288
|
|
|
|
# hr: videomatte_1920x1080
|
|
python evaluation/eval_hr.py \
|
|
--pred-dir ./data/results/videomatte_1920x1080 \
|
|
--true-dir ./data/VideoMatte/videomatte_1920x1080
|
|
```
|
|
|
|
## CRGNN
|
|
|
|
**📦 We provide the inference results with MatAnyone 2 on the CRGNN benchmark [here](https://drive.google.com/file/d/1JJyE4uPymEcijNa1Ok8ME_BY6oxJ3EXJ/view?usp=sharing).**
|
|
|
|
### Preparation
|
|
* [real_human_data](https://www.dropbox.com/sh/23uvsue5we7e7b5/AAB4GSSWIaKiSouvN3wuWiwWa?dl=0)
|
|
* [CRGNN_first_frame_seg_mask.zip (151K)](https://drive.google.com/file/d/1cDSf1kO_tdWy-q3CuX4IfuZ16ZYEER-d/view?usp=sharing)
|
|
|
|
To run the inference scripts, your files should be arranged as:
|
|
```
|
|
data
|
|
|- crgnn
|
|
|- alpha
|
|
|- image_allframe
|
|
|- mask # first frame seg mask
|
|
```
|
|
|
|
### Batch Inference
|
|
|
|
```shell
|
|
bash evaluation/infer_batch_crgnn.sh
|
|
```
|
|
|
|
### Evaluation
|
|
To run the evaluation scripts, your files should be arranged as:
|
|
|
|
```
|
|
data
|
|
|- crgnn
|
|
|- alpha
|
|
|
|
|- results
|
|
|- crgnn
|
|
```
|
|
|
|
```shell
|
|
python evaluation/eval_crgnn.py \
|
|
--pred-dir ./data/results/crgnn \
|
|
--true-dir ./data/crgnn/alpha
|
|
```
|