A Geospatial Video Analytic System with Spatial-Aware Optimizations

Absract

Videos that are shot using commodity hardware such as phones and surveillance cameras record various metadata such as time and location. We encounter such geospatial videos on a daily basis and such videos have been growing in volume significantly. Yet, we do not have data management systems that allow users to interact with such data effectively.

In this paper, we describe Spatialyze, a new framework for end-to-end querying of geospatial videos. Spatialyze comes with a domain-specific language where users can construct geospatial video analytic workflows using a 3-step, declarative, build-filter-observe paradigm. Internally, Spatialyze leverages the declarative nature of such workflows, the temporal-spatial metadata stored with videos, and physical behavior of real-world objects to optimize the execution of workflows. Our results using real-world videos and workflows show that Spatialyze can reduce execution time by up to 5.3x, while maintaining up to 97.1% accuracy compared to unoptimized execution.

Requirement

- python >= 3.10 (Prefer Conda/Mamba)
- docker
- cuda >= 11.7 (If using GPU)

How to Setup Spatialyze Repo

Clone the Spatialyze repo

git clone --recurse-submodules [email protected]:apperception-db/spatialyze.git
cd spatialyze

We use Conda/Mamba to manage our python environment

Install Mamba or install Conda

Setup Environment and Dependencies

# clone submodules
git submodule update --init --recursive

# setup virtual environment
# with conda
conda env create -f environment.yml
conda activate spatialyze
# OR with mamba
mamba env create -f environment.yml
mamba activate spatialyze

# install python dependencies
poetry install

If using DeepSORT (Optional)

Building rank_cylib will speed up DeepSORT.

cd ./spatialyze/video_processor/modules/yolo_deepsort/deep_sort/deep/reid/torchreid/metrics/rank_cylib
make
# If make does not work (use your current python interpreter)
python setup.py build_ext --inplace
rm -rf build

Spatialyze Demo

Start Spatialyze Geospatial Metadata Store MobilityDB

docker volume create spatialyze-gsstore-data
docker run --name     "spatialyze-gsstore"                        \
           --detach                                               \
           --publish  25432:5432                                  \
           --volume   spatialyze-gsstore-data:/var/lib/postgresql \
                      mobilitydb/mobilitydb

Setup the MobilityDB with customized functions

docker exec -it spatialyze-gsstore rm -rf /pg_extender
docker cp scripts/pg-extender spatialyze-gsstore:/pg_extender
docker exec -it -w /pg_extender spatialyze-gsstore python3 install.py

To run MobilityDB every system restart

docker update --restart unless-stopped spatialyze-gsstore

Try the demo (WIP 🚧)

In spatialyze repo:

jupyter-lab

The demo notebook first constructs the world. Then it queries for the trajectory of the cars that appeared once in an area of interests within some time interval.

Citing Spatialyze

This paper is currently under review for VLDB.

@misc{kittivorawong2023spatialyze,
    title={Spatialyze: A Geospatial Video Analytics System with Spatial-Aware Optimizations}, 
    author={Chanwut Kittivorawong and Yongming Ge and Yousef Helal and Alvin Cheung},
    year={2023},
    eprint={2308.03276},
    archivePrefix={arXiv},
    primaryClass={cs.DB}
}

Codecov