Model Evaluation of DPI-Net and other particle representation based work

[LittleFlyFish]

Hi,

This is very impressive work! I believe it will be like ImageNet for Physicals prediction. However, I face many difficulties when I try to follow it. The problems are mostly about model evaluations of DPI-NET/GNS etc. The questions are followings:

1. Did this dataset offers particle representations for evaluation and training of DPI-NET/GNS/HRN? Could we download it directly like the mp4 data as well? (That would be very very helpful!) How many particles are used? For each particle, is it contains information like mass, velocity, stiffness, etc? How could I obtain the particles data information?

2. For DPI-Net, it only takes one G_0 as the initial state to produce the future simulations. For Physion, how many time steps are used as inputs? How many time steps are used as predictions? (In the human accuracy evaluation as well) Is the last frame is used as initial state for DPI-NET?

3. This code offers the Physpot as tools to train the DPI-NET model. But I somehow find out it is very complex. Could you present the script to train DPI-NET and other particle-based models? It would be very important to provide more detailed explanations for this part for other researchers to follow this work.

Thank you very much if you would like to help!

[htung0101]

Thanks for your interests! We will open source our implementation for DPI-Net and GNS soon.

1. Yes, you will need to download the hdf5 files from our website. We have a script that will take these hdf5 files and transform them into the data format used by DPI/GNS. The procedure includes transforming saved object meshes in hdf5 into particles.
   We used around 500-30000 particles, depending on the scenarios. The information includes particle positions and velocities. Our dataset doesn't include variations from object mass and stiffness, thus this is not included. We plan to have it in our future version.

2. Currently we use 2 time steps as input (time t for the particle positions, and time t-1 for calculating the particle velocities). We unroll the model for 100-300 timesteps for prediction/evaluation, depending on the scenarios. The last 2 frames of the stimuli are used as the intial state for the models.

3. Yes, we will release our implementation for DPI/GNS soon.

[LittleFlyFish]

Thanks for your interests! We will open source our implementation for DPI-Net and GNS soon.

1. Yes, you will need to download the hdf5 files from our website. We have a script that will take these hdf5 files and transform them into the data format used by DPI/GNS. The procedure includes transforming saved object meshes in hdf5 into particles.
   We used around 500-30000 particles, depending on the scenarios. The information includes particle positions and velocities. Our dataset doesn't include variations from object mass and stiffness, thus this is not included. We plan to have it in our future version.
2. Currently we use 2 time steps as input (time t for the particle positions, and time t-1 for calculating the particle velocities). We unroll the model for 100-300 timesteps for prediction/evaluation, depending on the scenarios. The last 2 frames of the stimuli are used as the intial state for the models.
3. Yes, we will release our implementation for DPI/GNS soon.

Thank you very much for the detailed explanation! This information is very helpful for the other researchers. We are looking forward to the release of the rest code and keep going on improving the performance! : )

[htung0101]

Hi, the repo is now public: https://github.com/htung0101/Physion-particles
Let me know if you run into any issue