A combination of Euclidean Clustering and Shape Estimation fails to capture detected objects

[kminoda]

I am currently trying to use euclidean_cluster as the main object detector since my robot does not have a CUDA environment.

In my setup, shape_estimation node subscribes to the output of euclidean_cluster (/perception/object_recognition/detection/clustering/clusters) and estimate shape for each detected object.
euclidean_cluster seems to work fine, as far as looking at its output on RViz. (See the below figure. The colorful dots represent /perception/object_recognition/detection/clustering/debug/clusters)

However, the x&y dimensions calculated by the shape_estimation node are always 0.0 for all the detected objects, and this results in empty objects messages in the /perception/object_recognition/objects topic.

Interestingly, I also noticed that theshape_estimation node works fine by setting the label of all objects detected by euclidean_cluster from UNKNOWN to the other one, say CAR. Thus, currently, this is my temporary (and a bit forcible) solution to the above-mentioned issue, but I would like to know if there is any other solution to this.

Did anyone succeed in performing object detection that solely relies on euclidean_cluster in Autoware? If you did, did you encounter the same issue? And would you tell me how you solve it?

P.S.
I realized that, in shape_estimation, the shape of UNKNOWN type objects is estimated using a convex hull model, and it seems that the dimension of x&'y is always set to 0 (in this line).
My another question is: why are these values set to 0?

[yukkysaito]

@kminoda Can you show the perception.launch?

[yukkysaito]

@shmpwk Can you remove the merger node and publish from clustering directly as detection/objects.
Then, pls turn on the Detection visualization in Rviz.

[yukkysaito]

Yes. UNKNOWN objects appeared in Rviz as /perception/object_recognition/detection/objects.
But the object in /perception/object_recognition/tracking/objects is empty.

I see.
Pls put 1 at unknown-unknown matrix.
https://github.com/autowarefoundation/autoware.universe/blob/main/launch/tier4_perception_launch/config/object_recognition/tracking/multi_object_tracker/data_association_matrix.param.yaml#L5

[shmpwk]

Solved thank you!

Can you remove the merger node and publish from clustering directly as detection/objects.Then, pls turn on the Detection visualization in Rviz.

First, I interpreted objects removing merger node as getting input objects of merger node.
The detected/objects with skiped merger (=/perception/object_recognition/detection/clustering/clusters_without_feature for us) is as follows.

It seems UNKNOWN objects are detected.
But failed to visualize tracked objects. (They are not visualize but empty object topic is published.)

The key is

Pls put 1 at unknown-unknown matrix.

I can get tracked objects and also predicted objects.
The predicted objects is as follow

Before:

After

:

[shmpwk]

@yukkysaito
As you mentioned https://github.com/orgs/autowarefoundation/discussions/225#discussioncomment-2651812,
without specifying the parameter can_assign_matrix as 1, the target objects cannot be tracked.

But readme says unknown objects should be tracked.

Does UNKNOWN-UNKNOWN parameter can_assign_matrix is 0 match your expectation?
Or can I creat PR for it?

[yukkysaito]

@shmpwk I'm working about it, so pls wait a minutes
autowarefoundation/autoware.universe#870

[shmpwk]

As for the dimension x&y of convex hull model is 0, the convex hull model is Amorphous.
But it is not the cause of failure to capture detected objects.

Thus the title is our misunderstaning

A combination of Euclidean Clustering and Shape Estimation fails to capture detected objects

The correct title is "Tracking UNKNOWN object fails"

I'm debug now and at the moment I found that unknown object cannot be tracked well in multi_object_tracker.
I'm investigating it more now.

[shmpwk]

This problem is solved with #225 (comment).