Animal Collisions aren't detected

[kafe-it]

Hi Simon,

thanks for your insightful work! This will greatly contribute to future research.

I've integrated f2d as an additional benchmark to our framework and noticed some weird behavior regarding the animals, where collisions occasionally aren't counted.

I attach some images from GT boxes directly out of the simulator and model (tf++) predictions, which are visualized in Rerun.

It seems like the animal GT boxes are regular pedestrian boxes (at least for some, the elephant box is larger actually). Visually, the animal is hit in the front camera however, no collision is detected and the vehicle & animal boxes never actually intersect, resulting in 100 DS:

(light blue = model predictions, red = GT box, dark blue = car box)

Same scenario but different seed, the collision is detected (25 DS). Obviously the boxes do intersect here:

This only seems to happen occasionally, but coincides with the intersection of GT animal boxes and the vehicle box.

Now to the questions:

• Have you noticed this behavior as well?
• Are the collisions calculated using these GT boxes / Are the simulator internal meshes different from these boxes?
• How should we handle this in terms of benchmark contributions?

PS: Sorry for the bad front image quality (had to save some storage 😆)

Thanks in advance, Karol

[SimonGer]

Hey Karol,

thank you very much for your interest and the detailed issue!

I can tell you upfront that the bounding boxes returned by the simulator are incorrect for the newly added animals. The bounding boxes were determined by a capsule object in UE4, which I was unable to resize properly to reflect the animal bounding boxes (because it is a square from above). I might be able to revisit this in the future, if you have any insights about this I would greatly appreciate it.

However, the collisions with the animals should be determined by an additional bounding box (which might be inaccessible in the packaged version), and the actual mesh object. Are you seeing cases where the vehicle is very close to the object without the death animation being triggered? In the images you provided I cant clearly see a collision being present. In the camera image seems like there is still some room to the animal. It would be great if you could provide some spectator images of this case.

Another thing to consider is that there is a behavior in CARLA where collisions don't reduce the score if the ego vehicle is stationary. You can see this here this is a design decision of the original CARLA scenario runner. This might also explain some of the behaviors you are seeing?

I hope this helps, let me know what you find out!
Simon