Why do different coloured objects show up better/worse in the depth map?
Different colours (even on the same material) will affect how well an object is detected because of reflectivity.
(If you're unclear on how LiDAR and Time-of-Flight devices work, we suggest reading our short explainer on the subject)
Different coloured objects will absorb and reflect the IR light from the Tau differently, resulting in better or worse depth detection. Dark coloured objects and low reflectivity objects (grass, leather) will be harder to detect.
(There's no escaping from physics )
For better detection of these objects, try increasing the 3D integration time as much as possible.
The trade-off is that reflective or very close objects will be saturated.