Improving camera performance

JohnW

Sorry in advance for all the questions.

Is there a way to improve the camera performance in terms of frame rate, synchronicity, and noise?

Some observations:

The current API looks like a loop. The loop goes like this:

do
    fetch frame
    process frame
    display frame
    check for escape character
done

Depending on the host processor, and the ability of the OpenCV stack to display the frame, the frame rate can be lower or higher.

The interface used is a virtual serial port (and it seems that is running at 4Mbits/s). So, a frame is only captured after it is requested.

Is it possible or planned to use USB Video Class (UVC) to enable a constant stream of frames from the camera?

I could be wrong, but I think the sensor used here is the Espros EPC635:
https://www.espros.com/photonics/time-of-flight-chip-epc635/

If so, the maximum frame rate available is around between 128 and 512 fps depending on the number of DCS frames collected. This might be limited in the Onion Tau due to the MCU used to capture frame data.

Finally, is a noise filter implemented within the Tau camera?

Lazar Demin

@JohnW Your analysis of the API is correct.
For the 4 operations, it's the process frame stage that takes the most time and acts as the bottle-neck for framerate. (As an aside, it's also the operations that is most dependent on the capability of the host processor)

To speed up the frame rate you could instead have two programs or threads. One to fetch the frames from the camera. And the other to process the frame and optionally display with OpenCV.

This is briefly discussed in the API documentation.
And there's an example (non-threaded) implementation of this in the Record and Playback frames example programs.

@JohnW said in Improving camera performance:

I could be wrong, but I think the sensor used here is the Espros EPC635:

That's right, the Tau Camera uses the Espros EPC635

@JohnW said in Improving camera performance:

Finally, is a noise filter implemented within the Tau camera?

I'll check with the team and get back to you

JohnW

@Lazar-Demin Thanks or your reply.

@Lazar-Demin said in Improving camera performance:

@JohnW Your analysis of the API is correct.
For the 4 operations, it's the process frame stage that takes the most time and acts as the bottle-neck for framerate. (As an aside, it's also the operations that is most dependent on the capability of the host processor)

I'll need to do a little profiling here, but what is the theoretical maximum frame rate for the Tau camera? Is the virtual serial port not a bottleneck?

@JohnW said in Improving camera performance:

I could be wrong, but I think the sensor used here is the Espros EPC635:

That's right, the Tau Camera uses the Espros EPC635

Great. Good to know.

@JohnW said in Improving camera performance:

Finally, is a noise filter implemented within the Tau camera?

I'll check with the team and get back to you

This would be good to know. It would certainly be possible to implement this in software on the host processor, but if it being done on the Tau, all the better.

Lazar Demin

@JohnW said in Improving camera performance:

what is the theoretical maximum frame rate for the Tau camera? Is the virtual serial port not a bottleneck?

The theoretical maximum frame rate is 50fps. And the USB serial port shouldn't be a bottleneck because the bandwidth required is low.
Having said that, the realistic frame rate that can be achieved depends on lots of factors, such as the sensor configuration (integration time, operation mode etc.) and available resources in the host machine (CPU, cache, RAM etc.). The practical frame rate is usually between 20 and 30 fps.

On the noise filter:
There is no filter running on the Tau Camera hardware or in the driver.
If this is a requirement, you'll need to add it in your software.