Is there a fixed deviation between the LiDAR frame and camera frame?

[yunliuh]

Hello, upon reviewing your code, I’ve noticed that the time synchronization between the LiDAR and the camera is achieved by sharing the timestamp from the LiDAR scan header. This aligns the camera frame timestamps with each LiDAR scan timestamp. However, theoretically, the trigger moments for the LiDAR and the camera are not the same. Therefore, should there be a fixed deviation between them (since the camera is triggered by a 10Hz PWM signal, whereas the LiDAR is not triggered by a PPS signal)?

[xuankuzcr]

There is no fixed delay; the time deviation mainly consists of the variable exposure time and ISP processing time.
Let me explain the basic principle. All sensors are hard-synchronized using a 10 Hz trigger signal generated by synchronized timers on the STM32. Note that the Livox converter only supports a 1 Hz Pulse Per Second (PPS) trigger signal, and the Livox SDK automatically subdivides this into 10 Hz. Additionally, to segment the Livox LiDAR point cloud into scans at the rising edge of the square wave, we use the STM32 to generate a synthetic (false) GPRMC (containing an integer-second timestamp) and transmit it to the onboard PC via serial communication at 1 Hz. The Livox SDK then assigns the decoded timestamp to the LiDAR scan header, and the same timestamp is also applied to the corresponding camera image frame header, as the camera is triggered by the same rising edge. Since solid-state LiDAR lacks a scan synchronization signal, this approach ensures reliable hard synchronization, especially in indoor environments where GPS signals are unavailable.

[SanghyunPark01]

@yunliuh Maybe it's because the image timestamp is assigned after the camera triggering and after the exposure time.

[mkarklins]

Wanted to clarify.

In the livox_ros_driver2 I couldn't find code which would read the serial containing the fake GPRMC messages. This seems to be present only in livox_ros_driver (old).

Is it correct to assume that when using MID360 with livox_ros_driver2 the GPRMC messages should be sent from the STM32 directly to the MID360 via serial ( data/function cable ), so that the packets from the MID360 already contain the precise timestamps in the header, hence can be stored in /timeshare?