yarpdataplayer - variable sample rate data playing too fast

[kt10aan]

I have 2 sets of data recorded by yarpdatadumper (so, from two YARP ports).
One of them is from /icub/head/state:o. The other one comes from an Oculus Rift and contains just x,y,z orientation data. Both datasets were recorded using both Tx and Rx timestamps.

Both data sets are supposed to be ~30 seconds long.

I need to replay these datasets through yarpdataplayer. Upon playing them with yarpdataplayer, the /icub/head/state:o data have a steady 10ms sample rate and they indeed last for 30 seconds.

However, the oculus data set has a variable sample rate and they last for much less than 30 seconds (20-23 seconds).

My questions are:

• Why is this happening? (checking the timestamps of the oculus data, they are indeed 30 seconds long). Is it a limitation of yarpdataplayer when dealing with variable rate data or the error is on my side?
• is there a way to fix it?

The dataset (115KB in size, compressed) can be found here

Edit: I forgot to mention that I am using the Qt version of yarpdataplayer.

[vtikha]

@kt10aan thanks for the post and the datasets, will have a look at it asap and get back to you.

[vtikha]

@kt10aan I have made modifications to the yarpdatasetplayer that now follows the exact length and synchronisation stored within the acquisition logs. I also added a time taken column that accounts for the elapsed time. I tried it on your dataset and all seems fine.

[kt10aan]

thanks for the quick fix @vtikha. I ran the updated yarpdataplayer and I still have some issues:

• Starting the yarpdataplayer i get in the console:

The part icub_state, should last for: 30.005729 with 2978 frames                                                                                                                                                     
The part oculus_output, should last for: 30.002716 with 3595 frames 

which is correct, as I mentioned that the two datasets are 30secs long.

However, when running the dataset, both of them last for approximately 28seconds (according to the Time Taken column).
Is it an issue of wrongly calculating the Time Taken or is there still a bug in the actual replay of the dataset?

• When the two datasets start playing, the /icub/head/state one starts a few seconds after the /oculus/orientation one. As a result, the /oculus/orientation dataset finishes a few seconds ahead of the /icub/head/state one. Both datasets have the same initial timestamp and final timestamp so theoretically they should start and finish together.

You can find a screen recording of it here
Thanks!

[pattacini]

@kt10aan just a quick comment.

Your dataset contains both tx and rx time stamps in the two streams, where the former stamps are lagged of ~3.84 s each other, whereas the latter are in sync. I think tx stamps are selected by default, but you can choose them by means of --withExtraTimeCol command line option. Hence, can you do the test again with --withExtraTimeCol 2?

[pattacini]

More on this.

I've performed the following test:

• Replayed the dataset with --withExtraTimeCol 2
• Attached a yarpdatadumper to /icub/head/state:o

The outcome is that on the dumped data the timing is retained, that is the acquisition lasted 30 s, while on the GUI the time taken column actually showed a different overall lapse.

@kt10aan can you confirm this?

[vtikha]

@kt10aan they start a few seconds after but this is how your dataset had been recorded. @pattacini is right, you have to select the correct timestamp. Thanks @pattacini for the quick yarpdatadumper test: after chatting with him, the replay is actually correct, time wise and data wise, but I must have done something not quite right with the time taken display, it is probably a mis-synchronisation with the updating of the display and the data. Anyway, it was just a temporary debugging column, and will probably remove it as yarpdataplayer already displays percentages.

[kt10aan]

I replayed the datasets with --withExtraTimeCol 2 and attached a yarpdatadumper to /icub/head/state:o.
Indeed, the datasets are now being replayed in a synchronous manner, and the timing on the dumped data is retained (30 s).
The Time Taken column shows a lapse of 28 s (but it is inconsequential for my purposes anyway).

Also, I assume that the Tx column is not selected by default. In that case, the documentation is wrong, as it is now, as it states that --withExtraTimeCol 2 the Rx field is selected. So I assume that either the behaviour has to be inverted or the documentation updated.

Anyway, my problem is solved!
Thanks @pattacini, @vtikha for your help!

[pattacini]

@kt10aan 👍

The data are always played in sync. Simply, according to tx time stamps, one of the two acquisition started afterwards, or most likely, the two clock sources were not in sync.