Privately Owned Vehicle Work Group Meeting - 2025/01/13 - Slot 1

[m-zain-khawaja]

Agenda

• Update on SuperDepth Training
• Update on PathDet dataset curation
• Update on LaneDet dataset curation

Discussion

SuperDepth Network Training Update

@m-zain-khawaja :

A first version of SuperDepth was trained using the UrbanSyn and MUAD synthetic datasets as an initial test.

The network achieved impressive results on validation data from UrbanSyn and MUAD which it had not been shown during training, achieving a validation error on mAE of 0.031 overall.

This network was then tested on the KITTI dataset, which the network had not seen before and as expected, due to simulation-to-real domain gap, the validation results were not as robust, with an overall mAE of 0.121. Certain artefacts were also visible in the KITTI estimates caused by light/shadow effects on the road as these were not present in the UrbanSyn/MUAD training data.

In order to address this, a new scheme had to be developed to be able to account for real-world LIDAR based data whilst factoring in the noise characteristics of LIDAR data. Therefore, a 'validity mask' was calculated (1,0 binary mask) highlighting valid depth estimates projected onto the image plane vs non-valid depth estimates projected onto the image plane. The loss function was modified to only account for 'valid' pixels as identified in the validity mask.

Additionally, I was able to successfully parse the DDAD dataset by successfully building Docker on WSL2 (there was a conflict between IP addresses in WSL2 and Docker which had to be resolved - details here allowing me to utilize the Toyota Research Institue DGP Library to correctly project the LIDAR depth to the image plane. This yields a further 16,600 data samples from a combination of the front-facing and rear-facing vehicle cameras.

Phase 2 SuperDepth Training

I have made significant changes to the load_data_super_depth , super_depth_trainer , and augmentations classes to reflect the above strategy of utilizing both simulated data as well as real-world data using LIDAR projected and interpolated depth combined with a validity mask.

New Loss Function:

The new loss function now also includes a gradient matching loss to better preserve the prediction at boundary pixels, this is done by calculating the x and y gradients between the prediction and ground truth and performing an L1 Loss on the gradient difference. This loss was also utilized in DepthAnythingV2
 
I expect to begin Phase 2 training once the main training loop train_super_depth has been refactored accordingly as well.

PathDet Dataset Curation Update

Dataset curation tracking

• TuSimple - 6,394 training data samples
• CULane - 15,800 training data samples
• CurveLane - (parsing completed by @TranHuuNhatHuy, data to be uploaded soon)
• BDD100K - (parsing in progress by @sarun-hub)
• ROADWorks - (parsing in progress by @docjag)
• Comma2k19 - (parsing completed by @siddas27, data to be uploaded soon)

LaneDet Dataset Curation Update

• @devang-marvania has begun parsing the CULane dataset for the LaneDet neural network.

Dataset curation tracking

• TuSimple - 6,394 training data samples
• CULane - (parsing by @devang-marvania)
• CurveLane - to be started

Attendees

TBD

Zoom Meeting Video Recording

Video Meeting Link
Please contact the work group lead (@m-zain-khawaja) to request access to a recording of this meeting.