Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix image downloader failing with automl format on deserialize due to missing label transformations #2435

Conversation

imatiach-msft
Copy link
Contributor

Description

fix image downloader failing with automl format on deserialize due to missing label transformations

Seeing errors when deserializing RAIVisionInsights with automl format labels:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/azureml-envs/responsibleai-vision/lib/python3.8/site-packages/raiwidgets/responsibleai_dashboard.py", line 36, in __init__
    self.input = ResponsibleAIDashboardInput(
  File "/azureml-envs/responsibleai-vision/lib/python3.8/site-packages/raiwidgets/responsibleai_dashboard_input.py", line 41, in __init__
    self.dashboard_input = analysis.get_data()
  File "/azureml-envs/responsibleai-vision/lib/python3.8/site-packages/responsibleai_vision/rai_vision_insights/rai_vision_insights.py", line 558, in get_data
    dataset = self._get_dataset()
  File "/azureml-envs/responsibleai-vision/lib/python3.8/site-packages/responsibleai_vision/rai_vision_insights/rai_vision_insights.py", line 696, in _get_dataset
    dashboard_dataset.true_y = self._format_od_labels(
  File "/azureml-envs/responsibleai-vision/lib/python3.8/site-packages/responsibleai_vision/rai_vision_insights/rai_vision_insights.py", line 794, in _format_od_labels
    object_index = int(detection[0] - 1)
KeyError: 0

After extensive debugging, we discovered this was in the deserializer for true_y labels when image downloader is specified and the test dataset is reset, but the labels are not converted, which we could confirm based on the printed labels when adding debug:

...
detection:  [3.0, 296.3221740722656, 169.22532653808594, 383.11572265625, 305.3504333496094, 0.6043946146965027]
detection:  [2.0, 164.48257446289062, 220.76792907714844, 471.8297119140625, 508.36151123046875, 0.8620063662528992]
detection:  {'label': 'carton', 'topX': 0.2004008016032064, 'topY': 0.25975975975975973, 'bottomX': 0.46693386773547096, 'bottomY': 0.7822822822822822, 'isCrowd': 0}

The first two detections are in the converted common format for predicted labels. The third detection is for the true_y labels and is in the automl format and was not transformed. Transforming the labels on deserialize fixed the error.
Also new tests for object detection serialize + deserialize were added as part of this PR, which only previously existed for other tasks like image classification.

Checklist

  • I have added screenshots above for all UI changes.
  • I have added e2e tests for all UI changes.
  • Documentation was updated if it was needed.

@imatiach-msft imatiach-msft force-pushed the ilmat/fix-image-downloader-with-automl-format-deserialize branch 2 times, most recently from 23eb4ec to 08ab314 Compare December 1, 2023 18:01
5 similar comments
@imatiach-msft imatiach-msft force-pushed the ilmat/fix-image-downloader-with-automl-format-deserialize branch 2 times, most recently from 0fcf1c3 to 8b124b5 Compare December 1, 2023 19:29
1 similar comment
@imatiach-msft imatiach-msft force-pushed the ilmat/fix-image-downloader-with-automl-format-deserialize branch from 8b124b5 to c854a35 Compare December 1, 2023 20:02
@imatiach-msft imatiach-msft force-pushed the ilmat/fix-image-downloader-with-automl-format-deserialize branch from c854a35 to ba48d9a Compare December 4, 2023 15:37
@imatiach-msft imatiach-msft force-pushed the ilmat/fix-image-downloader-with-automl-format-deserialize branch from ba48d9a to 7af64fe Compare December 4, 2023 16:45
1 similar comment
@imatiach-msft imatiach-msft force-pushed the ilmat/fix-image-downloader-with-automl-format-deserialize branch from 513c6ab to 117f658 Compare December 4, 2023 20:22
@imatiach-msft imatiach-msft merged commit 85d6373 into main Dec 4, 2023
101 of 102 checks passed
@imatiach-msft imatiach-msft deleted the ilmat/fix-image-downloader-with-automl-format-deserialize branch December 4, 2023 22:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants