Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

POST submission attachments: reduce audit logging overhead #1334

Merged
merged 3 commits into from
Jan 15, 2025

Conversation

alxndrsn
Copy link
Contributor

@alxndrsn alxndrsn commented Dec 8, 2024

In the submission attachment POST endpoint and a few other submission-related endpoints, a full submission definition including XML is fetched from the database, where only one or two columns from that result are used.

This commit reduces the data fetched from the database to only those which are required.

What has been done to verify that this works as intended?

Tests.

Why is this the best possible solution? Were any other approaches considered?

There may be more idiomatic ways of defining query results?

How does this change affect users? Describe intentional changes to behavior and behavior that could have accidentally been affected by code changes. In other words, what are the regression risks?

It should not affect users.

Does this change require updates to the API documentation? If so, please update docs/api.yaml as part of this PR.

No.

Before submitting this PR, please make sure you have:

  • run make test and confirmed all checks still pass OR confirm CircleCI build passes
  • verified that any code from external sources are properly credited in comments or that everything is internally sourced

alxndrsn added 2 commits December 8, 2024 08:21
In the submission attachment POST endpoint and a few other submission-related endpoints, a full submission definition including XML is fetched from the database, where only one or two columns from that result are used.

This commit reduces the data fetched from the database to only those which are required.
@ktuite ktuite self-requested a review January 9, 2025 16:50
Copy link
Member

@ktuite ktuite left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope this improves submission attachments a little bit! Not sure how to measure it, but not fetching the whole XML of the submission def and only picking specific columns seems like it will be useful if the submission XML is large and there are a lot of attachments. This improvement is also part of submission editing and checking against the previous deprecated def instance ID.

Just wanted to note for whoever sees this in the future that one thing that also makes the audit logging slow here is the query to get the old attachment blob ID so it can be referenced in the audit log. That query is still present, but at least it is leaner now.

There may be more idiomatic ways of defining query results?

I think this is fine; there are some DB utils for joining and extending frames but not really for picking specific fields/cols of a Frame. The new helper functions make sense to me!

@alxndrsn alxndrsn merged commit 9e177f0 into getodk:master Jan 15, 2025
6 checks passed
@alxndrsn alxndrsn deleted the remove-comment branch January 15, 2025 05:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants