Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JSON] Raise exceptions when "sibling" keywords are unhandled #1063

Merged
merged 8 commits into from
Oct 29, 2024

Conversation

hudson-ai
Copy link
Collaborator

WIP BECAUSE THIS DEPENDS ON STILL-OPEN PR #1061

When an applicator has sibling keywords, JSON schema mandates that both sets of keywords are respected, e.g.

{
   "$defs": {"foo": {"type": "integer"}},
   "$ref": "#/$defs/foo",
   "minimum": 5
}

In this schema, "minimum" is a "sibling" to the "$ref", and we need to enforce BOTH the schema in the definition AND the "minimum" keyword.

There are some trivial examples of this general problem, and we can provide implementations for them (in this PR, I also include an implementation for combinations of the "type", "const", and "enum" keywords, which addresses the enum part of #1038). But when no implementation is provided, we need to raise an exception to prevent "type 1 errors" (the LLM is happy to generate an instance that might not validate against the full schema).

There are a number of tests in the TestRefs class that have to be xfailed due to this stricter behavior. In time, we can add more implementations that handle siblings, reducing the number of xfails.

@codecov-commenter
Copy link

codecov-commenter commented Oct 24, 2024

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 86.84211% with 5 lines in your changes missing coverage. Please review.

Project coverage is 64.16%. Comparing base (da80081) to head (8487e8b).

Files with missing lines Patch % Lines
guidance/library/_json.py 86.84% 5 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1063      +/-   ##
==========================================
- Coverage   65.40%   64.16%   -1.24%     
==========================================
  Files          63       63              
  Lines        4810     4842      +32     
==========================================
- Hits         3146     3107      -39     
- Misses       1664     1735      +71     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@riedgar-ms riedgar-ms left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will wait on other merge

@hudson-ai
Copy link
Collaborator Author

@riedgar-ms ready :)

@hudson-ai hudson-ai changed the title [WIP] [JSON] Raise exceptions when "sibling" keywords are unhandled [JSON] Raise exceptions when "sibling" keywords are unhandled Oct 29, 2024
@hudson-ai hudson-ai merged commit dc5a080 into guidance-ai:main Oct 29, 2024
100 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants