-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement deserialize #138
base: pz-serialize
Are you sure you want to change the base?
Conversation
809932a
to
ed940df
Compare
require 'test_helper' | ||
|
||
class TemplateTest < MiniTest::Test | ||
def test_serialize |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure how to (or need to) further improve test coverage.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One way to stress test it would be if we could run the tests with a monkey patch that automatically serializes and deserializes on Liquid::Template.parse. Even if we didn't always run CI with that, it would help us find missing tests by adding a test for any failure that it turns up.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think if we add this to the benchmarks, and run those as part of CI it would add coverage for free.
We also need to make sure Liquid::Template.load(serialize).render!
is, and will remain faster than Liquid::Template.parse(source).render!
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I implemented benchmarks in Shopify/liquid#1385. Deserialize is about 20% faster than parsing from source.
Great work Peter. |
Thank you Tobi! ♥ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wow! That was a big piece. Nice work 🙇
The only potential issue I see is w/ the assert(RTEST(tag_class));
ext/liquid_c/block.c
Outdated
VALUE markup = rb_utf8_str_new(tag_markup_header_markup(current_tag), current_tag->markup_len); | ||
|
||
VALUE tag_class = rb_funcall(tag_registry, intern_square_brackets, 1, tag_name); | ||
assert(RTEST(tag_class)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this should raise? Seems technically possible that tag could get removed between the time the template is parsed, cached (for a long time) & deserialized.
Maybe using Block.unknown_tag
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 Good point. On a unrelated note, should block.c
be renamed to block_body.c
since it actually contains the code for Liquid::C::BlockBody
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, block_body.c would be a more appropriate name for this file.
986bff5
to
bafe96f
Compare
0b504b7
to
07ffbfb
Compare
require 'test_helper' | ||
|
||
class TemplateTest < MiniTest::Test | ||
def test_serialize |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One way to stress test it would be if we could run the tests with a monkey patch that automatically serializes and deserializes on Liquid::Template.parse. Even if we didn't always run CI with that, it would help us find missing tests by adding a test for any failure that it turns up.
dump_load_eval('{% if test %}goodbye {% else %}hello {% endif %}world', 'test' => false)) | ||
assert_equal('hello world', dump_load_eval('{% if true %}hello {% endif %}{% if true %}world{% endif %}')) | ||
assert_equal('123', dump_load_eval('{% for i in (1..10) %}{{i}}{% if i == 3 %}{% break %}{% endif %}{% endfor %}')) | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could use a unit test that ensures that line numbers are being set properly. I think Liquid::ParseContext#line_number is being inherited by Liquid::C::SerializeParseContext without it ever being set on deserialization.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is e3f403c the kind of test you were thinking of?
const char *data = RSTRING_PTR(source); | ||
|
||
document_body_header_t *header = (document_body_header_t *)data; | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should at least have a serialization format version number in the header to check against in order be able to make backwards incompatible changes. We will need to actually raise an exception for that check to safely handle the incompatible version error by loading and parsing the template source.
Another check we could use would be an endianness check. Little-endian seems fairly standard these days, so if we don't want to extend the header and add a runtime check, we could have a compile-time check for it in extconf.rb. We can do the same if we add any more platform specific dependencies.
We should also document that this method needs to be passed data serialized by this library to be used safely, since we probably don't want to add significant overhead to provide runtime safety.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in 005b088. I added a Liquid::C::DeserializationError
class that is raised for all serialization incompatibility reasons.
bafe96f
to
0e8d1c2
Compare
07ffbfb
to
a406547
Compare
Co-Authored-By: Dylan Thacker-Smith <[email protected]>
Co-Authored-By: Dylan Thacker-Smith <[email protected]>
…ext_tag Co-Authored-By: Dylan Thacker-Smith <[email protected]>
Co-Authored-By: Dylan Thacker-Smith <[email protected]>
Co-Authored-By: Dylan Thacker-Smith <[email protected]>
…ion has completed
0e8d1c2
to
f26621c
Compare
e3f403c
to
5b1f9d8
Compare
@dylanahsmith I added a rake task that runs the liquid integration tests for serialization in a65f10e and it's now running as part of CI. The commits after that commit are fixes I made from bugs discovered when running the integration tests. |
Based on #137.
This PR implements
Liquid::Template.load
which reads and deserializes a parse liquid template.I added a
SerializeParseContext
class that is a subclass ofParseContext
that is used to store the information required for deserialization. The actual deserialization work is done in the functionblock_body_parse_from_serialize
that is called duringLiquid::BlockBody.parse
when aSerializeParseContext
is passed in.