You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I suspect Clojure's batching of lazy seqs into 32 items might cause a space leak here if a test-case returns a large lazy sequence of test case errors then it may not be released. Out of curiousity I tried hacking this by putting a (take 1000) over the results inside the with-open which eagerly loads data into a vector, however it seemed the leak was still occuring, so I suspect there may be other issues here.
It may be worth rewriting this code to be eager using transducers, or to put a configurable limit over the amount of test failures to report on for any given test case.
The text was updated successfully, but these errors were encountered:
e.g. running
$ clojure -M:pmd-qb:validate -e https://staging.gss-data.org.uk/sparql
In the pmd-rdf-validations repo.
I think one reason this may occur is because of this:
rdf-validator/src/rdf_validator/core.clj
Lines 92 to 93 in 5ecc5fa
I suspect Clojure's batching of lazy seqs into 32 items might cause a space leak here if a test-case returns a large lazy sequence of test case errors then it may not be released. Out of curiousity I tried hacking this by putting a
(take 1000)
over the results inside thewith-open
which eagerly loads data into a vector, however it seemed the leak was still occuring, so I suspect there may be other issues here.It may be worth rewriting this code to be eager using transducers, or to put a configurable limit over the amount of test failures to report on for any given test case.
The text was updated successfully, but these errors were encountered: