DCP Profiles (Standard, Applications) specific checks / error levels

[remia]

ClairMeta is used to check different DCP (Interop, SMPTE) in various context (Mastering, Distribution, Preservation, ...).

The requirements are not the same in all case and we need to adjust the checks performed, eg. an Interop DCP from early 2010 ingested for preservation would not be expected to strictly follow RDD 52 recommendation. At the same time, it can be useful to have access to these new checks to detect potential issues on existing DCPs.

We already provide the check profile functionality that allow per check error level specification, one idea from @matmat would be to allow per standard error levels. To go further we could imagine a concept of DCP profiles (IOP, SMPTE, SMPTE B2.1, ...) and potentially user defined ones with each their own overload default error levels. The default check profile would probably be the more strict version (eg. SMPTE B2.1 RDD52) and others could be progressively more lenient.

ClairMeta could select the profile to use automatically or from user override.

This was discussed recently for checks related to picture resolution #184 subtitles font #178.

What are your thoughts on this ?

[jamiegau]

Just some thoughts on this issue.
Comment on how its applied.
For example, if you wanted to check the DCP against different profiles. Does that mean you have to run it multiple times OR, do we take a snapshot of ALL checks, than apply a profile to it giving me the result for that specific profile.

I think of it more in terms of, the QC solution I have. I would prefer to store a full library or checks, then allow the user to see results against all or a selected profile the DCP is targeting. negating the need to run multi check passes.

Just an idea as implementation wise this could be major rewrite.

[matmat]

A related question is how granular the checks should be? Right now related checks are sometimes bundled togehter in the same function.

[remia]

Yes agreed @jamiegau it would be nice to check how the DCP fares with the different profiles in one go. That is something to keep in mind and should be relatively straightforward with the current implementation.

We need to find a good balance in check granularity, I agree that some checks could be more atomic. The more granular, the more control you have on the error report levels, but that can come at the price of code duplication / less performant code. Ideally the design goal would be to be as granular as possible...

Sometime I wish be had a way of defining the specs / checks in a higher level language, like XSD schema, and generate code automatically from that...

[matmat]

Maybe not totaly related, but it would be interesting to also explore property based testing with e.g. Hypothesis.

[remia]

We now have a way to check a DCP against multiple profiles without having to go through the whole test suite every time to save processing time. The only thing now would be to produce those different profiles.

[jamiegau]

Hi Remi,
good to see work on this moving forward.
A comment. How do the profiles come into existence. Who makes them, what do they check etc.
Interesting issue. One item I would like to point out here is that we need an easy way to help those wanting to make a profile to create one. For example, a new README in the repo that goes over this and specifically names all the possible checks.
For example a template file with them all and a profile creator can use as template and enable/disable items as required (comment out). Based on a target template profile.json file.

I would like to puth this forward to ISDCF proponents and ask them to potentially create the profiles for addition to the repo.. And see if we get any input.

For example, maybe a profile from deluxe (or multiple) based on purpose.
Or from ISDCF that is a minimum level to be expected to be considered SMPTE B2.1 etc ready, based on internal ISDCF SMPTE DCP supported feature requirements.

Give the propellerheads in the group something to do and helpful in contribution to the project.