ippfuzz: Fuzz test generator for ipptool

[michaelrsweet]

As part of our discussions during the February 2018 IPP F2F meeting, we discussed ways to introduce fuzz-testing into ipptool.

Two approaches were considered:

1. Adding support to ipptool to randomly generate requests, attributes, values, etc.
2. Adding a program to generate random ipptool test files that can be fed into ipptool.

Since the first approach would still require ipptool to be able to write out a failed test for later diagnosis (what the second approach will do as its primary function), it seems like creating a test file generator is the simpler approach.

The generator should provide options to limit the amount of randomness, for example "always use Get-Printer-Attributes", "always include the following attributes and values", etc. so that "mostly valid" requests can be generated to test specific areas.

The generator should also provide options for the expected responses, for example "expect the following status codes" or "expect the random attributes to be in the unsupported attributes group of the response."

Ideally ippfuzz should be able to automate running ipptool and iterate N times, storing the results in a directory and providing a summary of the test results on stdout.

[michaelrsweet]

Also, when automating the tool should stop after a certain number of repeated failures (default 3?) - at that point the Printer has probably crashed...

[michaelrsweet]

When writing the results to a test directory, rename tests and results, e.g. "pass-nnnnnnnn.ext" for passed tests and "fail-nnnnnnnn.ext" for failed tests? Be sure to include the .test file and the XML results plist file.

[michaelrsweet]

Maybe focus on supporting fuzzing templates which look like ipptool test files? Expand values like:

• "$[ascii(100)]": Random string of exactly 100 characters
• "$[ascii(0,32767)]": Random string of 0 to 32767 characters
• "$[utf8(1000)]": Random Unicode string of 1000 characters
• "$[bytes(10,5000)]": 10 to 5000 random bytes
• "$[integer(10,20)]": Random integer from 10 to 20
• "$[boolean]": Random boolean value
• "$[value1|value2|value3]": randomly choose between the three values
• "$[repeat(10000) ... ]": expand the "..." text 10000 times.

"$[...]" might not be the best choice, consider other markup options.

Need to see what other fuzzing tools do.

[michaelrsweet]

Resources:

• The Fuzzing Project
• American fuzzy lop
• Scapy

[michaelrsweet]

Moved to the libcups project.