Next phase of the fuzzer, adds some mechanics for the fuzzer itself to
add modifiers on the options.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
fix failing fuzz case involving binary string with a '\x' in it.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
add mechanics for the fuzzer to add custom options.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This is the next phase of the fuzzer. It runs a round trip and makes
sure that the config files generated by the app will load into the same
results, to test full round trip on the config files.
Issues fixed
- fix a bug in the string escape code caught by initial round trip tests
- resolve inconsistencies in handling of {} for empty vector indication
between config and cli parsing
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This adds a round trip test for config file generation to the fuzzer.
(the next step after this PR will be a fuzzer that verifies that the
round trip actually matches the results.
This change ended up requiring quite a few minor changes to fix the
ambiguities between the config file generation and config file reader.
1). There was a number of potential conflicts between positional names
and regular option names that could be triggered in config files, this
required a number of additional checks on the positional naming to
ensure no conflicts.
2). flag options with disable flag override can produce output results
that are not valid by themselves, resolving this required flag input to
be able to handle an array and output the original value set of results.
3). strings with non-printable characters could cause all sorts of chaos
in the config files. This was resolved by generating a binary string
conversion format and handling multiline comments and characters, and
handling escaped characters. Note; I think a better solution is to move
to fully supporting string formatting and escaping along with the binary
strings from TOML now that TOML 1.0 is finalized. That will not be this
PR though, maybe the next one.
4). Lot of ambiguities and edge cases in the string splitter, this was
reworked
5). handling of comments was not done well, especially comment characters in the
name of the option which is allowed.
6). non printable characters in the option naming. This would be weird
in practice but it also cause some big holes in the config file
generation, so the restricted character set for option naming was
expanded. (don't allow spaces or control characters).
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add the beginnings of a fuzzing system for CLI11. This commit adds the fuzzing code, a simple test, and two fixes to issues(seg faults) found by the initial round of fuzzing. It also adds a few tests and coverage issues uncovered in the process of developing the fuzz tests. As a side effect adjusts some of the azure tests to specify the vmImage which was being changed on azure.
* update license to match rest of code base
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
