Creating 'clang-mutate' for clang-tools-extra

I’d like a tool that applies plausible edits to code so I can check if tests pick up the change or not.

I think I can do this with a clang-tidy-like framework with a list of mutations (rather than checks).

The tool would produce a large list of single-change ‘fixes’ for clang-apply-fixes to make use of and to be set up as a series of mutated branches for a CI system. Applying mutations to only new code should keep the set manageable.

An example mutation would invert < in conditionals:

if(date > expiry ) { return 0.0; }

is mutated to

if(date < expiry ) { return 0.0; }

If our tests don’t spot this, we have a gap.

Is anyone working on, or interested in collaborating on a similar tool?

regards,

Jon

Haven’t heard of anyone working on it (& don’t have time myself) - but I’ve been looking forward to/wanting to see such a tool in LLVM/Clang for a while now. (if you could have it running on a buildbot or similar that tracked LLVM changes so we could vet code change test coverage… that’d be so awesome)

I’d always pictured such a tool as more like a sanitizer/run at the LLVM IR level rather than as source transformations - but I’ve not thought about it too much & maybe it makes more sense as a real source transformation (easier to communicate to the user “this change is not caught by test coverage”).

(idle thought: Presumably it’d be easier to just add a ! at the top of the boolean expression, so transforming “if (date > expiry)” into “if (!(date > expiry))”)

That is interesting idea. The other mistake that would be easy to change is finding +1 errors like

for (int i = 0 ; i < n ; i++ )
=>
for (int i = 0 ; i <= n ; i++)

or changing ++it to it++ in the cases where the result goes to other expr.

Piotr

I’d expect the problem with running it on the IR level to be to get the information what exactly changed back to the user.