Creating 'clang-mutate' for clang-tools-extra

Clang Frontend
#1

I’d like a tool that applies plausible edits to code so I can check if tests pick up the change or not.

I think I can do this with a clang-tidy-like framework with a list of mutations (rather than checks).

The tool would produce a large list of single-change ‘fixes’ for clang-apply-fixes to make use of and to be set up as a series of mutated branches for a CI system. Applying mutations to only new code should keep the set manageable.

An example mutation would invert < in conditionals:

if(date > expiry ) { return 0.0; }

is mutated to

if(date < expiry ) { return 0.0; }

If our tests don’t spot this, we have a gap.

Is anyone working on, or interested in collaborating on a similar tool?

regards,

Jon

#2

Haven’t heard of anyone working on it (& don’t have time myself) - but I’ve been looking forward to/wanting to see such a tool in LLVM/Clang for a while now. (if you could have it running on a buildbot or similar that tracked LLVM changes so we could vet code change test coverage… that’d be so awesome)

I’d always pictured such a tool as more like a sanitizer/run at the LLVM IR level rather than as source transformations - but I’ve not thought about it too much & maybe it makes more sense as a real source transformation (easier to communicate to the user “this change is not caught by test coverage”).

(idle thought: Presumably it’d be easier to just add a ! at the top of the boolean expression, so transforming “if (date > expiry)” into “if (!(date > expiry))”)

#3

That is interesting idea. The other mistake that would be easy to change is finding +1 errors like

for (int i = 0 ; i < n ; i++ )
=>
for (int i = 0 ; i <= n ; i++)

or changing ++it to it++ in the cases where the result goes to other expr.

Piotr

#4

I’d expect the problem with running it on the IR level to be to get the information what exactly changed back to the user.