Cloning Functions

I seem to recall havbing asked this before but I can't find it by searching.

What's the right way to clone a Function? I need to save off the text at a
certain point and spit it out later, after other optimizations have run. But
I need to spit out the original text, not the optimized version.

What would be the most effective way to do this?

                                                  -Dave

You can use CloneFunction (in llvm/Transforms/Utils/Cloning.h) to
duplicate a function within a module.

That said, I'm not completely sure what you're trying to do, so I
don't know if that's what you're looking for.

-Eli

Ok, I've looked at the cloning stuff and it's on the right track. Here's
what I need to do:

I need to implement functionality similar to llvm-gcc's -emit-llvm in
our compiler. I need to emit the llvm before any optimizations on the
IR have been performed.

But here's the catch. Our compiler processes one function at a time,
so the LLVM IR is generated and fully optimized before the next function
appears. This also means the global symbol table changes as we
process functions.

So what I need to do is somehow clone the final module (so I capture all
of the needed global symbols) and then replace the functions in the cloned
module (which have been optimized) with the unoptimized LLVM IR.

So it seems I need to do something like this:

1. Call CloneFunction for each function as it comes across to LLVM. This
   captures the unoptimized LLVM IR.

2. Call CloneModule at the end of all processing to capture all globals.

3. ??!?

4. Profit!

The ??!? is something like, replace the cloned optimized Functions in the
cloned Module with the cloned unoptimized Functions that were saved
off in step 1. But how do I correctly remap the values? I would have a
ValueMap for each function from step 1 but I somehow have to map those
Values onto the Values in the cloned module. And map the arguments as
well (what CloneFunctionInto does).

Can anyone with more familiarity that I have about the cloning mechanism
provide guidance?

Thanks!

                                             -Dave

                                                   -Dave

I seem to recall havbing asked this before but I can't find it by
searching.

What's the right way to clone a Function? I need to save off the text at
a certain point and spit it out later, after other optimizations have
run. But I need to spit out the original text, not the optimized
version.

Is it possible to explain intended use of original unoptimized version ?

You can use CloneFunction (in llvm/Transforms/Utils/Cloning.h) to

duplicate a function within a module.

That said, I'm not completely sure what you're trying to do, so I
don't know if that's what you're looking for.

Ok, I've looked at the cloning stuff and it's on the right track. Here's
what I need to do:

I need to implement functionality similar to llvm-gcc's -emit-llvm in
our compiler. I need to emit the llvm before any optimizations on the
IR have been performed.

But here's the catch. Our compiler processes one function at a time,
so the LLVM IR is generated and fully optimized before the next function
appears.

You do not need to optimize LLVM IR before processing next function.
If your compiler works like

  for each function
    generate_ir
    optimize

then it seems you're doing

  for each function
    generate_ir
    convert_to_llvm_ir
    optimize_llvm_ir

If you're doing this then you may try

  for each function
    generate_ir
    convert_to_llvm_ir
  optimize_llvm_ir_module

Now, you have access to un-optimized functions before you do optimize_llvm_ir_module. Would that work ?

Is it possible to explain intended use of original unoptimized version ?

bugpoint. I want to run it on the IR produced by our frontend. This will
help us generate new LLVM tests we can send upstream. We've fixed
bugs that aren't caught by the upstream tests and it would be nice to capture
the problem and make the test available to everyone.

You do not need to optimize LLVM IR before processing next function.

You do if the functions you compile are mega-humongous.

If your compiler works like

  for each function
    generate_ir
    optimize

Yep.

then it seems you're doing

  for each function
    generate_ir
    convert_to_llvm_ir
    optimize_llvm_ir

Yep.

If you're doing this then you may try

  for each function
    generate_ir
    convert_to_llvm_ir
  optimize_llvm_ir_module

We in fact did used to do it this way. Then we ran out of memory.

And there are other issues to. We'd added things in our copy of LLVM
that looks to our middle-end for various bits of information. Some of that
information goes away after we get each function so delaying optimization
means we don't have all of the useful information we'd like.

And BTW, there are many things we had to fix in LLVM to allow us to do
this. It seems LLVM keeps around a lot of global state that needs to be
reset if you do function-at-a-time processing. I'm hoping to send these
fixes back upstream as soon as I deal with the red tape that's recently
been put in place here.

Now, you have access to un-optimized functions before you do
optimize_llvm_ir_module. Would that work ?

It will work for this particular code I'm looking at but in general it
won't. And I'm loathe to change how our compiler flow works just
for this. It's one more difference we'd have to deal with when tracking
down bugs.

                                                     -Dave

Ok, I've mostly got a mechanism to do what I want:

1. As each function comes in for op/codegen, clone it and save off
   the clone and its associated ValueMap (I call these clones "pristine"
   functions.

2. After all processing is done, clone the resulting Module (this has all of
   the global data and the optimized functions). Save the ValueMap from this
   operation.

3. Merge the ValueMaps from the Module clone and all the pristine function
   ValueMaps so that if V->V' in the Module map and V->V'' in the pristine
   function map, make V''->V' in the Module map.

4. Fix up arguments to the pristine functions _a_la_ what is done in
   CloneModule.

5. Delete the function bodies of all functions in the cloned Module (these are
   the optimized bodies) by calling deleteBody on them.

6. Call CloneFunctionInto to clone the pristine functions into the cloned
   Module.

7. Fix up linkages (set function linkages in the Module to what they are in
   the pristine function clones (they were changed to External by deleteBody).

8. Delete all the pristine Function clones, they are now cloned into the
   cloned Module.

This should work as I understand things, but I have a question about
Function::deleteBody. It calls dropAllReferences which has the following
scary comment:

  /// dropAllReferences() - This method causes all the subinstructions to "let
  /// go" of all references that they are maintaining. This allows one to
  /// 'delete' a whole module at a time, even though there may be circular
  /// references... first all references are dropped, and all use counts go to
  /// zero. Then everything is deleted for real. Note that no operations are
  /// valid on an object that has "dropped all references", except operator
  /// delete.
  ///
  /// Since no other object in the module can have references into the body of
a
  /// function, dropping all references deletes the entire body of the
function,
  /// including any contained basic blocks.
  ///

Is it really true I can't do anything else to a Function after calling this?
Can I clone into it via CloneFunctionInto?

                                                       -Dave

The global data is also optimized at this point. Will it map properly with un-optimized pristine functions ?

I had wondered that myself. In what way is global data optimized?
My assumption is that global data might be added by optimizations
but that nothing would be deleted or drastically changed.

                                                  -Dave

Hmm...it seems you are right. When I try to merge the two value maps I
get a faullt mapping a ConstantValue from one of the pristine Function
clones. Apparently this ConstantValue got deleted by optimization
somewhere.

This actually doesn't make a lot of sense to me. What happens if I
clone a function and optimize it or the original function and ConstantValues
start getting deleted? That ConstantValue could still be referenced by
the unoptimized version of the function. I would think this situation would
be rather common.

Is it because the pristine Function clones live outside a Module that I'm
having this issue?

This seems much harder than it should be.

                                             -Dave

Ok, so I worked around this by changing the code that merges value
maps and also cloned the whole Module the pristine function is in
each time (I don't actually know if that latter bit is necessary).

Bottom line, I now have complete unoptimized LLVM out! Yay!

Deleting things and cleaning up is tricky though.

                                               -Dave