RFC for refactoring common code for OpenACC and OpenMP

This post is aimed at starting a discussion for the design and implementation of common code for OpenACC and OpenMP for some similar directives between the two. (https://openmpcon.org/wp-content/uploads/openmpcon2015-oscar-hernandez-portingacc.pdf Page#8 contains a list of few). So, for this whole discussion let us focus on example acc enter/update/exit data which is similar to omp enter/update/exit data and acc data which is similar to omp target data.

We have all the current implementation is in OpenACCToLLVMIRTranslation.cpp.

convertOperation() would call template function convertStandAloneDataOp() or convertOperation() based on the value of Operation. We will be ultimately calling processOperands() which has some specific OpenACC flags which might not all be applicable to OpenMP. I am currently trying to implement omp enter data.

Solution 1, Inorder to re-use some of the existing functionality of OpenACC I started with refactoring to place common code in a new Utils directory in a new file named DirectiveToLLVMIR which will be made commonly accessible to both OpenMP and OpenACC.

However the major problem with this approach is there is lot of openacc specific code especially for convertDataOp() and processOperands() existent in DirectiveToLLVMIR file and which will lead to not good refactoring. And, if I try to move these above two functions to OpenACC and keep the other functions in Utils/DirectiveToLLVMIR this is causing Cmake cyclic dependency errors between DirectiveToLLVMIR and OpenACCToLLVMIR libraries as both are inter-dependent on each other.

So, I backed out for little bit and started a different approach Solution 2

Moving only fragments of common code to OpenMPIRBuilder by creating new functions specific to each operation createEnterData(), createUpdateData() and createExitData(). These new functions would be called from OpenACC’s convertOperation() and OpenMP’s convertOperation() via template function convertStandAloneDataOp() and convertOmp[Enter|Exit|Update]Op(). As convertDataOp of both OpenACC and OpenMP differ a lot may be shall maintain different functions in their respective files. During implementation in future if we feel otherwise we can always follow the above approach to get common code to OpenMPIRBuilder

As OpenACC functions need to call processDataOperands() which is specific to OpenACC convertEnterData() function consists of code post processDataOperands() and eventhough we have some common code preceding processDataOperands() function I kind of maintained two copies of those in both OpenACC and OpenMP. I am open to any suggestions to get an alternate solution for taking away this mini-problem to resolve even the pre processDataOperands().

Also, if the community thinks we are refactoring only tiny pieces of code with a lot of refactoring effort we can go for Solution 3 which is not to go with refactoring efforts at all. Instead, do not change anything in OpenACCToLLVMIR.cpp and add all the code in OpenMPToLLVMIR.cpp eventhough we have some common code between the two.

To make solution 1 work seemed to be very complex. So, I personally I prefer solution 2 or solution 3 which ever community thinks more advantageous to go with not only for enter/update/exit data implementation but also for all the future implementations for more directives.

I will appreciate any feedback and any suggestions or any other solutions to this effort.

Thanks!

I was classified as a new user and I wasn’t allowed to include the following images in the above introduction so including them here.

Existing code

Solution 1:

Solution 2:

What should remain very specific to each dialect is the processDataOperands/processOperands part. I don’t see a way to make this generic. As I mentioned before the flags are tuned to fit the different behavior.

I don’t think you can do much more in the OpenMPIRBuilder. The part that process the operands is really tied to MLIR and you need its information to be able to perform the translation.

It should be easy to reuse the code and delegate the part that process operand to each dialects.

If you go for solution 2 I don’t think you need to touch the OpenACC part at all. At least I don’t see what will be gained for it.

FWIW, I would have expected the dialects to build the information (size array, base ptr array, ptr array, …) and then pass it to the OMPIRBuilder for runtime call generation.

FWIW, I would have expected the dialects to build the information (size array, base ptr array, ptr array, …) and then pass it to the OMPIRBuilder for runtime call generation.

That’s basically what is done in OpenACCToLLVMIRTranslation