Hello!
I’m a newbie in MLIR project 
Basically I need one of the following functionality when I run applyFullConversion:
The second option is more preferred
Is it possible or I’m speaking nonsense?
Thanks for the help in advance!
This isn’t directly possible, but maybe if you describe the problem to solve we could help with a strategy
Fair enough!
I’m trying to enable OpenACC conversions in Flang. I believe that the problem is tightly related to MLIR and/or can be solved with it.
OpenACC lowering to LLVM IR in MLIR is done in 2 phases:
OpenACCToLLVMConversion - as far as I understand, it converts a memref type in the operands to LLVM pointers. Actual operation conversions expect only memref or llvm pointer arguments: https://github.com/llvm/llvm-project/blob/main/mlir/lib/Conversion/OpenACCToLLVM/OpenACCToLLVM.cpp#L106-L135OpenACCDialectLLVMIRTranslationInterface - converts actual operations to runtime call directly in LLVM IROpenMP lowering done in a similar way and it is already enabled in Flang, so I just try to mimic it here: WIP · unterumarmung/llvm-project@8e6249d · GitHub
And it seems that it should work fine, but the problem occurs when we actually try to convert an OpenACC operation.
I use this Fortran program as an example because OpenACC’s enter data lowering to LLVM IR should be fully supported in MLIR.
subroutine foo
integer :: a
a = 42
!$acc enter data copyin(a)
end subroutine foo
Which has the following FIR (build/bin/flang-new -fc1 -fopenacc -emit-mlir c.f90) before any conversion to LLVM IR whatsoever:
module attributes {fir.defaultkind = "a1c4d8i4l4r4", fir.kindmap = "", llvm.target_triple = "x86_64-unknown-linux-gnu"} {
func.func @_QPfoo() {
%0 = fir.alloca i32 {bindc_name = "a", uniq_name = "_QFfooEa"}
%c42_i32 = arith.constant 42 : i32
fir.store %c42_i32 to %0 : !fir.ref<i32>
acc.enter_data copyin(%0 : !fir.ref<i32>)
return
}
}
As you can see, copyin in acc.enter_data accepts !fir.ref type which should be eliminated before we run OpenACCToLLVMConversion. This is should be done by AllocaOpConversion (basically converts fir.alloca to llvm.alloca) which is registered here: llvm-project/flang/lib/Optimizer/CodeGen/CodeGen.cpp at flang-openacc · unterumarmung/llvm-project · GitHub
Finally, when we try to run OpenACC conversion: llvm-project/mlir/lib/Conversion/OpenACCToLLVM/OpenACCToLLVM.cpp at flang-openacc · unterumarmung/llvm-project · GitHub
In the output I get this:
"llvm.func"() ({
%0 = "llvm.mlir.constant"() {value = 42 : i32} : () -> i32
%1 = "arith.constant"() {value = 42 : i32} : () -> i32
%2 = "llvm.mlir.constant"() {value = 1 : i64} : () -> i64
%3 = "llvm.alloca"(%2) {bindc_name = "a", in_type = i32, operand_segment_sizes = array<i32: 0, 0>, uniq_name = "_QFfooEa"} : (i64) -> !llvm.ptr<i32>
%4 = "fir.alloca"() {bindc_name = "a", in_type = i32, operand_segment_sizes = array<i32: 0, 0>, uniq_name = "_QFfooEa"} : () -> !fir.ref<i32>
"llvm.store"(%0, %3) : (i32, !llvm.ptr<i32>) -> ()
"fir.store"(%1, %4) : (i32, !fir.ref<i32>) -> ()
"acc.enter_data"(%4) {operand_segment_sizes = array<i32: 0, 0, 0, 0, 1, 0, 0, 0>} : (!fir.ref<i32>) -> ()
"func.return"() : () -> ()
}) {CConv = #llvm.cconv<ccc>, function_type = !llvm.func<void ()>, linkage = #llvm.linkage<external>, sym_name = "foo_"} : () -> ()
Which is weird - LLVM and FIR operations duplicated and acc.enter_data still accepts !fir.ref
Also, earlier I was getting <<<UNKNOWN SSA VALUE>>> in the last !fir.store and acc.enter_data.
What do you think the problem is? Is this MLIR-related or I accidentally violated some Flang-specific stuff?
I was thinking if tried to schedule the OpenACC conversions as later as possible the problem would be solved or, at least, become more clear. And, AFAIK, OpenACC conversions should be run as later as possible.
P. S. I’m sorry if something in my message is not clear - I’m very confused and tired..
Are you running the --fir-to-llvm-ir conversion here? If not can you try that?
I think what is happening is that it is going into the else case in LegalizeDataOpForLLVMTranslation. When I tried adding something like the following it worked.
diff --git a/mlir/lib/Conversion/OpenACCToLLVM/OpenACCToLLVM.cpp b/mlir/lib/Conversion/OpenACCToLLVM/OpenACCToLLVM.cpp
index 5a4da7bfd8c6..04f4d355a704 100644
--- a/mlir/lib/Conversion/OpenACCToLLVM/OpenACCToLLVM.cpp
+++ b/mlir/lib/Conversion/OpenACCToLLVM/OpenACCToLLVM.cpp
@@ -131,8 +131,11 @@ class LegalizeDataOpForLLVMTranslation : public ConvertOpToLLVMPattern<Op> {
} else if (originalDataOperand.getType().isa<LLVM::LLVMPointerType>()) {
convertedOperands.push_back(originalDataOperand);
} else {
+ llvm::outs() << "Entering here\n";
+ convertedOperands.push_back(adaptor.getOperands()[nonDataOperands.size()+idx]);
// Type not supported.
- return builder.notifyMatchFailure(op, "unsupported type");
+ //return builder.notifyMatchFailure(op, "unsupported type");
}
}
While trying to convert the OpenACC enter_data operation during fir-to-llvm conversion with the copyin operand as a fir.ref there is no matching case (the available cases being memref, llvm.pointer) so it goes to the else and fails. Also, the existing check is checking whether the original data operand is an llvm pointer type (originalDataOperand.getType().isa<LLVM::LLVMPointerType>()) and not the converted operand, may be we can change the check to also check the converted operand. I think it is done the way it is because we want to handle the case where the original operand is a memref type.
Yes, that’s exactly why it fails.
I do not quite understand what is going on here. Is this a way to postpone the conversion?
But I have another idea judging by the code, data operands are placed after non-data operands in the getOperands(), but the loop starts with 0 offset even though it should’ve started with an offset of nonDataOperands.size().
What do you think about this?
I’m going to check if this is going to solve the problem tomorrow.
adaptor.getOperands() contains all the converted operands (i.e the llvm.ptr type here) and op.getDataOperand gets us the original data (not others) operand (i.e the fir.ref type here). So basically the code above is pushing the converted copyin operand (the llvm.ptr one) into the convertedOperands vector.
While the loop starts at 0, it is using the getDataOperand (not the getOperand one) function to get the Data Operands specifically.
Yeah, I missed Data in the getDataOperand 
It works as a charm! I think it should be covered by tests, but I have no I idea how to do it. Maybe you have some thoughts?
FIR types should be converted in a dedicated pass since FIR is not in core MLIR. The idea is that the types are converted where the knowledge about them is available. memref is in core but not the FIR types.
And it was converted, but for some reason not replaced in the OpenACC operands.
In the previous messages you can observe that for some reason operations were duplicated: both FIR and LLVM operations are present in the same function. I do not understand why
It was converted by the FIR to LLVM pass but what we want is to extract the information before or during this pass. For fir.ref it is more or less ok because it will become a ptr but for other type like fir.box and so one, we need to do much more.
I do not understand why the OpenACC operand isn’t changed if it was actually converted.
basically LegalizeDataOpForLLVMTranslation is a pass that takes known types and transform them to either a ptr or a DataDescriptor (two pointers and a size). This is a preparation pass to be able to call the runtime. The runtime expect this kind of information. You cannot just pass any type to the runtime and expect things to work out of the box.
For FIR, there is a FIR to LLVM pass but this is not OpenACC aware or OpenMP aware in term of data offloading. During the conversion, there is some work to be done when you know the original type (FIR) and the target type (LLVM). At this point you have all the information to either just pass it to OpenACC as a ptr or create a DataDescriptor and extract the correct information from the original type. After the conversion happened all the knowledge about the original type is lost.
So the missing part is a pass or a hook in the fir-to-llvm pass to perform this work.
Note that this is specific to use the current runtime (extended OpenMP runtime in LLVM). It would probably be done differently if you want to plug another runtime.
Just to confirm, whatever I was suggesting was to get you past the error for fir.ref’s of scalar types. For complicated types we will have to do what @clementval is suggesting in the post above. I think @clementval had a rudimentary patch ([openacc] Conversion of FIR type data operands to llvm type by clementval · Pull Request #915 · flang-compiler/f18-llvm-project · GitHub) that was trying to solve this but I guess it needs lot more work.
The PR @kiranchandramohan pointed out is exactly an attempt at doing what I’m explaining before. As Kiran mentioned, it didn’t make it in before the upstreaming and it will require quite some work now.
As you can see there was a fir.cast operation introduced so the pass could be done before fir-to-llvm and when fir-to-llvm happen the fir.cast just become a no op and are removed.
If this is not clear, we can open a new topic in the flang section. We are quite far from the initial topic of this thread.