How to add a new rewriting option

Hi,

I would like to make some test about converting some objc code into C#
So I have started to locate the code responsible to handle objc-rewrite and I tried to do the same for my case.
Presently I just want have something that compiles so I have duplicated RewriteObjC.cpp and renamed it into RewriteObjC2Cs.cpp
and changed the class name (I didn't include it inside the diff below because it makes things hard to read).
So now when I compile I get the following error :

Scanning dependencies of target clangFrontend
[ 84%] Building CXX object tools/clang/lib/Frontend/CMakeFiles/clangFrontend.dir/CompilerInstance.cpp.o
/home/vincent/objc2cs/clang/llvm/tools/clang/lib/Frontend/CompilerInstance.cpp: In member function ‘bool clang::CompilerInstance::ExecuteAction(clang::FrontendAction&)’:
/home/vincent/objc2cs/clang/llvm/tools/clang/lib/Frontend/CompilerInstance.cpp:665:43: error: ‘RewriteObjC2Cs’ is not a member of ‘clang::frontend’
make[2]: *** [tools/clang/lib/Frontend/CMakeFiles/clangFrontend.dir/CompilerInstance.cpp.o] Error 1
make[1]: *** [tools/clang/lib/Frontend/CMakeFiles/clangFrontend.dir/all] Error 2
make: *** [all] Error 2

What am I missing ?

Index: lib/Frontend/InitPreprocessor.cpp

Actually I think I found I forgot to add the corresponding enum inside FrontendOptions.h :

Index: tools/clang/include/clang/Frontend/FrontendOptions.h

Hum still problem because when I try to use the new rewrite options clang shows me the command usage (with the
new option) :

With the working rewrite-objc it works because I end up with a .cpp file
  ~/objc2cs/gnustep/core/gui/Source $ clang -rewrite-objc NSButton.m -c -MMD -MP -DGNUSTEP_TARGET_DIR=\".\" -DGNUSTEP_TARGET_CPU=\"x86_64\" -DGNUSTEP_TARGET_OS=\"linux-gnu\" -DLIBRARY_COMBO=\"gnu-gnu-gnu\" -DBACKEND_BUNDLE=1 -DGNUSTEP -DGNUSTEP_BASE_LIBRARY=1 -DGNU_GUI_LIBRARY=1 -DGNU_RUNTIME=1 -DGNUSTEP_BASE_LIBRARY=1 -fno-strict-aliasing -fexceptions -fobjc-exceptions -D_NATIVE_OBJC_EXCEPTIONS -fobjc-nonfragile-abi -D_NONFRAGILE_ABI -pthread -fPIC -DDEBUG -fno-omit-frame-pointer -Wall -DGSWARN -DGSDIAGNOSE -Wno-import -g -fgnu-runtime -Wall -fconstant-string-class=NSConstantString -I../Headers/Additions -I../Headers -I./. -I. -I/home/vincent/GNUstep/Library/Headers -I/GNUstep/Local/Library/Headers -I/GNUstep/System/Library/Headers -I/usr/include/libpng12

However if I try with the same command line and replace with -rewrite-objc2cs it shows me the command usage ...

Any idea ?

I have attached my latest patch(clang 3.3) below and I am wondering if it wouldn't be easier to only modify the
RewriteObjc.cpp instead of trying to add a new rewriting option ...

Index: tools/clang/lib/Frontend/InitPreprocessor.cpp

Ok I found the last missing part.
Sorry for the noise.

Ok so last question, actually I have noticed a big difference between -rewrite-legacy-objc and -rewrite-objc because
when I try the two differents flags on the same file I get respectively a file of 40.8 kB and another of 392.4 kB.
The difference seems to be because when using -rewrite-objc flag the file is first preprocessed :

1) -rewrite-legacy-objc

clang -v -v -rewrite-legacy-objc NSButton.m -c -MMD -MP -DGNUSTEP_TARGET_DIR=\".\" -DGNUSTEP_TARGET_CPU=\"x86_64\" -DGNUSTEP_TARGET_OS=\"linux-gnu\" -DLIBRARY_COMBO=\"gnu-gnu-gnu\" -DBACKEND_BUNDLE=1 -DGNUSTEP -DGNUSTEP_BASE_LIBRARY=1 -DGNU_GUI_LIBRARY=1 -DGNU_RUNTIME=1 -DGNUSTEP_BASE_LIBRARY=1 -fno-strict-aliasing -fexceptions -fobjc-exceptions -D_NATIVE_OBJC_EXCEPTIONS -fobjc-nonfragile-abi -D_NONFRAGILE_ABI -pthread -fPIC -DDEBUG -fno-omit-frame-pointer -Wall -DGSWARN -DGSDIAGNOSE -Wno-import -g -fgnu-runtime -Wall -fconstant-string-class=NSConstantString -I../Headers/Additions -I../Headers -I./. -I. -I/home/vincent/GNUstep/Library/Headers -I/GNUstep/Local/Library/Headers -I/GNUstep/System/Library/Headers -I/usr/include/libpng12
clang version 3.3 (186145)
Target: x86_64-unknown-linux-gnu
Thread model: posix

**** File is transformed (-rewrite-objc)

  "/opt/bin/clang-3.3" -cc1 -triple x86_64-unknown-linux-gnu -rewrite-objc -disable-free -main-file-name NSButton.m -mrelocation-model pic -pic-level 2 -mdisable-fp-elim -relaxed-aliasing -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -v -v -g -coverage-file /home/vincent/objc2cs/gnustep/core/gui/Source/NSButton.cpp -resource-dir /opt/bin/../lib/clang/3.3 -dependency-file NSButton.d -MT NSButton.o -MP -D "GNUSTEP_TARGET_DIR=\".\"" -D "GNUSTEP_TARGET_CPU=\"x86_64\"" -D "GNUSTEP_TARGET_OS=\"linux-gnu\"" -D "LIBRARY_COMBO=\"gnu-gnu-gnu\"" -D BACKEND_BUNDLE=1 -D GNUSTEP -D GNUSTEP_BASE_LIBRARY=1 -D GNU_GUI_LIBRARY=1 -D GNU_RUNTIME=1 -D GNUSTEP_BASE_LIBRARY=1 -D _NATIVE_OBJC_EXCEPTIONS -D _NONFRAGILE_ABI -D DEBUG -D GSWARN -D GSDIAGNOSE -I ../Headers/Additions -I ../Headers -I ./. -I . -I /home/vincent/GNUstep/Library/Headers -I /GNUstep/Local/Library/Headers -I /GNUstep/System/Library/Headers -I /usr/include/libpng12 -internal-isystem /usr/local/include -internal-isystem /opt/bin/../lib/clang/3.3/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -Wall -Wno-import -Wall -fdebug-compilation-dir /home/vincent/objc2cs/gnustep/core/gui/Source -fconstant-string-class NSConstantString -ferror-limit 19 -fmessage-length 232 -pthread -mstackrealign -fblocks -fobjc-runtime=gnustep-1.6 -fobjc-dispatch-method=non-legacy -fobjc-default-synthesize-properties -fno-objc-infer-related-result-type -fobjc-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -backend-option -vectorize-loops -o NSButton.cpp -x objective-c NSButton.m

2)-rewrite-objc

clang -v -v -rewrite-objc NSButton.m -c -MMD -MP -DGNUSTEP_TARGET_DIR=\".\" -DGNUSTEP_TARGET_CPU=\"x86_64\" -DGNUSTEP_TARGET_OS=\"linux-gnu\" -DLIBRARY_COMBO=\"gnu-gnu-gnu\" -DBACKEND_BUNDLE=1 -DGNUSTEP -DGNUSTEP_BASE_LIBRARY=1 -DGNU_GUI_LIBRARY=1 -DGNU_RUNTIME=1 -DGNUSTEP_BASE_LIBRARY=1 -fno-strict-aliasing -fexceptions -fobjc-exceptions -D_NATIVE_OBJC_EXCEPTIONS -fobjc-nonfragile-abi -D_NONFRAGILE_ABI -pthread -fPIC -DDEBUG -fno-omit-frame-pointer -Wall -DGSWARN -DGSDIAGNOSE -Wno-import -g -fgnu-runtime -Wall -fconstant-string-class=NSConstantString -I../Headers/Additions -I../Headers -I./. -I. -I/home/vincent/GNUstep/Library/Headers -I/GNUstep/Local/Library/Headers -I/GNUstep/System/Library/Headers -I/usr/include/libpng12

**** FIRST file is preprocessed(-E flag) :

"/opt/bin/clang-3.3" -cc1 -triple x86_64-unknown-linux-gnu -E -disable-free -main-file-name NSButton.m -mrelocation-model pic -pic-level 2 -mdisable-fp-elim -relaxed-aliasing -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -v -v -g -coverage-file /tmp/NSButton-sxcmia.mi -resource-dir /opt/bin/../lib/clang/3.3 -dependency-file NSButton.d -MT NSButton.o -MP -D "GNUSTEP_TARGET_DIR=\".\"" -D "GNUSTEP_TARGET_CPU=\"x86_64\"" -D "GNUSTEP_TARGET_OS=\"linux-gnu\"" -D "LIBRARY_COMBO=\"gnu-gnu-gnu\"" -D BACKEND_BUNDLE=1 -D GNUSTEP -D GNUSTEP_BASE_LIBRARY=1 -D GNU_GUI_LIBRARY=1 -D GNU_RUNTIME=1 -D GNUSTEP_BASE_LIBRARY=1 -D _NATIVE_OBJC_EXCEPTIONS -D _NONFRAGILE_ABI -D DEBUG -D GSWARN -D GSDIAGNOSE -I ../Headers/Additions -I ../Headers -I ./. -I . -I /home/vincent/GNUstep/Library/Headers -I /GNUstep/Local/Library/Headers -I /GNUstep/System/Library/Headers -I /usr/include/libpng12 -internal-isystem /usr/local/include -internal-isystem /opt/bin/../lib/clang/3.3/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -Wall -Wno-import -Wall -fdebug-compilation-dir /home/vincent/objc2cs/gnustep/core/gui/Source -fconstant-string-class NSConstantString -ferror-limit 19 -fmessage-length 232 -pthread -mstackrealign -fblocks -fobjc-runtime=gnustep-1.6 -fobjc-dispatch-method=non-legacy -fobjc-default-synthesize-properties -fobjc-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -backend-option -vectorize-loops -o /tmp/NSButton-sxcmia.mi -x objective-c++-cpp-output NSButton.m

**** THEN file is transformed (-rewrite-objc)

"/opt/bin/clang-3.3" -cc1 -triple x86_64-unknown-linux-gnu -rewrite-objc -disable-free -main-file-name NSButton.m -mrelocation-model pic -pic-level 2 -mdisable-fp-elim -relaxed-aliasing -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -v -v -g -coverage-file /home/vincent/objc2cs/gnustep/core/gui/Source/NSButton.cpp -resource-dir /opt/bin/../lib/clang/3.3 -Wall -Wno-import -Wall -fdebug-compilation-dir /home/vincent/objc2cs/gnustep/core/gui/Source -fconstant-string-class NSConstantString -ferror-limit 19 -fmessage-length 232 -pthread -mstackrealign -fblocks -fobjc-runtime=gnustep-1.6 -fobjc-dispatch-method=non-legacy -fobjc-default-synthesize-properties -fno-objc-infer-related-result-type -fobjc-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -backend-option -vectorize-loops -o NSButton.cpp -x objective-c++-cpp-output /tmp/NSButton-sxcmia.mi

Is is a normal behavior to have a difference in preprocessing between -rewrite-legacy-objc and -rewrite-objc or is it one of my change that could explain it ?
What part of code is responsible for that ?
With my new flag -rewrite-objc2cs I don't want to have a preprocessing stage but I don't know how to do it ?
If you have any pointers about this part, it would help a lot.

Pleas note that for the moment the RewriteObjC2Cs.cpp is a copy of RewriteObjC.cpp and I am calling a method called CreateObjC2CsRewriter
that actually corresponds to the CreateObjCRewriter (fragile runtime) of the RewriteObjC.cpp

ASTConsumer *RewriteObjCAction::CreateASTConsumer(CompilerInstance &CI,
                                                   StringRef InFile) {
   if (raw_ostream *OS = CI.createDefaultOutputFile(false, InFile, "cpp")) {
     if (CI.getLangOpts().ObjCRuntime.isNonFragile())
       return CreateModernObjCRewriter(InFile, OS,
                                 CI.getDiagnostics(), CI.getLangOpts(),
                                 CI.getDiagnosticOpts().NoRewriteMacros,
                                 (CI.getCodeGenOpts().getDebugInfo() !=
                                  CodeGenOptions::NoDebugInfo));
     return CreateObjCRewriter(InFile, OS,
                               CI.getDiagnostics(), CI.getLangOpts(),
                               CI.getDiagnosticOpts().NoRewriteMacros);
   }
   return 0;
}

ASTConsumer *RewriteObjC2CsAction::CreateASTConsumer(CompilerInstance &CI,
                                                   StringRef InFile) {
   if (raw_ostream *OS = CI.createDefaultOutputFile(false, InFile, "cs")) {
    return CreateObjC2CsRewriter(InFile, OS,
                               CI.getDiagnostics(), CI.getLangOpts(),
                               CI.getDiagnosticOpts().NoRewriteMacros);
   }
   return 0;
}

Thanks

Hi Vincent,

-rewrite-legacy-objc is out of date. For up to date support for ObjectiveC features and translation of its
modern ABI, you may want to look into -rewrite-objc. -rewrite-objc has simplified the workflow by pre-processing
user program and translating it into C++. -rewrite-legacy-objc requires user to do the pre-processing step manually.

  • Fariborz

Ok but what I would like to achieve is closer to rewrite-legacy-objc than rewrite-objc
because I don't want to compile the code after transformation and I don't want preprocessing stage.

So now I have modified the Options.td for my new flag -rewrite-objc2cs and used the same parameters
as rewrite-legacy-objc but now the driver tells me : unrecognized

clang -rewrite-objc2cs NSButton.m -c -MMD -MP -DGNUSTEP_TARGET_DIR=\".\" -DGNUSTEP_TARGET_CPU=\"x86_64\" -DGNUSTEP_TARGET_OS=\"linux-gnu\" -DLIBRARY_COMBO=\"gnu-gnu-gnu\" -DBACKEND_BUNDLE=1 -DGNUSTEP -DGNUSTEP_BASE_LIBRARY=1 -DGNU_GUI_LIBRARY=1 -DGNU_RUNTIME=1 -DGNUSTEP_BASE_LIBRARY=1 -fno-strict-aliasing -fexceptions -fobjc-exceptions -D_NATIVE_OBJC_EXCEPTIONS -fobjc-nonfragile-abi -D_NONFRAGILE_ABI -pthread -fPIC -DDEBUG -fno-omit-frame-pointer -Wall -DGSWARN -DGSDIAGNOSE -Wno-import -g -fgnu-runtime -Wall -fconstant-string-class=NSConstantString -I../Headers/Additions -I../Headers -I./. -I. -I/home/vincent/GNUstep/Library/Headers -I/GNUstep/Local/Library/Headers -I/GNUstep/System/Library/Headers -I/usr/include/libpng12

vincent@vincent-virtual-machine ~/objc2cs/gnustep/core/gui/Source $ clang -v -v -rewrite-objc2cs NSButton.m -c -MMD -MP -DGNUSTEP_TARGET_DIR=\".\" -DGNUSTEP_TARGET_CPU=\"x86_64\" -DGNUSTEP_TARGET_OS=\"linux-gnu\" -DLIBRARY_COMBO=\"gnu-gnu-gnu\" -DBACKEND_BUNDLE=1 -DGNUSTEP -DGNUSTEP_BASE_LIBRARY=1 -DGNU_GUI_LIBRARY=1 -DGNU_RUNTIME=1 -DGNUSTEP_BASE_LIBRARY=1 -fno-strict-aliasing -fexceptions -fobjc-exceptions -D_NATIVE_OBJC_EXCEPTIONS -fobjc-nonfragile-abi -D_NONFRAGILE_ABI -pthread -fPIC -DDEBUG -fno-omit-frame-pointer -Wall -DGSWARN -DGSDIAGNOSE -Wno-import -g -fgnu-runtime -Wall -fconstant-string-class=NSConstantString -I../Headers/Additions -I../Headers -I./. -I. -I/home/vincent/GNUstep/Library/Headers -I/GNUstep/Local/Library/Headers -I/GNUstep/System/Library/Headers -I/usr/include/libpng12
clang version 3.3 (186145)
Target: x86_64-unknown-linux-gnu
Thread model: posix
  "/opt/bin/clang-3.3" -cc1 -triple x86_64-unknown-linux-gnu -rewrite-objc2cs -disable-free -main-file-name NSButton.m -mrelocation-model pic -pic-level 2 -mdisable-fp-elim -relaxed-aliasing -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -v -v -g -coverage-file /home/vincent/objc2cs/gnustep/core/gui/Source/NSButton.cs -resource-dir /opt/bin/../lib/clang/3.3 -dependency-file NSButton.d -MT NSButton.o -MP -D "GNUSTEP_TARGET_DIR=\".\"" -D "GNUSTEP_TARGET_CPU=\"x86_64\"" -D "GNUSTEP_TARGET_OS=\"linux-gnu\"" -D "LIBRARY_COMBO=\"gnu-gnu-gnu\"" -D BACKEND_BUNDLE=1 -D GNUSTEP -D GNUSTEP_BASE_LIBRARY=1 -D GNU_GUI_LIBRARY=1 -D GNU_RUNTIME=1 -D GNUSTEP_BASE_LIBRARY=1 -D _NATIVE_OBJC_EXCEPTIONS -D _NONFRAGILE_ABI -D DEBUG -D GSWARN -D GSDIAGNOSE -I ../Headers/Additions -I ../Headers -I ./. -I . -I /home/vincent/GNUstep/Library/Headers -I /GNUstep/Local/Library/Headers -I /GNUstep/System/Library/Headers -I /usr/include/libpng12 -internal-isystem /usr/local/include -internal-isystem /opt/bin/../lib/clang/3.3/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -Wall -Wno-import -Wall -fdebug-compilation-dir /home/vincent/objc2cs/gnustep/core/gui/Source -fconstant-string-class NSConstantString -ferror-limit 19 -fmessage-length 159 -pthread -mstackrealign -fblocks -fobjc-runtime=gnustep-1.6 -fobjc-dispatch-method=non-legacy -fobjc-default-synthesize-properties -fno-objc-infer-related-result-type -fobjc-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -backend-option -vectorize-loops -o NSButton.cs -x objective-c NSButton.m
error: unknown argument: '-rewrite-objc2cs'

Before I use the same arguments as rewrite-legacy-objc, the command was working even if the result wasn't what I wanted to (non processed file).
So I suppose I forgot to add my new option somewhere in the code but I don't know where ...

And another thing I don't understand because when looking at generated code when using -rewrite-legacy-objc
it seems the rewriter used is the one implemented inside RewriteModernObjC.cpp.

I don't understand why because if I look int Tools.cpp :

} else if (JA.getType() == types::TY_RewrittenObjC) {
       CmdArgs.push_back("-rewrite-objc");
       rewriteKind = RK_NonFragile;
     } else if (JA.getType() == types::TY_RewrittenLegacyObjC) {
       CmdArgs.push_back("-rewrite-objc");
       rewriteKind = RK_Fragile;
     } else if (JA.getType() == types::TY_RewrittenObjC2Cs) {
       CmdArgs.push_back("-rewrite-objc2cs");
       rewriteKind = RK_Fragile;
     } else {
       assert(JA.getType() == types::TY_PP_Asm &&
              "Unexpected output type!");
     }

the associated rewriteKind of TY_RewrittenLegacyObjC is RK_Fragile BUT the code that use RewriteModernObjC is the following :

if (CI.getLangOpts().ObjCRuntime.isNonFragile())
       return CreateModernObjCRewriter(InFile, OS,
                                 CI.getDiagnostics(), CI.getLangOpts(),
                                 CI.getDiagnosticOpts().NoRewriteMacros,
                                 (CI.getCodeGenOpts().getDebugInfo() !=
                                  CodeGenOptions::NoDebugInfo));
     return CreateObjCRewriter(InFile, OS,
                               CI.getDiagnostics(), CI.getLangOpts(),
                               CI.getDiagnosticOpts().NoRewriteMacros);

so does it mean that the fragile meaning of ObjCRuntime.isNonFragile() is different from the rewriteKind meaning ?
Because let's say I cannot debug clang and just by reading the code I would have thought than when using -rewrite-legacy-objc (RK_Fragile)
it would call the fragile branch ie CreateObjCRewriter() ...

And another question about debugging I tried to put some breakpoints inside CreateModernObjCRewriter and CreateObjCRewriter and inside
some other methods inside RewriteModernObjC.cpp but they are never triggered. Any idea why ?