[clang-repl] Implement autocompletion in clang-repl

Description

The Clang compiler is part of the LLVM compiler infrastructure and supports various languages such as C, C++, ObjC and ObjC++. The design of LLVM and Clang enables them to be used as libraries, and has led to the creation of an entire compiler-assisted ecosystem of tools. The relatively friendly codebase of Clang and advancements in the JIT infrastructure in LLVM further enable research into different methods for processing C++ by blurring the boundary between compile time and runtime. Challenges include incremental compilation and fitting compile/link time optimizations into a more dynamic environment.

Incremental compilation pipelines process code chunk-by-chunk by building an ever-growing translation unit. Code is then lowered into the LLVM IR and subsequently run by the LLVM JIT. Such a pipeline allows creation of efficient interpreters. The interpreter enables interactive exploration and makes the C++ language more user friendly. The incremental compilation mode is used by the interactive C++ interpreter, Cling, initially developed to enable interactive high-energy physics analysis in a C++ environment.

Our group puts efforts to incorporate and possibly redesign parts of Cling in Clang mainline through a new tool, clang-repl. The project aims at the design and implementation of robust autocompletion when users type C++ at the prompt of clang-repl. For example:

[clang-repl] class MyLongClassName {};
[clang-repl] My<tab>
// list of suggestions.

Expected results

There are several foreseen tasks:

  • Research the current approaches for autocompletion in clang such as clang -code-completion-at=file:col1:col2.
  • Implement a version of the autocompletion support using the partial
    translation unit infrastructure in clang’s libInterpreter.
  • Investigate the requirements for semantic autocompletion which takes into
    account the exact grammar position and semantics of the code. Eg:
  [clang-repl] struct S {S* operator+(S&) { return nullptr;}};
  [clang-repl] S a, b;
  [clang-repl] v = a + <tab> // shows b as the only acceptable choice here.
  • Present the work at the relevant meetings and conferences

Desirable skills

Intermediate C++; Understanding of Clang and the Clang API in particular

Project type

Large

Mentors

@vvassilev

Further reading

clang-repl
clang-repl RFC
Cling

1 Like

Hello @vvassilev, I am interested in working on this project. I just graduated, and now I am working at Oracle. My end-of-studies project (at Oracle Labs) was about compiling a domain-specific language by analyzing and transforming an intermediate representation of programs written in this DSL, it was my first interaction with IRs and compilers and the project was a success.

Hi @mabbay,

Thanks for reaching out and thanks for your interest.

Since GSoC22 is over I was wondering under what form you want to be involved in the project? I am happy to mentor you as a unfunded contributor for example.

My goal is to contribute to LLVM and a project like this one is a great opportunity to get started, right?

Absolutely! I am happy to discuss design or review patches in that area. I’d propose to start by building clang-repl and maybe preparing an draft design document and an implementation plan.

Thanks for accepting to mentor me.
At this point, do you think I need to learn about Clang?

I think that’s a prerequisite.

So, I will take some time to learn about Clang. What concepts do I need to learn?

Hi @vvassilev I am a phd student at University of Kansas. My fundamental research focus is binary analysis. Although my research is more on binary side of things, I believe this project suits me, as I gained some experience on clang and llvm api (out of tree passes) during my internship at Meta (2022) and during my research at KU. As you suggested I built clang-repl and played with it a little. I will also work on creating the proposal. How do you want me to share the link? Also, I never contributed to llvm before, could you suggest me a small task to get started with?

Hi @Ruturaj4, thanks for reaching out.

Could you send me (in private) your CV?

This document Contributing to LLVM — LLVM 17.0.0git documentation gives a good initial overview of how to contribute to llvm.

1 Like

@vvassilev Thanks!

Hi @vvassilev,
I want to learn more about compilers, I created compiler for an assembly-like language from scratch using C and it felt super fun to do (Vyantra-v2) and I also followed llvm’s kaleidoscope.
This project looks really cool but intimidating at the same time. I’m comfortable with recursive logic/graph algorithms… I’m not sure if that’s an important thing to know but it helped me understand a lot of stuff when I built the parser and visitor for the AST.
I also know C/C++ but ofc not like an expert.
How should I get started with this project? If you think this will be hard at this point is there something in llvm that I can look into? (non-GSoC projects will also work).

I think your assessment is right. If you look for some projects for beginners outside of GSoC perhaps would be a good idea to reach out to me in private as this thread discusses GSoC.

Hi @vvassilev,

I have a strong interest in this project, but I am not quite sure how to get started with it. I have sent you an email at “vvasilev at cern.ch”. Is this email address correct?

1 Like