[RFC] lldb-vscode evaluate repl behavior and improvements

Introduction

The repl in lldb-vscode currently will try to evaluate expressions in the context of the selected frame. There is a sort of escape hatch to allow users to run lldb commands instead of evaluating expressions by prefixing the expression with a ‘`’ character.

This expression evaluation mode is not very discoverable and can be unexpected if a user is familiar with the lldb CLI.

The Debug Adapter Protocol is a little light on the specific expectations of the evaluate request with a repl context. The full specification can be found here Specification

Proposal

I think it would be useful to give users a little more control over the expression evaluation behavior. I think it would make sense to have a runtime flag that can adjust the behavior. I’d like to introduce a new runtime lldb command and lldb-vscode CLI argument for specifying this behavior.

The runtime lldb command to get or set the repl behavior would be a command like: lldb-vscode repl-mode that returns the mode and lldb-vscode repl-mode <mode> to set the mode.

Proposed modes:

  • variable - Essentially the current behavior. If an express has a ` prefix then assume its an lldb command, otherwise evaluate the expression in terms of the current frame.\

  • command - Assumes all evaluation requests with the repl context are lldb commands. There is a bit of a usability issue with command mode however. Since we evaluate all requests as lldb commands this mode would mean the return values will always be a stringified value of the command output. This does limit the ability for users to inspect output from an expression evaluation. For example, if I try to print a structure with multiple fields then in command mode I would only see the stringified output compared to variable mode where I can inspect individual fields and values.

  • auto - A heuristic driven mode to try to combine the two. When evaluating the expression this will first try to determine if the expression is an lldb command, otherwise evaluate the expression in the context of the current frame. This does mean that lldb commands would prevent users from easily inspecting a variable that happens to have the same name as an lldb command. To mitigate this, I think it would make sense to additionally check if the expression is either a frame variable or expression command and instead evaluate the expressions within the frame as opposed to lldb commands. The rough logic if this flow is as follows:

g_vsc.debugger.GetCommandInterpreter().ResolveCommand(request["expression"])
if (result.GetStatus() != lldb::eReturnStatusSuccessFinishResult) {
  /* Evaluate as an expression, e.g. "1 + 2" or "myVar.length" */
}

llvm::StringRef output = result.GetOutput();
/* Check if this is a user command to inspect a variable hidden by a command, e.g. "var help" or "p n" */
if (output.starts_with("frame variable") || 
    output.starts_with("expression") || 
    output.starts_with("dwim-print")) {
 /* Evaluate as an expression. */
}

/* Evaluate the command as an lldb command. */

Additionally, for the exposed lldb commands to be more useful, I would like to enable completions again.

From my testing, they are working well but I see some comments in the code that performance may be a concern.

Does anyone know any additional info about this?

I have a working prototype of this new behavior in ⚙ D154030 [lldb-vscode] Creating a new flag for adjusting the behavior of evaluation repl expressions to allow users to more easily invoke lldb commands. if you’d like to try this out.

Given this, is this change looking to address the discoverability or the usability or both?

I haven’t used lldb with vscode (sounds like I should!), so I assume the lldb-vscode command is something a user is likely to experiment with? Or would something in vscode itself run this command for them? Perhaps after changing a setting in vscode.

On the usability side, having a specific command mode would be great if you know you just want to muck about with lldb commands. GDB’s text user interface has a command pane and it’s really useful. Plus you don’t have to add any escape chars there.

I assume the existing ‘`’ method will remain? So even this auto mode will have an escape hatch.

Finally a side question. I assume the answer is no otherwise you wouldn’t be doing this, but does the adapter protocol have a channel for these debugger specific commands? I could imagine it being used to reset a JTAG probe for example, that sort of thing.

A bit of both, the ` prefix is not always known by new users and not very discoverable.

VS Code launches the executable, but it can be configured to have some additional flags. Also as part of my implementation the repl mode can be toggled at runtime using the lldb command:

lldb-vscode repl-mode <command|variable|auto>

You could also have this in your launch.json like:

{
  "type": "lldb-vscode",
  "program": "./exe",
  "initCommands": ["lldb-vscode repl-mode auto"]
}

Yea, that would be good to always keep the ` as an escape hatch, I can update my prototype with this.

No, not at the moment, there have been requests in the DAP to have a ‘command’ mode but so far they haven’t stuck yet (e.g. Support execution of debugger commands · Issue #231 · microsoft/debug-adapter-protocol · GitHub).


To expand on the auto behavior a little, here is an example of when you might see the variable being hidden by a lldb command:

# MyView.swift

func doWork() {
  var help = "Hello"
  print("\(help)") // breakpoint 1
}
> `lldb-vscode repl-mode auto
(lldb) lldb-vscode repl-mode auto
lldb-vscode repl-mode auto set.
> help
(lldb) help
Debugger commands:
  apropos           -- List debugger commands related to a word or subject.
... truncated
> v help
< "Hello" (this reply turns into a VS Code inspectable/expandable result, see screenshot)

Thanks for explaining. Sounds like a good change overall.

Which mode will be the default? variable is the conservative choice but if auto works 99% of the time it could become the default without anyone noticing.

The issue might be if I had a variable with the same name as a destructive lldb command. What if I have step for example. Anything that would change the debug state and potentially desync lldb and vscode (if that’s possible). You could flip the logic to be try expression, then try command. That would mitigate that some.

Potentially you could emit some messages when you autodetect command vs. expression, but it’s liable to get in the way and best done only if users find it confusing in practice.

Some recent interest in just this topic: [lldb-vscode] should provide a full Debugger Console · Issue #63654 · llvm/llvm-project · GitHub

Thanks for coming up with such a useful feature. A few comments:

  • I love the idea of having these 3 modes.
  • Regarding auto: I think that, in order to prevent unexpected bad behaviors, this mode should only consider frame var and command, and not expr, because some malformed expressions might cause LLDB to crash or some unexpected commands might cause bad side effects. So we should try to be on side of safety. If the user wants to execute something as an expression anyway, they could issue an actual expression evaluation command with p or expr. Not only that, you shouldn’t resolve the command just to make sure that the input is a command or not. This can lead to unwanted side effects. You should instead resolve the input as a frame var (which doesn’t cause JITting and thus it’s safe), and if failed, then you resolve it as a command. In this mode, if the user wants to override the behavior of the repl, you could provide some new escape prefixes like command or variable to distinguish how to resolve the input.

What do you think?

I hope that auto works well enough to be made the default, but I wanted to leave the flag in place to allow users to control the behavior.

I think this makes sense and it looks like @wallace had a similar comment below.

It sounds like it might also be good to add some basic usage information to the console once lldb-vscode is initialized to help users in this regard, I’ll see about that in a follow up commit.

I think the adjustments to auto make sense, let me try updated my prototype and test it out. I added ` as an escape hatch to ensure an expression is evaluated as an lldb command so I’m less worried about preferring variables over lldb commands. I’ll see about adding some help messages if we encounter a scenario like a variable that might be shadowing an lldb command.

I couldn’t tell if ResolveCommand had side effects, but it sounds like it might so I’ll avoid that for now. Would it be to expensive to resolve the command to detect collisions? If its not to expensive (or has side effects) then I could detect when a variable is hiding a command and give a one time warning about the situation.

I think I was wrong. ResolveCommand shouldn’t have side effects, so you should feel free to use it.

What about this?

  • do not evaluate JIT expressions, just do frame var, and if an input can be resolved by both frame var and the command resolver, then you show an error and ask the user to use some escape character to distinguish which want they want?

If there is no harm in evaluating a frame var, could we use a one time warning? E.g.

int main() {
  int n = 4;
  return n; // breakpoint 1
}
> n
(warning) Local variable detected with the same name as an lldb command ('n'), use a ` prefix to run the lldb command.
=> 4
> n
=> 4 (no warning)
> `n 
=> (step past the end of main).

We could only print the warning once (or maybe once per frame? I’m not sure what would be best to track in that situation).

There’s no harm with frame vars, as they try to avoid JITting almost always.

A one time warning per debug session session might be enough.

SGTM, I’ll get these changes working and update my review after validating the behavior.