Remote connection expansion?

I'm looking for support running lldb over ssh. I can forward the
originating connection, but the run command is attempting to use
random ports on localhost to attain another connection. This fails as
the localhost's are not the same.

Is there a platform, preferably real time, for lldb support?

One might ask why ssh, the basic answer is I don't want to open
*another port on my remote host... Even if I did I'd still have the
same problem, a random port would fail to connect(this time because of
a firewall). The main answer is, without ssh, lldb is limited to
running on local or VPN networks. I'd rather use ssh than configure a
VPN for this one use case.

* lldb doesn't sound like something one would want to host, even if
connections were blocked from everywhere "else."

Now I'm attempting forward error correction by guessing where this
topic could lead. I would be willing to expand the network code to
include domain sockets, to replace the whole idea of using, IMHO
barbaric, port numbers. This work could potentially include direct
support for ssh. I understand that this would likely be a breaking
change, is there version negotiation?

I'm looking for support running lldb over ssh. I can forward the
originating connection, but the run command is attempting to use
random ports on localhost to attain another connection. This fails as
the localhost's are not the same.

When you say you want to run lldb over ssh, do you mean run "lldb-server" remotely and then have a local LLDB connect to that lldb-server? You are looking to avoid "lldb-server" from having to bind to port 0 and then tell you which port it was actually bound to?

>
> I'm looking for support running lldb over ssh. I can forward the
> originating connection, but the run command is attempting to use
> random ports on localhost to attain another connection. This fails as
> the localhost's are not the same.

When you say you want to run lldb over ssh, do you mean run "lldb-server"

Is there really an issue with saying these are both lldb? Seems like
my statements were unambiguous without noting a distinction.

remotely and then have a local LLDB connect to that lldb-server?

That's the usual way lldb is used remotely, so I don't really get that
lldb does not, in some cases, mean lldb-server.

You are looking to avoid "lldb-server" from having to bind to port 0 and then tell you which port it was actually bound to?

This is indeed a *feature* of lldb-server and not lldb, so I don't
really get your confusion when I claim lldb has a feature that's
entirely implemented with regard to lldb-server.

>
> Is there a platform, preferably real time, for lldb support?
>

This is a really important question of mine! If we can only go back
and forth on email I'd like ppl to make a best effort to lessen the
effects this has. One tip I have, even for when ppl are chatting, is
to phrase questions in the form of an answer... Don't ask yes/no
questions or even questions with say less than 7 answers. Instead
provide bullet points of all the possible answers, one at a time.
There are many good reasons for everyone doing this, but here the most
relevant is to avoid round-trips that are expansive.

Can you answer your question twice? Once If I claim that "Yes" I do
plan on having lldb choose a port to listen on and have ssh forward
that port and again for if I answer "No".

I hope you can appreciate that having ssh forward a handful of ports
for a single service is not optimal. I'd actually do better by
writing another set of client-server applications that tunnel the IPC
over ssh... Something I'd only do if the program was closed source.
As lldb is not, the obvious path forward is to re-implement the lldb
IPC so it's more friendly to ssh.

I’m looking for support running lldb over ssh. I can forward the
originating connection, but the run command is attempting to use
random ports on localhost to attain another connection. This fails as
the localhost’s are not the same.

When you say you want to run lldb over ssh, do you mean run “lldb-server”

Is there really an issue with saying these are both lldb? Seems like
my statements were unambiguous without noting a distinction.

remotely and then have a local LLDB connect to that lldb-server?

That’s the usual way lldb is used remotely, so I don’t really get that
lldb does not, in some cases, mean lldb-server.

I am trying to figure out what exactly you would like to see happen here. There are many people that run ssh to open a terminal to another computer and run lldb on the command line on that remote machine, hence why I was trying to clarify what you are are asking for with regards to ssh.

You are looking to avoid “lldb-server” from having to bind to port 0 and then tell you which port it was actually bound to?

This is indeed a feature of lldb-server and not lldb, so I don’t
really get your confusion when I claim lldb has a feature that’s
entirely implemented with regard to lldb-server.

Again, trying to understand exactly what you are asking for any not sure why the snippy answer.

Is there a platform, preferably real time, for lldb support?

This is a really important question of mine! If we can only go back
and forth on email I’d like ppl to make a best effort to lessen the
effects this has. One tip I have, even for when ppl are chatting, is
to phrase questions in the form of an answer… Don’t ask yes/no
questions or even questions with say less than 7 answers. Instead
provide bullet points of all the possible answers, one at a time.
There are many good reasons for everyone doing this, but here the most
relevant is to avoid round-trips that are expansive.

LLDB does have an IRC channel on the LLVM IRC chatbot named “lldb”. Many LLDB developers do hang out on this chat and can provide some realtime answers for LLDB and lldb-server.

Can you answer your question twice? Once If I claim that “Yes” I do
plan on having lldb choose a port to listen on and have ssh forward
that port and again for if I answer “No”.

I hope you can appreciate that having ssh forward a handful of ports
for a single service is not optimal. I’d actually do better by
writing another set of client-server applications that tunnel the IPC
over ssh… Something I’d only do if the program was closed source.
As lldb is not, the obvious path forward is to re-implement the lldb
IPC so it’s more friendly to ssh.

We are happy to help make remote debugging better and more friendly to ssh. We don’t have a lot of ssh expertise in the LLDB core group of developers as far as I know, so if you have expertise here and can add ways for us to be better in that respect, we would love to see any patches. See more detailed solution suggestion below on where I think this would fit the best in LLDB.

One might ask why ssh, the basic answer is I don’t want to open
*another port on my remote host… Even if I did I’d still have the
same problem, a random port would fail to connect(this time because of
a firewall). The main answer is, without ssh, lldb is limited to
running on local or VPN networks. I’d rather use ssh than configure a
VPN for this one use case.

  • lldb doesn’t sound like something one would want to host, even if
    connections were blocked from everywhere “else.”

Now I’m attempting forward error correction by guessing where this
topic could lead. I would be willing to expand the network code to
include domain sockets, to replace the whole idea of using, IMHO
barbaric, port numbers. This work could potentially include direct
support for ssh.

lldb-server is not limited to port connections. LLDB has a plug-in mechanism for communications that is URL based. For example, if you are going to launch lldb-server on a remote system and specify a port number of 1234, you would connect to it with:

(lldb) process connect connect://remote.foo.com:1234

“process connect” uses a communication plug-in that is specified by the URL for the first argument: “connect://”. Each communication plug-in built into lldb and lldb-server registers a URL with the LLDB codebase so it can be easily found and used. “connect://” will get you a direct TCP connection using IPV4 of IPV6 depending on the rest of the URL. There isn’t a valid URL for direct TCP connections so we made one up (“connect://”).

The reason I mention this is this is where I would envision any ssh support could easily be added to LLDB. So my theory would be we would add a new “ssh” communication plug-in, if that makes sense as I have no SSH experience other than using a lot of command line tools that can use it, and then when connecting to the remote via ssh we would end up using:

(lldb) process connect ssh://

where is anything we need to make the connection, like “remote.foo.com”. We would probably add a new option to lldb-server to specify it would use ssh as well when it was launched like:

remote$ lldb-server gdbserver --ssh – /bin/ls -lAF

Does that seam feasible?

I understand that this would likely be a breaking
change, is there version negotiation?

I don’t think this needs to be a breaking change. For lldb-server, we just need a way to move packets over a communication layer, so we should be able to use our communication plug-ins to do this.

Another example of using different communication plug-ins is when lldb-server is running on the same machine as lldb: lldb calls socketpair() and gets 2 socket file handles that are connected and dupes one of them into the lldb-server using posix_spawn attributes. When we actually spawn the process we send the file descriptor that it was duped to down as an argument: “–fd 12”. So when lldb-server is starting up, it will parse the options, and discover it is just needs to use the specified file descriptor.

Greg Clayton

I'm looking for support running lldb over ssh. I can forward the
originating connection, but the run command is attempting to use
random ports on localhost to attain another connection. This fails as
the localhost's are not the same.

When you say you want to run lldb over ssh, do you mean run "lldb-server"

Is there really an issue with saying these are both lldb? Seems like
my statements were unambiguous without noting a distinction.

"Remote debugging" can mean different things to different people. Please assume good faith. I'm sure Greg asked this question because he was genuinely not sure what you meant, and not just to annoy you.

As lldb is not, the obvious path forward is to re-implement the lldb
IPC so it's more friendly to ssh.

I've been wanting to do something like that for a while, since the current design has a very 1970 (the decade FTP was invented) feel to it. However, the issue never came up on the projects that I worked on, so I couldn't find time to do that.

The way this currently works is that lldb sends a packet like "qSpawnGdbServer", which causes lldb-server platform to spawn a gdb server (either lldb-server gdbserver, or debugserver) and return the port number it is listening on. One way to change that would be to have lldb open *another* connection to the same lldb-server, and then issue something like "qExecGdbserver" (a new command). This command would cause the platform to exec (without forking) the debug server and pass the already established connection to it (something which we already support).

Then there would be no need for two ports, as both connections would be established through the same one.

Now I'm attempting forward error correction by guessing where this
topic could lead. I would be willing to expand the network code to
include domain sockets, to replace the whole idea of using, IMHO
barbaric, port numbers. This work could potentially include direct
support for ssh. I understand that this would likely be a breaking
change, is there version negotiation?

Direct support for ssh might be interesting as well, though I am not sure what exactly would that mean. As for version negotiation, the way that's generally handled is by making a new gdb-remote packet or a "feature" (see https://sourceware.org/gdb/current/onlinedocs/gdb/General-Query-Packets.html#qSupported) and then checking for that.

So, for example, in order to implement my idea, we could have the lldb-server platform send qSpawnGdbServer+ in its qSupported response, and then have lldb key off of that.

cheers,
pavel