does vFile:Open actually work?

The comments in File.h say:

// NB this enum is used in the lldb platform gdb-remote packet
// vFile:open: and existing values cannot be modified.
enum OpenOptions {
eOpenOptionRead = (1u << 0), // Open file for reading
eOpenOptionWrite = (1u << 1), // Open file for writing
eOpenOptionAppend =
(1u << 2), // Don’t truncate file when opening, append to end of file

And In GDBRemoteCommunicationServerCommon.cpp it says:

uint32_t flags = packet.GetHexMaxU32(false, 0);
if (packet.GetChar() == ‘,’) {
mode_t mode = packet.GetHexMaxU32(false, 0600);
FileSpec path_spec(path);
// Do not close fd.
auto file = FileSystem::Instance().Open(path_spec, flags, mode, false);

But in the GDB documentation it says:

@node Open Flags
@unnumberedsubsubsec Open Flags
@cindex open flags, in file-i/o protocol

All values are given in hexadecimal representation.

O_RDWR 0x2
O_CREAT 0x200
O_TRUNC 0x400
O_EXCL 0x800
@end smallexample

Does vFile:Open actually work? Are there any tests that cover it?

Yeah, this is a bug in lldb's implementation of vFile:open. lldb talks to lldb-server (in platform mode) so it will work with itself, but it will not interoperate with any other implementations. That's the down side to having the client and server literally built from the same sources. :slight_smile:

I have a small self-contained platform implementation I wrote locally from the specification, and I stumbled across the bug last winter. We'll need to add a packet so lldb can request standards-correct vFile:open: behavior, and fall back to its original implementation if it that is not understood for a while so we interoperate with existing platforms in the wild.

A similar type of bug is lldb's incorrect implementation of the A packet, see 42471 – LLDB debugserver - GDB remote protocol 'A' packet format is not spec-compliant - Spencer had the good suggestion of creating a protocol fixes packet to query for these so additional ones can be added in the future, he suggested:

Request: "qProtocolFixes:fix;…" where each 'fix' is one of the following strings:
  - "GDBFlagsInvFileOpen"
  - "GDBBaseInAPacket"
  - ...
Unknown strings are acceptable, but ignored. If a fix string is not present, it is assumed that that fix is not present.

Response: "fix;…", same definition as 'fix' above.

I have a little TODO on myself to do this (<rdar://problem/46788934>).