Hi all,
I've been hacking on a virtual file system for clang and this seemed like
the right time to start getting some feedback. Briefly, the idea is to
interpose a virtual file system layer between llvm::sys::fs and Clang's
FileManager that allows us to mix virtual files/links/etc. with the 'real'
file system in a general way.Motivation
The use case that I have in mind is to allow a build system to provide a
file/directory layout to clang without having to construct it "for real" on
disk. For example, I am building a project containing two modules, and
module A imports module B. It would be useful if we could bundle up the
headers and module.map file for module B from wherever they may exist in
the source directories and provide clang with a notion of the file layout
of B _as it will be installed_. Right now, I know of two existing ways to
accomplish this:1) Copy the files into a fake installation during build. This is
unsatisfying, as it requires tracking and copying files every time they are
changed. And diagnostics, debug info, etc. do not refer back to the
original source file.2) Header maps provide this functionality for header files. However,
header maps work from within the header search logic, which does not extend
well to other kinds of files. They are also insufficient for bundling
modules, as clang needs to see the framework for the module laid out as
described in the module map.Description
The idea is to abstract the view of the file system using an
AbstractFileSystem class that mimics the llvm::sys::fs interface:class AbstractFileSystem {
public:
class Status { ... };
// openFileForRead
// status, and maybe 'stat'
// recursive iteration
// getBuffer
// getBufferForOpenFile
// recursive directory iteration
};that can be implemented by any concrete file system that we want. Clients
that want to lookup files/directories (notably the FileManager) can operate
on an AbstractFileSystem object. One leaky part of this interface is that
clients that need to care whether they are working with a 'real path' will
need to explicitly ask for it. For example, debug information and
diagnostics should ask for the real path. I suggest putting that
information into the AbstractFIleSystem::Status object.Some non-goals (at least for a first iteration):
1) File system modification operations (create_directory, rename, etc.).
Clients will continue to use the real file system for these operations,
and we don't intend to detect any conflicts this might create.
2) Completely virtual file buffers that do not exist on disk.One implementation of the AbstractFileSystem interface would be a wrapper
over the 'real' file system, which would just defer to llvm::sys::fs.class RealFileSystem : public AbstractFileSystem { ... };
And to provide a unified view of the file system, we can create an overlay
file system, similar to [1].class OverlayFileSystem : public AbstractFileSystem { ... };
This reminds me a lot of Plan 9. Plan 9 may be a good inspiration for the
primitive API operations (see also the 9p protocol).
-- Sean Silva
