I would like to move the Python sources currently under
mlir/lib/Bindings/Python to a new
mlir/python directory. I’ve been meaning to simplify this directory tree for a while and now that we have real users, I wanted to ask before just sending a patch. I’d like to make some more comprehensive enhancements to docs, packaging, etc, and I would like to do that in the context of a directory structure we intend to keep for a while.
We would still keep the native libraries in the
mlir/lib trees, as with other C/C++ library code, and we would continue to just build the libraries to the right location in the project-wide
python directory in the build tree. Build rules and artifacts for generating python code (i.e. for dialects, transforms, etc) would move to the new location. Python tests would move to a new
test/python directory tree and be organized by Python package.
mlir/lib directory is really for C/C++ code. When we started the project, the only pure Python code we had was an
__init__.py to set things up and we just put that there as well. Since then, the pure Python code has become its own center of mass, providing public APIs and tools for the project, and it is obnoxious to have it so deeply nested in a C/C++ oriented tree. All of these sources are already copied/symlinked to the
python directory in the build and install trees, so this is more just aligning the source tree.
As a project that also has a fair amount of Python code, NPComp lays things out this way, and removing the three levels of nesting and presenting a more canonical Python source tree definitely helps the ergonomics of working on it.