LLDB Website

Hey Tanya,

Thanks again for migrating the LLDB website so it is generated with Sphinx!

I made a change yesterday that hasn’t been propagated yet. It looks like it might have something to do with http://lists.llvm.org/pipermail/www-scripts/2019-April/007524.html.

Also, as the result of this change the following two links are broken:

https://lldb.llvm.org/cpp_reference/

https://lldb.llvm.org/python_reference/

Could we make the script generate those two folders as well? The corresponding CMake target are lldb-cpp-doc and lldb-python-doc.

Thank you,
Jonas

PS: I’ve included lldb-dev in CC so everyone knows we’re working on the missing documentation.

Friendly ping.

There’s a bunch of people that are annoyed by the missing documentation. I’ve already addressed most of the other comments about the broken URLs and missing top level links, but unfortunately that doesn’t take effect because the website isn’t updating.

Anytime new targets are added, the script has to be modified. Is there a way you can put them all under a top level html target? Or is there a reason not to?

-Tanya

Jonas,

Ignore what I said before as these do need to be separate targets. It appears the new targets are running doxygen. This isn’t something we typically do as a post commit hook since it takes awhile. I’ll need to do this via the doxygen nightly script. Any concerns?

-Tanya

Hey Tanya,

Jonas,

Ignore what I said before as these do need to be separate targets. It appears the new targets are running doxygen. This isn’t something we typically do as a post commit hook since it takes awhile. I’ll need to do this via the doxygen nightly script. Any concerns?

That sounds perfect. Can we still do the regular website post commit?

-Tanya

Thank you!

Hey Tanya,

Jonas,

Ignore what I said before as these do need to be separate targets. It appears the new targets are running doxygen. This isn’t something we typically do as a post commit hook since it takes awhile. I’ll need to do this via the doxygen nightly script. Any concerns?

That sounds perfect. Can we still do the regular website post commit?

Yes, so it will do docs-lldb-html on every commit.

So I am able to generate the cpp reference docs: https://lldb.llvm.org/cpp_reference/index.html

However, the main website links to https://lldb.llvm.org/cpp_reference/html/index.html. Do you want the html in that url? I can change the alias. We strip for other doxygen.

As for python docs, what is required to build those? It’s not showing up as a target for me.

Thanks,
Tanya

Hey Tanya,

Jonas,

Ignore what I said before as these do need to be separate targets. It appears the new targets are running doxygen. This isn’t something we typically do as a post commit hook since it takes awhile. I’ll need to do this via the doxygen nightly script. Any concerns?

That sounds perfect. Can we still do the regular website post commit?

Yes, so it will do docs-lldb-html on every commit.

Perfect!

So I am able to generate the cpp reference docs: https://lldb.llvm.org/cpp_reference/index.html

However, the main website links to https://lldb.llvm.org/cpp_reference/html/index.html. Do you want the html in that url? I can change the alias. We strip for other doxygen.

Let’s keep it without the html. I’ll update a link on the website and add a redirect.

As for python docs, what is required to build those? It’s not showing up as a target for me.

This is probably because you don’t have epydoc installed (sudo pip install epydoc).
I think you’ll have to re-run cmake after for it to pick it up. The corresponding target should then be lldb-python-doc.

https://lldb.llvm.org/cpp_reference/index.html

Thanks,
Tanya

Thanks again for doing this.

Cheers,
Jonas

Hey Tanya,

Jonas,

Ignore what I said before as these do need to be separate targets. It appears the new targets are running doxygen. This isn’t something we typically do as a post commit hook since it takes awhile. I’ll need to do this via the doxygen nightly script. Any concerns?

That sounds perfect. Can we still do the regular website post commit?

Yes, so it will do docs-lldb-html on every commit.

Perfect!

So I am able to generate the cpp reference docs: https://lldb.llvm.org/cpp_reference/index.html

However, the main website links to https://lldb.llvm.org/cpp_reference/html/index.html. Do you want the html in that url? I can change the alias. We strip for other doxygen.

Let’s keep it without the html. I’ll update a link on the website and add a redirect.

As for python docs, what is required to build those? It’s not showing up as a target for me.

This is probably because you don’t have epydoc installed (sudo pip install epydoc).
I think you’ll have to re-run cmake after for it to pick it up. The corresponding target should then be lldb-python-doc.

https://lldb.llvm.org/cpp_reference/index.html

Well installing epydoc did the trick, but I don’t think the doxygen script is the right place for this target. I have not dug into it yet but it appears to require some LLVM libraries and is building those. I’m letting it finish to verify it builds but I’ll have to sort out the best way of doing this on the server. We have other scripts that generate other documentation that build parts of LLVM. Ideally, I would want to leverage that and reduce build times.

-Tanya

Hey Tanya,

Jonas,

Ignore what I said before as these do need to be separate targets. It appears the new targets are running doxygen. This isn’t something we typically do as a post commit hook since it takes awhile. I’ll need to do this via the doxygen nightly script. Any concerns?

That sounds perfect. Can we still do the regular website post commit?

Yes, so it will do docs-lldb-html on every commit.

Perfect!

So I am able to generate the cpp reference docs: https://lldb.llvm.org/cpp_reference/index.html

However, the main website links to https://lldb.llvm.org/cpp_reference/html/index.html. Do you want the html in that url? I can change the alias. We strip for other doxygen.

Let’s keep it without the html. I’ll update a link on the website and add a redirect.

As for python docs, what is required to build those? It’s not showing up as a target for me.

This is probably because you don’t have epydoc installed (sudo pip install epydoc).
I think you’ll have to re-run cmake after for it to pick it up. The corresponding target should then be lldb-python-doc.

https://lldb.llvm.org/cpp_reference/index.html

Well installing epydoc did the trick, but I don’t think the doxygen script is the right place for this target. I have not dug into it yet but it appears to require some LLVM libraries and is building those. I’m letting it finish to verify it builds but I’ll have to sort out the best way of doing this on the server. We have other scripts that generate other documentation that build parts of LLVM. Ideally, I would want to leverage that and reduce build times.

Yeah, the annoying thing about the Python documentation is that it builds the C++ API, then runs swig to generate the Python wrapper, and finally generates the docs from that.
I wonder if we can just use the static bindings that are checked-in instead. I will look into that later today/tomorrow.

Hey Tanya,

Jonas,

Ignore what I said before as these do need to be separate targets. It appears the new targets are running doxygen. This isn’t something we typically do as a post commit hook since it takes awhile. I’ll need to do this via the doxygen nightly script. Any concerns?

That sounds perfect. Can we still do the regular website post commit?

Yes, so it will do docs-lldb-html on every commit.

Perfect!

So I am able to generate the cpp reference docs: https://lldb.llvm.org/cpp_reference/index.html

However, the main website links to https://lldb.llvm.org/cpp_reference/html/index.html. Do you want the html in that url? I can change the alias. We strip for other doxygen.

Let’s keep it without the html. I’ll update a link on the website and add a redirect.

As for python docs, what is required to build those? It’s not showing up as a target for me.

This is probably because you don’t have epydoc installed (sudo pip install epydoc).
I think you’ll have to re-run cmake after for it to pick it up. The corresponding target should then be lldb-python-doc.

https://lldb.llvm.org/cpp_reference/index.html

Well installing epydoc did the trick, but I don’t think the doxygen script is the right place for this target. I have not dug into it yet but it appears to require some LLVM libraries and is building those. I’m letting it finish to verify it builds but I’ll have to sort out the best way of doing this on the server. We have other scripts that generate other documentation that build parts of LLVM. Ideally, I would want to leverage that and reduce build times.

Yeah, the annoying thing about the Python documentation is that it builds the C++ API, then runs swig to generate the Python wrapper, and finally generates the docs from that.
I wonder if we can just use the static bindings that are checked-in instead. I will look into that later today/tomorrow.

Right, so the reason is that we don’t have the static bindings on llvm.org (we have them for swift-lldb on GitHub).
Maybe we should check them in upstream too? That’s something the community will have to weigh in on…

            Hey Tanya,

                Jonas,

                Ignore what I said before as these do need to be
                separate targets. It appears the new targets are
                running doxygen. This isn’t something we typically do
                as a post commit hook since it takes awhile. I’ll
                need to do this via the doxygen nightly script. Any
                concerns?

            That sounds perfect. Can we still do the regular website
            post commit?

            Yes, so it will do docs-lldb-html on every commit.

        Perfect!

            So I am able to generate the cpp reference docs:
            https://lldb.llvm.org/cpp_reference/index.html

            However, the main website links to
            https://lldb.llvm.org/cpp_reference/html/index.html. Do
            you want the html in that url? I can change the alias. We
            strip for other doxygen.

        Let's keep it without the html. I'll update a link on the
        website and add a redirect.

            As for python docs, what is required to build those? It's
            not showing up as a target for me.

        This is probably because you don't have `epydoc` installed
        (sudo pip install epydoc).
        I think you'll have to re-run cmake after for it to pick it
        up. The corresponding target should then be `lldb-python-doc`.

        https://lldb.llvm.org/cpp_reference/index.html

        Well installing epydoc did the trick, but I don’t think the
        doxygen script is the right place for this target. I have not
        dug into it yet but it appears to require some LLVM libraries
        and is building those. I’m letting it finish to verify it builds
        but I’ll have to sort out the best way of doing this on the
        server. We have other scripts that generate other documentation
        that build parts of LLVM. Ideally, I would want to leverage that
        and reduce build times.

    Yeah, the annoying thing about the Python documentation is that it
    builds the C++ API, then runs swig to generate the Python wrapper,
    and finally generates the docs from that.

It should be possible to solve this by tweaking the dependency graph a bit. There's no fundamental reason why you need to build anything in order to run swig. It is purely a textual step -- it ingests header files and interface definitions and spits out python and cpp files. The inputs are present as static checked in source, so the swig step could theoretically be the very first build command that we run.

    I wonder if we can just use the static bindings that are checked-in
    instead. I will look into that later today/tomorrow.

Right, so the reason is that we don't have the static bindings on llvm.org <http://llvm.org> (we have them for swift-lldb on GitHub).
Maybe we should check them in upstream too? That's something the community will have to weigh in on...

I think it would be good to avoid that...

pl

That's the issue - lldb-python-doc depends on liblldb. From docs/CMakeLists.txt:

if(EPYDOC_EXECUTABLE)
  find_program(DOT_EXECUTABLE dot)
    if(DOT_EXECUTABLE)
      set(EPYDOC_OPTIONS ${EPYDOC_OPTIONS} --graph all --dotpath ${DOT_EXECUTABLE})
    endif()
    set(DOC_DIR "${CMAKE_CURRENT_SOURCE_DIR}/doc")
    file(MAKE_DIRECTORY "${DOC_DIR}")
    #set(ENV{PYTHONPATH} ${CMAKE_CURRENT_BINARY_DIR}/../../../lib/python2.7/site-packages)
    add_custom_target(lldb-python-doc
      ${EPYDOC_EXECUTABLE}
      --html
      lldb
      -o ${CMAKE_CURRENT_BINARY_DIR}/python_reference
      --name "LLDB python API"
      --url "http://lldb.llvm.org"
      ${EPYDOC_OPTIONS}
      DEPENDS swig_wrapper liblldb
      WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/../../../lib${LLVM_LIBDIR_SUFFIX}/python2.7/site-packages
      COMMENT "Generating LLDB Python API reference with epydoc" VERBATIM
    )
endif(EPYDOC_EXECUTABLE)

I’ve put up a patch to make it possible to generate the python reference without building lldb at all: https://reviews.llvm.org/D61216

PS: The website isn’t updating anymore, is that because of the python reference generation?

I’ve merged the aforementioned patch.

Tanya, can you give generating the python docs another shot?

Thanks,
Jonas

I will give this a shot. I did remove the changes before to prevent any issue.

-Tanya

Hey Tanya,

It appears the website is still stuck. It hasn’t picked up my changes from earlier this week. Please let me know if there’s anything I can do to help.

Thanks,
Jonas

Friendly ping.

I’m not sure it is working. To clarify, nothing in LLVM should be compiled to build the python docs correct?

So I shouldn’t see this?

Scanning dependencies of target liblldb_exports
[ 0%] Creating export file for liblldb
[ 0%] Built target liblldb_exports
Scanning dependencies of target LLVMDemangle
[ 0%] Building CXX object lib/Demangle/CMakeFiles/LLVMDemangle.dir/Demangle.cpp.o
[ 0%] Building CXX object lib/Demangle/CMakeFiles/LLVMDemangle.dir/ItaniumDemangle.cpp.o
[ 0%] Building CXX object lib/Demangle/CMakeFiles/LLVMDemangle.dir/MicrosoftDemangle.cpp.o
[ 0%] Building CXX object lib/Demangle/CMakeFiles/LLVMDemangle.dir/MicrosoftDemangleNodes.cpp.o
[ 0%] Linking CXX static library …/libLLVMDemangle.a
[ 0%] Built target LLVMDemangle
Scanning dependencies of target LLVMSupport
[ 0%] Building CXX object lib/Support/CMakeFiles/LLVMSupport.dir/AArch64TargetParser.cpp.o
[ 0%] Building CXX object lib/Support/CMakeFiles/LLVMSupport.dir/ARMTargetParser.cpp.o
[ 0%] Building CXX object lib/Support/CMakeFiles/LLVMSupport.dir/AMDGPUMetadata.cpp.o
[ 0%] Building CXX object lib/Support/CMakeFiles/LLVMSupport.dir/APFloat.cpp.o
[ 0%] Building CXX object lib/Support/CMakeFiles/LLVMSupport.dir/APInt.cpp.o

Do I need any additional config options?

Thanks,
Tanya

Ignore this. svn wasn’t actually updating the src tree. It works! I just need doxygen script to finish and it will be confirmed tonight.

-Tanya

Hey Tanya,

That’s great. I see the Python documentation is online now!

Unfortunately it appears that the Sphinx part still isn’t updating. I pushed a bunch of changes last week and none have made it to the homepage yet. I checked the www-scripts mailing list but I don’t see any failures for LLDB. Do you know what’s up here?

Thanks,
Jonas