We’ve started using LNT and found it very handy. However, we found that LNT web UI is missing some features to help our use-cases.
We’d like to add them (and already started doing so) but it’s hard to add them without breaking existing features/workflows because while LNT has some python CLI test coverage the dynamic Web part (mainly JS UI) is not really tested.
So our question is – are there any plans to introduce testing to this part of LNT? Were there any discussions on this topic? Maybe the LNT developers has already decided on the testing framework or have some candidates in mind?
We’d want to help implement this so it’s important to use the same tools.
For checking HTML output: There are some tests in lnt/tests/server/ui/V4Pages.py (and similar) that look at the produced html output. I also recommend installing pytidylib and running the testsuite with --use-tidylib to catch additional html errors.
There was this PR to use Jasmine:
But I think it did not make it into tree. Kristof, any update on the issues you were encountering? We could move forward with that. I think we were pretty agnostic to which framework to use as long as it was not hard to install.
JS Testing would be *really* good. I have debugged a lot of issues recently that ended up in the JS.
I think I’ve listed all my reasons for going with Jasmine as a framework on that review, so please have a look there.
After getting one issue fixed in jasmine-py (see review for details), I remember I found that the tests could be run, but had some troubles with some combinations of web browsers (see last comment on the review).
I’m afraid I’ve been kept busy with lots of other stuff since, so haven’t followed up since then.
Danila: if you’d like to review/try out/improve/make D20161 committable: I’d very much welcome that!
I think a major first step to achieve that would be to try out D20161 on your system and check which browsers it works with and which ones it doesn’t. We could document the combinations we’ve found to work and which ones we’ve found to have issues.
Sorry for the delayed response. Thanks for pointing this review out to me. I’ve tried it out and managed to run the tests using phantomjs and recent firefox (had no luck with 52 esr) with minimal modifications (just to adjust to updated code). Although I’ve also had to use 2.99 versions since I’ve encountered some issues with 3.x which I’m still investigating. I can submit the updated patch once I’ve figured it out.
However, if I understood correctly, this will only cover unit testing, while I’d like to test the UI/page layout as well. At least something basic, like that the page loads, shows expected controls and diffs. Has anyone looked at this side of things or have any experience with testing it? I’ve looked at some frameworks like Galen, but they’ve shared similar selenium/browser/webdriver compatibility quirkiness, although they’ve generally worked fine with the latest browser versions.
By the way, I’m not a web developer myself (seems to be a common theme) so I’m hesitant to make the final call. Any advice/direction in that regard would be appreciated.
I don’t really have much to add. I noticed requests-html is using pyppeteer for page rendering. That is chrome only and does not directly provide a testing interface. I’d be hesitant to resort to any image diffing.
I’d be interested to hear what else is out there. I know some people like Selenium.