I’m in the process of writing an editor based upon libclang, and a major issue I’m hitting is the inability to cancel long running tasks. The primary tasks are parsing and creation of auto-complete results.
Is there already functionality to do this that I’ve missed?
If not, I would like to implement this as its a requirement to get good responsiveness out of my editor. Has there been any prior discussions on how this should be implemented?
Have you tried running libclang on a separate thread?
Yes it already runs on a separate thread; however when its known the result is no longer necessary (for example the user changed the cursor location - negating the need for autocomplete at the previous location) the separate thread still runs consuming resources.
On slower devices like laptops these operations can take a long time. Even if we are willing to accept the waste of cycles (and battery life!), the processor has a limited number of cores and responsiveness will be poor if too many “zombie” jobs are still running.
I’m willing to add this functionality and submit it upstream if necessary, but prior to doing so I wanted to reach out to current developers. It would be unfortunate to invest significant effort only to find my patch can’t be accepted for an avoidable reason.
John,
It should be fairly straightforward for you to implement this behavior in your application using std::future::wait_for.
Jon
Hi Jonathan,
I don’t understand how std::future::wait_for would allow me to cancel a running libclang operation.
I’m not asking how to do this asynchronously - I already have that working. I need a way to stop performing work when the results become obsolete. I could always shutdown the thread its on but that would leave things in an undefined state.
I’ll add my +1 to it being unfortunate that libclang operations cannot be canceled (living in a code base where some TUs take > 30 seconds to parse).
I’m pretty confident there is no current way to do it, so this week I’ll start on my own implementation. For reasons I don’t quite understand libclang likes to spawn jobs onto seperate threads (and therefore ignores my thread priorities…) which will make this somewhat more difficult.
I think the sanest way is to pass a bool FCheckAbort() style function pointer as an optional parameter to jobs which may run a long time. I’ll have to do some work to marshal this across thread boundaries. Either that or remove the spawned thread entirely.
I’m pretty confident there is no current way to do it, so this week I’ll start on my own implementation. For reasons I don’t quite understand libclang likes to spawn jobs onto seperate threads (and therefore ignores my thread priorities…) which will make this somewhat more difficult.
I think the sanest way is to pass a bool FCheckAbort() style function pointer as an optional parameter to jobs which may run a long time. I’ll have to do some work to marshal this across thread boundaries. Either that or remove the spawned thread entirely.
The problem is that most of the time is spent throughout various functions of Sema in clang, so the only way I see is to have every function in Sema check for abortion, which sounds like it would horribly clutter the code…
The problem is that most of the time is spent throughout various functions of Sema in clang, so the only way I see is to have every function in Sema check for abortion, which sounds like it would horribly clutter the code…
I don’t think it needs to be that extreme. Once every 100ms would be sufficient for my purposes which would drastically reduce the number of checks needed. My own profiles are also showing a fair amount of time in the Lex code as well. I’m still learning the code but I suspect it would be feasible to check in-between major phases.
The problem is that most of the time is spent throughout various functions of Sema in clang, so the only way I see is to have every function in Sema check for abortion, which sounds like it would horribly clutter the code…
I don’t think it needs to be that extreme. Once every 100ms would be sufficient for my purposes which would drastically reduce the number of checks needed. My own profiles are also showing a fair amount of time in the Lex code as well. I’m still learning the code but I suspect it would be feasible to check in-between major phases.
I suspect it won’t, but feel free to give it a try - for really long TUs we see all of the time spent in Sema trying to parse C++, and it’s basically a flat profile.
A check in Sema at the start of processing each FunctionDecl might be enough. Even long functions with a lot of template instantiations probably don't take more than 100ms in most code. Unless you're trying to use libclang with the raytracer written in C++ templates or something equally silly (boost, perhaps)...
David
The problem is that most of the time is spent throughout various functions of Sema in clang, so the only way I see is to have every function in Sema check for abortion, which sounds like it would horribly clutter the code…
I don’t think it needs to be that extreme. Once every 100ms would be sufficient for my purposes which would drastically reduce the number of checks needed. My own profiles are also showing a fair amount of time in the Lex code as well. I’m still learning the code but I suspect it would be feasible to check in-between major phases.
I suspect it won’t, but feel free to give it a try - for really long TUs we see all of the time spent in Sema trying to parse C++, and it’s basically a flat profile.
A check in Sema at the start of processing each FunctionDecl might be enough. Even long functions with a lot of template instantiations probably don’t take more than 100ms in most code. Unless you’re trying to use libclang with the raytracer written in C++ templates or something equally silly (boost, perhaps)…
That would definitely help already. Great idea.
Sorry, for some reason I thought std::future had a way to kill the task. The idea was to loop on the future's wait_for, giving it some small interval, then to kill the future if the result is no longer needed.
Jon
Thanks for the help everyone. I’ll check back in a few days when I’ve had a chance to prototype it.
One other question, does anybody know the rationale for libclang spawning new threads to do the work on? I plan to change this at least for my own builds, but if its something that could be changed upstream that would be much better.
Spawning a separate thread causes it to ignore the thread priority of the thread making the original call. It also has non-trivial overhead.