How to use MCJIT by default for a target

Hi, I'm looking into making MCJIT the default for the ARM-Linux-EABI target for lli.

(WHY? the old JIT doesn't really work on ARM, and in particular there are some regression tests of interesting things -- such as profiling -- that fail purely because the default old JIT doesn't work. So a consensus has been reached that the best way forward is to try to make the MCJIT the default for ARM-Linux-EABI and see if that's workable.)

Looking at the lli source code, there's the option UseMCJIT that defaults to false. You can also use -use-mcjit, -use-mcjit=true, -use-mcjit=false. Things would be fine if we could make the option default architecture dependent. However, these defaults are all set-up BEFORE we commit to what architectural triple we're using (as the target triple can be specified via argument). If we try to mess with the option based on the triple after it's gone through parsing rather than the default, we can't distinguish between "false (from default)" and "false (set deliberately)". All this does assume, of course, that it's important to provide a way for the user to stop lli using the MCJIT; I think that's probably wise but maybe it's unnecessary.

Anyway, this sounds like making a default choice based on a target-triple is something that probably ought to be done "using canonical idiom" so it's easily grokked rather than using an ad-hoc approach. Does anyone have any thoughts?

Many thanks,
David Tweed

-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

Hi David,

in particular there are some regression tests of interesting things
-- such as profiling -- that fail purely because the default old JIT
doesn't work.

I've actually got LLVM currently compiling within an ARM QEmu install to look at an assert within the ARM JIT code. Profiling tests that I submitted a few weeks ago are failing on ARM build-bots (assert in lli), perhaps this is what you are referring to.

If I can verify that the tests pass when using MCJIT is there any interest in fixing it? Obviously if they still break with MCJIT there is something which definitely needs fixed.

All I really know so far is that when using profiling metadata a pseudo-instruction is generated which the ARM JIT does not support.

Regards,
Alastair.

It seems to me that MCJIT would be a nice default on the platforms that support it. On the other hand, I don't like the idea of the default behavior being platform dependent.

Perhaps the biggest obstacle to changing the default is that it would have complicated implications for the automated tests. The testing of JIT and MCJIT is already a bit of a mess, and changing the default to MCJIT for some platforms while leaving it as JIT for other platforms would make things considerably worse. I think at this point we'd need to consider cleaning up the JIT/MCJIT nightly tests as a pre-requisite to changing the default, regardless of other considerations.

-Andy

Hi Andrew,

in particular there are some regression tests of interesting things
-- such as profiling -- that fail purely because the default old JIT
doesn't work.

I've actually got LLVM currently compiling within an ARM QEmu install to
look at an assert within the ARM JIT code. Profiling tests that I
submitted a few weeks ago are failing on ARM build-bots (assert in lli),
perhaps this is what you are referring to.

If I can verify that the tests pass when using MCJIT is there any
interest in fixing it? Obviously if they still break with MCJIT there
is something which definitely needs fixed.

Yep, this is the issue. Running on an ARM pandaboard I can confirm that it's
due
to issues with the old JIT in general which are fixed in MCJIT, and nothing
to do
with the profiling code being tested, as discussed in this thread:

http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20120910/150406.
html

As you can see, I'm very interested in getting this fixed, but it's one of
those situations where there's not really any nice mechanism in place for
the
obvious solution, which is to use MCJIT when needed without requiring the
user (or for regression tests, tester) to do anything special.

Any ideas anyone has would be great!

Cheers,
David

I agree that we should clean up JIT/MCJIT tests, then change to MCJIT
until it pass those tests.

Regards,
chenwj

That's a big idea, how this has come up is that there are some tests (in particular profiling) which AREN'T testing JITing itself, but need to use the JIT in order to run stuff. On ARM these don't run with the old JIT but do with the MCJIT. We could just disable the tests, but then we're losing coverage of the profiling stuff due to a "configuration limitation"... Particularly if more such tests are likely to be added, it'd be preferable to figure a way to "use the most robust JIT on the current platform" and run them.

Cheers,
Dave

Hi David,

One possible way to solve the problem is to add an extra substitution argument to the lli invocation in the test files, then in the lit config files define that substitution to be "--use-mcjit" if the host OS architecture is ARM and empty otherwise.

It's not a particularly elegant solution, but I think it would work.

-Andy

Anything to remove the current duplication in test code between
ExecutionEngine & ExecutionEngine/MCJIT is a step forward. While that
mess is tidied up, we may as well factor in a solution to allow
setting default JITs. IMO target support for MCJIT would improve
faster if at some point the transition was made completely. It's going
to happen eventually, and the sooner for those targets with sufficient
support the better, in terms of user testing and patches.

I think this would mean that the target default JIT options should end
up living in lli rather than in the lit config files, overridable of
course.

Amara

Some time ago an extension to lit that would allow multiple embedded subtests in a file with different run lines was agreed upon. This was intended to solve the duplication in the ExecutionEngine tests. One run line would test JIT, one would test MCJIT.

I don't know if the lit extension was committed, but I know the tests haven't been updated to use it.

-Andy

> I agree that we should clean up JIT/MCJIT tests, then change to MCJIT
>until it pass those tests.

That's a big idea, how this has come up is that there are some tests (in particular profiling) which AREN'T testing JITing itself, but need to use the JIT in order to run stuff. On ARM these don't run with the old JIT but do with the MCJIT. We could just disable the tests, but then we're losing coverage of the profiling stuff due to a "configuration limitation"... Particularly if more such tests are likely to be added, it'd be preferable to figure a way to "use the most robust JIT on the current platform" and run them.

  I see test cases under Analysis/Profiling/ directory just call `lli`
WITHOUT "-use-mcjit" option. Do you mean on ARM, lli use MCJIT by
default already? I thought we have to explicitly use "-use-mcjit"
option to make lli use MCJIT...

  As Eli said in [1], ExecutionEngine/* and ExecutionEngine/MCJIT/*
are same test cases for old JIT and MCJIT separately. I guess we can
choose the most robust JIT base on their result.

Regards,
chenwj

[1] http://lists.cs.uiuc.edu/pipermail/llvmdev/2012-June/050878.html

Yep, this is the issue. Running on an ARM pandaboard I can confirm that it's
due
to issues with the old JIT in general which are fixed in MCJIT, and nothing
to do
with the profiling code being tested, as discussed in this thread:

http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20120910/150406.
html

  I don't see this patch get committed. Do you want to ping it? :wink:

Regards,
chenwj

Or we can make Eli's lit SUBTEST patch in, so that we don't have to
duplicate test cases for old JIT and MCJIT.

Regards,
chenwj

[1]
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20120521/143186.html

I think the response said that unconditionally using MCJIT everywhere was not considered viable. On the other hand, it looks like converting ARM to use MCJIT is something people will need persuading about. I'll try to find time to have a look at Andrew Kaylor's suggestion of adding a per-triple jitVariety variable to lit-cfg.

Yeah, that patch is what I was referring to before. It's definitely needed to clean up the ExecutionEngine tests. The profiling tests can probably be made to do something reasonable without it.

I'm not sure what Eli's priorities are these days. That patch may need a new owner.

-Andy