[Windows] Cleaning up dialog box suppression code.

We have multiple different places where we handle dialog box
suppression to various degrees. I recently discovered another place we
need to do it, and so I wanted to unify all of them.

We currently have most of this suppression logic behind the
LLVM_DISABLE_CRASH_REPORT environment variable. I would like to make
it unconditional. I don't believe we have any users that desire the
dialog box behavior, as all of our tools are used via the command line
and can report crashes via stderr.

I wanted to check with other Windows users/vendors for exactly which
behavior they wanted.

The case where I need to add it now is the lto shared library. The CRT
crash reporting state doesn't seem to be replicated across DLL
boundaries, so the main application can't actually control this
currently. Should we unconditionally disable here? or add an API to
control it?

- Michael Spencer

For my purposes, I don't need it to be conditionally controlled. That
being said, it may make sense to conditionally control it if we ever
decide to code sign the executable and sign up for WinQual, since we
may *want* the dialogs in that case (since it will allow people to
send crash reports that we could then obtain). But if we have *one*
location for this API, we can solve that issue another day.

~Aaron

In theory, I am happy to suppress more dialog boxes more often. In practice, suppressing the dialog often prevents our stack dumper from kicking in. If you can try to preserve any stack dumping we have, then yeah, let’s turn them off unconditionally. :slight_smile:

I am concerned about disabling crash diagnostics inside lib LTO. We don’t generally call the signal setup code outside main. I’d like to understand why you need to change global settings like this from a library before doing it.

In theory, I am happy to suppress more dialog boxes more often. In practice, suppressing the dialog often prevents our stack dumper from kicking in. If you can try to preserve any stack dumping we have, then yeah, let’s turn them off unconditionally. :slight_smile:

I am concerned about disabling crash diagnostics inside lib LTO. We don’t generally call the signal setup code outside main. I’d like to understand why you need to change global settings like this from a library before doing it.

It’s actually not global. It’s per image. Setting it in the dll doesn’t change what happens when the main app crashes. Same for the other direction.
My main concern is a user of lib lto that doesn’t have stderr hooked up anywhere. If we don’t have that, then i don’t really see a problem.

Are you sure it's per image, or just per CRT? I'm imagining that your DLL
uses one CRT DLL and the exe uses another.

I am concerned about disabling crash diagnostics inside lib LTO. We don't
generally call the signal setup code outside main. I'd like to understand
why you need to change global settings like this from a library before
doing it.

On the PS4 toolchain we're using LTO.dll called from an LTO process via
LoadLibrary. Despite setting all of the various flags you need to do to
prevent pop-up boxes on Windows from the calling process it's having no
effect in suppressing the evil modal boxes from crashes/aborts coming from
inside the dll. We also verified with a very simple testcase of a process
loading a dll that aborts that the settings do not appear to apply across
dlls (in either direction). This area is not particularly well documented
though if anyone has a better way to achieve this.

The question is whether there is ever going to be any case where having a
modal dialog pop up is actually desirable behavior? Certainly one bad
crash bug in clang or LTO would be capable of bringing the entire PS4
compiler testing pool to a grinding halt until someone manually went into
each of our boxes and clicked a button to close the box (and the same would
happen for any of our users who hit one on their own automated game
builds). For that reason we've been disabling all the crash dialogs
unconditionally for PS4 since the beginning.

On a related note, I'm sitting on a local change that I'll start a
discussion on soon that will automatically write Windows minidumps without
any form of dialog box required so that we can do post-mortem debugging
when just the stack-trace isn't enough. This is a more intrusive change
though, so I imagine that one will have to be opt-in via a config setting
though.

-Greg

OK. This seems like a reasonable thing to do in llvm/tools/lto under #ifdef LLVM_ON_WIN32, given that that code is designed to loaded as a plugin.