LLVM Discourse migration: goals justify means?

Hi all.

As most of us here learned on Jan 7, apparently, we,
the LLVM community, have overwhelmingly supported
the decision to move to Discourse.

It already raises a question as to how said decision was made,
and what exactly said "majority of the community" is.
While it is true that the LLVM RFC process is unclear at best,
in this particular case the problem becomes exceptionally egregious.

While it may be a selection bias, as a data point,
everybody, that i regularly talk to, in #llvm IRC
were just as surprised to learn of said development as I was.

There was no indication on e.g. llvm-dev@,
and in fact the last mention of the migration was:
https://lists.llvm.org/pipermail/cfe-dev/2021-June/068449.html
(over half a year ago!), but even if you just look at said thread,
there were certain comments that weren't addressed, e.g.
https://lists.llvm.org/pipermail/cfe-dev/2021-June/068406.html

Hopefully, the "vote" wasn't held at the discourse itself,
otherwise it very much mirrors certain recent & future world events,
and does not paint the LLVM in a good light.

I'm fearful that the same story is bound to happen yet again
with GitHub Pull Request migration, that all the multitude of complaints
that were received each time they were requested (and that happened
a number of times, hopefully not to exhaust those providing said feedback!)
will be just swept away and ignored, and the switch be pushed through
regardless, in the name of a noble "lowering the barrier of entry" goal.
(There's similar question about discord "RFC")

So the first point I would like to raise is:
such painful, community-wide decisions **can not** be made in secret.
One way or another, it's going to affect every single LLVM developer,
be it one working on the upstream LLVM, or some downstream fork,
or those just wishing to keep up with the project.
**There should be transparency and accountability.**

The second question I would like to raise is:
the blog post claims transparent, first-class email support,
but the mailing list mode can not actually be toggled on.
There is just no such checkbox, unlike some other discourse forum.
For me personally, that is a deal-breaker, and unless I'm able to
keep up to date with the discussions in the lists format,
I'm simply going to stop following discussions, period.

While, I, personally, have not had much hands-on experience with
LLVM's discourse, mainly it's email side, I hear the situation
is not what the blogpost claims it to be, and there are other things
that aren't "just work", and that was known months ago, e.g.:
https://llvm.discourse.group/t/discourse-as-mailing-list-replacement-some-questions/3713/4

Given that the hard switch point of Feb 1'st has already been set,
and is less than a week away, i'd like to hear some clarification
as to what is going on, and strongly recommend doing either of the following:
* STOP migration(s) due to "false start", the end status already being decided
  before the process even begun, and using the process just as a means
  to legalize the decision made beforehand.
* postponing the switch by a month (until March 1'st), or however long needed,
  effectively immediately, in order to make the migration actually possible
  by working out the issues that have come up during the migration.

While what is written above is my personal view on things,
I do **not** believe the said view is unique to me.

What are the foundation's thoughts on this?

Roman

Hi all.

As most of us here learned on Jan 7, apparently, we,
the LLVM community, have overwhelmingly supported
the decision to move to Discourse.

It already raises a question as to how said decision was made,
and what exactly said "majority of the community" is.
While it is true that the LLVM RFC process is unclear at best,
in this particular case the problem becomes exceptionally egregious.

While it may be a selection bias, as a data point,
everybody, that i regularly talk to, in #llvm IRC
were just as surprised to learn of said development as I was.

There was no indication on e.g. llvm-dev@,
and in fact the last mention of the migration was:
https://lists.llvm.org/pipermail/cfe-dev/2021-June/068449.html
(over half a year ago!), but even if you just look at said thread,
there were certain comments that weren't addressed, e.g.
https://lists.llvm.org/pipermail/cfe-dev/2021-June/068406.html

Hopefully, the "vote" wasn't held at the discourse itself,
otherwise it very much mirrors certain recent & future world events,
and does not paint the LLVM in a good light.

I'm fearful that the same story is bound to happen yet again
with GitHub Pull Request migration, that all the multitude of complaints
that were received each time they were requested (and that happened
a number of times, hopefully not to exhaust those providing said feedback!)
will be just swept away and ignored, and the switch be pushed through
regardless, in the name of a noble "lowering the barrier of entry" goal.
(There's similar question about discord "RFC")

So the first point I would like to raise is:
such painful, community-wide decisions **can not** be made in secret.
One way or another, it's going to affect every single LLVM developer,
be it one working on the upstream LLVM, or some downstream fork,
or those just wishing to keep up with the project.
**There should be transparency and accountability.**

The second question I would like to raise is:
the blog post claims transparent, first-class email support,
but the mailing list mode can not actually be toggled on.
There is just no such checkbox, unlike some other discourse forum.
For me personally, that is a deal-breaker, and unless I'm able to
keep up to date with the discussions in the lists format,
I'm simply going to stop following discussions, period.

There seems to be a lot of confusion about what "Mailing List Mode"
is, so let me try to clear this up:

"Mailing List Mode" is a convenient way to watch all Discourse categories
at once. The mailman equivalent of this would be if you subscribed to
all the LLVM mailing lists at the same time.

There is another way to watch all the Discourse categories. You can go
to your notification preferences and add every category (and sub-category)
to the list of categories that you want to watch. Watching a category is
the mailman equivalent of subscribing to a single mailing list.

When you watch a category, you will receive emails for every new post in
the category. Turning "Mailing List Mode" off does not prevent users from
receiving emails for new posts.

This is relevant now because "Mailing List Mode" was recently disabled for
everyone on Discourse. This was necessary because we were running up against
the daily email limits for free accounts on Discourse.

Whether or not the "Mailing List Mode" feature is available it is
not recommended that people use this feature. It is recommended
that instead users watch only the categories that they are interested
in. This will help reduce your own email load and also reduce the load
on the Discourse server (number of emails per month is limited based on
the organization's subscription type).

I hope this helps clear things up.

-Tom

I want to chime in to say that Roman is definitely not alone in his impressions here. I have previously shared my objections to the original proposal, and will not repeat myself.

I don't have the energy to engage in this discussion, and have already decided to put up with this and deal with the fallout. Given that, please do not expect further response from me on this topic.

Philip

(reply inline, in the only way that Outlook can manage ... look for the non-indented text)

    > Hi all.
    >
    > As most of us here learned on Jan 7, apparently, we,
    > the LLVM community, have overwhelmingly supported
    > the decision to move to Discourse.
    >
    > It already raises a question as to how said decision was made,
    > and what exactly said "majority of the community" is.
    > While it is true that the LLVM RFC process is unclear at best,
    > in this particular case the problem becomes exceptionally egregious.
    >
    > While it may be a selection bias, as a data point,
    > everybody, that i regularly talk to, in #llvm IRC
    > were just as surprised to learn of said development as I was.
    >
    > There was no indication on e.g. llvm-dev@,
    > and in fact the last mention of the migration was:
    > https://lists.llvm.org/pipermail/cfe-dev/2021-June/068449.html
    > (over half a year ago!), but even if you just look at said thread,
    > there were certain comments that weren't addressed, e.g.
    > https://lists.llvm.org/pipermail/cfe-dev/2021-June/068406.html
    >
    > Hopefully, the "vote" wasn't held at the discourse itself,
    > otherwise it very much mirrors certain recent & future world events,
    > and does not paint the LLVM in a good light.
    >
    > I'm fearful that the same story is bound to happen yet again
    > with GitHub Pull Request migration, that all the multitude of complaints
    > that were received each time they were requested (and that happened
    > a number of times, hopefully not to exhaust those providing said feedback!)
    > will be just swept away and ignored, and the switch be pushed through
    > regardless, in the name of a noble "lowering the barrier of entry" goal.
    > (There's similar question about discord "RFC")
    >
    > So the first point I would like to raise is:
    > such painful, community-wide decisions **can not** be made in secret.
    > One way or another, it's going to affect every single LLVM developer,
    > be it one working on the upstream LLVM, or some downstream fork,
    > or those just wishing to keep up with the project.
    > **There should be transparency and accountability.**
    >
    > The second question I would like to raise is:
    > the blog post claims transparent, first-class email support,
    > but the mailing list mode can not actually be toggled on.
    > There is just no such checkbox, unlike some other discourse forum.
    > For me personally, that is a deal-breaker, and unless I'm able to
    > keep up to date with the discussions in the lists format,
    > I'm simply going to stop following discussions, period.

    There seems to be a lot of confusion about what "Mailing List Mode"
    is, so let me try to clear this up:

    "Mailing List Mode" is a convenient way to watch all Discourse categories
    at once. The mailman equivalent of this would be if you subscribed to
    all the LLVM mailing lists at the same time.

    There is another way to watch all the Discourse categories. You can go
    to your notification preferences and add every category (and sub-category)
    to the list of categories that you want to watch. Watching a category is
    the mailman equivalent of subscribing to a single mailing list.

    When you watch a category, you will receive emails for every new post in
    the category. Turning "Mailing List Mode" off does not prevent users from
    receiving emails for new posts.

    This is relevant now because "Mailing List Mode" was recently disabled for
    everyone on Discourse. This was necessary because we were running up against
    the daily email limits for free accounts on Discourse.

    Whether or not the "Mailing List Mode" feature is available it is
    not recommended that people use this feature. It is recommended
    that instead users watch only the categories that they are interested
    in. This will help reduce your own email load and also reduce the load
    on the Discourse server (number of emails per month is limited based on
    the organization's subscription type).

The big difference I see between watching a category and mailing list mode is
that watching a category only emails me for *new* posts in that category,
whereas mailing list mode emails me for all replies as well (or at least
that's what I understood the post about disabling it implied). The latter is
really what I want: keep up with all activity the same way I would on the
mailing list, making the transition seamless (which was one of the big selling
points of Discourse, at least as I understood it).

    I hope this helps clear things up.

    -Tom

(reply inline again, unindented)

    (reply inline, in the only way that Outlook can manage ... look for the non-indented text)

        > Hi all.
        >
        > As most of us here learned on Jan 7, apparently, we,
        > the LLVM community, have overwhelmingly supported
        > the decision to move to Discourse.
        >
        > It already raises a question as to how said decision was made,
        > and what exactly said "majority of the community" is.
        > While it is true that the LLVM RFC process is unclear at best,
        > in this particular case the problem becomes exceptionally egregious.
        >
        > While it may be a selection bias, as a data point,
        > everybody, that i regularly talk to, in #llvm IRC
        > were just as surprised to learn of said development as I was.
        >
        > There was no indication on e.g. llvm-dev@,
        > and in fact the last mention of the migration was:
        > https://lists.llvm.org/pipermail/cfe-dev/2021-June/068449.html
        > (over half a year ago!), but even if you just look at said thread,
        > there were certain comments that weren't addressed, e.g.
        > https://lists.llvm.org/pipermail/cfe-dev/2021-June/068406.html
        >
        > Hopefully, the "vote" wasn't held at the discourse itself,
        > otherwise it very much mirrors certain recent & future world events,
        > and does not paint the LLVM in a good light.
        >
        > I'm fearful that the same story is bound to happen yet again
        > with GitHub Pull Request migration, that all the multitude of complaints
        > that were received each time they were requested (and that happened
        > a number of times, hopefully not to exhaust those providing said feedback!)
        > will be just swept away and ignored, and the switch be pushed through
        > regardless, in the name of a noble "lowering the barrier of entry" goal.
        > (There's similar question about discord "RFC")
        >
        > So the first point I would like to raise is:
        > such painful, community-wide decisions **can not** be made in secret.
        > One way or another, it's going to affect every single LLVM developer,
        > be it one working on the upstream LLVM, or some downstream fork,
        > or those just wishing to keep up with the project.
        > **There should be transparency and accountability.**
        >
        > The second question I would like to raise is:
        > the blog post claims transparent, first-class email support,
        > but the mailing list mode can not actually be toggled on.
        > There is just no such checkbox, unlike some other discourse forum.
        > For me personally, that is a deal-breaker, and unless I'm able to
        > keep up to date with the discussions in the lists format,
        > I'm simply going to stop following discussions, period.

        There seems to be a lot of confusion about what "Mailing List Mode"
        is, so let me try to clear this up:

        "Mailing List Mode" is a convenient way to watch all Discourse categories
        at once. The mailman equivalent of this would be if you subscribed to
        all the LLVM mailing lists at the same time.

        There is another way to watch all the Discourse categories. You can go
        to your notification preferences and add every category (and sub-category)
        to the list of categories that you want to watch. Watching a category is
        the mailman equivalent of subscribing to a single mailing list.

        When you watch a category, you will receive emails for every new post in
        the category. Turning "Mailing List Mode" off does not prevent users from
        receiving emails for new posts.

        This is relevant now because "Mailing List Mode" was recently disabled for
        everyone on Discourse. This was necessary because we were running up against
        the daily email limits for free accounts on Discourse.

        Whether or not the "Mailing List Mode" feature is available it is
        not recommended that people use this feature. It is recommended
        that instead users watch only the categories that they are interested
        in. This will help reduce your own email load and also reduce the load
        on the Discourse server (number of emails per month is limited based on
        the organization's subscription type).

    The big difference I see between watching a category and mailing list mode is
    that watching a category only emails me for *new* posts in that category,
    whereas mailing list mode emails me for all replies as well (or at least
    that's what I understood the post about disabling it implied). The latter is
    really what I want: keep up with all activity the same way I would on the
    mailing list, making the transition seamless (which was one of the big selling
    points of Discourse, at least as I understood it).

Correction: it seems like watching a category does email you about replies as
well, which addresses my concern. https://llvm.discourse.group/t/6022/12

        I hope this helps clear things up.

        -Tom

Same here.

I would like to make sure that we separate the two issues here:

  - The decision to move to Discourse.
  - The way in which decisions are made for the project.

There are a lot of problems with LLVM mailing lists. People don't know which lists to post things on, things are cross-posted to cfe-dev and llvm-dev (for example) but not all replies go to both, which makes following discussions difficult because they essentially end up in two forks. Some things only go to llvm-dev, so you have to subscribe to the fire hose and try to skip the 90% of messages that are likely to be irrelevant to you.

Given how low the bar is for the starting point, Discourse seems like it is definitely a less bad solution. I'm even willing to accept that it is the least-bad solution currently available.

That said, I completely agree with the comments by Roman, Philip, and Renato in this thread. This is not the first decision where my perception of the consensus of the broader LLVM community and the consensus of the folks that turn up to Silicon Valley socials have been in opposite directions and the group in the valley's decision has been pushed through with everyone else then having to live with the fallout.

There is a significant need for a more transparent decision process that reflects all stakeholders on multiple axes:

  - Industrial, academic, or individual contributors.
  - Contributors to core LLVM libraries, to tightly coupled components and to largely independent projects.
  - Direct LLVM contributors and downstream consumers.
  - Groups shipping a complete LLVM toolchain and those shipping some LLVM components.

The LLVM Foundation board is heavily skewed in most of these axes and, as a self-selecting entity, is not likely to address this without an intentional policy of doing so and without a broader effort to explicitly engage with the segments of the LLVM community that are not directly represented.

David

Hi Roman - thank you for flagging that mailing list mode was disabled.
I thought things had gone a little quiet!

Firstly, I wanted to explicitly recognise that maintaining or evolving
project infrastructure can be a thankless job (or even worse - it's
easy for it to feel like every action attracts criticism!). I'm
genuinely grateful to everyone who has put time into trying to improve
the way we communicate within LLVM. I was quite happy with mailing
lists, but Discourse with mailing list mode didn't seem to really
degrade my experience in any meaningful way. I'll drop a note on
<https://llvm.discourse.group/t/disabling-site-wide-mailing-list-mode-not-reply-by-email-or-watching-categories-via-email/6022>
about how important mailing list mode is to me.

One suggestion for future such decisions would be to more explicitly
follow something based around LLVM's contentious decision making
process <https://github.com/llvm/llvm-www/blob/main/proposals/LP0001-LLVMDecisionMaking.md>.

Best,

Alex

I also found this decision to be really surprising and disappointing.

I was surprised to hear a decision had been made at all, because the
last public discussion about the switch was over six months ago, there
was no clear consensus that I could see, and there were several
unanswered questions and concerns raised on that thread. To me, this
says that consensus was either not formed or was formed via a process
that is completely opaque to me as a code owner and active
contributor. It also gives me the impression that asking questions or
providing feedback during an infrastructure RFC is largely a waste of
time. Frankly, I find our RFC process to be untenable when it comes to
decisions that impact the whole community; we have no idea what
consensus looks like so the end result is continually "do it and the
community will adapt or the people who disagree will leave."

I was also surprised to get an email after 2am on a Friday night (East
Coast, US) telling me that the switch was happening and I should sign
up to Discourse within two days or risk disruption. Coupled with the
lack of communication that any decision was even being considered, I
thought this could have been handled better with a more reasonable
timeline.

Unfortunately, I don't see a good path forward from here. We now have
Discourse, people are using it and folks who are happy about it will
very reasonably wish to continue to do so, and anyway, we have no good
(trivial) way to migrate back to a mailing list without losing the
information now contained only on Discourse. We now also have people
who are not able to use Discourse for whatever variety of reasons. So
we've fractured our communication channels and caused some hard
feelings, again. However, unlike with Discord, the decision to move to
Discourse impacts everyone in the community, not just the people
opting to use an alternate means of ad hoc discussion, because our
current RFC process now means you have to be on Discourse. I think
pausing the timeline to give the infrastructure team the time and
space to work out the usability issues with the service is a
reasonable measure, but if the answer winds up being "sorry, we can't
do that" (as happened a few times with the switch from Bugzilla to
GitHub Issues), I don't know what we do aside from accepting it as the
new reality and potentially losing input from more members of the
community as a result.

So I very much share Roman's concern about the discussions around code
review tools, as that's another "impacts everyone in the community"
decision where judging consensus will be hard. Because of that, and
orthogonal to the discussions about Discourse, I would very much
appreciate it if we could have some idea of how consensus is being
judged for decisions that impact the entire community. I do not have
faith that the current process is working or tenable, and it seems
critical (to me, at least) to solve that before making further
infrastructure decisions.

~Aaron

I also found this decision to be really surprising and disappointing.

I was surprised to hear a decision had been made at all, because the
last public discussion about the switch was over six months ago, there
was no clear consensus that I could see, and there were several
unanswered questions and concerns raised on that thread. To me, this
says that consensus was either not formed or was formed via a process
that is completely opaque to me as a code owner and active
contributor. It also gives me the impression that asking questions or
providing feedback during an infrastructure RFC is largely a waste of
time. Frankly, I find our RFC process to be untenable when it comes to
decisions that impact the whole community; we have no idea what
consensus looks like so the end result is continually "do it and the
community will adapt or the people who disagree will leave."

I was also surprised to get an email after 2am on a Friday night (East
Coast, US) telling me that the switch was happening and I should sign
up to Discourse within two days or risk disruption. Coupled with the
lack of communication that any decision was even being considered, I
thought this could have been handled better with a more reasonable
timeline.

Unfortunately, I don't see a good path forward from here. We now have
Discourse, people are using it and folks who are happy about it will
very reasonably wish to continue to do so, and anyway, we have no good
(trivial) way to migrate back to a mailing list without losing the
information now contained only on Discourse. We now also have people
who are not able to use Discourse for whatever variety of reasons. So
we've fractured our communication channels and caused some hard
feelings, again. However, unlike with Discord, the decision to move to
Discourse impacts everyone in the community, not just the people
opting to use an alternate means of ad hoc discussion, because our
current RFC process now means you have to be on Discourse. I think
pausing the timeline to give the infrastructure team the time and
space to work out the usability issues with the service is a
reasonable measure, but if the answer winds up being "sorry, we can't
do that" (as happened a few times with the switch from Bugzilla to
GitHub Issues), I don't know what we do aside from accepting it as the
new reality and potentially losing input from more members of the
community as a result.

Hi,

Do you have more information about who is unable to use Discourse and why?

-Tom

I would like to make sure that we separate the two issues here:

  - The decision to move to Discourse.
  - The way in which decisions are made for the project.

There are a lot of problems with LLVM mailing lists. People don't know
which lists to post things on, things are cross-posted to cfe-dev and
llvm-dev (for example) but not all replies go to both, which makes
following discussions difficult because they essentially end up in two
forks. Some things only go to llvm-dev, so you have to subscribe to the
fire hose and try to skip the 90% of messages that are likely to be
irrelevant to you.

Given how low the bar is for the starting point, Discourse seems like it
is definitely a less bad solution. I'm even willing to accept that it
is the least-bad solution currently available.

About Discourse itself:

Given the number of people who seem to have opted for email notification,
and that we're bumping up against some limit imposed by the provider,
it's clear that email has advantages over a forum. I've been dutifully
trying to engage with Discourse through the web UI, and it has serious
drawbacks compared to a mail client, for people who are mostly reading
it and not posting a lot. For example:
- AFAICT the only way to mark a post read, is to read it. With a mail
  client, I can mark it read, or just delete it from the mail folder.
  Having to read a post takes extra time; Discourse doesn't think you've
  read a post unless you spend enough time with it visible in your browser.
- My mail client shows many threads in a small amount of screen real estate.
  Discourse gives each thread a relatively huge amount of space, which
  again reduces the efficiency of looking at What's New Today.
- Reading a topic/post and then going back to the list of unread stuff is
  not particularly efficient; page loading feels slower than with gmail,
  for example. There are other poor choices that are harder to describe.
- I'm never 100% sure that I've actually seen everything new in the web UI;
  Discourse seems to distinguish an unread new topic from an unread post,
  presenting them separately, which just feels weird and counter-intuitive.

All this means the time I spend on Discourse is *higher* than what I spent
reading the dev lists.

I do agree with David Chisnall's remark about cross-posting, which the
unified Discourse eliminates; arguably posts might still align with
multiple categories, but many categories are no longer artificially
constrained to 'llvm-dev' or 'cfe-dev' so I think that the situation
there has improved overall.

Clearly, the most effective way to handle reading all the traffic is
- mute the categories you're pretty sure you'll never care about
- set up email notifications
- go back to reading the traffic efficiently in the email client.
And at least you can be sure you've been notified about everything,
rather than hoping you found everything in the web UI.

That said, I completely agree with the comments by Roman, Philip, and
Renato in this thread. This is not the first decision where my
perception of the consensus of the broader LLVM community and the
consensus of the folks that turn up to Silicon Valley socials have been
in opposite directions and the group in the valley's decision has been
pushed through with everyone else then having to live with the fallout.

Regarding the decision-making process:

I think it's not really fair to chalk it up to "the consensus of the
folks that turn up to Silicon Valley socials." Early on they were small
convivial affairs, but the last few I went to were mob scenes full of
people only vaguely associated with LLVM, drawn by the prospect of free
food and drink. And of course there haven't been any at all, since the
start of the pandemic.

That said, I don't doubt that there's a relatively small circle of folks
centered around the board membership who talk to each other a lot and
come to their own perception of the consensus. This is a pretty natural
process, I'm sure well understood in psychology/sociology, and it takes
genuine, conscious and persistent effort to overcome it. Having the
good of the community at heart is not enough.

There is a significant need for a more transparent decision process that
reflects all stakeholders on multiple axes:

  - Industrial, academic, or individual contributors.
  - Contributors to core LLVM libraries, to tightly coupled components
and to largely independent projects.
  - Direct LLVM contributors and downstream consumers.
  - Groups shipping a complete LLVM toolchain and those shipping some
LLVM components.

The LLVM Foundation board is heavily skewed in most of these axes and,
as a self-selecting entity, is not likely to address this without an
intentional policy of doing so and without a broader effort to
explicitly engage with the segments of the LLVM community that are not
directly represented.

Hear, hear. This has been my chief problem with the board ever since
the Foundation asserted itself into existence. The board's selection
process is not transparent, and the board is probably too small to
adequately represent all the diverse stakeholder axes that you've
described. There's a structural deficiency, here. There is plenty
of research into nonprofit and open-source governance structures, and
the Foundation would benefit from taking a hard look at it.

--paulr

I want to chime in to say that Roman is definitely not alone in his impressions here. I have previously shared my objections to the original proposal, and will not repeat myself.

I don't have the energy to engage in this discussion, and have already decided to put up with this and deal with the fallout. Given that, please do not expect further response from me on this topic.

I would like to make sure that we separate the two issues here:

- The decision to move to Discourse.
- The way in which decisions are made for the project.

There are a lot of problems with LLVM mailing lists. People don't know which lists to post things on, things are cross-posted to cfe-dev and llvm-dev (for example) but not all replies go to both, which makes following discussions difficult because they essentially end up in two forks. Some things only go to llvm-dev, so you have to subscribe to the fire hose and try to skip the 90% of messages that are likely to be irrelevant to you.

Given how low the bar is for the starting point, Discourse seems like it is definitely a less bad solution. I'm even willing to accept that it is the least-bad solution currently available.

I don't get this point. On the one hand you seem to say things get lost (e.g., answers) as people pick a list from the (limited number of) options, on the other hand you argue the trees are hidden in the woods, is that a reasonable summary?

With discourse you have two choices, neither of which solves both problems you bring up. IMHO, the choices make one of the problems worse:
A) I have to use mailing list mode (somewhat equivalent to watching 20+ categories and sub-categories) to find the things I want to see for sure. We are back to the fire hose, email filters, and skipping things, except that we now have 1 mailing list rather than 10. (Arguably, we could have merged llvm-dev and cfe-dev to avoid cross posting as well.)
B) I only watch my categories, sub-categories, and tags and miss out on everything relevant not posted in there. While there might not be cross posting, we should reasonably assume that 10 people posting about the same topic will not all choose the same categories, sub-categories, and tags if there is any overlap between them. As an example, "How to offload to AMD GPUs with OpenMP" is either a beginner question, an OpenMP one, or an AMDGPU one, not to mention it might end up in the generic category or any other one if the message also asks about optimizing the code, e.g., with MLIR.

I agree with the rest below though.

~ Johannes

This is relevant now because “Mailing List Mode” was recently disabled for
everyone on Discourse. This was necessary because we were running up against
the daily email limits for free accounts on Discourse.

!!!

Finding out that all my notifications have been disabled right before the planned transition date is EXTREMELY unwelcome news!

In anticipation of the transition, I had already gotten everything setup for myself – with mailing list mode enabled, email filters to sort things the way I like, etc.

To hear that option is now gone and I need to figure out a new workflow, and furthermore, that apparently the last N days of conversations are now missing for me (since, the outgoing emails were disabled with no warning!), is very disheartening.

Whether or not the “Mailing List Mode” feature is available it is
not recommended that people use this feature. It is recommended
that instead users watch only the categories that they are interested
in. This will help reduce your own email load and also reduce the load
on the Discourse server (number of emails per month is limited based on
the organization’s subscription type).

Email is critical – for me, for others. If discourse is not viable (or, not economically viable) when too many people want to be emailed “too much”, then I don’t think this transition as planned should be undertaken. Maybe a self-hosted discourse instance, or something else…

> I also found this decision to be really surprising and disappointing.
>
> I was surprised to hear a decision had been made at all, because the
> last public discussion about the switch was over six months ago, there
> was no clear consensus that I could see, and there were several
> unanswered questions and concerns raised on that thread. To me, this
> says that consensus was either not formed or was formed via a process
> that is completely opaque to me as a code owner and active
> contributor. It also gives me the impression that asking questions or
> providing feedback during an infrastructure RFC is largely a waste of
> time. Frankly, I find our RFC process to be untenable when it comes to
> decisions that impact the whole community; we have no idea what
> consensus looks like so the end result is continually "do it and the
> community will adapt or the people who disagree will leave."
>
> I was also surprised to get an email after 2am on a Friday night (East
> Coast, US) telling me that the switch was happening and I should sign
> up to Discourse within two days or risk disruption. Coupled with the
> lack of communication that any decision was even being considered, I
> thought this could have been handled better with a more reasonable
> timeline.
>
> Unfortunately, I don't see a good path forward from here. We now have
> Discourse, people are using it and folks who are happy about it will
> very reasonably wish to continue to do so, and anyway, we have no good
> (trivial) way to migrate back to a mailing list without losing the
> information now contained only on Discourse. We now also have people
> who are not able to use Discourse for whatever variety of reasons. So
> we've fractured our communication channels and caused some hard
> feelings, again. However, unlike with Discord, the decision to move to
> Discourse impacts everyone in the community, not just the people
> opting to use an alternate means of ad hoc discussion, because our
> current RFC process now means you have to be on Discourse. I think
> pausing the timeline to give the infrastructure team the time and
> space to work out the usability issues with the service is a
> reasonable measure, but if the answer winds up being "sorry, we can't
> do that" (as happened a few times with the switch from Bugzilla to
> GitHub Issues), I don't know what we do aside from accepting it as the
> new reality and potentially losing input from more members of the
> community as a result.
>

Hi,

Do you have more information about who is unable to use Discourse and why?

I can speak for myself -- I'm still evaluating it.

Discourse comes with a Terms of Service and Privacy Policy I have to
evaluate and decide whether I can accept. For example, the TOS
requires me to accept that California law is the way all disputes are
to be resolved and places restrictions on what legal processes I'm
allowed to follow
(https://llvm.discourse.group/tos#heading--disputes). Further, the TOS
imposes requirements on me even after I leave the community
(https://llvm.discourse.group/tos#heading--termination). Both the ToS
and the privacy policies can change at any time, without my consent,
and if I no longer agree to those changes, I'm blocked from
communicating with the open source community.

(Note, many of the provisions of the Discourse TOS are also things we
want enforced for the mailing list, such as the LLVM Code of Conduct
policy. I don't have concerns about those aspects of the TOS as
they're perfectly reasonable for the health of the community.)

Given that email already works for me, requiring me to switch to
Discourse is an imposition. It's one I'm certainly willing to consider
spending the time on, but given how many issues people have already
identified with using Discourse from an email-only workflow, I have
less confidence that what I'll end up with will meet my needs. But I
have to accept legal risk in order to even try to find that out, and
if trying it out discovers that the service isn't usable for me, I
still carry legal obligations.

~Aaron

I agree with virtually everything else that has been said in this thread, so I'll limit this to saying things that I haven't seen said yet.

If you look at the history of the big infrastructure changes LLVM has made in the recent past, there's a worrying trend. The first big change I'm thinking of was the move from SVN to git via github. The discussion period for this change was quite long (several years), but the actual migration I remember as being relatively smooth. More recently, we had the move from Bugzilla to Github issues. The discussion period was similarly long, but the migration was far from smooth: the final notification (including things contributors needed to do) seemed to come out of nowhere, with short timetables, and over a holiday week, and the actual conversion process ran into several technical issues (to be fair, many of them were not easily foreseeable).

Now we have the migration to Discourse, where the previous discussion was arguably more contentious than the bug move and seemed to be left in a "no consensus" state. And again we have a very-little-notice announcement of the move, including a late Friday night or early Saturday morning announcement on a holiday weekend. And again, there are technical issues--the "mailing list mode" feature. However, this one really ought to have been foreseen: the amount of emails that llvm mailing lists send out a day should be *really* easy to estimate, so how is it a surprise that we're using too many emails?

The worrying thing is the extrapolation to the "next" infrastructure change, the move to PRs... which is the most contentious of the lot, with several contributors outright saying that it may cause them to stop contributing altogether. The infrastructure process clearly *isn't* working well right now, and I think we need to step back and fix that process before risking contributor loss.

I originally wasn't going to bring this up, but I think the decision to disable "mailing list mode" absolutely needs to part of the "what went wrong" postmortem. There may be a good reason why the problem of LLVM discourse sending too many emails wasn't foreseeable beforehand, but I'm not seeing it right now--it's important to understand where the blind spots of the infrastructure group exist right now. But the communication of the disabling of this feature really leaves something to be desired: it was announced on Discourse, after it had been disabled, so that everybody who was solely relying on it for email *never saw they had been cut off*. I myself only found out about this because it was mentioned in the IRC channel.

In a broader sense, I want to part with this observation. In my experience, large projects develop a kind of "in-group", a set of people who need to be interacted with to get things done in the project. Of the projects I've worked with, LLVM has had the most opaque "in-group", in the sense that it's difficult for a beginner (or even more experienced contributors) to figure out who you need to get to review a patch, or when you've got enough agreement on an RFC to move forward with implementation. This is a bigger issue with LLVM in general, but the risk with respect to infrastructure in particular is that I am extremely worried that the LLVM infrastructure group is pushing away much or all of the "in-group", and that has incumbent risks for the future health of the project as a whole.

To be clear, email notifications are not disabled. The only thing that has been changed
is how you enable them in your Discourse preferences.

-Tom

As I replied on Discourse, this is still not working for me, even following all the instructions on the thread and other docs.

Let me just clarify a point here… The bugzilla migration process was very hard.

All those years waiting for the majority of people in the community were actually spent by Anton K painstakingly solving every issue that was raised.

The number of back and forth emails between Anton and Github people and the level of support he got from them is appalling.

Most people probably had forgotten about it all when Anton finally broke through and saw the light at the end of the tunnel.

So, while I agree with everything else you said above and below, the bugzilla migration wasn’t quite like the others.

What that means to the Phab->PR move is left as an exercise to the reader…

Roman,

I would really appreciate if you would ask questions about the migration instead of making assumptions, accusations, and demands. Those involved in the migration are happy to answer them.

In any best laid out plan, there are unexpected things that pop up. In this particular case, we found out that mailing list mode was generating a ton of email that caused us to go over our current email limits and impact the sites functionality. We asked for the limit to be overridden until the migration was complete and that was not possible. In an effort to keep things functioning, we disabled “mailing list mode”. This is not a feature that was present before. There was no button you could click on lists.llvm.org that automatically subscribed you to all the mailing lists. Most people are not subscribed to every mailing list.

I’ve been working on LLVM for over 15 years and maintaining the LLVM lists for the majority of that time. I understand the importance of email for people in our community. That importance has been made very clear in the many discussions about Discourse (one the lists, in working groups, in round tables, at dev mtgs, to me, to the board as a whole).

There are ample ways for people to get involved:

  1. Participate in the Infrastructure Working Group
  2. Read the LLVM Foundation board meeting minutes to see what the board is talking about. Ask questions if you want to know more
  3. Email the LLVM Foundation board directly or use the mailing list (now Discourse)
  4. Email me personally. Ask me for a video chat or phone call. I am here to answer questions and to take feedback.

I hope that you can have some patience and compassion for people who are doing the migration. I am very sorry that we had to take this step, but it does not mean its permanent and it does not prevent you from using email.

-Tanya

I would like to make sure that we separate the two issues here:

- The decision to move to Discourse.
- The way in which decisions are made for the project.

There are a lot of problems with LLVM mailing lists. People don't know
which lists to post things on, things are cross-posted to cfe-dev and
llvm-dev (for example) but not all replies go to both, which makes
following discussions difficult because they essentially end up in two
forks. Some things only go to llvm-dev, so you have to subscribe to the
fire hose and try to skip the 90% of messages that are likely to be
irrelevant to you.

Given how low the bar is for the starting point, Discourse seems like it
is definitely a less bad solution. I'm even willing to accept that it
is the least-bad solution currently available.

About Discourse itself:

Given the number of people who seem to have opted for email notification,
and that we're bumping up against some limit imposed by the provider,
it's clear that email has advantages over a forum. I've been dutifully
trying to engage with Discourse through the web UI, and it has serious
drawbacks compared to a mail client, for people who are mostly reading
it and not posting a lot. For example:
- AFAICT the only way to mark a post read, is to read it. With a mail
client, I can mark it read, or just delete it from the mail folder.
Having to read a post takes extra time; Discourse doesn't think you've
read a post unless you spend enough time with it visible in your browser.
- My mail client shows many threads in a small amount of screen real estate.
Discourse gives each thread a relatively huge amount of space, which
again reduces the efficiency of looking at What's New Today.
- Reading a topic/post and then going back to the list of unread stuff is
not particularly efficient; page loading feels slower than with gmail,
for example. There are other poor choices that are harder to describe.
- I'm never 100% sure that I've actually seen everything new in the web UI;
Discourse seems to distinguish an unread new topic from an unread post,
presenting them separately, which just feels weird and counter-intuitive.

All this means the time I spend on Discourse is *higher* than what I spent
reading the dev lists.

I do agree with David Chisnall's remark about cross-posting, which the
unified Discourse eliminates; arguably posts might still align with
multiple categories, but many categories are no longer artificially
constrained to 'llvm-dev' or 'cfe-dev' so I think that the situation
there has improved overall.

Clearly, the most effective way to handle reading all the traffic is
- mute the categories you're pretty sure you'll never care about
- set up email notifications
- go back to reading the traffic efficiently in the email client.
And at least you can be sure you've been notified about everything,
rather than hoping you found everything in the web UI.

That said, I completely agree with the comments by Roman, Philip, and
Renato in this thread. This is not the first decision where my
perception of the consensus of the broader LLVM community and the
consensus of the folks that turn up to Silicon Valley socials have been
in opposite directions and the group in the valley's decision has been
pushed through with everyone else then having to live with the fallout.

Regarding the decision-making process:

I think it's not really fair to chalk it up to "the consensus of the
folks that turn up to Silicon Valley socials." Early on they were small
convivial affairs, but the last few I went to were mob scenes full of
people only vaguely associated with LLVM, drawn by the prospect of free
food and drink. And of course there haven't been any at all, since the
start of the pandemic.

That said, I don't doubt that there's a relatively small circle of folks
centered around the board membership who talk to each other a lot and
come to their own perception of the consensus. This is a pretty natural
process, I'm sure well understood in psychology/sociology, and it takes
genuine, conscious and persistent effort to overcome it. Having the
good of the community at heart is not enough.

There is a significant need for a more transparent decision process that
reflects all stakeholders on multiple axes:

- Industrial, academic, or individual contributors.
- Contributors to core LLVM libraries, to tightly coupled components
and to largely independent projects.
- Direct LLVM contributors and downstream consumers.
- Groups shipping a complete LLVM toolchain and those shipping some
LLVM components.

The LLVM Foundation board is heavily skewed in most of these axes and,
as a self-selecting entity, is not likely to address this without an
intentional policy of doing so and without a broader effort to
explicitly engage with the segments of the LLVM community that are not
directly represented.

Hear, hear. This has been my chief problem with the board ever since
the Foundation asserted itself into existence. The board's selection
process is not transparent, and the board is probably too small to
adequately represent all the diverse stakeholder axes that you've
described. There's a structural deficiency, here. There is plenty
of research into nonprofit and open-source governance structures, and
the Foundation would benefit from taking a hard look at it.

Paul,

I have spent a great deal of time researching how nonprofits are structured and I am well versed in the different ways. There are many open source nonprofits that are member based and many that are not. Many that are member based are actually 501c6 which is very different. One way is not the best way for all. Given the role of this board, we felt that this was the best approach.

I have also answered questions about the board election process and we have tried to make the process as transparent as we can but also maintaining confidentiality of those who applied but did not get selected (we could change that). If you have further questions or clarifications, please let me know.

Thanks,
Tanya