[RFC][offload][AMDGPU] Device-specific runtime defaults for the OpenMP AMDGPU plugin

Hello Community,

We would like to propose a per-device configuration which will be used during AMDGPUDeviceTy construction for selecting the device-specific default values.

Motivation

The AMDGPU plugin currently uses one default value for each runtime option. However, the best default can differ between GPU architectures. The runtime should select appropriate defaults for each GPU so users need environment variables only when they explicitly want to override those defaults.

Our downstream implementation already maintains per-device defaults for runtime optimizations. Moving this logic upstream would reduce downstream divergence and lower the associated maintenance cost.

Upstreaming this feature would provide:

  • Better defaults for each architecture.
  • Unchanged user control through environment variables.
  • One place for hardware-specific runtime configuration.
  • Less downstream code and fewer merge conflicts.

Proposed behavior

During device construction, the plugin would:

  1. Query device properties.
  2. Select a device configuration.
  3. Apply its values only when the user did not set the corresponding environment variable.

Initial scope

We plan to first onboard the configure for LIBOMPTARGET_AMDGPU_USE_MULTIPLE_SDMA_ENGINES

  • The upstream plugin already supports this option and its implementation.
  • Explicit environment settings retain their current behavior.
  • Unknown devices retain conservative defaults.
  • No environment variable is removed or renamed.

Other defaults that downstream currently maintains should remain outside the scope of the initial patch. Once the dependent components are upstream, these defaults can use the same configuration feature.

Thanks in advance for your suggestions.

1 Like

This is totally reasonable, but I think we need to refactor how we handle a lot of these options. Historically, these were internal and primarily controlled through environment variables. Now that we have the offloading library we really should factor most of these into a per-device config and factor out a lot of the environment variables into this.

2 Likes

I think this is the right thing to do to get device-specific settings.

Agreed that we should try to make these things into some sort of config file that we ship. But, I do believe that we want escape hatches for users to override every option of they feel a need for it without changing a config file.

2 Likes

+1

We could go very generic, if we want to, or have predetermined options per architecture.
We load this only once and ignore most of it, so, initialization cost is not really my main concern.

Thanks @Kewen12 for driving this. I would suggest you pick 3 values that “are different” and make a horizontal prototype. Then you push that as draft PR and start adding them as real PRs one by one. Benefit is that we can see how the “generalization” will make everything fit without being too generic.
If you find splitting is not necessary, we can go with the first and add them one by one after. Simple PRs, simple reviews, fast upstream time.

1 Like

Thank you all for sharing the valuable insights! I will follow the suggested direction and create a prototype so that we can evaluate and determine the next steps.