It appears that TOSA MLIR validation pass lack sufficient visibility to users. I’d like to provide a brief introduction to it. TOSA validation pass is designed to verify a TOSA MLIR graph is compliant to a given TOSA specification profile, extension, level and specific operation compliance.
Use --tosa-validate to enable TOSA MLIR validation pass, i.e. without passing in this option, all checks within the pass are disabled.
In this article, I will focus on the two main components:
- Level check validation
- Profile-based operation validation
Level check validation
Validate if operator parameters are within specification for the given level.
There are two levels defined in TOSA 1.0.0 draft specification.
Available level option:
=8k - Ranges are expected to be sufficient for applications with frame \
sizes up to 8K.
=none - Allows the full range of arguments specified by the operations \
according to the operation data type
How to specify Tosa Level to the validation pass
// Set level to 8k.
$mlir-opt ... --tosa-validate="level=8k"
// Set level to none.
$mlir-opt ... --tosa-validate="level=none"
Profile-based operation validation
This feature was introduced in this pull request and is part of the existing validation pass, which already includes level checks and other spec-specific validations. It is designed to identify the profile and extension requirements of each operation in the TOSA MLIR graph input and verify whether these profiles are implemented and enabled in the target.
Available profiles and extensions:
Profiles = [ pro_int | pro_fp ]
Extensions = [ int16 | int4 | bf16 | fp8e4m3 | fp8e5m2 | fft | variable | controlflow | dynamic | doubleround | inexactround ]
For detail, please see TOSA 1.0.0 draft specification
Note that some of extension works has not been upstreamed yet.
How to plug the Tosa validation pass in the passes pipeline but turn off the particular profile-based validation by default:
// The first parameter means none profile enabled. And the second parameter means none extension enabled.
auto validationOptions = tosa::TosaValidationOptions{{"none"}, {"none"}, ...});
pm.addPass(tosa::createTosaValidation(validationOptions));
How to specify the profile/extension capability of target to profile-based validation:
The profile-based validation is ONLY effective when strict-op-spec-alignment is provided. Other checks, including level checks, constant tensor validation, and operation-specific verifications, are enabled by default in the validation pass. However, profile-based operation validation may be too strict for experimentation. To manage this, define strict-op-spec-alignment option to control. This allows users to disable all exacting (strict) validation via the command line, even if the custom profile or extension configuration is enabled by default in the pass.
// If `strict-op-spec-alignment` is not provided, the profile-based validation
// remains disabled. Ignoring the default profile or extension configuration
// and the configuration passed in.
$mlir-opt ... -tosa-validate
$mlir-opt ... -tosa-validate="profile=pro_int"
// Enable Integer Profile only
$mlir-opt ... -tosa-validate="strict-op-spec-alignment profile=pro_int"
// Enable Floating-Point Profile only
$mlir-opt ... -tosa-validate="strict-op-spec-alignment profile=pro_fp"
// Enable Integer and Floating-Point Profile, and fp8 extension support
$mlir-opt ... -tosa-validate="strict-op-spec-alignment profile=pro_int,pro_fp \
extension=fp8e4m3,fp8e5m2"
// Enable all profiles and extensions
$mlir-opt ... -tosa-validate="strict-op-spec-alignment profile=pro_int,pro_fp \
extension=int16,int4,bf16,fp8e4m3,fp8e5m2,fft,variable,controlflow,dynamic,\
doubleround,inexactround"
The overall workflow of TOSA profile-based validation:
