Repository navigation
Finalize the semantics of storage_min, storage_max #1406
Description
The goal of this ticket is to investigate whether or not the quantized type specification should limit the allowed values of storage_min and storage_max. Currently, the allowed values are limited to [min_value(storage_type, max_value(storage_type], but this limit may not be followed by some use cases.
Please refer to ref to follow the discussion.
update
The current spec expects default values of min_value(storage_type) and max_value(storage_type) for storage_min and storage_max resp., if they are not explicitly provided. Moreover, the specification documents the following general constraint on storage_min and storage_max
`min_value(storage_type) <= storage_min < storage_max <= max_value(storage_type)`.
Metadata
Metadata
Assignees
Type
Projects
- Status
- Status
Activity
[-]Figuring out the values for `storage_min`, `storage_max`[/-][+]Figuring out the limits for `storage_min`, `storage_max`[/+][-]Figuring out the limits for `storage_min`, `storage_max`[/-][+]Finalize the semantics of `storage_min`, `storage_max`[/+]sdasgup3 commentedon Apr 28, 2023
From the POV of the semantics of an op producing quantized results, the relevance
storage_min/storage_maxvalues are to clamp the result(s) . The current spec expects default values ofmin_value(storage_type)andmax_value(storage_type)forstorage_minandstorage_maxresp., if they are not explicitly provided. Moreover, the specification documents the following general constraint onstorage_minandstorage_maxMy personal opinion: The above constraint seems sufficiently general to accommodate the existing implementations (e.g. TFLite) as well as allow the adoption of new or experimental quantization schemes. The op semantics should not add additional constraints unless there is a strong reason to do that.
Having said that,
let meI am re-proposing a draft PR for the specification of quantized add. As some of you might remember that, at some point, we decided to drop the introduction of its specification mainly because we were unsure about the fate of #1406.The proposal will be almost same as before except that it does not have any additional constraint imposed by the op's semantics on
storage_minorstorage_max.Please let me know your feedback and/or feel free to review the
upcoming PRPR.Specification for quantized AddOp (#1446)
jingpu commentedon Jun 8, 2023
Here are some pointers regarding the "narrow_range" INT8 feature in TFLite.
Op specs: https://www.tensorflow.org/lite/performance/quantization_spec
Essentially, the symmetric INT8 quantization is designed not to use the value 128 (I heard it was for leveraging some fast ARM instruction to implement the INT8 kernels).
The quantizer and TFL dialect MLIR importer also incorporate this feature.
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/compiler/mlir/lite/quantization/quantization_driver.cc#L390-L393
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/compiler/mlir/lite/flatbuffer_import.cc#L173
sdasgup3 commentedon Jun 12, 2023
@jingpu Thanks for your comment and explaining on the narrow range on INT8 quantization feature in TFLite.
I note that the current spec expects default values of
min_value(storage_type)andmax_value(storage_type)forstorage_minandstorage_maxresp., if they are not explicitly provided.Having said that, I am wondering, as the default value of
storage_typelike[-128,127]does not work in this case, how about theQuantizedTypeto explicitly encode[-127, 127]for the weights.Please let me know your feedback!
jingpu commentedon Jun 12, 2023
Sorry. I am a bit confused by what the proposal is. During the meeting last week, I thought the proposal was to drop
storage_minandstorage_maxfrom the StableHLO quantize type, and use [min_value(storage_type), max_value(storage_type)] as the range. Could you clarify it?sdasgup3 commentedon Jun 13, 2023
Hi @jingpu
Sorry for the confusion :(. Actually, my goal in the meeting is to brainstorm on
storage_minandstorage_maxshould have their values constraint for each op, as the case in TFLite spec, Ormin_value(storage_type)andmax_value(storage_type)as default value, ifstorage_minandstorage_max, resp., are absent. If they are present, then the following constraint need to be satisfied.I agree with you that if the allowed values are only
min_value(storage_type)andmax_value(storage_type), then it would be a problem expressing the use-case that you provided.I think the description of the issue is misleading. I have updated the issue description to avoid any confusion.
jingpu commentedon Jun 13, 2023
Thanks for the clarification. I don't have a strong preference between the two because I don't depend on the TFLite narrow_range feature in my work and don't have the full context about it either.
sdasgup3 commentedon Apr 9, 2024
The current spec allows the
storage_min/storage_maxvalues to be provided explicitly to cover cases like exciting use cases (like narrow_range mentioned above). With that let us close this issue.