Skip to content

fromFlatTerm impl of PeekTokenType is insufficently precise #324

Closed

Description

@dcoutts

The implementation of PeekTokenType in fromFlatTerm only makes fairly coarse token type distinctions:

tokenTypeOf (TkInt n)
| n >= 0 = TypeUInt
| otherwise = TypeNInt
tokenTypeOf TkInteger{} = TypeInteger

tokenTypeOf (TkInt n)
    | n >= 0                = TypeUInt
    | otherwise             = TypeNInt
tokenTypeOf TkInteger{}     = TypeInteger

In particular, it doesn't ever us TypeUInt64 or TypeNInt64:

data TokenType
  = TypeUInt
  | TypeUInt64
  | TypeNInt
  | TypeNInt64
  | TypeInteger

This creates a problem for some decoders that use peekTokenType and only look for the Type(U|N)Int(64) tokens and do not consider TypeInteger. Such decoders will not correctly round-trip via FlatTerm, where the will correctly round-trip via the binary CBOR representation.

The solution is to return TypeUInt64 and TypeNInt64 when the integer fits within the range, e.g.

tokenTypeOf (TkInteger n)
  | n >= 0 && n <  2^64 = TypeUInt64
  | n <  0 && n > -2^64 = TypeNInt64
  | otherwise           = TypeInteger

Alternatively, we could change the representation of FlatTerm to preserve more token type information, e.g. by splitting TkInt into TkUInt, TkNInt, TkUInt64, TkNInt64.

Activity

  1. added a commit that references this issue on Aug 31, 2023
  2. added a commit that references this issue on Sep 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions