Skip to content

[REQ] Optimize scalar fallback for shared tile loads #1239

Closed

Description

@daedalus5

Tile load/store scalar path uses expensive per-element div/mod and lacks benchmark coverage

Problem

The scalar fallback path in copy_from_global / copy_to_global (tile.h) calls coord_from_linear on every element, which performs repeated integer division and modulo to convert a linear thread index into N-D tile coordinates. This is unnecessarily expensive — particularly for 1D tiles (where no coordinate math is needed) and for N-D tiles (where coordinates can be maintained incrementally across iterations).

Missing benchmark coverage

Existing ASV benchmarks (load_store.py, bounds_check.py) and the example benchmark (benchmark_tile_load_store.py) only use POT (power-of-two) tile sizes on aligned arrays, which always hit the vectorized float4 path. The scalar fallback path has zero ASV coverage, meaning regressions would go undetected. Specific gaps:

  • No NPOT tile sizes (which force the scalar path)
  • No 1D or 3D tile benchmarks
  • No structured dtypes (e.g. vec3)
  • No distinction between L2-cached and DRAM-bound regimes
  • Example benchmark is hardcoded to 2D / tile=32 / float32 with no CLI configurability

Expected outcome

  • Scalar path should avoid per-element div/mod where possible
  • 1D tiles should use flat indexing with no coordinate math
  • N-D tiles should use incremental coordinate iteration instead of recomputing from scratch each iteration
  • ASV benchmarks should cover both scalar and vectorized paths across 1D/2D/3D, multiple dtypes, and memory regimes
  • Example benchmark should be configurable for profiling arbitrary tile configurations

Metadata

Metadata

Assignees

Labels

feature requestRequest for something to be added

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions