You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
rewrite_tag loses records when emitter buffer limit is reached, even with filesystem buffering #12667
When emitter_mem_buf_limit is reached, rewrite_tag stops re-emitting records, even when emitter_storage.type filesystem is enabled.
Issue #12026 was addressed by PR #12029, which preserves the original record when re-emission fails. However, the record keeps its original tag, effectively skipping the intended rewrite_tag processing.
If downstream outputs only match the rewritten tag, those records are never delivered.
Observed behavior
In our tests:
230 records sent through the OpenTelemetry input.
Only 49 records reached S3.
Increasing emitter_mem_buf_limit resulted in all 230 records being delivered.
Filesystem buffering was enabled on the input and all rewrite_tag emitters.
Debug log:
[input:emitter:datacontenttype_emitter] emitter memory buffer limit reached. Not accepting record.
Expected behavior
Reaching the emitter memory limit should not cause records to skip the required rewrite_tag processing.
With filesystem buffering enabled, Fluent Bit should queue records on disk and retry re-emission, or propagate backpressure upstream until the records can be processed.
Preserving the original record without retrying the failed rewrite is not sufficient to prevent end-to-end data loss.
Activity