Overview

This processor samples your logs for a representative subset at the rate that you define, dropping the remaining events. As an example, you can use this processor to sample 20% of events from a noisy non-critical service.

The sampling only applies to events that match your filter query and does not impact other events. If an event is dropped at this processor, the event is not sent to subsequent processors.

Setup

To set up the sample processor:

1. Define a filter query. See Logs Search Syntax for more information.
   • Only events that match the specified filter query are sampled at the specified retention rate.
   • The sampled events and the events that do not match the filter query are sent to the next step in the pipeline.
2. Enter your desired sampling rate in the Retain field. For example, entering 2 means 2% of events are retained out of all events that match the filter query.
3. Optionally, enter a Group By field to create separate sampling groups for each unique value for that field. For example, status:error and status:info are two unique field values. Each bucket of events with the same field is sampled independently. Click Add Field if you want to add more fields to partition by. See the group-by example.

Group-by example

If you have the following setup for the sample processor:

• Filter query: env:staging
• Retain: 40% of matching events
• Group by: status and service

Then, 40% of events for each unique combination of status and service from env:staging is retained. For example:

• 40% of events with status:info and service:networks are retained.
• 40% of events with status:info and service:core-web are retained.
• 40% of events with status:error and service:networks are retained.
• 40% of events with status:error and service:core-web are retained.