Forward
Forward is the protocol used by Fluent Bit and Fluentd to route messages between peers. This plugin implements the input service to listen for Forward messages.
Configuration parameters
The plugin supports the following configuration parameters:
buffer_chunk_size
By default the buffer to store the incoming Forward messages, don't allocate the maximum memory allowed, instead it allocate memory when it's required. The rounds of allocations are set by buffer_chunk_size. The value must be according to the Unit Size specification.
1024000
buffer_max_size
Specify the maximum buffer memory size used to receive a Forward message. This limit also applies to incoming payloads and decompressed data; payloads exceeding this size are rejected and the connection is closed. The value must be according to the Unit Size specification.
6144000
empty_shared_key
Enable secure forward protocol with a zero-length shared key. Use this to enable user authentication without requiring a shared key, or to connect to Fluentd with a zero-length shared key.
false
listen
Listener network interface.
0.0.0.0
msgpack_memory_limit
In v5.1.4 or later, sets the maximum memory used to decode the MessagePack data of an incoming Forward message, including the entries of PackedForward payloads. If a message exceeds this limit, the plugin logs the warning invalid frame or MessagePack allocation budget exceeded and closes the connection. A value of 0 sets the limit to 16 times buffer_max_size, with a minimum of 64 KiB. The value must be according to the Unit Size specification.
0
port
TCP port to listen for incoming connections.
24224
security.users
Specify the username and password pairs for secure forward authentication. Requires shared_key or empty_shared_key to be set.
none
self_hostname
Hostname for secure forward authentication.
localhost
shared_key
Shared key for secure forward authentication.
none
signal_object_memory_limit
In v5.1.5 or later, sets the estimated maximum memory used to decode the metrics or traces in an incoming PackedForward or CompressedPackedForward message. For CompressedPackedForward messages, the limit applies to the decompressed data. Each top-level MessagePack object, array element, and map entry counts as 1024 bytes, so this is an estimate, not an exact memory limit. If a message exceeds this limit, the plugin logs the warning PackedForward signal object budget exceeded or invalid payload and closes the connection. This limit is independent of msgpack_memory_limit. The value must be at least 1024 bytes and must be according to the Unit Size specification.
100M
tag
Override the tag of the forwarded events with the defined value. In v5.1.3 or later, if tag and tag_prefix aren't set and an incoming tag is empty, the input name is used. If tag_prefix is set, it's applied to the incoming tag instead.
none
tag_prefix
Prefix incoming tag with the defined value.
none
unix_path
Specify the path to Unix socket to receive a Forward message. If set, listen and port are ignored.
none
unix_perm
Set the permission of the Unix socket file. If unix_path isn't set, this parameter is ignored.
none
workers
The number of listener workers that accept and process incoming connections. Not supported with unix_path; a value greater than 1 while unix_path is set causes startup to fail.
1
Payload validation
In v5.1.3 or later, the plugin validates PackedForward and CompressedPackedForward payloads before it ingests them. If an entry isn't a valid log event, the plugin logs the warning invalid PackedForward entry and rejects the payload.
TLS / SSL
The Forward input plugin supports TLS/SSL. For more details about the properties available and general configuration, refer to Transport Security.
Get started
To receive Forward messages, you can run the plugin from the command line or through the configuration file as shown in the following examples.
Command line
From the command line you can let Fluent Bit listen for Forward messages with the following options:
By default, the service listens on all interfaces (0.0.0.0) through TCP port 24224. You can change this by passing parameters to the command:
In the example, the Forward messages arrive only through network interface 192.168.3.2 address and TCP Port 9090.
Configuration file
In your main configuration file append the following:
Fluent Bit and secure forward setup
In Fluent Bit v3 or later, in_forward can handle secure forward protocol.
For shared key authentication, specify shared_key in both forward output and forward input. For user-password authentication, specify security.users with at least one user-password pair along with a shared key. To use user authentication without requiring clients to know a shared key, set empty_shared_key to true.
The self_hostname value can't be the same between Fluent Bit servers and clients.
User authentication with empty_shared_key
To use username and password authentication without requiring clients to know a shared key, set empty_shared_key to true:
Testing
After Fluent Bit is running, you can send some messages using the fluent-cat tool, provided by Fluentd:
When you run the plugin with the following command:
In Fluent Bit you should see the following output:
Last updated
Was this helpful?
