Repository navigation
Fluent Bit Splunk output: Fluent Bit starts; events take their sourcetype from SPLUNK_SOURCETYPE - #120
Draft
talwgx wants to merge 1 commit into
Draft
Fluent Bit Splunk output: Fluent Bit starts; events take their sourcetype from SPLUNK_SOURCETYPE#120talwgx wants to merge 1 commit into
talwgx wants to merge 1 commit into
Conversation
…ents sourcetype from SPLUNK_SOURCETYPE
The templates output passed Splunk properties to the `es` plugin. Fluent Bit
5.1.5 stops at startup on an unknown property ("es: unknown configuration
property 'splunk_token'", "output initialization failed"), so no event was
sent. It now uses the `splunk` plugin.
The events this output sends are the full text of each event, as
{"text": "<event>"}. They were labelled `tenx_encoded`, the sourcetype the
10x Splunk app reads as compact events. They now take their sourcetype from
SPLUNK_SOURCETYPE, `_json` when it is not set.
Checked with the engine (1.1.153 jar, stdin input, Receiver app) and Fluent
Bit 5.1.5 against a stand-in HEC endpoint (TLS, verify on), 6 input lines:
before, Fluent Bit stopped at startup and 0 requests arrived; after, 6 of 6
events arrived as {"event": {"text": "<line>"}}, each line equal to the
input, sourcetype `_json` by default and `aws:cloudtrail` with
SPLUNK_SOURCETYPE=aws:cloudtrail. `node --test tests/`: 15 of 15.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With
pipelines/run/output/event/fluentbit/splunk.yaml, Fluent Bit now starts and the events reach Splunk HEC, under the sourcetype set inSPLUNK_SOURCETYPE(_jsonwhen it is not set). Before, the templates output passed Splunk properties to theesplugin, Fluent Bit 5.1.5 stopped at startup (es: unknown configuration property 'splunk_token',output initialization failed), and no event was sent.What changed
-o esbecomes-o splunk, which is the plugin its properties belong to.event_sourcetypecomes fromSPLUNK_SOURCETYPE, default_json. The events this output sends are the full text of each event, as{"text": "<event>"}, not compact events, so they are no longer labelledtenx_encoded. In the 10x Splunk app (tenx-for-splunk/default), every event of that sourcetype is treated as compact:tenx-eventsselectssourcetype::tenx_encoded,REPORT-tenxreads a template hash from the text before the first comma (the leading~is optional in its regex), andTRANSFORMS-tenx_timecan set_timefrom a 13- or 19-digit number that follows that comma._jsondescribes the body as sent;SPLUNK_SOURCETYPElets a deployment use the sourcetype its searches expect for this data.Checked
Engine 1.1.153 (jar, stdin input, Receiver app) with this file as the output, Fluent Bit 5.1.5, and a stand-in HEC endpoint (TLS,
tls.verify=onagainst its own CA) that stores each request; 6 input lines. For the run, the output'scommandpointed at a script that starts Fluent Bit 5.1.5 in a container; nothing else in the file differed._json; eachevent.textequal to its input line, in orderSPLUNK_SOURCETYPE=aws:cloudtrailaws:cloudtrailNo template record was sent in any run:
writeTemplatesisfalsein this file, so the templates output receives nothing. In one more run of this branch the engine reached the end of its input, started Fluent Bit and closed its input within the same second, and Fluent Bit stopped before sending (0 requests); that is in the engine, not in this file.node --test tests/: 15 of 15.🤖 Generated with Claude Code