Skip to content

GreyNoise-Intelligence/greynoise-fluentbit-lua

BranchesTags

Repository files navigation

main License: MIT

GreyNoise Fluent Bit Lua Filter

This is a prototype Fluent Bit container using a filter plugin which calls the GreyNoise API to drop, re-route, or enrich records. This specific filter leverages the Fluent Bit Lua script filter.

Getting Started

These instructions will cover usage information and for the docker container

Prerequisities

In order to run this container you'll need docker installed.

In order to run make stats you will need jq installed

GreyNoise Sign-Up

  1. Sign-Up for GreyNoise
  2. Copy .env_example to .env
  3. Copy your GreyNoise API key from the Account section in the top right corner
  4. Replace the <REPLACE_ME> in .env with your API key

Docker

Usage

Container Parameters

docker run --env-file .env -it -p 2020:2020 -v $(PWD):/app greynoise/greynoise-fluentbit-lua:latest -c /app/conf/myconfig.conf

Environment Variables (required)

  • GREYNOISE_API_KEY - GreyNoise API key to use for HTTP requests.
  • GREYNOISE_IP_FIELD - Named field from the Fluent Bit parser to use for IP lookups.
  • GREYNOISE_LUA_LOG_LEVEL - Lua logging level (info/error/warning/debug)
  • GREYNOISE_LUA_CACHE_SIZE - The number of IP records to cache in-memory before overwriting.

Volumes

  • /app - Core working directory (mounted from the base repo folder)

Sample Data Testing

The sample data tests are meant to be run from the repo base folder.

Example 1 - Dummy Data

conf/dummy.conf

This example just generates the same JSON line over and over.

  1. Copy .env_example to .env
  2. Copy your GreyNoise API key from the Account section in the top right corner
  3. Replace the <REPLACE_ME> in .env with your API key
  4. Run make build
  5. Run make run

Example 2 - Auth.log

conf/tail.conf

This example watches reads a log file in and watches for new lines.

  1. Run make run-tail
  2. Run make stats in another terminal to see metrics

Example 3 - Auth.log With RewriteTag Rules

conf/rewrite.conf

This is the same as #2 except this leverages rewrite_tag filter to drop records. This config drops invalid IPv4 records, bogon address space, GreyNoise RIOT records, and GreyNoise Noise records.

  1. Run make run-rewrite
  2. Run make stats in another terminal to see metrics (note the drop rates)

Running in your environment

  1. Create a directory for your configs mkdir conf
  2. Add your Fluent Bit configs and parsers to