Plugins

Introduction

The Falco libraries and Falco itself can be extended by using Plugins. Plugins are shared libraries that conform to a documented API and allow for:

• adding new event sources that can be evaluated using filtering expressions/Falco rules.
• adding the ability to define new fields that can extract information from events.

This page describes how plugins fit into the existing event processing pipeline and how to enable/configure plugins in Falco.

Plugins Architecture Concepts

Plugins

Plugins are dynamic shared libraries (.so files in Unix, .dll files in Windows) that export C calling convention functions. Programs like Falco dynamically load these libraries and call the exported functions to extend Falco's support for event sources/fields.

Plugins are versioned using semantic versioning to minimize regressions and compatibility issues.

Plugins can be written in any language, as long as they export the required functions. Go, however, is the preferred language to write plugins, followed by C/C++.

There are two kinds of plugins: source plugins and extractor plugins.

Plugin SDKs

To make it easier to write plugins, there are Go and C++ SDKs that handle the details of memory management and type conversion. These SDKs provide a streamlined way to implement plugins without having to deal with all the details of the lower-level functions that make up the Plugin API.

These SDKs are optional, but using them is highly recommended.

Source Plugin

A source plugin provides a new sinsp/scap event source. It has the ability to "open" and "close" a stream of events. It also has the ability to return an event to the plugin framework via a next() method.

Source plugins also have the ability to extract information from events based on fields. For example, a field (e.g. proc.name) extracts a value (e.g. process name like nginx) from a syscall event. The plugin returns a set of supported fields, and there are functions to extract a value given an event and field. The plugin framework can then build filtering expressions/Falco rule conditions based on these fields combined with relational and/or logical operators. For example, given an expression ct.name=root and ct.region=us-east-1, the plugin framework handles parsing the expression, calling the plugin to extract values for fields ct.name/ct.region for a given event, and determining the result of the expression. In a Falco output string like An EC2 Node was created (name=%ct.name region=%ct.region), the plugin framework handles parsing the output string, calling the plugin to extract values for fields, and building the resolved string, replacing the template field names (e.g. %ct.region) with values (e.g. us-east-1).

Source plugins also provide a plugin ID, which is globally unique and is used in capture files (see below). They also provide an event source, which is tied to events generated by the plugin and is used by Falco rules/extractor plugins (see below).

Extractor Plugin

An extractor plugin focuses only on field extraction from events generated by other plugins, or by the core libraries. It does not provide an event source, but can extract fields from other event sources. An example is json field extraction, where a plugin might be able to extract fields from arbitrary json payloads.

Plugins are Coresident with Falco

The libraries will do everything possible to validate the data coming from the plugins and protect Falco and the other consumers from corrupted data. However, for performance reasons, plugins are "trusted": they run in the same thread and address space as Falco and they could crash the program. We assume that the user will be in control of plugin loading and will make sure only trusted plugins are loaded/packaged with Falco.

Plugin Event IDs

Every source plugin requires its own, unique plugin event ID to interoperate with Falco and the other plugins. This ID is used in the following ways:

• The ID is saved in in-memory event objects and is used to identify the associated plugin that injected the event.
• The ID is saved in capture files and is used to recreate in-memory event objects when reading capture files.

The ID must be unique to ensure that events written by a given plugin will be properly associated with that plugin (and its event sources, see below).

Source plugin authors must register the plugin with the Falcosecurity organization by creating a PR to modify the PLUGINS-REGISTRY.md file with details on the new plugin. This ensures that a given ID is used by exactly one source plugin.

Plugin Event Sources and Interoperability

Events returned by source plugins have an "event source" which describes the information in the event. This is distinct from the plugin name to allow for multiple kinds of plugins to generate the same kind of events. For example, there might be plugins gke-audit-bridge, eks-audit-bridge, ibmcloud-audit-bridge, etc. that all fetch K8s Audit information. The plugins would have different names and IDs but would have the same event source "k8s_audit".

An extractor plugin optionally provides a set of event sources. When the framework receives an event with an event source in the plugin's set of event sources, fields in expressions/Falco outputs will be extracted from events using the plugin. An extractor plugin can also not name a set of event sources. In this case, all events will be presented to the extractor plugin, regardless of source. In this case, the extractor plugin must detect the format of arbitrary payloads and be able to return NULL/no value when the payload is not supported.

Extractor plugin authors should register the plugin with the Falcosecurity organization by creating a PR to modify the PLUGINS-REGISTRY.md file with details on the new plugin. This allows source plugin authors and extractor plugin authors to coordinate about event source formats.

Handling Duplicate/Overlapping Fields in Plugins/Libraries Core

At an initial glance, adding plugins introduces the possibility of tens/hundreds of new filtercheck fields that could potentially overlap/conflict. For example, what happens if a plugin defines a proc.name field? However, the notion of event source makes these potential conflicts manageable.

Remember that field extraction is always done in the context of an event, and each event can be mapped back to an event source. So we only need to ensure that filtercheck fields are non-overlapping for a given event source. For example, it's perfectly valid for an AWS Cloudtrail plugin to define a proc.name field, as the events generated by that plugin are wholly separate from syscall events. For syscall events, the AWS Cloudtrail plugin is not involved and the core libraries extract the process name for the tid performing a syscall. For AWS Cloudtrail events, the core libraries are not involved in field extraction. Extraction is performed by the AWS Cloudtrail plugin instead.

When managing plugins, we only need to ensure the following:

• That only one plugin is loaded at a time that exports a given event source. For example, the libraries can load either a gke-audit-bridge plugin with event source k8s_audit, or eks-audit-bridge with event source k8s_audit, but not both.
• That for a mix of source and extractor plugins having the same event source, that the fields are distinct. For example, a source plugin with source k8s_audit can export ka.* fields, and an extractor plugin with event source k8s_audit can export a jevt.value[/...] field, and the appropriate plugin will be used to extract fields from k8s_audit events as fields are parsed from condition expressions/output format strings.

Plugin API

Here is an overview of the functions that comprise the plugins API. This list is not complete--the developer's guide has full documentation of the source and extractor plugin APIs.

In almost all cases, a plugin author can use the SDKs which provide a more streamlined interface. This still provides a good overview of the functionality a plugin provides.

Info Functions

A set of functions provide information about the plugin and its compatibility with the plugin framework:

• plugin_get_required_api_version: Return the version of the plugin API used by this plugin.
• plugin_get_type: Return the plugin type.
• plugin_get_id: Return the unique ID of the plugin.
• plugin_get_name: Return the name of the plugin.
• plugin_get_description: Return a short description of the plugin.
• plugin_get_contact: Return a contact url/email/twitter account for the plugin authors.
• plugin_get_version: Return the version of the plugin itself.
• plugin_get_event_source: Return a string describing the events generated by this source plugin.

Instance/Capture Management Functions

Plugins have functions to initialize/destroy a plugin, as well as functions to open/close streams of events:

• plugin_init: Initialize the plugin and, if needed, allocate its state.
• plugin_destroy: Destroy the plugin and, if plugin state was allocated, free it.
• plugin_open: Open the source and start a stream of events.
• plugin_close: Close a stream of events.

Events/Fields Related Functions

Source plugins have functions to provide events to the plugin framework:

• plugin_next_batch: Return one or more events to the plugin framework.
• plugin_get_progress: (Optional) Provide feedback on how much of the event stream has been read.
• plugin_event_to_string: Return a text representation of an event generated by this source plugin.

Both source and extractor plugins have functions to define the set of fields that can be used to extract information from events, to actually extract values from events, and to return printable representations of events:

• plugin_get_fields: Return the list of extractor fields exported by this plugin.
• plugin_extract_fields: Extract one or more a filter field values from an event.

How Falco Uses Plugins

Falco loads plugins based on configuration in falco.yaml. Currently, if a source plugin is loaded the only events processed are from that plugin--syscall and k8s_audit events are disabled. There are other restrictions on loaded plugins (see below).

Loading plugins in Falco

Falco configures plugins via the new "plugins" property in falco.yaml. Here's an example:

plugins:
  - name: aws_cloudtrail
    library_path: aws_cloudtrail/plugin.so
    init_config: "..."
    open_params: "..."
  - name: http_json
    library_path: http_json/plugin.so
    init_config_file: http_json/config.txt
    open_params_file: http_json/params.txt

# Optional
load_plugins: [aws_cloudtrail]

A new "plugins" property in falco.yaml will define the set of plugins that can be loaded by Falco, and a new "load_plugins" property in falco.yaml will control which plugins are actually loaded when Falco starts.

For more information, see Configuration.

The mechanics of loading a plugin are implemented in the libraries and leverage the dynamic library functionality of the operating system (dlopen/dlsym in unix, LoadLibrary/GetProcAddress in Windows). The plugin loading code also ensures that:

• the plugin is valid, i.e. that it exports the set of expected symbols
• the plugin has an api version number that is compatible with the plugin framework.
• that only one source plugin is loaded at a time for a given event source
• if a mix of source and extractor plugins are loaded for a given event source, that the exported fields have unique names that don't overlap across plugins

Event Sources and Falco Rules

Falco rules already have the notion of a "source", using the source property in YAML rules objects, and there are currently two kinds of event sources: "syscall" and "k8s_audit". The source property in Falco rules maps a given rule to the event source on which the rule runs.

For example, given a source plugin with event source "cloudtrail", and a Falco rule with source property "cloudtrail", the rule will be evaluated for any events returned by the plugin.

Similarly, an extractor plugin that includes "cloudtrail" in its set of event sources will have the opportunity to extract information from cloudtrail events. As a result, fields exported by the extractor plugin can be put in a rule's condition, exception, or output properties when the rule has a source "cloudtrail".

Falco compiles rules/macros/lists selectively based on the set of loaded plugins (specifically, their event sources), instead of unconditionally as Falco is started. This is especially important for macros, which do not contain a source property, but might contain fields that are only implemented by a given plugin.

Plugin Versions and Falco Rules

To allow rules files to document the plugin versions they are compatible with, rules files can have a new top-level field required_plugin_versions. The field is optional, and if not provided no plugin compatibility checks will be performed. The syntax of required_plugin_versions is the following:

- required_plugin_versions:
  - name: <plugin_name>
    version: x.y.z
  ...

Below required_plugin_versions is a list of objects, where each object has name and version properties. If a plugin is loaded, and if an entry in required_plugin_versions has a matching name, then the loaded plugin version must be semver compatible with the version property.

Falco can load multiple rules files, and each file may contain its own required_plugin_versions property. In this case, name+version pairs across all files will be merged, and in the case of duplicate names all provided versions must be compatible.

Plugin Developer's Guide

If you are interested in authoring your own plugin, or modifying an existing plugin to add new functionality, we've written a developer's guide that documents the full plugin APIs and walks through two existing plugins to show how the API is used.

Where's the Code?

Plugins

Plugins authored by the Falcosecurity community are at the plugins github repository. The current plugins are:

• cloudtrail: Reads Cloudtrail JSON logs from files/S3 and injects as events.
• dummy: Reference plugin use to document plugins interface.
• dummy_c: Like dummy, but written in C++.
• json: Extracts values from any JSON payload.

Plugin SDKs

To facilitate the development of plugins written in Go, we've written a SDK that provides support code for writing plugins. The SDK provides Go structs/enums corresponding to the C structs/enums used by the API, has utility packages that manage the details of memory management/type conversion, and presents abstract interfaces that provide a more streamlined interface to potential plugin authors.

There is also a C++ SDK that defines abstract C++ base classes for source plugins and plugin instances. Plugin authors can derive from these base classes and implement the abstract methods to provide demographic information, events, and extract fields from events.

All of the Falcosecurity-provided plugins use these SDKs.

Falco

Falco itself includes the cloudtrail and json plugins in its packages/container images. The plugins are defined in falco.yaml but by default no plugins are loaded when Falco starts.

To add plugins, you can put them as shared libraries below /usr/share/falco/plugins, and use a relative path in the value for library_path in falco.yaml.

References

If you're interested in how this feature came about, you can view the original proposal for the plugin system.