Graylog Processing Pipeline functions to enrich log messages with IoC information from threat intelligence databases
Required Graylog version: 2.4.0
This Plugin use external sources to enrich your data - read the documentation before you run this in production
This plugin adds Processing Pipeline functions to enrich log messages with threat intelligence data.
let src_addr_intel = threat_intel_lookup_ip(to_string($message.src_addr), "src_addr"); set_fields(src_addr_intel);
Please read the usage instructions below for more information and specific guides.
Since Graylog Version 2.4.0 this plugin is already included in the Graylog server installation package as default plugin.
Download the plugin and place the
.jarfile in your Graylog plugin directory. The plugin directory is the
plugins/folder relative from your
graylog-serverdirectory by default and can be configured in your
graylog.conffile.
Restart
graylog-serverand you are done.
Example Processing Pipeline rules are following:
This is the recommended way to use this plugin. The
threat_intel_lookup_*function will run an indicator like an IP address or domain name against all enabled threat intel sources and return a combined result. (Except OTX lookups)
let src_addr_intel = threat_intel_lookup_ip(to_string($message.src_addr), "src_addr"); set_fields(src_addr_intel);let dns_question_intel = threat_intel_lookup_domain(to_string($message.dns_question), "dns_question"); set_fields(dns_question_intel);
This will lead to the fields
src_addr_threat_indicated:true|falseand
dns_question_threat_indicated:true|falsebeing added to the processed message. It will also add fields like
testing_threat_indicated_abusech_ransomware:true(Abuse.ch Ransomware tracker OSINT) to indicate threat intel sources returned matches.
Add a second pipeline step that adds the field
threat_indicated:trueif either of the above fields was true to allow easier queries for all messages that indicated any kind of threat:
rule "inflate threat intel results" when to_bool($message.src_threat_indicated) || to_bool($message.dst_threat_indicated) then set_field("threat_indicated", true); end
You can look up WHOIS information about IP addresses. The method will return the registered owner and country code. The lookup results are heavily cached and invalidated after 12 hours or when the
graylog-serverprocess restarts.
let whois_intel = whois_lookup_ip(to_string($message.src_addr), "src_addr") set_fields(whois_intel);
Note: The plugin will use the ARIN WHOIS servers for the first lookup because they have the best redirect to other registries in case they are not responsible for the block of the requested IP address. Graylog will follow the redirect to other registries like RIPE-NCC, AFRINI, APNIC or LACNIC. Future versions will support initial lookups in other registries, but for now, you might experience longer latencies if your Graylog cluster is not located in Nort America.
let intel = otx_lookup_ip(to_string($message.src_addr)); // let intel = otx_lookup_domain(to_string($message.dns_question))set_field("threat_indicated", intel.otx_threat_indicated); set_field("threat_ids", intel.otx_threat_ids); set_field("threat_names", intel.otx_threat_names);
You'll need at least Java 8 (u101) to make this work. The exit node information is hosted on a Tor website that uses Let's Encrypt for SSL and only Java 8 (u101 or newer) supports it.
let intel = tor_lookup(to_string($message.src_addr)); set_field("src_addr_is_tor_exit_node", intel.threat_indicated);
let intel = spamhaus_lookup_ip(to_string($message.src_addr)); set_field("threat_indicated", intel.threat_indicated);
let intel = abusech_ransom_lookup_domain(to_string($message.dns_domain)); // let intel = abusech_ransom_lookup_ip(to_string($message.src_addr)); set_field("request_domain_is_ransomware", intel.threat_indicated);
Note that you can combine these and change field names as you wish.
in_private_net(ip_address) : Booleanfor any manual lookups of the same kind.