IntelMQ – Framework to Collect and Process Security Feeds

IntelMQ is a solution for IT security teams (CERTs & CSIRTs, SOCs, abuse departments, etc.) for collecting and processing security feeds (such as log files) using a message queuing protocol. It’s a community driven initiative called IHAP (Incident Handling Automation Project) which was conceptually designed by European CERTs/CSIRTs during several InfoSec events.

Its main goal is to give to incident responders an easy way to collect & process threat intelligence thus improving the incident handling processes of CERTs.

IntelMQ – Framework to Collect and Process Security Feeds

IntelMQ can be used for – automated incident handling – situational awareness – automated notifications – as data collector for other tools – etc.

IntelMQ’s aims at:

  • Reducing the complexity of system administration
  • Reducing the complexity of writing new bots for new data feeds
  • Reducing the probability of events lost in all process with persistence functionality (even system crash)
  • Use and improve the existing Data Harmonization Ontology
  • Use JSON format for all messages
  • Provide easy way to store data into Log Collectors like ElasticSearch, Splunk, databases (such as PostgreSQL)
  • Provide easy way to create your own black-lists
  • Provide easy communication with other systems via HTTP RESTful API

It follows the following basic meta-guidelines:

  • Don’t break simplicity – KISS
  • Keep it open source – forever
  • Strive for perfection while keeping a deadline
  • Reduce complexity/avoid feature bloat
  • Embrace unit testing
  • Code readability: test with unexperienced programmers
  • Communicate clearly

There are several components evolved around IntelMQ for easy use and installation it will be important to check the Ecosystem document.

Current supported feeds are: , AlienVault , AnubisNetworks , Autoshun , Bambenek , Bitcash , , Blueliv , CERT.PL , CINSscore , Calidog , CleanMX , CyberCrime Tracker , DShield , Danger Rulez , Dataplane , DynDNS , Fraunhofer , HPHosts , Have I Been Pwned , Malc0de , Malware Domain List , Malware Domains , MalwarePatrol , MalwareURL , McAfee Advanced Threat Defense, Microsoft , Netlab 360 , Nothink , OpenPhish Commercial , PhishTank , PrecisionSec , ShadowServer , Spamhaus , Sucuri , Surbl , Taichung , Team Cymru , Threatminer, Turris , URLVir , University of Toulouse , VXVault , ViriBack , WebInspektor , ZoneH.

You can read more and download this framework over here:

Notify of
Inline Feedbacks
View all comments