An overview of some of the most popular open-source tools for threat intelligence and threat hunting
As the term threat intelligence can be easily confounded with threat hunting, we will first endeavor to outline some of the differences between them.
Threat intelligence refers to the aggregation and enrichment of data to create a recognizable profile of what a specific cyberattack, malicious campaign, or attacker’s capability looks like.
Threat hunting, meanwhile, refers to the process of analyzing event data for abnormal and malicious behaviors in a network that could indicate the intrusion of an attacker, the theft of data, or other damage. Although threat intelligence does not have the same objectives as threat hunting, it serves as an excellent point of departure for threat hunting.
Now let’s look at a selection of open-source tools used in both disciplines:
Figure 1. Seven popular open-source tools for threat intelligence and threat hunting
Threat intelligence tools
Yeti
Your everyday threat intelligence (Yeti) is a platform born from the need of security analysts to centralize multiple threat data feeds. Analysts frequently deal with questions such as: “Where was this indicator observed?” and “Is this information related to a specific attack or malware family?” To answer these questions, Yeti helps analysts to organize Indicators of Compromise (IoCs) and information on the tactics, techniques, and procedures (TTPs) employed by attackers in a single, unified repository. Once ingested, Yeti automatically enriches the indicators, for instance, by resolving domains or geolocating IP addresses.
Figure 2. Listing observables in Yeti
Figure 3. Tracking malicious campaigns in Yeti
Yeti stands out for its ability to ingest data (even blogposts), enrich them, and then export the enriched data to other tools used in an organization’s threat intelligence ecosystem. This allows analysts to focus on using this tool to aggregate threat information instead of worrying about how to import and export data in a machine-readable format. The enriched data can then be shared with other systems for incident management, malware analysis, or monitoring.
To further streamline the workflow of analysts, Yeti also offers an HTTP API with access to the full power of the tool both from a command shell and from other threat intelligence tools.
MISP
MISP, Open Source Threat Intelligence and Sharing Platform (formerly called Malware Information Sharing Platform), is a free tool for sharing IoCs and vulnerability information between organizations, thus promoting collaborative work on threat intelligence. The platform is used by organizations around the world to form trusted communities that share data so as to correlate it and achieve a better understanding of threats targeting specific sectors or areas.
Figure 4. MISP dashboard
Instead of sending IoCs via email and as PDF documents, the platform helps collaborating organizations better manage how information is shared and centralized between them. The information shared in MISP communities can then be fed into Yeti for further enrichment.
OpenCTI
Similar to Yeti, Open Cyber Threat Intelligence (OpenCTI) is a platform for ingesting and aggregating data so as to enrich an organization’s knowledge about threats. It is supported by France’s national cybersecurity agency ANSSI, the Computer Emergency Response Team for the EU (CERT-EU), and Luatix.
In addition to manually entering threat data, OpenCTI offers connectors to automatically ingest threat data feeds and information from popular threat intelligence sources, including MISP, MITRE ATT&CK, and VirusTotal. Other connectors are available to enrich data with sources like Shodan and export data into platforms like Elastic and Splunk.
Figure 5. OpenCTI dashboard
Harpoon
Harpoon is a command line tool that comes with a set of Python plugins to automate open-source intelligence tasks. Each plugin provides a command that analysts can use to consult platforms such as MISP, Shodan, VirusTotal, and Have I Been Pawned, via their APIs. Analysts can use higher level commands to gather information related to an IP address or domain from all these platforms at once. Finally, other commands can query URL shortener services and search social media platforms, GitHub repositories, and web caches.
Figure 6. Harpoon running in a command shell
Threat hunting tools
Sysmon
Although it is not open source, System Monitor (Sysmon) is a free Windows tool that monitors and logs activities such as process creations, network connections, loading of drivers and DLLs, and modifications of file creation timestamps to the Windows Event Log. As Sysmon does not analyze system data, threat hunters typically use a Security Information and Event Management (SIEM) tool to collect and analyze the data logged by Sysmon for suspicious and malicious activities happening in the network.
APT-Hunter
Since SIEM solutions require a paid license, a free alternative is APT-Hunter. Released in 2021, APT-Hunter is an open source tool that can analyze the Windows Event Log to detect threats and suspicious activities. The tool currently contains a set of more than 200 detection rules to identify malicious activity such as pass-the-hash and password spraying attacks, as well as other suspicious activity for manual inspection by threat hunters. Many of the rules map directly to the MITRE ATT&CK knowledge base.
APT-Hunter can collect Windows logs in both the EVTX and CSV formats. Upon execution, APT-Hunter generates two output files:
- A .xlsx file that contains all events detected as suspicious or malicious.
- A .csv file that can be loaded into Timesketch to display the progress of an attack chronologically.
DeepBlueCLI
DeepBlueCLI is an open source tool provided in the SANS Blue Team GitHub repository that can analyze EVTX files from the Windows Event Log. The tool parses logged Command shell and PowerShell command lines to identify suspicious indicators like long command lines, regex searches, obfuscation, and unsigned EXEs and DLLs; attacks on user accounts like password guessing and password spraying; and tools like Mimikatz, PowerSploit, and BloodHound.
Originally released as a PowerShell module, DeepBlueCLI has also been written in Python for use on Unix-like machines.
Final word
Threat intelligence and threat hunting are complementary activities in the daily workflow of an organization’s security team. As new malicious campaigns arise in the threatscape, it is critical that organizations are able to share knowledge about what they are seeing so as to paint a more detailed picture both of the latest activities of known threats and of new attackers appearing on the scene. Security analysts are tasked with organizing and correlating data from multiple and sometimes disparate sources. Based on the enriched threat data, threat hunters can then more easily identify any threats in their networks and neutralize them.
