⚠️ cyber_threat

Cyber threat intelligence, exploit data, IOC feeds, MITRE ATT&CK taxonomy, and standards/compliance mappings. Part of the govdata cyber sub-domain.

12 datasets · 124 columns

attack_techniques · table

MITRE ATT&CK technique and sub-technique records from STIX2 bundle. Technique ID is extracted from external_references where source_name == "mitre-attack". Sub-technique parent resolved via subtechnique-of relationship objects. Domain controlled by CYBER_ATTACK_DOMAIN env var (enterprise, ics, mobile).

Column	Type	Null	Description
technique_id	string	no	ATT&CK technique ID (e.g., T1055.011)
stix_id	string	yes	STIX2 object ID
name	string	yes	Technique name
description	string	yes	Technique description
tactic_short_names	string	yes	Pipe-delimited tactic short names from kill_chain_phases
platforms	string	yes	Pipe-delimited target platforms (x_mitre_platforms)
data_sources	string	yes	Pipe-delimited data sources (x_mitre_data_sources)
is_subtechnique	boolean	yes	True if this is a sub-technique (x_mitre_is_subtechnique)
parent_technique_id	string	yes	Parent technique ID for sub-techniques (via subtechnique-of relationship)
domain	string	yes	ATT&CK domain (enterprise-attack, ics-attack, mobile-attack)
detection	string	yes	Detection guidance
mitre_version	string	yes	ATT&CK version string (x_mitre_version)

ioc_urls · table

URLhaus malicious URL indicators. Delivered as ZIP archive containing CSV; must decompress before parsing. Comment lines prefixed with #. Tags are comma-delimited. Partitioned by first_seen date for TTL pruning.

Column	Type	Null	Description
url_id	string	yes	URLhaus numeric ID
url	string	no	Malicious URL
url_status	string	yes	URL status (online, offline)
date_added	string	yes	Date added to URLhaus
last_online	string	yes	Last time URL was observed online
threat	string	yes	Threat classification
tags	string	yes	Comma-delimited tags (as provided by URLhaus)
urlhaus_link	string	yes	URLhaus entry page
reporter	string	yes	Reporter handle
source	string	yes	Feed source (urlhaus)
first_seen	string	yes	Partition column (date of first observation, mapped from dateadded)

ioc_hashes · table

MalwareBazaar malware sample hashes. Plain CSV (no ZIP). Comment lines prefixed with #. vtpercent and imphash may be string "n/a" — coerce to null. Partitioned by first_seen date for TTL pruning.

Column	Type	Null	Description
sha256	string	no	SHA-256 hash (transformer emits final column names; no source remap)
md5	string	yes	MD5 hash
sha1	string	yes	SHA-1 hash
first_seen_utc	string	yes	First seen UTC timestamp
reporter	string	yes	Reporter handle
file_name	string	yes	Original file name
file_type	string	yes	Detected file type
mime_type	string	yes	MIME type
signature	string	yes	Malware family signature
clamav	string	yes	ClamAV detection name
vt_percent	double	yes	VirusTotal detection percentage (null when n/a)
imphash	string	yes	Import hash (null when n/a)
ssdeep	string	yes	ssdeep fuzzy hash
tlsh	string	yes	TLSH fuzzy hash
source	string	yes	Feed source (malwarebazaar)
first_seen	string	yes	Partition column (date of first observation)

ioc_ips · table

Feodo Tracker botnet C2 IP blocklist. JSON from /downloads/ipblocklist.json (not botnet_ip_blacklist.json). ~5 active entries typically. Partitioned by first_seen date for TTL pruning.

Column	Type	Null	Description
ip_address	string	no	C2 IP address
port	integer	yes	C2 port
status	string	yes	Status (online, offline)
hostname	string	yes	Resolved hostname
as_number	integer	yes	Autonomous system number
as_name	string	yes	Autonomous system name
country	string	yes	Country code (ISO 3166-1 alpha-2)
first_seen	string	yes	First observed date (partition column)
last_online	string	yes	Last observed online date
malware_family	string	yes	Associated malware family
source	string	yes	Feed source (feodo)

ioc_mixed · table

ThreatFox IOC feed covering ip:port, domain, url, md5_hash, and sha256_hash indicator types. Auth-Key header is mandatory even for free tier; anonymous POST returns {"error":"Unauthorized"}. Partitioned by first_seen date for TTL pruning.

Column	Type	Null	Description
ioc_id	string	yes	ThreatFox IOC ID
ioc_value	string	no	IOC value (IP:port, domain, URL, or hash)
ioc_type	string	yes	IOC type (ip:port, domain, url, md5_hash, sha256_hash)
threat_type	string	yes	Threat type classification
malware_key	string	yes	Malware identifier key
malware_printable	string	yes	Human-readable malware name
malware_aliases	string	yes	Pipe-delimited malware aliases
first_seen_utc	string	yes	First seen UTC timestamp
last_seen_utc	string	yes	Last seen UTC timestamp
confidence_level	integer	yes	Confidence level (0-100)
is_compromised	boolean	yes	True if indicator is a compromised legitimate asset
reference	string	yes	Reference URL
tags	string	yes	Pipe-delimited tags
reporter	string	yes	Reporter handle
anonymous	boolean	yes	True if submitted anonymously
source	string	yes	Feed source (threatfox)
first_seen	string	yes	Partition column (date of first observation)

nist_controls · table

NIST SP 800-53 Rev 5 security controls. Sourced from the usnistgov/oscal-content GitHub repository in OSCAL JSON format. Includes both base controls and enhancements (e.g., AC-2(1)). Description assembled from statement/item parts recursively.

Column	Type	Null	Description
control_id	string	no	Control identifier (e.g., AC-1, AC-2(1))
family_code	string	yes	Two-letter family code (e.g., AC, SI)
family_name	string	yes	Control family name (e.g., Access Control)
title	string	yes	Control title
description	string	yes	Assembled control statement prose (up to 4000 chars)
enhancement_of	string	yes	Parent control_id for enhancements; null for base controls
framework	string	yes	Framework name (NIST SP 800-53)
version	string	yes	Framework version (Rev 5)
source	string	yes	Data source identifier

nist_csf_functions · table

NIST CSF 2.0 subcategories. One row per subcategory (most granular level). 3-level hierarchy: Function → Category → Subcategory. 6 functions: GV (Govern), ID (Identify), PR (Protect), DE (Detect), RS (Respond), RC (Recover).

Column	Type	Null	Description
function_id	string	no	Function code (GV, ID, PR, DE, RS, RC)
function_name	string	yes	Function name (GOVERN, IDENTIFY, etc.)
category_id	string	yes	Category identifier (e.g., GV.OC)
category_name	string	yes	Category label
subcategory_id	string	no	Subcategory identifier (e.g., GV.OC-01)
subcategory_description	string	yes	Subcategory statement prose
framework_version	string	yes	CSF version (2.0)
source	string	yes	Data source identifier

cis_controls · table

CIS Controls v8 safeguards from the CISecurity/ControlsAssessmentSpecification GitHub repository (RST format). 18 controls, ~153 safeguards. The transformer fetches individual RST files for each safeguard and parses title, description, asset type, security function, and implementation groups.

Column	Type	Null	Description
control_id	string	no	CIS Control number (1-18)
control_title	string	yes	CIS Control title
safeguard_id	string	no	Safeguard identifier (e.g., 1.1, 2.3)
title	string	yes	Safeguard title
description	string	yes	Safeguard description (up to 2000 chars)
asset_type	string	yes	Asset type from implementation table (e.g., Devices, Applications)
security_function	string	yes	Security function (Identify, Protect, Detect, Respond, Recover)
ig_group	string	yes	Implementation groups string (e.g., 1, 2, 3 or 2, 3)
version	string	yes	CIS Controls version (v8)
source	string	yes	Data source identifier

owasp_top10 · table

OWASP Top 10 2021 entries parsed from Markdown files in the OWASP/Top10 GitHub repo. No canonical JSON exists; 10 Markdown files are fetched and parsed by the transformer.

Column	Type	Null	Description
entry_id	string	no	OWASP entry ID (e.g., A01:2021)
rank	integer	yes	Rank (1-10)
name	string	yes	Entry name (e.g., Broken Access Control)
overview	string	yes	Overview section prose (up to 2000 chars)
cwe_ids	string	yes	Pipe-delimited CWE IDs mentioned in the entry
year	integer	yes	OWASP Top 10 year (2021)
source	string	yes	Data source identifier

attack_to_nist_mappings · table

MITRE ATT&CK to NIST SP 800-53 control mappings from the center-for-threat-informed-defense/mappings-explorer GitHub repository. ATT&CK v16.1 mapped to NIST 800-53 Rev 5 enterprise domain. Records with status=non_mappable are excluded.

Column	Type	Null	Description
technique_id	string	no	ATT&CK technique ID (e.g., T1566, T1566.001)
attack_object_name	string	yes	ATT&CK technique name
nist_control_id	string	no	NIST 800-53 control ID (e.g., CM-03)
capability_group	string	yes	Control family code (e.g., CM, AC)
capability_description	string	yes	Control name
mapping_type	string	yes	Mapping relationship type (mitigates)
comments	string	yes	Mapping rationale
status	string	yes	Mapping status (complete)
source_version	string	yes	ATT&CK and NIST versions used for mapping
source	string	yes	Data source identifier

threat_pulses · table

AlienVault OTX subscribed-pulse threat intelligence. Requires CYBER_OTX_API_KEY environment variable (free registration at otx.alienvault.com). Cursor-based pagination via the "next" URL in the response envelope. Partitioned by first_seen (date portion of created) for TTL pruning.

Column	Type	Null	Description
pulse_id	string	no	OTX pulse UUID
name	string	yes	Pulse name
author	string	yes	Author username
tags	string	yes	Pipe-delimited tags
targeted_countries	string	yes	Pipe-delimited ISO country codes
malware_families	string	yes	Pipe-delimited malware family display names
attack_ids	string	yes	Pipe-delimited ATT&CK technique IDs
ioc_count	integer	yes	Number of IOC indicators in this pulse
created	string	yes	Creation timestamp (ISO-8601)
modified	string	yes	Last modified timestamp (ISO-8601)
tlp	string	yes	Traffic Light Protocol classification
source	string	yes	Feed source (otx)
first_seen	string	yes	Partition column (date portion of created)

active_threat_intel · view

Unified indicator-of-compromise view across IP, URL, and hash IOC tables. Normalises all three sources to a common schema for cross-indicator analysis.

View — columns are resolved by the query engine at runtime.