curben
/
splunk-malware-filter
mirror of https://gitlab.com/malware-filter/splunk-malware-filter
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

fix: normalise "updated" column 

with ISO-8601 format
- fix command example
This commit is contained in:
Ming Di Leom 2023-01-30 10:08:25 +00:00
parent a3ca016974
commit 31638daa85
No known key found for this signature in database
GPG Key ID: 32D3E28E96A695E8
3 changed files with 27 additions and 16 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
README.md
View File

@ -89,7 +89,7 @@ Output columns are listed here https://gitlab.com/malware-filter/phishing-filter
## getpupfilter
```
| getphishingfilter message=<string>
| getpupfilter wildcard_prefix=<string> wildcard_suffix=<string> wildcard_affix=<string> message=<string>
| outputlookup override_if_empty=false pup-filter-splunk.csv
```
@ -98,7 +98,7 @@ Output columns are listed here https://gitlab.com/malware-filter/pup-filter#splu
## getvnbadsitefilter
```
| getphishingfilter wildcard_prefix=<string> wildcard_suffix=<string> wildcard_affix=<string> message=<string>
| getvnbadsitefilter wildcard_prefix=<string> wildcard_suffix=<string> wildcard_affix=<string> message=<string>
| outputlookup override_if_empty=false vn-badsite-filter-splunk.csv
```
@ -109,7 +109,7 @@ Output columns are listed here https://gitlab.com/malware-filter/vn-badsite-filt
Highly recommend to use [`getbotnetip`](#getbotnetip) instead.
```
| getphishingfilter message=<string>
| getbotnetfilter message=<string>
| outputlookup override_if_empty=false botnet-filter-splunk.csv
```
@ -120,27 +120,32 @@ Output columns are listed here https://gitlab.com/malware-filter/botnet-filter#s
Recommend to update the lookup file "botnet_ip.csv" every 5 minutes (cron `*/5 * * * *`).
```
| getphishingfilter wildcard_prefix=<string> wildcard_suffix=<string> wildcard_affix=<string> message=<string>
| getbotnetip message=<string>
| outputlookup override_if_empty=false botnet_ip.csv
```
Source: https://feodotracker.abuse.ch/downloads/ipblocklist.csv
Columns:
| first_seen_utc | dst_ip | dst_port | c2_status | last_online | malware | last_updated_utc |
| ------------------- | ------------- | -------- | --------- | ----------- | ------- | ------------------- |
| 2021-01-17 07:44:46 | 51.178.161.32 | 4643 | online | 2023-01-26 | Dridex | 2023-01-25 17:41:16 |
| first_seen_utc | dst_ip | dst_port | c2_status | last_online | malware | updated |
| ------------------- | ------------- | -------- | --------- | ----------- | ------- | -------------------- |
| 2021-01-17 07:44:46 | 51.178.161.32 | 4643 | online | 2023-01-26 | Dridex | 2023-01-25T17:41:16Z |
Source: https://feodotracker.abuse.ch/downloads/ipblocklist.csv
## getopendbl
Recommend to update the lookup file "opendbl_ip.csv" every 15 minutes (cron `*/15 * * * *`).
```
| getopendbl wildcard_prefix=<string> wildcard_suffix=<string> wildcard_affix=<string> message=<string>
| getopendbl message=<string>
| outputlookup override_if_empty=false opendbl_ip.csv
```
| start | end | netmask | cidr | name | updated |
| --------------- | --------------- | ------- | ------------------ | ----------------------------------------- | -------------------- |
| 187.190.252.167 | 187.190.252.167 | 32 | 187.190.252.167/32 | Emerging Threats: Known Compromised Hosts | 2023-01-30T08:03:00Z |
| 89.248.163.0 | 89.248.163.255 | 24 | 89.248.163.0/24 | Dshield | 2023-01-30T08:01:00Z |
Source: https://opendbl.net/
## Disable individual commands

10
bin/getbotnetip.py
View File

@ -26,21 +26,27 @@ class GetBotnetIP(Utility, GeneratingCommand):
def generate(self):
feodo_csv = self.download(DOWNLOAD_URL)
last_updated_utc = datetime.now(timezone.utc).isoformat(timespec="seconds")
updated = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
# parse updated time from header comment
for line in filter(lambda row: row[0] == "#", feodo_csv.splitlines()):
if line.startswith("# Last updated:"):
last_updated_utc = search(
r"\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}", line
).group()
updated = (
datetime.strptime(last_updated_utc, "%Y-%m-%d %H:%M:%S")
.replace(tzinfo=timezone.utc)
.strftime("%Y-%m-%dT%H:%M:%SZ")
)
break
# parse input csv, remove '#' comments and output as events
for row in self.csv_reader(feodo_csv):
row["last_updated_utc"] = last_updated_utc
row["updated"] = updated
if isinstance(self.custom_message, str) and len(self.custom_message) >= 1:
row["custom_message"] = self.custom_message
yield self.gen_record(**row)
break
if __name__ == "__main__":

8
bin/getopendbl.py
View File

@ -36,7 +36,7 @@ class GetOpenDBL(Utility, GeneratingCommand):
def generate(self):
for name, dl_path in OPENDBL_LIST.items():
blocklist = self.download(f"https://opendbl.net/lists/{dl_path}")
last_updated_utc = datetime.now(timezone.utc).isoformat(timespec="seconds")
updated = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
# parse updated time from header comment
for line in filter(lambda row: row[0] == "#", blocklist.splitlines()):
if "Last updated" in line:
@ -44,10 +44,10 @@ class GetOpenDBL(Utility, GeneratingCommand):
r"\d{4}-\d{2}-\d{2} \d{2}:\d{2}", line
).group()
# Assume UTC timezone
last_updated_utc = (
updated = (
datetime.strptime(last_updated, "%Y-%m-%d %H:%M")
.replace(tzinfo=timezone.utc)
.isoformat()
.strftime("%Y-%m-%dT%H:%M:%SZ")
)
break
@ -58,7 +58,7 @@ class GetOpenDBL(Utility, GeneratingCommand):
"netmask": "32",
"cidr": f"{line}/32",
"name": name,
"last_updated_utc": last_updated_utc,
"updated": updated,
}
if "-" in line: