curben
/
urlhaus-filter
mirror of https://gitlab.com/malware-filter/urlhaus-filter
1
0
Fork 0
Code Issues Releases Wiki Activity
2,362 Commits 2 Branches 0 Tags 657 MiB
Shell 90.2%
JavaScript 9.8%
Go to file
curben-bot 916a0af5c2 Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
.gitlab Success pipeline 2021-07-27 00:10:17 +00:00
src fix(exclude): 104.244.72.54 2021-09-05 07:20:04 +00:00
utils fix(status-badge): remove dependency on Shields.io 2020-04-01 10:33:08 +01:00
.gitignore chore: ignore folders 2020-10-08 04:48:56 +00:00
.gitlab-ci.yml chore(ci): enable secret detection 2021-06-20 00:20:34 +00:00
LICENSE.md Initial commit 2018-10-09 16:48:46 +10:30
README.md feat: dnscrypt-proxy blocklists 2021-07-18 09:01:55 +00:00
script.sh fix: stricter IPv4 matching 2021-07-18 09:06:03 +00:00
urlhaus-filter-ag-online.txt Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-ag.txt Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-agh-online.txt Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-agh.txt Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-bind-online.conf Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-bind.conf Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-dnscrypt-blocked-ips-online.txt Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-dnscrypt-blocked-ips.txt Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-dnscrypt-blocked-names-online.txt Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-dnscrypt-blocked-names.txt Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-dnsmasq-online.conf Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-dnsmasq.conf Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-domains-online.txt Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-domains.txt Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-hosts-online.txt Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-hosts.txt Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-online.tpl Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-online.txt Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-rpz-online.conf Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-rpz.conf Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-snort2-online.rules Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-snort3-online.rules Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-suricata-online.rules Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-unbound-online.conf Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-unbound.conf Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-vivaldi-online.txt Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter-vivaldi.txt Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter.tpl Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00
urlhaus-filter.txt Filter updated: Tue, 14 Sep 2021 12:10:43 +0000 2021-09-14 12:10:44 +00:00

README.md

Malicious URL Blocklist

A blocklist of malicious websites that are being used for malware distribution, based on the Database dump (CSV) of Abuse.ch URLhaus. Blocklist is updated twice a day.

There are multiple formats available, refer to the appropriate section according to the program used:

Not sure which format to choose? See Compatibility page in the wiki.

Check out my other filters:

URL-based

Import the following URL into uBO to subscribe (includes online and offline malicious websites):

Mirrors

Lite version (online links only):

enabled by default in uBO >=1.28.2

Mirrors

Note: Lite version is 99% smaller by excluding offline urls. The status of urls is determined by the upstream Abuse.ch. However, the test is not 100% accurate and some malicious urls that are otherwise accessible may be missed. If bandwidth (9 MB/day) is not a constraint, I recommend the regular version; browser extensions may utilise HTTP compression that can save 70% of bandwidth.

Regular version contains >260K filters, do note that uBO can easily handle 500K filters.

If you've installed the lite version but prefer to use the regular version, it's better to remove it beforehand. Having two versions at the same time won't cause any conflict issue, uBO can detect duplicate network filters and adjust accordingly, but it's a waste of your bandwidth.

AdGuard Home users should use this blocklist.

URL-based (AdGuard)

Import the following URL into AdGuard browser extensions to subscribe (includes online and offline malicious websites):

Mirrors

Lite version (online links only):

Mirrors

URL-based (Vivaldi)

Requires Vivaldi Desktop/Android 3.3+, blocking level must be at least "Block Trackers"

Import the following URL into Vivaldi's Tracker Blocking Sources to subscribe (includes online and offline malicious websites):

Mirrors

Lite version (online links only):

Mirrors

Domain-based

This blocklist includes domains and IP addresses.

Mirrors

Lite version (online domains/IPs only):
Mirrors

Domain-based (AdGuard Home)

This AdGuard Home-compatible blocklist includes domains and IP addresses.

Mirrors

Lite version (online domains/IPs only):
Mirrors

Hosts-based

This blocklist includes domains only.

Mirrors

Lite version (online domains only):
Mirrors

Dnsmasq

This blocklist includes domains only.

Install

# Create a new folder to store the blocklist
mkdir -p /usr/local/etc/dnsmasq/

# Create a new cron job for daily update
printf '#!/bin/sh\ncurl -L "https://curben.gitlab.io/malware-filter/urlhaus-filter-dnsmasq.conf" -o "/usr/local/etc/dnsmasq/urlhaus-filter-dnsmasq.conf"\n' > /etc/cron.daily/urlhaus-filter

# cron job requires execution permission
chmod 755 /etc/cron.daily/urlhaus-filter

# Configure dnsmasq to use the blocklist
printf "\nconf-file=/usr/local/etc/dnsmasq/urlhaus-filter-dnsmasq.conf\n" >> /etc/dnsmasq.conf
Mirrors

Lite version (online domains only):
Mirrors

BIND

This blocklist includes domains only.

Install

# Create a new folder to store the blocklist
mkdir -p /usr/local/etc/bind/

# Create a new cron job for daily update
printf '#!/bin/sh\ncurl -L "https://curben.gitlab.io/malware-filter/urlhaus-filter-bind.conf" -o "/usr/local/etc/bind/urlhaus-filter-bind.conf"\n' > /etc/cron.daily/urlhaus-filter

# cron job requires execution permission
chmod 755 /etc/cron.daily/urlhaus-filter

# Configure BIND to use the blocklist
printf '\ninclude "/usr/local/etc/bind/urlhaus-filter-bind.conf";\n' >> /etc/bind/named.conf

Add this to "/etc/bind/null.zone.file" (skip this step if the file already exists):

$TTL    86400   ; one day
@       IN      SOA     ns.nullzone.loc. ns.nullzone.loc. (
               2017102203
                    28800
                     7200
                   864000
                    86400 )
                NS      ns.nullzone.loc.
                A       0.0.0.0
@       IN      A       0.0.0.0
*       IN      A       0.0.0.0

Zone file is derived from here.

Mirrors

Lite version (online domains only):
Mirrors

Response Policy Zone

This blocklist includes domains only.

Mirrors

Lite version (online domains only):
Mirrors

Unbound

This blocklist includes domains only.

Install

# Create a new folder to store the blocklist
mkdir -p /usr/local/etc/unbound/

# Create a new cron job for daily update
printf '#!/bin/sh\ncurl -L "https://curben.gitlab.io/malware-filter/urlhaus-filter-unbound.conf" -o "/usr/local/etc/unbound/urlhaus-filter-unbound.conf"\n' > /etc/cron.daily/urlhaus-filter

# cron job requires execution permission
chmod 755 /etc/cron.daily/urlhaus-filter

# Configure Unbound to use the blocklist
printf '\n  include: "/usr/local/etc/unbound/urlhaus-filter-unbound.conf"\n' >> /etc/unbound/unbound.conf
Mirrors

Lite version (online domains only):
Mirrors

dnscrypt-proxy

Install

# Create a new folder to store the blocklist
mkdir -p /etc/dnscrypt-proxy/

# Create a new cron job for daily update
printf '#!/bin/sh\ncurl -L "https://curben.gitlab.io/malware-filter/urlhaus-filter-dnscrypt-blocked-names.txt" -o "/etc/dnscrypt-proxy/urlhaus-filter-dnscrypt-blocked-names.txt"\n' > /etc/cron.daily/urlhaus-filter
printf '\ncurl -L "https://curben.gitlab.io/malware-filter/urlhaus-filter-dnscrypt-blocked-ips.txt" -o "/etc/dnscrypt-proxy/urlhaus-filter-dnscrypt-blocked-ips.txt"\n' >> /etc/cron.daily/urlhaus-filter

# cron job requires execution permission
chmod 755 /etc/cron.daily/urlhaus-filter

Configure dnscrypt-proxy to use the blocklist:

[blocked_names]
+  blocked_names_file = '/etc/dnscrypt-proxy/urlhaus-filter-dnscrypt-blocked-names.txt'

[blocked_ips]
+  blocked_ips_file = '/etc/dnscrypt-proxy/urlhaus-filter-dnscrypt-blocked-ips.txt'
Mirrors

Lite version (online domains only):
Mirrors

Tracking Protection List (IE)

This blocklist includes domains only. Supported in Internet Explorer 9+.

Mirrors

Lite version (online domains only):
Mirrors

Snort2

This ruleset includes online URLs only. Not compatible with Snort3.

Install

# Download ruleset
curl -L "https://curben.gitlab.io/malware-filter/urlhaus-filter-snort2-online.rules" -o "/etc/snort/rules/urlhaus-filter-snort2-online.rules"

# Create a new cron job for daily update
printf '#!/bin/sh\ncurl -L "https://curben.gitlab.io/malware-filter/urlhaus-filter-snort2-online.rules" -o "/etc/snort/rules/urlhaus-filter-snort2-online.rules"\n' > /etc/cron.daily/urlhaus-filter

# cron job requires execution permission
chmod 755 /etc/cron.daily/urlhaus-filter

# Configure Snort to use the ruleset
printf "\ninclude \$RULE_PATH/urlhaus-filter-snort2-online.rules\n" >> /etc/snort/snort.conf
Mirrors

Snort3

This ruleset includes online URLs only. Not compatible with Snort2.

Install

# Download ruleset
curl -L "https://curben.gitlab.io/malware-filter/urlhaus-filter-snort3-online.rules" -o "/etc/snort/rules/urlhaus-filter-snort3-online.rules"

# Create a new cron job for daily update
printf '#!/bin/sh\ncurl -L "https://curben.gitlab.io/malware-filter/urlhaus-filter-snort3-online.rules" -o "/etc/snort/rules/urlhaus-filter-snort3-online.rules"\n' > /etc/cron.daily/urlhaus-filter

# cron job requires execution permission
chmod 755 /etc/cron.daily/urlhaus-filter

Configure Snort to use the ruleset:

# /etc/snort/snort.lua
ips =
{
  variables = default_variables,
+  include = 'rules/urlhaus-filter-snort3-online.rules'
}
Mirrors

Suricata

This ruleset includes online URLs only.

Install

# Download ruleset
curl -L "https://curben.gitlab.io/malware-filter/urlhaus-filter-suricata-online.rules" -o "/etc/suricata/rules/urlhaus-filter-suricata-online.rules"

# Create a new cron job for daily update
printf '#!/bin/sh\ncurl -L "https://curben.gitlab.io/malware-filter/urlhaus-filter-suricata-online.rules" -o "/etc/suricata/rules/urlhaus-filter-suricata-online.rules"\n' > /etc/cron.daily/urlhaus-filter

# cron job requires execution permission
chmod 755 /etc/cron.daily/urlhaus-filter

Configure Suricata to use the ruleset:

# /etc/suricata/suricata.yaml
rule-files:
  - local.rules
+  - urlhaus-filter-suricata-online.rules
Mirrors

Third-party mirrors

iosprivacy/urlhaus-filter-mirror

Issues

This blocklist operates by blocking the whole website, instead of specific webpages; exceptions are made on popular websites (e.g. https://docs.google.com/), in which webpages are specified instead (e.g. https://docs.google.com/malware-page). Malicious webpages are only listed in the URL-based filter, popular websites are excluded from other filters.

Popular websites are as listed in the Umbrella Popularity List (top 1M domains + subdomains), Tranco List (top 1M domains) and this custom list.

If you wish to exclude certain website(s) that you believe is sufficiently well-known, please create an issue or merge request. If the website is quite obscure but you still want to visit it, you can add a new line ||legitsite.com^$badfilter to "My filters" tab of uBO; use a subdomain if relevant, ||sub.legitsite.com^$badfilter.

This filter only accepts new malware URLs from URLhaus.

Please report new malware URL to the upstream maintainer through https://urlhaus.abuse.ch/api/#submit.

Cloning

Since the filter is updated frequently, cloning the repo would become slower over time as the revision grows.

Use shallow clone to get the recent revisions only. Getting the last five revisions should be sufficient for a valid MR.

git clone --depth 5 https://gitlab.com/curben/urlhaus-filter.git

FAQ

See FAQ.

License

Creative Commons Zero v1.0 Universal

badge.sh & .gitlab/ contain badges that are licensed by Shields.io under CC0 1.0

URLhaus: CC0

Tranco List: MIT License

Umbrella Popularity List: Available free of charge by Cisco Umbrella

This repository is not endorsed by Abuse.ch.