Go to file

curben 958fc5c45c docs: add URLhaus' license		2020-07-07 06:29:30 +01:00
.gitlab	Success pipeline	2020-06-06 00:09:59 +00:00
src	fix(exclude): add 'hampsteadclinic.co.uk'	2020-06-08 01:57:21 +01:00
utils	fix(status-badge): remove dependency on Shields.io	2020-04-01 10:33:08 +01:00
.gitignore	chore(gitignore): remove unnecessary regex	2020-03-30 03:34:58 +01:00
.gitlab-ci.yml	feat: remove URLhaus.csv	2020-05-14 10:44:25 +01:00
LICENSE.md	Initial commit	2018-10-09 16:48:46 +10:30
README.md	docs: add URLhaus' license	2020-07-07 06:29:30 +01:00
script.sh	fix: more accurate static filter	2020-07-07 02:23:10 +01:00
urlhaus-filter-bind-online.conf	Filter updated: Tue, 07 Jul 2020 00:09:59 UTC	2020-07-07 00:10:00 +00:00
urlhaus-filter-bind.conf	Filter updated: Tue, 07 Jul 2020 00:09:59 UTC	2020-07-07 00:10:00 +00:00
urlhaus-filter-dnsmasq-online.conf	Filter updated: Tue, 07 Jul 2020 00:09:59 UTC	2020-07-07 00:10:00 +00:00
urlhaus-filter-dnsmasq.conf	Filter updated: Tue, 07 Jul 2020 00:09:59 UTC	2020-07-07 00:10:00 +00:00
urlhaus-filter-domains-online.txt	Filter updated: Tue, 07 Jul 2020 00:09:59 UTC	2020-07-07 00:10:00 +00:00
urlhaus-filter-domains.txt	Filter updated: Tue, 07 Jul 2020 00:09:59 UTC	2020-07-07 00:10:00 +00:00
urlhaus-filter-hosts-online.txt	Filter updated: Tue, 07 Jul 2020 00:09:59 UTC	2020-07-07 00:10:00 +00:00
urlhaus-filter-hosts.txt	Filter updated: Tue, 07 Jul 2020 00:09:59 UTC	2020-07-07 00:10:00 +00:00
urlhaus-filter-online.txt	Filter updated: Tue, 07 Jul 2020 00:09:59 UTC	2020-07-07 00:10:00 +00:00
urlhaus-filter-unbound-online.conf	Filter updated: Tue, 07 Jul 2020 00:09:59 UTC	2020-07-07 00:10:00 +00:00
urlhaus-filter-unbound.conf	Filter updated: Tue, 07 Jul 2020 00:09:59 UTC	2020-07-07 00:10:00 +00:00
urlhaus-filter.txt	Filter updated: Tue, 07 Jul 2020 00:09:59 UTC	2020-07-07 00:10:00 +00:00

README.md

URLhaus Malicious URL Blocklist

A blocklist of malicious websites that are being used for malware distribution, based on the Database dump (CSV) of Abuse.ch URLhaus. Blocklist is updated twice a day.

There are multiple formats available, refer to the appropriate section according to the program used:

uBlock Origin (uBO) -> URL-based section (recommended)
Pi-hole -> Domain-based or Hosts-based section
Hosts file -> Hosts-based section
Dnsmasq -> Dnsmasq section
BIND -> BIND section
Unbound -> Unbound section

Not sure which format to choose? See Compatibility page in the wiki.

URL-based

Import the following URL into uBO to subscribe (includes online and offline malicious websites):

https://gitlab.com/curben/urlhaus-filter/raw/master/urlhaus-filter.txt

Mirrors

Lite version (online links only):

https://gitlab.com/curben/urlhaus-filter/raw/master/urlhaus-filter-online.txt

Mirrors

Note: Lite version is ~95% smaller by excluding offline urls. The status of urls is determined by the upstream Abuse.ch. However, the test is not 100% accurate and some malicious urls that are otherwise accessible may be missed. If bandwidth (4 MB/day) is not a constraint, I recommend the regular version.

PS: While regular version contains roughly >145K filters, uBO can easily handle half a million filters.

Domain-based

This blocklist includes domains and IP addresses.

https://gitlab.com/curben/urlhaus-filter/raw/master/urlhaus-filter-domains.txt

Mirrors

Lite version (online domains/IPs only):

https://gitlab.com/curben/urlhaus-filter/raw/master/urlhaus-filter-domains-online.txt

Mirrors

Hosts-based

This blocklist includes domains only.

https://gitlab.com/curben/urlhaus-filter/raw/master/urlhaus-filter-hosts.txt

Mirrors

Lite version (online domains only):

https://gitlab.com/curben/urlhaus-filter/raw/master/urlhaus-filter-hosts-online.txt

Mirrors

Dnsmasq

This blocklist includes domains only.

Install

# Create a new folder to store the blocklist
mkdir -p /usr/local/etc/dnsmasq/

# Create a new cron job for daily update
printf '#!/bin/sh\ncurl -L "https://gitlab.com/curben/urlhaus-filter/raw/master/urlhaus-filter-dnsmasq.conf" -o "/usr/local/etc/dnsmasq/urlhaus-filter-dnsmasq.conf"\n' > /etc/cron.daily/urlhaus-filter

# cron job requires execution permission
chmod 755 /etc/cron.daily/urlhaus-filter

# Configure dnsmasq to use the blocklist
printf "\nconf-file=/usr/local/etc/dnsmasq/urlhaus-filter-dnsmasq.conf\n" >> /etc/dnsmasq.conf

Mirrors

Lite version (online domains only):

https://gitlab.com/curben/urlhaus-filter/raw/master/urlhaus-filter-dnsmasq-online.conf

Mirrors

BIND

This blocklist includes domains only.

Install

# Create a new folder to store the blocklist
mkdir -p /usr/local/etc/bind/

# Create a new cron job for daily update
printf '#!/bin/sh\ncurl -L "https://gitlab.com/curben/urlhaus-filter/raw/master/urlhaus-filter-bind.conf" -o "/usr/local/etc/bind/urlhaus-filter-bind.conf"\n' > /etc/cron.daily/urlhaus-filter

# cron job requires execution permission
chmod 755 /etc/cron.daily/urlhaus-filter

# Configure BIND to use the blocklist
printf '\ninclude "/usr/local/etc/bind/urlhaus-filter-bind.conf";\n' >> /etc/bind/named.conf

Add this to "/etc/bind/null.zone.file" (skip this step if the file already exists):

$TTL    86400   ; one day
@       IN      SOA     ns.nullzone.loc. ns.nullzone.loc. (
               2017102203
                    28800
                     7200
                   864000
                    86400 )
                NS      ns.nullzone.loc.
                A       0.0.0.0
@       IN      A       0.0.0.0
*       IN      A       0.0.0.0

Zone file is derived from here.

Mirrors

Lite version (online domains only):

https://gitlab.com/curben/urlhaus-filter/raw/master/urlhaus-filter-bind-online.conf

Mirrors

Unbound

This blocklist includes domains only.

Install

# Create a new folder to store the blocklist
mkdir -p /usr/local/etc/unbound/

# Create a new cron job for daily update
printf '#!/bin/sh\ncurl -L "https://gitlab.com/curben/urlhaus-filter/raw/master/urlhaus-filter-unbound.conf" -o "/usr/local/etc/unbound/urlhaus-filter-unbound.conf"\n' > /etc/cron.daily/urlhaus-filter

# cron job requires execution permission
chmod 755 /etc/cron.daily/urlhaus-filter

# Configure Unbound to use the blocklist
printf '\n  include: "/usr/local/etc/unbound/urlhaus-filter-unbound.conf"\n' >> /etc/unbound/unbound.conf

Mirrors

Lite version (online domains only):

https://gitlab.com/curben/urlhaus-filter/raw/master/urlhaus-filter-unbound-online.conf

Mirrors

Issues

This blocklist operates by blocking the whole website, instead of specific webpages; exceptions are made on popular websites (e.g. https://docs.google.com/), in which webpages are specified instead (e.g. https://docs.google.com/malware-page). Malicious webpages are only listed in the URL-based filter, popular websites are excluded from other filters.

Popular websites are as listed in the Umbrella Popularity List (top 1M domains + subdomains), Tranco List (top 1M domains) and this custom list.

If you wish to exclude certain website(s) that you believe is sufficiently well-known, please create an issue or merge request.

This filter only accepts new malware URLs from URLhaus.

Please report new malware URL to the upstream maintainer through https://urlhaus.abuse.ch/api/#submit.

Cloning

Since the filter is updated frequently, cloning the repo would become slower over time as the revision grows.

Use shallow clone to get the recent revisions only. Getting the last five revisions should be sufficient for a valid MR.

git clone --depth 5 https://gitlab.com/curben/urlhaus-filter.git

FAQ

See FAQ.

License

Creative Commons Zero v1.0 Universal

URLhaus: CC0

Tranco List: MIT License

Umbrella Popularity List: Available free of charge by Cisco Umbrella

This repository is not endorsed by Abuse.ch.