feat: add AdGuard-specific blocklist

- 7878367eeb
This commit is contained in:
MDLeom 2020-09-03 01:59:03 +00:00
parent eecb957b97
commit c9feacdc2e
3 changed files with 13213 additions and 10 deletions

View File

@ -7,10 +7,11 @@ There are multiple formats available, refer to the appropriate section according
- uBlock Origin (uBO) -> [URL-based](#url-based) section (recommended)
- Pi-hole -> [Domain-based](#domain-based) or [Hosts-based](#hosts-based) section
- AdGuard Home -> [Domain-based (AdGuard Home)](#domain-based-adguard-home) or [Hosts-based](#hosts-based) section
- Hosts file -> [Hosts-based](#hosts-based) section
- Dnsmasq -> [Dnsmasq](#dnsmasq) section
- BIND -> [BIND](#bind) section
- Unbound -> [Unbound](#unbound) section
- AdGuard browser extension -> [URL-based (AdGuard)](#url-based-adguard)
- [Hosts](#hosts-based)
- [Dnsmasq](#dnsmasq)
- [BIND](#bind)
- [Unbound](#unbound)
Not sure which format to choose? See [Compatibility](https://gitlab.com/curben/urlhaus-filter/wikis/compatibility) page.
@ -30,7 +31,25 @@ Import the following URL into uBO to subscribe:
- https://gitcdn.xyz/repo/curbengh/phishing-filter/master/dist/phishing-filter.txt
- https://cdn.jsdelivr.net/gh/curbengh/phishing-filter/dist/phishing-filter.txt
**Note:** AdGuard Home is not compatible with the URL filter (`||baddomain.com/bad/page$all`) of this blocklist, although it is still compatible with the domain filter (`||baddomain.com^`) also used in this blocklist, consider using a [smaller blocklist](#domain-based-adguard-home) (it's around half smaller).
</details>
**AdGuard Home** users should use [this blocklist](#domain-based-adguard-home), compatibility with URL-based blocklist is retained temporarily until in 30 Sep 2020 (UTC).
## URL-based (AdGuard)
Import the following URL into AdGuard browser extension to subscribe:
- https://gitlab.com/curben/phishing-filter/raw/master/dist/phishing-filter-ag.txt
<details>
<summary>Mirrors</summary>
- https://cdn.statically.io/gl/curben/phishing-filter/master/dist/phishing-filter-ag.txt
- https://glcdn.githack.com/curben/phishing-filter/raw/master/dist/phishing-filter-ag.txt
- https://raw.githubusercontent.com/curbengh/phishing-filter/master/dist/phishing-filter-ag.txt
- https://cdn.statically.io/gh/curbengh/phishing-filter/master/dist/phishing-filter-ag.txt
- https://gitcdn.xyz/repo/curbengh/phishing-filter/master/dist/phishing-filter-ag.txt
- https://cdn.jsdelivr.net/gh/curbengh/phishing-filter/dist/phishing-filter-ag.txt
</details>

13175
dist/phishing-filter-ag.txt vendored Normal file

File diff suppressed because it is too large Load Diff

View File

@ -18,8 +18,6 @@ fi
mkdir -p "tmp/"
cd "tmp/"
PHISHTANK_API="f6d24512707c96f8d01875ace77d52950a5b9157e1535ea70e7f704efb3e4066"
## Prepare datasets
curl -L "https://data.phishtank.com/data/$PHISHTANK_API/online-valid.csv.bz2" -o "phishtank.bz2"
curl -L "https://openphish.com/feed.txt" -o "openphish-raw.txt"
@ -116,19 +114,30 @@ COMMENT_UBO="$FIRST_LINE\n$SECOND_LINE\n$THIRD_LINE\n$FOURTH_LINE\n$FIFTH_LINE\n
# curben/urlhaus-filter#19
cat "phishing-notop-domains.txt" | \
sed "s/^/||/g" | \
sed "s/$/^/g" > "phishing-domains-adguard.txt"
sed "s/$/^/g" > "phishing-domains-adguard-home.txt"
# curben/urlhaus-filter#22
cat "phishing-domains-adguard.txt" | \
cat "phishing-domains-adguard-home.txt" | \
sort | \
sed '1 i\'"$COMMENT_UBO"'' | \
sed "1s/Blocklist/Blocklist (AdGuard Home)/" > "../dist/phishing-filter-agh.txt"
cat "phishing-domains-adguard.txt" "phishing-url-top-domains.txt" | \
cat "phishing-domains-adguard-home.txt" "phishing-url-top-domains.txt" | \
sort | \
sed '1 i\'"$COMMENT_UBO"'' > "../dist/phishing-filter.txt"
# Adguard browser extension compatibility
cat "phishing-notop-domains.txt" | \
sed "s/^/||/g" | \
sed "s/$/\$all/g" > "phishing-domains-adguard.txt"
cat "phishing-domains-adguard.txt" "phishing-url-top-domains.txt" | \
sort | \
sed '1 i\'"$COMMENT_UBO"'' | \
sed "1s/Blocklist/Blocklist (AdGuard)/" > "../dist/phishing-filter-ag.txt"
## Domains-only blocklist
# awk + head is a workaround for sed prepend
COMMENT=$(printf "$COMMENT_UBO" | sed "s/^!/#/g" | sed "1s/URL/Domains/" | awk '{printf "%s\\n", $0}' | head -c -2)