curben
/
phishing-filter
mirror of https://gitlab.com/malware-filter/phishing-filter
1
0
Fork 0
Code Issues Releases Wiki Activity

fix: remove mitchellkrogza/Phishing.Database source 

- close #35
This commit is contained in:
MDLeom 2022-09-27 09:43:30 +00:00
parent 6c27bf33aa
commit e68268f506
No known key found for this signature in database
GPG Key ID: 32D3E28E96A695E8
2 changed files with 3 additions and 13 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@ -1,6 +1,6 @@
# Phishing URL Blocklist
A blocklist of phishing websites, curated from [PhishTank](https://www.phishtank.com/), [OpenPhish](https://openphish.com/), [phishunt.io](https://phishunt.io/) and [mitchellkrogza/Phishing.Database](https://github.com/mitchellkrogza/Phishing.Database/blob/master/phishing-domains-ACTIVE.txt). Blocklist is updated twice a day.
A blocklist of phishing websites, curated from [PhishTank](https://www.phishtank.com/), [OpenPhish](https://openphish.com/), [phishunt.io](https://phishunt.io/). Blocklist is updated twice a day.
There are multiple formats available, refer to the appropriate section according to the program used:
@ -485,6 +485,4 @@ _PhishTank is either trademark or registered trademark of Cisco Systems, Inc._
[phishunt.io](https://phishunt.io/): All rights reserved by [Daniel López](https://twitter.com/0xDanielLopez)
[mitchellkrogza/Phishing.Database](https://github.com/mitchellkrogza/Phishing.Database): MIT License
This repository is not endorsed by PhishTank/OpenDNS and OpenPhish.

12
src/script.sh
View File

@ -39,7 +39,6 @@ cd "tmp/"
curl -L "https://data.phishtank.com/data/$PHISHTANK_API/online-valid.csv.bz2" -o "phishtank.bz2"
curl -L "https://openphish.com/feed.txt" -o "openphish-raw.txt"
curl -L "https://phishunt.io/feed.txt" -o "phishunt-raw.txt"
curl -L "https://github.com/mitchellkrogza/Phishing.Database/raw/master/phishing-links-ACTIVE.txt" -o "phishing.db-raw.txt"
curl -L "https://s3-us-west-1.amazonaws.com/umbrella-static/top-1m.csv.zip" -o "top-1m-umbrella.zip"
curl -L "https://tranco-list.eu/top-1m.csv.zip" -o "top-1m-tranco.zip"
@ -76,15 +75,8 @@ grep -F "." | \
sed "s/^www\.//g" | \
sed "s/ /%20/g" > "phishunt.txt"
cat "phishing.db-raw.txt" | \
tr "[:upper:]" "[:lower:]" | \
cut -f 3- -d "/" | \
grep -F "." | \
sed "s/^www\.//g" | \
sed "s/ /%20/g" > "phishing.db.txt"
## Combine all sources
cat "phishtank.txt" "openphish.txt" "phishunt.txt" "phishing.db.txt" | \
cat "phishtank.txt" "openphish.txt" "phishunt.txt" | \
sort -u > "phishing.txt"
## Parse domain and IP address only
@ -181,7 +173,7 @@ SECOND_LINE="! Updated: $CURRENT_TIME"
THIRD_LINE="! Expires: 1 day (update frequency)"
FOURTH_LINE="! Homepage: https://gitlab.com/malware-filter/phishing-filter"
FIFTH_LINE="! License: https://gitlab.com/malware-filter/phishing-filter#license"
SIXTH_LINE="! Sources: phishtank.com, openphish.com, phishunt.io, github.com/mitchellkrogza/Phishing.Database"
SIXTH_LINE="! Sources: phishtank.com, openphish.com, phishunt.io"
COMMENT_UBO="$FIRST_LINE\n$SECOND_LINE\n$THIRD_LINE\n$FOURTH_LINE\n$FIFTH_LINE\n$SIXTH_LINE"
mkdir -p "../public/"