curben
/
phishing-filter
mirror of https://gitlab.com/malware-filter/phishing-filter
1
0
Fork 0
Code Issues Releases Wiki Activity

chore: remove remaining phishunt 

no longer used since #43 #45
This commit is contained in:
MDLeom 2024-03-07 10:14:08 +00:00
parent b3f6e90b9a
commit 93b85b00f9
No known key found for this signature in database
GPG Key ID: 32D3E28E96A695E8
2 changed files with 2 additions and 17 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
README.md
View File

@ -258,10 +258,6 @@ https://gitlab.com/curben/blog#repository-mirrors
filters: [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/)
[PhishTank](https://www.phishtank.com/): Available [free of charge](https://www.phishtank.com/faq.php#isitoktousetheapifor) by Cisco for commercial and non-commercial use.
_PhishTank is either trademark or registered trademark of Cisco Systems, Inc._
[OpenPhish](https://openphish.com/): Available [free of charge](https://openphish.com/terms.html) by OpenPhish
[Tranco List](https://tranco-list.eu/): [MIT License](https://choosealicense.com/licenses/mit/)
@ -270,8 +266,6 @@ _PhishTank is either trademark or registered trademark of Cisco Systems, Inc._
[csvquote](https://github.com/dbro/csvquote): MIT License
[phishunt.io](https://phishunt.io/): All rights reserved by [Daniel López](https://twitter.com/0xDanielLopez)
[Cloudflare Radar](https://developers.cloudflare.com/radar/investigate/domain-ranking-datasets/): Available to free Cloudflare account
This repository is not endorsed by PhishTank/OpenDNS and OpenPhish.

13
src/script.sh
View File

@ -56,7 +56,6 @@ cd "tmp/"
## Prepare datasets
curl "https://openphish.com/feed.txt" -o "openphish-raw.txt"
curl "https://github.com/0xDanielLopez/TweetFeed/raw/master/week.csv" -o "phishunt.csv"
curl "https://s3-us-west-1.amazonaws.com/umbrella-static/top-1m.csv.zip" -o "top-1m-umbrella.zip"
curl "https://tranco-list.eu/top-1m.csv.zip" -o "top-1m-tranco.zip"
@ -99,14 +98,6 @@ sed "s/^www\.//g" | \
# url encode space #11
sed "s/ /%20/g" > "openphish.txt"
cat "phishunt.csv" | \
tr "[:upper:]" "[:lower:]" | \
grep -F "#phishing" | \
cut -f 4 -d "," | \
cut -f 3- -d "/" | \
grep -F "." | \
sed "s/^www\.//g" > "phishunt.txt"
## Combine all sources
sort -u "openphish.txt" > "phishing.txt"
@ -231,7 +222,7 @@ SECOND_LINE="! Updated: $CURRENT_TIME"
THIRD_LINE="! Expires: 1 day (update frequency)"
FOURTH_LINE="! Homepage: https://gitlab.com/malware-filter/phishing-filter"
FIFTH_LINE="! License: https://gitlab.com/malware-filter/phishing-filter#license"
SIXTH_LINE="! Sources: openphish.com, phishunt.io"
SIXTH_LINE="! Sources: openphish.com"
COMMENT_UBO="$FIRST_LINE\n$SECOND_LINE\n$THIRD_LINE\n$FOURTH_LINE\n$FIFTH_LINE\n$SIXTH_LINE"
mkdir -p "../public/"
@ -420,7 +411,7 @@ sed "2s/Domains Blocklist/Hosts Blocklist (IE)/" > "../public/phishing-filter.tp
## Clean up artifacts
rm "top-1m-umbrella.zip" "top-1m-umbrella.txt" "top-1m-tranco.txt" "openphish-raw.txt" "phishunt.csv" "cf/" "top-1m-radar.txt"
rm "top-1m-umbrella.zip" "top-1m-umbrella.txt" "top-1m-tranco.txt" "openphish-raw.txt" "cf/" "top-1m-radar.txt"
cd ../