fix: make PHISHTANK_API optional

- add TOC

README.md

@@ -1,5 +1,27 @@
 # Phishing URL Blocklist
 
+- Formats
+  * [URL-based](#url-based)
+  * [Domain-based](#domain-based)
+  * [Hosts-based](#hosts-based)
+  * [Domain-based (AdGuard Home)](#domain-based-adguard-home)
+  * [URL-based (AdGuard)](#url-based-adguard)
+  * [URL-based (Vivaldi)](#url-based-vivaldi)
+  * [Dnsmasq](#dnsmasq)
+  * [BIND zone](#bind)
+  * [RPZ](#response-policy-zone)
+  * [Unbound](#unbound)
+  * [dnscrypt-proxy](#dnscrypt-proxy)
+  * [Tracking Protection List (IE)](#tracking-protection-list-ie)
+  * [Snort2](#snort2)
+  * [Snort3](#snort3)
+  * [Suricata](#suricata)
+- [Reporting issues](#issues)
+- [See alse](#see-also)
+- [FAQ and Guides](#faq-and-guides)
+- [CI Variables](#ci-variables)
+- [License](#license)
+
 A blocklist of phishing websites, curated from [PhishTank](https://www.phishtank.com/), [OpenPhish](https://openphish.com/), [phishunt.io](https://phishunt.io/). Blocklist is updated twice a day.
 
 There are multiple formats available, refer to the appropriate section according to the program used:
@@ -465,6 +487,14 @@ Please report new phishing URL to [PhishTank](https://www.phishtank.com/add_web_
 
 See [wiki](https://gitlab.com/malware-filter/malware-filter/-/wikis/home)
 
+## CI Variables
+
+Optional variables:
+
+- `PHISHTANK_API`: Recommended if you intend to run [script.sh](src/script.sh) >5 times daily. Register an account at [phishtank.org](https://phishtank.org/developer_info.php) to generate an application key.
+- `CLOUDFLARE_BUILD_HOOK`: Deploy to Cloudflare Pages.
+- `NETLIFY_SITE_ID`: Deploy to Netlify.
+
 ## License
 
 [src/](src/): [CC0](LICENSE.md)

src/script.sh

@@ -2,7 +2,7 @@
 
 # works best on busybox sh
 
-set -efux -o pipefail
+set -efx -o pipefail
 
 alias rm="rm -f"
 
@@ -36,7 +36,12 @@ mkdir -p "tmp/"
 cd "tmp/"
 
 ## Prepare datasets
-curl -L "https://data.phishtank.com/data/$PHISHTANK_API/online-valid.csv.bz2" -o "phishtank.bz2"
+if [ -n "$PHISHTANK_API" ]; then
+  curl -L "https://data.phishtank.com/data/$PHISHTANK_API/online-valid.csv.bz2" -o "phishtank.bz2"
+else
+  curl -L "https://data.phishtank.com/data/online-valid.csv.bz2" -o "phishtank.bz2"
+fi
+
 curl -L "https://openphish.com/feed.txt" -o "openphish-raw.txt"
 curl -L "https://phishunt.io/feed.txt" -o "phishunt-raw.txt"
 curl -L "https://s3-us-west-1.amazonaws.com/umbrella-static/top-1m.csv.zip" -o "top-1m-umbrella.zip"