docs: unify wikis to curben/malware-filter

- https://gitlab.com/curben/malware-filter/-/wikis/home
MDLeom 2022-01-11 06:57:08 +00:00
parent 5b355ac3bb
commit ea72a46a00
No known key found for this signature in database
GPG Key ID: 32D3E28E96A695E8
4 changed files with 1 additions and 102 deletions

@ -1,43 +0,0 @@
The following list is derived from [FilterLists](https://filterlists.com/).
[URL-based](https://gitlab.com/curben/urlhaus-filter#url-based):
- [AdNauseam](https://adnauseam.io/)
- [uBlock Origin](https://github.com/gorhill/uBlock)
[Domain-based](https://gitlab.com/curben/urlhaus-filter#domain-based):
- [AdGuard](https://adguard.com/)
* Recommend to use this [blocklist](https://gitlab.com/curben/urlhaus-filter#url-based-adguard)
- [AdGuard Home](https://github.com/AdguardTeam/AdGuardHome)
* Recommend to use this [blocklist](https://gitlab.com/curben/urlhaus-filter#domain-based-adguard-home)
- [AdNauseam](https://adnauseam.io/)
- [Blockada](https://blokada.org/)
- [Diversion](https://diversion.ch/)
- [DNS66](https://github.com/julian-klode/dns66)
- [Google Hit Hider](https://www.jeffersonscher.com/gm/google-hit-hider/)
- [hostsmgr](https://www.henrypp.org/product/hostsmgr)
- [Personal Blocklist](https://addons.mozilla.org/firefox/addon/personal-blocklist/)
- [personalDNSfilter](https://zenz-solutions.de/personaldnsfilter/)
- [pfBlockerNG](https://www.reddit.com/r/pfBlockerNG/)
- [Pi-hole](https://pi-hole.net/)
- [Samsung Knox](https://www.samsungknox.com/)
- [uBlock Origin](https://github.com/gorhill/uBlock)
- [uMatrix](https://github.com/gorhill/uMatrix)
[Hosts-based](https://gitlab.com/curben/urlhaus-filter#hosts-based) aka hosts file:
- [AdAway](https://adaway.org/)
- [AdGuard Home](https://github.com/AdguardTeam/AdGuardHome)
- [AdNauseam](https://adnauseam.io/)
- [Blockada](https://blokada.org/)
- [Diversion](https://diversion.ch/)
- [DNS66](https://github.com/julian-klode/dns66)
- [Gas Mark](https://github.com/2ndalpha/gasmask)
- [Hosts File Editor](https://hostsfileeditor.com/)
- [hostsmgr](https://www.henrypp.org/product/hostsmgr)
- [personalDNSfilter](https://zenz-solutions.de/personaldnsfilter/)
- [pfBlockerNG](https://www.reddit.com/r/pfBlockerNG/)
- [Pi-hole](https://pi-hole.net/)
- [uBlock Origin](https://github.com/gorhill/uBlock)
- [uMatrix](https://github.com/gorhill/uMatrix)

16
faq.md

@ -1,16 +0,0 @@
- Can you add this *very-bad-url.com* to the filter?
+ Please report to the [upstream](https://urlhaus.abuse.ch/api/#submit).
- How is the filter created?
1. Grab the URLhaus **Database dump (CSV)** and save it to [URLhaus.csv](https://gitlab.com/curben/urlhaus-filter/blob/master/src/URLhaus.csv).
2. Extract the domains.
3. Exclude popular domains using [Umbrella Popularity List](https://s3-us-west-1.amazonaws.com/umbrella-static/index.html) (top 1M domains + subdomains), [Tranco List](https://tranco-list.eu/) (top 1M domains) and some well-known domains (if not listed by Umbrella & Tranco, see [exclude.txt](https://gitlab.com/curben/urlhaus-filter/blob/master/src/exclude.txt)).
4. Extract the URLs (from step 1) that include popular domains (Umbrella and exclude.txt).
5. Merge the files from step 3 and 4.
6. Lite version only parses online urls from that database. Status of an URL (online or offline) is determined by URLhaus and can be found in the fourth column of the database.
- Why there is an issue running the script locally?
+ Install busybox (bundled in Alpine and Ubuntu) and run the script using `busybox sh src/script.sh`.
- Script terminated halfway with "grep broken pipe" or "grep killed" error
+ This is an out-of-memory (OOM) issue due to large input file. Recommend 3GB+ RAM.

@ -1,3 +1 @@
- [FAQ](faq)
- [Compatibility](compatibility) list
- [Mirroring Guide](mirroring)
See https://gitlab.com/curben/malware-filter/-/wikis/home

@ -1,40 +0,0 @@
For [phishing-filter](https://gitlab.com/curben/phishing-filter), create a new API key. Then, in GitLab, go to **CI/CD** > **Variables**, add `PHISHTANK_API` key and paste the API key value.
## GitLab Pages
- Fork the repository.
- Go to **Repository Settings** and set the **Default branch** to `main`.
- Go to CI/CD **Schedules**, add `0 0,12 * * *` custom pattern, set **UTC** timezone and set `main` target branch.
- If you want to manually trigger a deployment before the scheduled time, go to CI/CD **Pipelines** and **Run pipeline**.
- After deployment (either scheduled or manual), filters will be available at `<username>.gitlab.io/<project>/<filename>`.
## Github Pages
- Fork the repository in GitLab.
- Create an *empty* GitHub repository.
- In GitLab, go to **Repository Settings** > **Mirror repositories**.
+ URL: `ssh://github.com/<username>/<project>.git`
+ Direction: **Push**
+ Authentication method: **SSH public key**
+ Tick **Keep divergent refs** (otherwise GitLab will remove the `gh-pages` branch of your GitHub mirror)
+ Click **Mirror repository**.
+ Click **Copy SSH public key** button.
- In GitHub repository, go to **Settings** > **Deploy keys**. Click **Add deploy key**, paste the public key, tick **Allow write access** and **Add key**.
- Back in GitLab, click **Update now**. This only needs to be done once. New commits will be automatically pushed to your GitHub.
- If you want to manually trigger a deployment before the scheduled time, go to **Actions** > **Pages**, under **Run workflow** dropdown, select `main` branch and click **Run workflow**.
- Filters will be deployed to `gh-pages` branch.
- Go to **Settings** > **Pages** > **Source**. Select `gh-pages` and **Save**.
- After deployment (either scheduled or manual), filters will be available at `<username>.github.io/<project>/<filename>`.
## Cloudflare Pages/Netlify
- Fork the repository in GitLab.
- Connect Cloudflare Pages or Netlify to the forked repository.
- Create deploy/build hook URL. Note the last part of the URL.
+ `https://api.cloudflare.com/client/v4/pages/webhooks/deploy_hooks/<copy-here>`
+ `https://api.netlify.com/build_hooks/<copy-here>`
- In GitLab, go to **CI/CD** > **Variables**:
+ Add `CLOUDFLARE_BUILD_HOOK` or `NETLIFY_BUILD_HOOK` key and paste the value.
+ Add `CLOUDFLARE` or `NETLIFY` key with **true** value.
- If you *haven't* add a schedule, go to CI/CD **Schedules**, add `0 0,12 * * *` custom pattern, set **UTC** timezone and set `main` target branch.
- Deployment will be triggered on schedule.