From ea72a46a00a667d1afcc483fbcc35e0f1981768c Mon Sep 17 00:00:00 2001 From: MDLeom <2809763-curben@users.noreply.gitlab.com> Date: Tue, 11 Jan 2022 06:57:08 +0000 Subject: [PATCH] docs: unify wikis to curben/malware-filter - https://gitlab.com/curben/malware-filter/-/wikis/home --- compatibility.md | 43 ------------------------------------------- faq.md | 16 ---------------- home.md | 4 +--- mirroring.md | 40 ---------------------------------------- 4 files changed, 1 insertion(+), 102 deletions(-) delete mode 100644 compatibility.md delete mode 100644 faq.md delete mode 100644 mirroring.md diff --git a/compatibility.md b/compatibility.md deleted file mode 100644 index be6142a..0000000 --- a/compatibility.md +++ /dev/null @@ -1,43 +0,0 @@ -The following list is derived from [FilterLists](https://filterlists.com/). - -[URL-based](https://gitlab.com/curben/urlhaus-filter#url-based): - -- [AdNauseam](https://adnauseam.io/) -- [uBlock Origin](https://github.com/gorhill/uBlock) - -[Domain-based](https://gitlab.com/curben/urlhaus-filter#domain-based): - -- [AdGuard](https://adguard.com/) - * Recommend to use this [blocklist](https://gitlab.com/curben/urlhaus-filter#url-based-adguard) -- [AdGuard Home](https://github.com/AdguardTeam/AdGuardHome) - * Recommend to use this [blocklist](https://gitlab.com/curben/urlhaus-filter#domain-based-adguard-home) -- [AdNauseam](https://adnauseam.io/) -- [Blockada](https://blokada.org/) -- [Diversion](https://diversion.ch/) -- [DNS66](https://github.com/julian-klode/dns66) -- [Google Hit Hider](https://www.jeffersonscher.com/gm/google-hit-hider/) -- [hostsmgr](https://www.henrypp.org/product/hostsmgr) -- [Personal Blocklist](https://addons.mozilla.org/firefox/addon/personal-blocklist/) -- [personalDNSfilter](https://zenz-solutions.de/personaldnsfilter/) -- [pfBlockerNG](https://www.reddit.com/r/pfBlockerNG/) -- [Pi-hole](https://pi-hole.net/) -- [Samsung Knox](https://www.samsungknox.com/) -- [uBlock Origin](https://github.com/gorhill/uBlock) -- [uMatrix](https://github.com/gorhill/uMatrix) - -[Hosts-based](https://gitlab.com/curben/urlhaus-filter#hosts-based) aka hosts file: - -- [AdAway](https://adaway.org/) -- [AdGuard Home](https://github.com/AdguardTeam/AdGuardHome) -- [AdNauseam](https://adnauseam.io/) -- [Blockada](https://blokada.org/) -- [Diversion](https://diversion.ch/) -- [DNS66](https://github.com/julian-klode/dns66) -- [Gas Mark](https://github.com/2ndalpha/gasmask) -- [Hosts File Editor](https://hostsfileeditor.com/) -- [hostsmgr](https://www.henrypp.org/product/hostsmgr) -- [personalDNSfilter](https://zenz-solutions.de/personaldnsfilter/) -- [pfBlockerNG](https://www.reddit.com/r/pfBlockerNG/) -- [Pi-hole](https://pi-hole.net/) -- [uBlock Origin](https://github.com/gorhill/uBlock) -- [uMatrix](https://github.com/gorhill/uMatrix) diff --git a/faq.md b/faq.md deleted file mode 100644 index f72e957..0000000 --- a/faq.md +++ /dev/null @@ -1,16 +0,0 @@ -- Can you add this *very-bad-url.com* to the filter? - + Please report to the [upstream](https://urlhaus.abuse.ch/api/#submit). - -- How is the filter created? - 1. Grab the URLhaus **Database dump (CSV)** and save it to [URLhaus.csv](https://gitlab.com/curben/urlhaus-filter/blob/master/src/URLhaus.csv). - 2. Extract the domains. - 3. Exclude popular domains using [Umbrella Popularity List](https://s3-us-west-1.amazonaws.com/umbrella-static/index.html) (top 1M domains + subdomains), [Tranco List](https://tranco-list.eu/) (top 1M domains) and some well-known domains (if not listed by Umbrella & Tranco, see [exclude.txt](https://gitlab.com/curben/urlhaus-filter/blob/master/src/exclude.txt)). - 4. Extract the URLs (from step 1) that include popular domains (Umbrella and exclude.txt). - 5. Merge the files from step 3 and 4. - 6. Lite version only parses online urls from that database. Status of an URL (online or offline) is determined by URLhaus and can be found in the fourth column of the database. - -- Why there is an issue running the script locally? - + Install busybox (bundled in Alpine and Ubuntu) and run the script using `busybox sh src/script.sh`. - -- Script terminated halfway with "grep broken pipe" or "grep killed" error - + This is an out-of-memory (OOM) issue due to large input file. Recommend 3GB+ RAM. diff --git a/home.md b/home.md index 5363c5a..9dbf87f 100644 --- a/home.md +++ b/home.md @@ -1,3 +1 @@ -- [FAQ](faq) -- [Compatibility](compatibility) list -- [Mirroring Guide](mirroring) +See https://gitlab.com/curben/malware-filter/-/wikis/home diff --git a/mirroring.md b/mirroring.md deleted file mode 100644 index 9fb4f01..0000000 --- a/mirroring.md +++ /dev/null @@ -1,40 +0,0 @@ -For [phishing-filter](https://gitlab.com/curben/phishing-filter), create a new API key. Then, in GitLab, go to **CI/CD** > **Variables**, add `PHISHTANK_API` key and paste the API key value. - -## GitLab Pages - -- Fork the repository. -- Go to **Repository Settings** and set the **Default branch** to `main`. -- Go to CI/CD **Schedules**, add `0 0,12 * * *` custom pattern, set **UTC** timezone and set `main` target branch. -- If you want to manually trigger a deployment before the scheduled time, go to CI/CD **Pipelines** and **Run pipeline**. -- After deployment (either scheduled or manual), filters will be available at `.gitlab.io//`. - -## Github Pages - -- Fork the repository in GitLab. -- Create an *empty* GitHub repository. -- In GitLab, go to **Repository Settings** > **Mirror repositories**. - + URL: `ssh://github.com//.git` - + Direction: **Push** - + Authentication method: **SSH public key** - + Tick **Keep divergent refs** (otherwise GitLab will remove the `gh-pages` branch of your GitHub mirror) - + Click **Mirror repository**. - + Click **Copy SSH public key** button. -- In GitHub repository, go to **Settings** > **Deploy keys**. Click **Add deploy key**, paste the public key, tick **Allow write access** and **Add key**. -- Back in GitLab, click **Update now**. This only needs to be done once. New commits will be automatically pushed to your GitHub. -- If you want to manually trigger a deployment before the scheduled time, go to **Actions** > **Pages**, under **Run workflow** dropdown, select `main` branch and click **Run workflow**. -- Filters will be deployed to `gh-pages` branch. -- Go to **Settings** > **Pages** > **Source**. Select `gh-pages` and **Save**. -- After deployment (either scheduled or manual), filters will be available at `.github.io//`. - -## Cloudflare Pages/Netlify - -- Fork the repository in GitLab. -- Connect Cloudflare Pages or Netlify to the forked repository. -- Create deploy/build hook URL. Note the last part of the URL. - + `https://api.cloudflare.com/client/v4/pages/webhooks/deploy_hooks/` - + `https://api.netlify.com/build_hooks/` -- In GitLab, go to **CI/CD** > **Variables**: - + Add `CLOUDFLARE_BUILD_HOOK` or `NETLIFY_BUILD_HOOK` key and paste the value. - + Add `CLOUDFLARE` or `NETLIFY` key with **true** value. -- If you *haven't* add a schedule, go to CI/CD **Schedules**, add `0 0,12 * * *` custom pattern, set **UTC** timezone and set `main` target branch. -- Deployment will be triggered on schedule.