curben
/
urlhaus-filter
mirror of https://gitlab.com/malware-filter/urlhaus-filter
1
0
Fork 0
Code Issues Releases Wiki Activity

docs: mirroring guide

MDLeom 2022-01-10 10:08:04 +00:00
parent daa143d69a
commit 5b355ac3bb
No known key found for this signature in database
GPG Key ID: 32D3E28E96A695E8
3 changed files with 44 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
faq.md

@ -1,3 +1,6 @@
- Can you add this *very-bad-url.com* to the filter?
+ Please report to the [upstream](https://urlhaus.abuse.ch/api/#submit).
- How is the filter created?
1. Grab the URLhaus **Database dump (CSV)** and save it to [URLhaus.csv](https://gitlab.com/curben/urlhaus-filter/blob/master/src/URLhaus.csv).
2. Extract the domains.

1
home.md

@ -1,2 +1,3 @@
- [FAQ](faq)
- [Compatibility](compatibility) list
- [Mirroring Guide](mirroring)

40
mirroring.md Normal file

@ -0,0 +1,40 @@
For [phishing-filter](https://gitlab.com/curben/phishing-filter), create a new API key. Then, in GitLab, go to **CI/CD** > **Variables**, add `PHISHTANK_API` key and paste the API key value.
## GitLab Pages
- Fork the repository.
- Go to **Repository Settings** and set the **Default branch** to `main`.
- Go to CI/CD **Schedules**, add `0 0,12 * * *` custom pattern, set **UTC** timezone and set `main` target branch.
- If you want to manually trigger a deployment before the scheduled time, go to CI/CD **Pipelines** and **Run pipeline**.
- After deployment (either scheduled or manual), filters will be available at `<username>.gitlab.io/<project>/<filename>`.
## Github Pages
- Fork the repository in GitLab.
- Create an *empty* GitHub repository.
- In GitLab, go to **Repository Settings** > **Mirror repositories**.
+ URL: `ssh://github.com/<username>/<project>.git`
+ Direction: **Push**
+ Authentication method: **SSH public key**
+ Tick **Keep divergent refs** (otherwise GitLab will remove the `gh-pages` branch of your GitHub mirror)
+ Click **Mirror repository**.
+ Click **Copy SSH public key** button.
- In GitHub repository, go to **Settings** > **Deploy keys**. Click **Add deploy key**, paste the public key, tick **Allow write access** and **Add key**.
- Back in GitLab, click **Update now**. This only needs to be done once. New commits will be automatically pushed to your GitHub.
- If you want to manually trigger a deployment before the scheduled time, go to **Actions** > **Pages**, under **Run workflow** dropdown, select `main` branch and click **Run workflow**.
- Filters will be deployed to `gh-pages` branch.
- Go to **Settings** > **Pages** > **Source**. Select `gh-pages` and **Save**.
- After deployment (either scheduled or manual), filters will be available at `<username>.github.io/<project>/<filename>`.
## Cloudflare Pages/Netlify
- Fork the repository in GitLab.
- Connect Cloudflare Pages or Netlify to the forked repository.
- Create deploy/build hook URL. Note the last part of the URL.
+ `https://api.cloudflare.com/client/v4/pages/webhooks/deploy_hooks/<copy-here>`
+ `https://api.netlify.com/build_hooks/<copy-here>`
- In GitLab, go to **CI/CD** > **Variables**:
+ Add `CLOUDFLARE_BUILD_HOOK` or `NETLIFY_BUILD_HOOK` key and paste the value.
+ Add `CLOUDFLARE` or `NETLIFY` key with **true** value.
- If you *haven't* add a schedule, go to CI/CD **Schedules**, add `0 0,12 * * *` custom pattern, set **UTC** timezone and set `main` target branch.
- Deployment will be triggered on schedule.