Commit Graph

66 Commits

Author SHA1 Message Date
ember ana b4c91c78af fix(exclude): add hst.sh to domain exclusions 2025-12-06 18:56:09 +00:00
MDLeom 7d752c6167
feat(clean_url): esvalabs 2025-11-25 07:05:00 +00:00
MDLeom d42792a433
feat(clean_url): trendmicro 2025-09-09 08:10:39 +00:00
MDLeom 8dba535c91
feat(clean_url): calendly 2025-08-20 11:47:20 +00:00
MDLeom a3bf9f6a7a
feat(clean_url): awstrack.me & duckduckgo 2025-07-15 10:32:41 +00:00
MDLeom 85d43577a6
fix(clean_url): possible invalid adurl value 2025-07-03 06:37:08 +00:00
MDLeom 5873315cfe
fix(exclude): x0.at
closes #109
2025-06-25 09:49:45 +00:00
MDLeom 287241eaa7
fix(exclude): uguu.se
closes #105
2025-06-25 09:05:09 +00:00
MDLeom 5c71a5e35b
fix(exclude): r2.e-z.host
closes #106
2025-06-25 08:57:41 +00:00
MDLeom af67343335
fix: add ipv6 to dnscrypt-proxy blocklist 2025-06-22 10:05:17 +00:00
MDLeom 355338e407
feat: url exclusion list
fixes #110
2025-06-10 10:03:51 +00:00
MDLeom d5daecab9d
fix: reduce expiry to 12h
to match filter update frequency.
uBO supports 4h interval
https://github.com/ameshkov/diffupdates/blob/master/README.md
2025-05-29 09:56:10 +00:00
MDLeom e0bee1eb88
feat: SNI inspection using Suricata
inspired by https://blog.nviso.eu/wp-content/uploads/2025/04/NVISO-BRICKSTORM-Report.pdf
2025-05-24 23:54:48 +00:00
MDLeom 496967732f
feat(clean_url): shopmy, disqus, vkontakte, whatsapp, google ads/search links 2025-05-24 03:12:03 +00:00
MDLeom 8f95fd3182
feat: parse barracuda & sophos link protection 2025-04-04 08:43:54 +00:00
MDLeom c07a0e4e11
feat: include IPs in TPL
tested support IPv4/6
2025-03-23 10:13:12 +00:00
MDLeom fd2659431e
feat: more robust url parsing
percent-encode, hostname/pathname/query-string parsing
- increase nodejs requirement to v20 due to URL.canParse()
- IPv6 support
- include query string in IDS blocklists
  * and escape ; on query string
- add address separator to adblock filters
  * f4377f1fe6
- remove unnecessary global flag in sed
- remove unnecessary sort
- clean tmp at start
2025-03-23 04:17:27 +00:00
MDLeom d2f18b753c
perf: rewrite IDS rule creation in javascript
"while do" can be inefficient
previously took >20s is now 0.2s
2025-03-17 12:15:37 +00:00
MDLeom c28f23bc2a
fix(exclude): landley.net
closes #104
2025-03-16 06:12:39 +00:00
MDLeom 9524001afb
fix(exclude): res.cloudinary.com
closes #73
2025-03-10 08:21:52 +00:00
MDLeom f2abca8ba9
fix(exclude): *.contabostorage.com
closes !10
2025-03-10 08:21:52 +00:00
MDLeom f4d428b56d
chore(exclude): cleanup entries
104.244.72.54 no longer points to git.freezer.life #45
a.uguu.se does not have dns record
srv-*.gofile.io do not have dns record
others have offline entries and older than 4 years
2025-03-10 08:21:41 +00:00
Ming Di Leom db82a16e91 Merge branch 'publicarray-main-patch-52812' into 'main'
Add storage.bunnycdn.com to exclude list

See merge request malware-filter/urlhaus-filter!11
2025-03-10 05:44:35 +00:00
publicarray - e28cc3d57b fix(exclude): storage.bunnycdn.com 2025-03-10 05:44:35 +00:00
MDLeom 70bf08a46d
fix: skip tranco if download fails 2025-03-08 01:23:59 +00:00
MDLeom b4acfdbd81
style(rpz): generic syntax 2025-02-16 00:44:01 +00:00
MDLeom 25e4e165ee
feat: wildcard asterisk 2025-02-16 00:40:54 +00:00
MDLeom a4374b02e0
refactor: extract zip from stream 2024-10-27 06:04:51 +00:00
MDLeom 02ddeffd36
fix: unzip alternatives 2024-07-15 09:43:45 +00:00
MDLeom 30a5ab5909
refactor: set pipefail conditionally 2024-07-15 08:03:30 +00:00
MDLeom 43b0193907
fix: expand alias in bash 2024-06-03 08:24:03 +00:00
MDLeom 7c4b71bbbf
refactor: replace got with fetch 2024-04-07 01:13:34 +00:00
MDLeom dea0eceb2b
fix: disable regex whenever possible 2023-12-31 04:06:41 +00:00
MDLeom 57f8afdc3d
fix: less strict check on pipeline status
it can be "success, "running" or "unknown"
2023-10-02 00:08:34 +00:00
MDLeom 1cbea2cd24
refactor: esm
got is esm only since v12
2023-10-02 00:05:58 +00:00
MDLeom d47888a8a3
fix: check existent of busybox
if dos2unix is not installed
2023-05-20 11:25:09 +00:00
MDLeom 385f6b450e
fix: check installed grep is GNU variant 2023-05-20 11:24:38 +00:00
MDLeom c4421d25d9
fix: dash does not support pipefail 2023-05-20 11:24:04 +00:00
MDLeom ca0cb81902
fix: sed syntax to recognise newline
close #79
2023-04-29 04:08:15 +00:00
MDLeom 7000159da1
fix(exclude): workbyspeech.com
- close #82
- https://urlhaus.abuse.ch/url/2542435/
2023-04-29 03:53:03 +00:00
MDLeom 1dbbe43121
fix(exclude): update s3 endpoints
add new regions
https://docs.aws.amazon.com/general/latest/gr/s3.html#regional-endpoints
2023-04-29 03:22:34 +00:00
MDLeom a34a4a3993
fix(exclude): link.storjshare.io
- close #83
- close !9
2023-04-29 03:21:24 +00:00
MDLeom 21faec52b1
fix(exclude): litter.catbox.moe
close #84
2023-04-29 03:19:34 +00:00
MDLeom 8c60652bba
fix: cloudflare radar dataset is now in csv format
instead of zip
2023-01-16 07:07:34 +00:00
MDLeom 612cfcabdc
style(sed): avoid backslash in insert option
- simpler and more readable
- https://unix.stackexchange.com/a/99351
2022-12-17 00:23:32 +00:00
MDLeom 214cfb7be8
docs(header): switch date format from RFC 5322 to ISO 8601
- universally readable
2022-12-16 08:16:29 +00:00
MDLeom 00a6e3029a
feat: add csv file for Splunk lookup
- https://docs.splunk.com/Documentation/Splunk/9.0.2/Knowledge/Aboutlookupsandfieldactions
- "set -u" is disabled due to CF_API variable
2022-12-16 08:07:38 +00:00
MDLeom df8fe36d39
style: set default options for curl & rm
- curl -L
- rm -rf
2022-11-26 01:36:17 +00:00
MDLeom 9b478347f8
feat: add Cloudflare Radar top 1m domains dataset 2022-11-25 07:13:50 +00:00
MDLeom 9965b32171
feat: fallback to busybox dos2unix 2022-11-03 08:50:36 +00:00