Commit Graph

1679 Commits

Author SHA1 Message Date
Luis Neto 9bc6cbd06d TUN-8818: update changes document to reflect newly added diag subcommand
Closes TUN-8818
2024-12-18 04:26:38 -08:00
Devin Carr bc9c5d2e6e TUN-8817: Increase close session channel by one since there are two writers
When closing a session, there are two possible signals that will occur,
one from the outside, indicating that the session is idle and needs to
be closed, and the internal error condition that will be unblocked
with a net.ErrClosed when the connection underneath is closed. Both of
these routines write to the session's closeChan.

Once the reader for the closeChan reads one value, it will immediately
return. This means that the channel is a one-shot and one of the two
writers will get stuck unless the size of the channel is increased to
accomodate for the second write to the channel.

With the channel size increased to two, the second writer (whichever
loses the race to write) will now be unblocked to end their go routine
and return.

Closes TUN-8817
2024-12-17 14:55:09 -08:00
Luis Neto 1859d742a8 TUN-8724: Add CLI command for diagnostic procedure
## Summary
Adds a new CLI subcommand, under the tunnel command, the `diag`. This command has as function the automatic collection of different data points, such as, logs, metrics, network information, system information, tunnel state, and runtime information which will be written to a single zip file.

Closes TUN-8724
2024-12-13 10:07:56 -08:00
Luis Neto 8ed19222b9 TUN-8797: update CHANGES.md with note about semi-deterministic approach used to bind metrics server
Closes TUN-8797
2024-12-13 01:53:11 -08:00
Luis Neto 02e7ffd5b7 TUN-8792: Make diag/system endpoint always return a JSON
## Summary
Change the system information collector and respective http handler so that it always returns a JSON.

Closes [TUN-8792](https://jira.cfdata.org/browse/TUN-8792)
2024-12-11 02:48:41 -08:00
Luis Neto ba9f28ef43 TUN-8786: calculate cli flags once for the diagnostic procedure
## Summary

The flags were always being computed when their value is static.

 Closes TUN-8786
2024-12-11 01:29:20 -08:00
Luis Neto 77b99cf5fe TUN-8784: Set JSON encoder options to print formatted JSON when writing diag files
## Summary

The initial implementation produced correct JSON however it was not formatted which would make it harder to read the file by an user.

 Closes TUN-8784
2024-12-10 13:01:24 -08:00
Luis Neto d74ca97b51 TUN-8785: include the icmp sources in the diag's tunnel state
Closes TUN-8785
2024-12-10 10:42:33 -08:00
Luis Neto 29f0cf354c TUN-8783: fix log collectors for the diagnostic procedure
## Summary

* The host log collector now verifies if the OS is linux and has systemd if so it will use journalctl to get the logs
* In linux systems docker will write the output of the command logs to the stderr therefore the function that handles the execution of the process will copy both the contents of stdout and stderr; this also affect the k8s collector

Closes TUN-8783
2024-12-10 09:53:13 -08:00
lneto e7dcb6edca Release 2024.12.1 2024-12-10 16:07:33 +00:00
Luis Neto 14cf0eff1d TUN-8795: update createrepo to createrepo_c to fix the release_pkgs.py script
## Summary
The default-flavour of cfsetup changed from bullseye to bookworm and in the latter the createrepo package was renamed to createrepo_c.

Closes TUN-8795
2024-12-10 08:07:02 -08:00
lneto a00c80f9e1 Release 2024.12.0 2024-12-09 16:55:31 +00:00
Luis Neto 12d878531c TUN-8789: make python package installation consistent
## Summary

The previous changes regarding python's distribution which broke CI the installation of python packages.

Python packages in cfsetup are now installed via virtual environment. The dependency python3-venv was added as builddep to allow the creation of venv and the python packages installation was moved to the post-cache resulting in the removal of 

* anchor build_release_pre_cache
* anchor component_test_pre_cache

Closes TUN-8789
2024-12-09 08:52:48 -08:00
Devin Carr 588ab7ebaa TUN-8640: Add ICMP support for datagram V3
Closes TUN-8640
2024-12-09 07:23:11 -08:00
Joe Groocock dfbccd917c TUN-8781: Add Trixie, drop Buster. Default to Bookworm
Closes TUN-8781
2024-12-06 05:04:16 -08:00
Devin Carr 37010529bc TUN-8775: Make sure the session Close can only be called once
The previous capture of the sync.OnceValue was re-initialized for each
call to `Close`. This needed to be initialized during the creation of
the session to ensure that the sync.OnceValue reference was held for
the session's lifetime.

Closes TUN-8775
2024-12-05 14:12:53 -08:00
Luis Neto f07d04d129 TUN-8768: add job report to diagnostic zipfile
## Summary

Add a new job that write to a file the result of all of the other tasks along with possible errors. This file is also added to the root of the diagnostic zip file.

 Closes TUN-8768
2024-12-04 10:32:49 -08:00
Luis Neto f12036c2da TUN-8770: add cli configuration and tunnel configuration to diagnostic zipfile
## Summary

Adds two new jobs which will export the cli configuration and tunnel configuration in separate files. These files will also be added to the zipfile's root.

 Closes TUN-8770
2024-12-04 05:26:51 -08:00
Luis Neto 520e266411 TUN-8767: include raw output from network collector in diagnostic zipfile
## Summary

Export raw format of traceroute is widely known and useful for debugging. This raw output is written to the zipfile's root at the end of the diagnostic.

Closes TUN-8767
2024-12-04 04:40:51 -08:00
Luis Neto 7bd86762a7 TUN-8725: implement diagnostic procedure
## Summary
Add a function that orchestrates the diagnostic jobs producing a zip file at the end.

Closes TUN-8725
2024-12-04 03:37:57 -08:00
lneto 451f98e1d1 TUN-8727: extend client to include function to get cli configuration and tunnel configuration 2024-12-03 21:39:03 +00:00
Luis Neto 60fe4a0800 TUN-8769: fix k8s log collector arguments
## Summary

The equal signs were making the exec command to fail removing them fixes the issue.

Closes TUN-8769
2024-12-03 06:24:14 -08:00
Luis Neto 1ef109c042 TUN-8762: fix argument order when invoking tracert and modify network info output parsing.
## Summary

The windows code path has three bugs:

* the -4 and -6 option cannot be passed in the last position
* since there are some lines in the command output that are not parsable the collection fails to parse any kind of output
* the timeout hop is not correctly parsed

This PR also guards the parsing code against empty domains

Closes TUN-8762
2024-12-03 04:56:28 -08:00
Luis Neto 65786597cc TUN-8732: implement port selection algorithm
## Summary
Implements the discovery of the metrics server given an array of addresses (known addresses), tunnelID, and connectorID.

Closes TUN-8732
2024-12-03 04:07:55 -08:00
Luis Neto f884b29d0d TUN-8726: implement compression routine to be used in diagnostic procedure
## Summary
Basic routine to compress the diagnostic files into the root of a zipfile.

Closes TUN-8726
2024-12-03 03:27:04 -08:00
Luis Neto b3304bf05b TUN-8727: implement metrics, runtime, system, and tunnelstate in diagnostic http client
## Summary
The diagnostic procedure needs to extract information available in the metrics server via HTTP calls. 
These changes add to the diagnostic client the remaining endpoints.

Closes TUN-8727
2024-11-29 09:08:42 -08:00
Luis Neto 28796c659e TUN-8729: implement network collection for diagnostic procedure
## Summary
This PR adds implementation for windows & unix that collect the tracert.exe & traceroute output in the form of hops.

Closes TUN-8729
2024-11-29 07:43:36 -08:00
Devin Carr 9da15b5d96 TUN-8640: Refactor ICMPRouter to support new ICMPResponders
A new ICMPResponder interface is introduced to provide different
implementations of how the ICMP flows should return to the QUIC
connection muxer.

Improves usages of netip.AddrPort to leverage the embedded zone
field for IPv6 addresses.

Closes TUN-8640
2024-11-27 12:46:08 -08:00
Luis Neto 46dc6316f9 TUN-8734: add log collection for kubernetes
## Summary
Adds the log collector for K8s based deployments.

Closes TUN-8734
2024-11-27 03:15:15 -08:00
Luis Neto 16e65c70ad TUN-8733: add log collection for docker
## Summary
Adds the log collector for docker based deployments

Closes TUN-8733
2024-11-27 01:08:54 -08:00
Luis Neto a6f9e68739 TUN-8735: add managed/local log collection
## Summary
Adds a log collector for the managed/local runtimes.


Closes TUN-8735 TUN-8736
2024-11-26 10:30:44 -08:00
Luis Neto f85c0f1cc0 TUN-8730: implement diag/configuration
Implements the endpoint that retrieves the configuration of a running instance.

The configuration consists in a map of cli flag to the provided value along with the uid that of the user that started the process
2024-11-25 11:24:51 -08:00
Luis Neto 4b0b6dc8c6 TUN-8728: implement diag/tunnel endpoint
## Summary
The new endpoint returns the current information to be used when calling the diagnostic procedure.
This also adds:
- add indexed connection info and method to extract active connections from connTracker
- add edge address to Event struct and conn tracker
- remove unnecessary event send
- add tunnel configuration handler
- adjust cmd and metrics to create diagnostic server

Closes TUN-8728
2024-11-25 10:43:32 -08:00
Luis Neto aab5364252 TUN-8731: Implement diag/system endpoint
## Summary
This PR will add a new endpoint, "diag/system" to the metrics server that collects system information from different operating systems.

Closes TUN-8731
2024-11-22 08:10:05 -08:00
Luis Neto e2c2b012f1 TUN-8737: update metrics server port selection
## Summary
Update how metrics server binds to a listener by using a known set of ports whenever the default address is used with the fallback to a random port in case all address are already in use. The default address changes at compile time in order to bind to a different default address when the final deliverable is a docker image.

Refactor ReadyServer tests.

Closes TUN-8737
2024-11-22 07:23:46 -08:00
Devin Carr d779394748 TUN-8748: Migrated datagram V3 flows to use migrated context
Previously, during local flow migration the current connection context
was not part of the migration and would cause the flow to still be listening
on the connection context of the old connection (before the migration).
This meant that if a flow was migrated from connection 0 to
connection 1, and connection 0 goes away, the flow would be early
terminated incorrectly with the context lifetime of connection 0.

The new connection context is provided during migration of a flow
and will trigger the observe loop for the flow lifetime to be rebound
to this provided context.
Closes TUN-8748
2024-11-21 12:56:47 -08:00
Chung-Ting Huang c59d56c655 Release 2024.11.1
## Summary
Some description...
2024-11-19 08:21:37 -08:00
chungthuang 3480a33fce
Merge pull request #1135 from firecow/tunnel-health
Add cloudflared tunnel ready command
2024-11-18 10:08:45 -06:00
chungthuang a26b2a0097
Merge pull request #1355 from pkillarjun/fuzzing
add: new go-fuzz targets
2024-11-18 09:53:12 -06:00
chungthuang 37eee7e727
Merge pull request #1346 from emilioarvidisaksson/patch-1
Fixed 404 in README.md to TryCloudflare
2024-11-18 09:52:44 -06:00
chungthuang 9771f3309e
Merge pull request #1329 from cloudflare/hrushikeshdeshpande-updating-semgrep-yml
Update semgrep.yml
2024-11-18 09:52:05 -06:00
Mads Jon Nielsen c39f0ae317 Make metrics a requirement for tunnel ready command 2024-11-14 18:38:17 +01:00
Mads Jon Nielsen 95dff74fc8
Merge branch 'cloudflare:master' into tunnel-health 2024-11-14 18:19:43 +01:00
Devin Carr ab3dc5f8fa TUN-8701: Simplify flow registration logs for datagram v3
To help reduce the volume of logs during the happy path of flow registration, there will only be one log message reported when a flow is completed.

There are additional fields added to all flow log messages:
1. `src`: local address
2. `dst`: origin address
3. `durationMS`: capturing the total duration of the flow in milliseconds

Additional logs were added to capture when a flow was migrated or when cloudflared sent off a registration response retry.

Closes TUN-8701
2024-11-12 10:54:37 -08:00
Arjun 53c523444e add: new go-fuzz targets
Signed-off-by: Arjun <pkillarjun@protonmail.com>
2024-11-11 20:45:49 +05:30
Devin Carr 1f3e3045ad TUN-8701: Add metrics and adjust logs for datagram v3
Closes TUN-8701
2024-11-07 11:02:55 -08:00
Devin Carr 952622a965 TUN-8709: Add session migration for datagram v3
When a registration response from cloudflared gets lost on it's way back to the edge, the edge service will retry and send another registration request. Since cloudflared already has bound the local UDP socket for the provided request id, we want to re-send the registration response.

There are three types of retries that the edge will send:

1. A retry from the same QUIC connection index; cloudflared will just respond back with a registration response and reset the idle timer for the session.
2. A retry from a different QUIC connection index; cloudflared will need to migrate the current session connection to this new QUIC connection and reset the idle timer for the session.
3. A retry to a different cloudflared connector; cloudflared will eventually time the session out since no further packets will arrive to the session at the original connector.

Closes TUN-8709
2024-11-06 12:06:07 -08:00
lneto 70393b6de4 Release 2024.11.0 2024-11-06 07:09:41 +00:00
Luis Neto e8e824a730 VULN-66059: remove ssh server tests
## Summary
The initial purpose of this PR was to bump the base image from buster to bookworm however these tests are no longer exercised hence the removal

Closes VULN-66059
2024-11-05 23:00:35 -08:00
Gonçalo Garcia 3d33f559b1 TUN-8641: Expose methods to simplify V3 Datagram parsing on the edge 2024-11-04 15:23:36 -08:00