Commit Graph

1661 Commits

Author SHA1 Message Date
Luis Neto 520e266411 TUN-8767: include raw output from network collector in diagnostic zipfile
## Summary

Export raw format of traceroute is widely known and useful for debugging. This raw output is written to the zipfile's root at the end of the diagnostic.

Closes TUN-8767
2024-12-04 04:40:51 -08:00
Luis Neto 7bd86762a7 TUN-8725: implement diagnostic procedure
## Summary
Add a function that orchestrates the diagnostic jobs producing a zip file at the end.

Closes TUN-8725
2024-12-04 03:37:57 -08:00
lneto 451f98e1d1 TUN-8727: extend client to include function to get cli configuration and tunnel configuration 2024-12-03 21:39:03 +00:00
Luis Neto 60fe4a0800 TUN-8769: fix k8s log collector arguments
## Summary

The equal signs were making the exec command to fail removing them fixes the issue.

Closes TUN-8769
2024-12-03 06:24:14 -08:00
Luis Neto 1ef109c042 TUN-8762: fix argument order when invoking tracert and modify network info output parsing.
## Summary

The windows code path has three bugs:

* the -4 and -6 option cannot be passed in the last position
* since there are some lines in the command output that are not parsable the collection fails to parse any kind of output
* the timeout hop is not correctly parsed

This PR also guards the parsing code against empty domains

Closes TUN-8762
2024-12-03 04:56:28 -08:00
Luis Neto 65786597cc TUN-8732: implement port selection algorithm
## Summary
Implements the discovery of the metrics server given an array of addresses (known addresses), tunnelID, and connectorID.

Closes TUN-8732
2024-12-03 04:07:55 -08:00
Luis Neto f884b29d0d TUN-8726: implement compression routine to be used in diagnostic procedure
## Summary
Basic routine to compress the diagnostic files into the root of a zipfile.

Closes TUN-8726
2024-12-03 03:27:04 -08:00
Luis Neto b3304bf05b TUN-8727: implement metrics, runtime, system, and tunnelstate in diagnostic http client
## Summary
The diagnostic procedure needs to extract information available in the metrics server via HTTP calls. 
These changes add to the diagnostic client the remaining endpoints.

Closes TUN-8727
2024-11-29 09:08:42 -08:00
Luis Neto 28796c659e TUN-8729: implement network collection for diagnostic procedure
## Summary
This PR adds implementation for windows & unix that collect the tracert.exe & traceroute output in the form of hops.

Closes TUN-8729
2024-11-29 07:43:36 -08:00
Devin Carr 9da15b5d96 TUN-8640: Refactor ICMPRouter to support new ICMPResponders
A new ICMPResponder interface is introduced to provide different
implementations of how the ICMP flows should return to the QUIC
connection muxer.

Improves usages of netip.AddrPort to leverage the embedded zone
field for IPv6 addresses.

Closes TUN-8640
2024-11-27 12:46:08 -08:00
Luis Neto 46dc6316f9 TUN-8734: add log collection for kubernetes
## Summary
Adds the log collector for K8s based deployments.

Closes TUN-8734
2024-11-27 03:15:15 -08:00
Luis Neto 16e65c70ad TUN-8733: add log collection for docker
## Summary
Adds the log collector for docker based deployments

Closes TUN-8733
2024-11-27 01:08:54 -08:00
Luis Neto a6f9e68739 TUN-8735: add managed/local log collection
## Summary
Adds a log collector for the managed/local runtimes.


Closes TUN-8735 TUN-8736
2024-11-26 10:30:44 -08:00
Luis Neto f85c0f1cc0 TUN-8730: implement diag/configuration
Implements the endpoint that retrieves the configuration of a running instance.

The configuration consists in a map of cli flag to the provided value along with the uid that of the user that started the process
2024-11-25 11:24:51 -08:00
Luis Neto 4b0b6dc8c6 TUN-8728: implement diag/tunnel endpoint
## Summary
The new endpoint returns the current information to be used when calling the diagnostic procedure.
This also adds:
- add indexed connection info and method to extract active connections from connTracker
- add edge address to Event struct and conn tracker
- remove unnecessary event send
- add tunnel configuration handler
- adjust cmd and metrics to create diagnostic server

Closes TUN-8728
2024-11-25 10:43:32 -08:00
Luis Neto aab5364252 TUN-8731: Implement diag/system endpoint
## Summary
This PR will add a new endpoint, "diag/system" to the metrics server that collects system information from different operating systems.

Closes TUN-8731
2024-11-22 08:10:05 -08:00
Luis Neto e2c2b012f1 TUN-8737: update metrics server port selection
## Summary
Update how metrics server binds to a listener by using a known set of ports whenever the default address is used with the fallback to a random port in case all address are already in use. The default address changes at compile time in order to bind to a different default address when the final deliverable is a docker image.

Refactor ReadyServer tests.

Closes TUN-8737
2024-11-22 07:23:46 -08:00
Devin Carr d779394748 TUN-8748: Migrated datagram V3 flows to use migrated context
Previously, during local flow migration the current connection context
was not part of the migration and would cause the flow to still be listening
on the connection context of the old connection (before the migration).
This meant that if a flow was migrated from connection 0 to
connection 1, and connection 0 goes away, the flow would be early
terminated incorrectly with the context lifetime of connection 0.

The new connection context is provided during migration of a flow
and will trigger the observe loop for the flow lifetime to be rebound
to this provided context.
Closes TUN-8748
2024-11-21 12:56:47 -08:00
Chung-Ting Huang c59d56c655 Release 2024.11.1
## Summary
Some description...
2024-11-19 08:21:37 -08:00
chungthuang 3480a33fce
Merge pull request #1135 from firecow/tunnel-health
Add cloudflared tunnel ready command
2024-11-18 10:08:45 -06:00
chungthuang a26b2a0097
Merge pull request #1355 from pkillarjun/fuzzing
add: new go-fuzz targets
2024-11-18 09:53:12 -06:00
chungthuang 37eee7e727
Merge pull request #1346 from emilioarvidisaksson/patch-1
Fixed 404 in README.md to TryCloudflare
2024-11-18 09:52:44 -06:00
chungthuang 9771f3309e
Merge pull request #1329 from cloudflare/hrushikeshdeshpande-updating-semgrep-yml
Update semgrep.yml
2024-11-18 09:52:05 -06:00
Mads Jon Nielsen c39f0ae317 Make metrics a requirement for tunnel ready command 2024-11-14 18:38:17 +01:00
Mads Jon Nielsen 95dff74fc8
Merge branch 'cloudflare:master' into tunnel-health 2024-11-14 18:19:43 +01:00
Devin Carr ab3dc5f8fa TUN-8701: Simplify flow registration logs for datagram v3
To help reduce the volume of logs during the happy path of flow registration, there will only be one log message reported when a flow is completed.

There are additional fields added to all flow log messages:
1. `src`: local address
2. `dst`: origin address
3. `durationMS`: capturing the total duration of the flow in milliseconds

Additional logs were added to capture when a flow was migrated or when cloudflared sent off a registration response retry.

Closes TUN-8701
2024-11-12 10:54:37 -08:00
Arjun 53c523444e add: new go-fuzz targets
Signed-off-by: Arjun <pkillarjun@protonmail.com>
2024-11-11 20:45:49 +05:30
Devin Carr 1f3e3045ad TUN-8701: Add metrics and adjust logs for datagram v3
Closes TUN-8701
2024-11-07 11:02:55 -08:00
Devin Carr 952622a965 TUN-8709: Add session migration for datagram v3
When a registration response from cloudflared gets lost on it's way back to the edge, the edge service will retry and send another registration request. Since cloudflared already has bound the local UDP socket for the provided request id, we want to re-send the registration response.

There are three types of retries that the edge will send:

1. A retry from the same QUIC connection index; cloudflared will just respond back with a registration response and reset the idle timer for the session.
2. A retry from a different QUIC connection index; cloudflared will need to migrate the current session connection to this new QUIC connection and reset the idle timer for the session.
3. A retry to a different cloudflared connector; cloudflared will eventually time the session out since no further packets will arrive to the session at the original connector.

Closes TUN-8709
2024-11-06 12:06:07 -08:00
lneto 70393b6de4 Release 2024.11.0 2024-11-06 07:09:41 +00:00
Luis Neto e8e824a730 VULN-66059: remove ssh server tests
## Summary
The initial purpose of this PR was to bump the base image from buster to bookworm however these tests are no longer exercised hence the removal

Closes VULN-66059
2024-11-05 23:00:35 -08:00
Gonçalo Garcia 3d33f559b1 TUN-8641: Expose methods to simplify V3 Datagram parsing on the edge 2024-11-04 15:23:36 -08:00
emilioarvidisaksson aa7abe7581
Fixed 404 in README.md to TryCloudflare 2024-11-04 23:25:53 +01:00
Devin Carr 589c198d2d TUN-8646: Allow experimental feature support for datagram v3
Closes TUN-8646
2024-11-04 13:59:32 -08:00
Devin Carr 5891c0d955 TUN-8700: Add datagram v3 muxer
The datagram muxer will wrap a QUIC Connection datagram read-writer operations to unmarshal datagrams from the connection to the origin with the session manager. Incoming datagram session registration operations will create new UDP sockets for sessions to proxy UDP packets between the edge and the origin. The muxer is also responsible for marshalling UDP packets and operations into datagrams for communication over the QUIC connection towards the edge.

Closes TUN-8700
2024-11-04 11:20:35 -08:00
lneto d29017fac9 TUN-8553: Bump go to 1.22.5 and go-boring 1.22.5-1
update docker files with go1.22.5
update windows scripts with go1.22.5
2024-11-04 01:25:49 -08:00
Devin Carr 6a6c890700 TUN-8667: Add datagram v3 session manager
New session manager leverages similar functionality that was previously
provided with datagram v2, with the distinct difference that the sessions
are registered via QUIC Datagrams and unregistered via timeouts only; the
sessions will no longer attempt to unregister sessions remotely with the
edge service.

The Session Manager is shared across all QUIC connections that cloudflared
uses to connect to the edge (typically 4). This will help cloudflared be
able to monitor all sessions across the connections and help correlate
in the future if sessions migrate across connections.

The UDP payload size is still limited to 1280 bytes across all OS's. Any
UDP packet that provides a payload size of greater than 1280 will cause
cloudflared to report (as it currently does) a log error and drop the packet.

Closes TUN-8667
2024-10-31 14:05:15 -07:00
Devin Carr 599ba52750 TUN-8708: Bump python min version to 3.10
With the recent bump of the windows CI to python 3.10, we will bump the minimum required python version for component testing to 3.10.

Closes TUN-8708
2024-10-31 13:33:24 -07:00
Mads Jon Nielsen 2cbe125e0b
Merge branch 'cloudflare:master' into tunnel-health 2024-10-28 10:33:08 +01:00
Luis Neto 0eddb8a615 TUN-8692: remove dashes from session id
Closes TUN-8692
2024-10-25 05:45:24 -07:00
Devin Carr 16ecf60800 TUN-8661: Refactor connection methods to support future different datagram muxing methods
The current supervisor serves the quic connection by performing all of the following in one method:
1. Dial QUIC edge connection
2. Initialize datagram muxer for UDP sessions and ICMP
3. Wrap all together in a single struct to serve the process loops

In an effort to better support modularity, each of these steps were broken out into their own separate methods that the supervisor will compose together to create the TunnelConnection and run its `Serve` method.

This also provides us with the capability to better interchange the functionality supported by the datagram session manager in the future with a new mechanism.

Closes TUN-8661
2024-10-24 11:42:02 -07:00
Luis Neto eabc0aaaa8 TUN-8694: Rework release script
## Summary
This modifies the release script to only create the github release after verifying the assets version

Closes TUN-8694
2024-10-24 09:43:02 -07:00
GoncaloGarcia 374a920b61 Release 2024.10.1 2024-10-23 15:46:48 +01:00
lneto 6ba0c25a92 TUN-8694: Fix github release script
Remove parameter from extractall function since it does not exist in python 3.7
2024-10-23 11:08:17 +01:00
GoncaloGarcia 48f703f990 Release 2024.10.1 2024-10-22 10:08:58 +01:00
GoncaloGarcia f407dbb712 Revert "TUN-8592: Use metadata from the edge to determine if request body is empty for QUIC transport"
This reverts commit e2064c820f.
2024-10-21 16:07:52 +01:00
Devin Carr 92e0f5fcf9 TUN-8688: Correct UDP bind for IPv6 edge connectivity on macOS
For macOS, we want to set the DF bit for the UDP packets used by the QUIC
connection; to achieve this, you need to explicitly set the network
to either "udp4" or "udp6". When determining which network type to pick
we need to use the edge IP address chosen to align with what the local
IP family interface we will use. This means we want cloudflared to bind
to local interfaces for a random port, so we provide a zero IP and 0 port
number (ex. 0.0.0.0:0). However, instead of providing the zero IP, we
can leave the value as nil and let the kernel decide which interface and
pick a random port as defined by the target edge IP family.

This was previously broken for IPv6-only edge connectivity on macOS and
all other operating systems should be unaffected because the network type
was left as default "udp" which will rely on the provided local or remote
IP for selection.

Closes TUN-8688
2024-10-18 14:38:05 -07:00
Devin Carr d608a64cc5 TUN-8685: Bump coredns dependency
Closes TUN-8685
2024-10-17 13:09:39 -07:00
Devin Carr abb3466c31 TUN-8638: Add datagram v3 serializers and deserializers
Closes TUN-8638
2024-10-16 12:05:55 -07:00
Devin Carr a3ee49d8a9 chore: Remove h2mux code
Some more legacy h2mux code to be cleaned up and moved out of the way.
The h2mux.Header used in the serialization for http2 proxied headers is moved to connection module. Additionally, the booleanfuse structure is also moved to supervisor as it is also needed. Both of these structures could be evaluated later for removal/updates, however, the intent of the proposed changes here is to remove the dependencies on the h2mux code and removal.

Approved-by: Chung-Ting Huang <chungting@cloudflare.com>
Approved-by: Luis Neto <lneto@cloudflare.com>
Approved-by: Gonçalo Garcia <ggarcia@cloudflare.com>

MR: https://gitlab.cfdata.org/cloudflare/tun/cloudflared/-/merge_requests/1576
2024-10-15 13:10:30 -07:00