Commit Graph

86 Commits

Author SHA1 Message Date
Devin Carr 16ecf60800 TUN-8661: Refactor connection methods to support future different datagram muxing methods
The current supervisor serves the quic connection by performing all of the following in one method:
1. Dial QUIC edge connection
2. Initialize datagram muxer for UDP sessions and ICMP
3. Wrap all together in a single struct to serve the process loops

In an effort to better support modularity, each of these steps were broken out into their own separate methods that the supervisor will compose together to create the TunnelConnection and run its `Serve` method.

This also provides us with the capability to better interchange the functionality supported by the datagram session manager in the future with a new mechanism.

Closes TUN-8661
2024-10-24 11:42:02 -07:00
GoncaloGarcia f407dbb712 Revert "TUN-8592: Use metadata from the edge to determine if request body is empty for QUIC transport"
This reverts commit e2064c820f.
2024-10-21 16:07:52 +01:00
Devin Carr 92e0f5fcf9 TUN-8688: Correct UDP bind for IPv6 edge connectivity on macOS
For macOS, we want to set the DF bit for the UDP packets used by the QUIC
connection; to achieve this, you need to explicitly set the network
to either "udp4" or "udp6". When determining which network type to pick
we need to use the edge IP address chosen to align with what the local
IP family interface we will use. This means we want cloudflared to bind
to local interfaces for a random port, so we provide a zero IP and 0 port
number (ex. 0.0.0.0:0). However, instead of providing the zero IP, we
can leave the value as nil and let the kernel decide which interface and
pick a random port as defined by the target edge IP family.

This was previously broken for IPv6-only edge connectivity on macOS and
all other operating systems should be unaffected because the network type
was left as default "udp" which will rely on the provided local or remote
IP for selection.

Closes TUN-8688
2024-10-18 14:38:05 -07:00
chungthuang e2064c820f TUN-8592: Use metadata from the edge to determine if request body is empty for QUIC transport
If the metadata is missing, fallback to decide based on protocol, http
method, transferring and content length
2024-10-07 10:51:21 -05:00
GoncaloGarcia e251a21810 TUN-8621: Prevent QUIC connection from closing before grace period after unregistering
Whenever cloudflared receives a SIGTERM or SIGINT it goes into graceful shutdown mode, which unregisters the connection and closes the control stream. Unregistering makes it so we no longer receive any new requests and makes the edge close the connection, allowing in-flight requests to finish (within a 3 minute period).
 This was working fine for http2 connections, but the quic proxy was cancelling the context as soon as the controls stream ended, forcing the process to stop immediately.

 This commit changes the behavior so that we wait the full grace period before cancelling the request
2024-10-07 10:51:21 -05:00
GoncaloGarcia 2437675c04 Reverts the following:
Revert "TUN-8621: Fix cloudflared version in change notes."
Revert "PPIP-2310: Update quick tunnel disclaimer"
Revert "TUN-8621: Prevent QUIC connection from closing before grace period after unregistering"
Revert "TUN-8484: Print response when QuickTunnel can't be unmarshalled"
Revert "TUN-8592: Use metadata from the edge to determine if request body is empty for QUIC transport"
2024-09-10 16:50:32 +01:00
GoncaloGarcia e05939f1c9 TUN-8621: Prevent QUIC connection from closing before grace period after unregistering
Whenever cloudflared receives a SIGTERM or SIGINT it goes into graceful shutdown mode, which unregisters the connection and closes the control stream. Unregistering makes it so we no longer receive any new requests and makes the edge close the connection, allowing in-flight requests to finish (within a 3 minute period).
 This was working fine for http2 connections, but the quic proxy was cancelling the context as soon as the controls stream ended, forcing the process to stop immediately.

 This commit changes the behavior so that we wait the full grace period before cancelling the request
2024-09-05 13:15:00 +00:00
chungthuang d6b0833209 TUN-8592: Use metadata from the edge to determine if request body is empty for QUIC transport
If the metadata is missing, fallback to decide based on protocol, http
method, transferring and content length
2024-08-26 15:53:24 -05:00
Devin Carr eb2e4349e8 TUN-8415: Refactor capnp rpc into a single module
Combines the tunnelrpc and quic/schema capnp files into the same module.

To help reduce future issues with capnp id generation, capnpids are
provided in the capnp files from the existing capnp struct ids generated
in the go files.

Reduces the overall interface of the Capnp methods to the rest of
the code by providing an interface that will handle the quic protocol
selection.

Introduces a new `rpc-timeout` config that will allow all of the
SessionManager and ConfigurationManager RPC requests to have a timeout.
The timeout for these values is set to 5 seconds as non of these operations
for the managers should take a long time to complete.

Removed the RPC-specific logger as it never provided good debugging value
as the RPC method names were not visible in the logs.
2024-05-17 11:22:07 -07:00
Devin Carr 1b02d169ad TUN-8374: Close UDP socket if registration fails
If cloudflared was unable to register the UDP session with the
edge, the socket would be left open to be eventually closed by the
OS, or garbage collected by the runtime. Considering that either of
these closes happened significantly after some delay, it was causing
cloudflared to hold open file descriptors longer than usual if continuously
unable to register sessions.
2024-04-22 21:59:43 +00:00
João "Pisco" Fernandes 76badfa01b TUN-8236: Add write timeout to quic and tcp connections
## Summary
To prevent bad eyeballs and severs to be able to exhaust the quic
control flows we are adding the possibility of having a timeout
for a write operation to be acknowledged. This will prevent hanging
connections from exhausting the quic control flows, creating a DDoS.
2024-02-15 17:54:52 +00:00
Chung-Ting 8068cdebb6 TUN-8006: Update quic-go to latest upstream 2023-12-04 17:09:40 +00:00
João Oliveirinha fc0ecf4185 TUN-7776: Remove warp-routing flag from cloudflared 2023-09-18 10:02:56 +01:00
Sudarsan Reddy 39847a70f2 TUN-7558: Flush on Writes for StreamBasedOriginProxy
In the streambased origin proxy flow (example ssh over access), there is
a chance when we do not flush on http.ResponseWriter writes. This PR
guarantees that the response writer passed to proxy stream has a flusher
embedded after writes. This means we write much more often back to the
ResponseWriter and are not waiting. Note, this is only something we do
when proxyHTTP-ing to a StreamBasedOriginProxy because we do not want to
have situations where we are not sending information that is needed by
the other side (eyeball).
2023-07-06 14:22:29 +00:00
Devin Carr b243602d1c TUN-7550: Add pprof endpoint to management service 2023-07-05 20:29:00 +00:00
Sudarsan Reddy 1abd22ef0a TUN-7480: Added a timeout for unregisterUDP.
I deliberately kept this as an unregistertimeout because that was the
intent. In the future we could change this to a UDPConnConfig if we want
to pass multiple values here.

The idea of this PR is simply to add a configurable unregister UDP
timeout.
2023-06-20 06:20:09 +00:00
João Oliveirinha 9c6fbfca18 TUN-7471: Fixes cloudflared not closing the quic stream on unregister UDP session
This code was leaking streams because it wasn't closing the quic stream
after unregistering from the edge.
2023-06-15 10:52:32 +01:00
Devin Carr 9426b60308 TUN-7227: Migrate to devincarr/quic-go
The lucas-clemente/quic-go package moved namespaces and our branch
went stale, this new fork provides support for the new quic-go repo
and applies the max datagram frame size change.

Until the max datagram frame size support gets upstreamed into quic-go,
this can be used to unblock go 1.20 support as the old
lucas-clemente/quic-go will not get go 1.20 support.
2023-05-10 19:44:15 +00:00
Devin Carr 0b5b9b8297 TUN-7130: Categorize UDP logs for streaming logs 2023-04-18 20:49:36 +00:00
Devin Carr 87f81cc57c TUN-7324: Add http.Hijacker to connection.ResponseWriter
Allows connection.ResponseWriter implemenations to be Hijacked to properly
handle WebSocket connection downgrades from proper HTTP requests.
2023-03-29 09:21:19 -07:00
Devin Carr be64362fdb TUN-7124: Add intercept ingress rule for management requests 2023-03-21 11:42:25 -07:00
Devin Carr bf3136debb TUN-7253: Adopt http.ResponseWriter for connection.ResponseWriter 2023-03-08 09:56:47 -08:00
iBug fed60ae4c3
GH-352: Add Tunnel CLI option "edge-bind-address" ()
* Add Tunnel CLI option "edge-bind-address"
2023-02-28 16:11:42 +00:00
João Oliveirinha 68ef4ab2a8 TUN-7197: Add connIndex tag to debug messages of incoming requests 2023-02-22 16:08:24 +00:00
Devin Carr bd046677e5 TUN-7158: Correct TCP tracing propagation
Previously QUIC would send TCP tracing response header that was empty regardless if prompted from origintunneld.
2023-02-03 18:01:27 -08:00
João Oliveirinha 513855df5c TUN-7073: Fix propagating of bad stream request from origin to downstream
This changes fixes a bug where cloudflared was not propagating errors
when proxying the body of an HTTP request.

In a situation where we already sent HTTP status code, the eyeball would
see the request as sucessfully when in fact it wasn't.

To solve this, we need to guarantee that we produce HTTP RST_STREAM
frames.
This change was applied to both http2 and quic transports.
2023-01-23 13:00:58 +00:00
João Oliveirinha 64f15d9992 TUN-6981: We should close UDP socket if failed to connecto to edge 2022-11-29 15:13:34 +00:00
cthuang 495f9fb8bd TUN-6856: Refactor to lay foundation for tracing ICMP
Remove send and return methods from Funnel interface. Users of Funnel can provide their own send and return methods without wrapper to comply with the interface.
Move packet router to ingress package to avoid circular dependency
2022-10-17 19:48:35 +01:00
cthuang 225c344ceb TUN-6855: Add DatagramV2Type for IP packet with trace and tracing spans 2022-10-17 19:45:01 +01:00
João Oliveirinha b01006fe46 TUN-6853: Reuse source port when connecting to the edge for quic connections 2022-10-13 11:50:44 +01:00
cthuang 49438f30f5 TUN-6813: Only proxy ICMP packets when warp-routing is enabled 2022-09-30 19:08:12 +01:00
cthuang eacc8c648d TUN-6812: Drop IP packets if ICMP proxy is not initialized 2022-09-30 14:10:32 +00:00
cthuang be0305ec58 TUN-6741: ICMP proxy tries to listen on specific IPv4 & IPv6 when possible
If it cannot determine the correct interface IP, it will fallback to all interfaces.
This commit also introduces the icmpv4-src and icmpv6-src flags
2022-09-26 11:37:08 +01:00
cthuang 8a53c1aa1d TUN-6592: Decrement TTL and return ICMP time exceed if it's 0 2022-09-15 17:53:26 +01:00
Devin Carr f5f3e6a453 TUN-6689: Utilize new RegisterUDPSession to begin tracing 2022-09-13 14:56:08 +00:00
cthuang 2ffff0687b TUN-6696: Refactor flow into funnel and close idle funnels
A funnel is an abstraction for 1 source to many destinations.
As part of this refactoring, shared logic between Darwin and Linux are moved into icmp_posix
2022-09-09 13:06:00 +01:00
Devin Carr e380333520 TUN-6688: Update RegisterUdpSession capnproto to include trace context 2022-09-08 21:50:58 +00:00
cthuang 59f5b0df83 TUN-6530: Implement ICMPv4 proxy
This proxy uses unprivileged datagram-oriented endpoint and is shared by all quic connections
2022-08-24 17:33:03 +01:00
João Oliveirinha f6bd4aa039 TUN-6676: Add suport for trailers in http2 connections 2022-08-24 15:16:30 +00:00
cthuang d2bc15e224 TUN-6667: DatagramMuxerV2 provides a method to receive RawPacket 2022-08-24 14:56:08 +01:00
cthuang bad2e8e812 TUN-6666: Define packet package
This package defines IP and ICMP packet, decoders, encoder and flow
2022-08-24 11:36:57 +01:00
João Oliveirinha 20ed7557f9 TUN-6679: Allow client side of quic request to close body
In a previous commit, we fixed a bug where the client roundtrip code
could close the request body, which in fact would be the quic.Stream,
thus closing the write-side.
The way that was fixed, prevented the client roundtrip code from closing
also read-side (the body).

This fixes that, by allowing close to only close the read side, which
will guarantee that any subsquent will fail with an error or EOF it
occurred before the close.
2022-08-23 10:43:45 +01:00
cthuang 278df5478a TUN-6584: Define QUIC datagram v2 format to support proxying IP packets 2022-08-12 08:06:56 +00:00
João Oliveirinha 4016334efc TUN-6642: Fix unexpected close of quic stream triggered by upstream origin close
This commit guarantees that stream is only closed once the are finished
handling the stream. Without it, we were seeing closes being triggered
by the code that proxies to the origin, which was resulting in failures
to actually send downstream the status code of the proxy request to the
eyeball.

This was then subsequently triggering unexpected retries to cloudflared
in situations such as cloudflared being unable to reach the origin.
2022-08-10 09:50:27 +01:00
Devin Carr b9cba7f2ae TUN-6576: Consume cf-trace-id from incoming TCP requests to create root span
(cherry picked from commit f48a7cd3dd)
2022-08-02 14:56:31 -07:00
João Oliveirinha 7f1c890a82 Revert "TUN-6576: Consume cf-trace-id from incoming TCP requests to create root span"
This reverts commit f48a7cd3dd.
2022-08-02 11:13:24 +01:00
Devin Carr f48a7cd3dd TUN-6576: Consume cf-trace-id from incoming TCP requests to create root span 2022-08-01 20:22:39 +00:00
Igor Postelnik 1733fe8c65 TUN-6517: Use QUIC stream context while proxying HTTP requests and TCP connections 2022-07-07 18:06:57 -05:00
Devin Carr c7a6304d32 TUN-6007: Implement new edge discovery algorithm
(cherry picked from commit 4f468b8a5d)
2022-06-23 16:55:03 +00:00
Igor Postelnik 3da1c25471 TUN-6381: Write error data on QUIC stream when we fail to talk to the origin; separate logging for protocol errors vs. origin errors. 2022-06-15 13:06:45 +00:00