Commit Graph

57 Commits

Author SHA1 Message Date
Devin Carr ab3dc5f8fa TUN-8701: Simplify flow registration logs for datagram v3
To help reduce the volume of logs during the happy path of flow registration, there will only be one log message reported when a flow is completed.

There are additional fields added to all flow log messages:
1. `src`: local address
2. `dst`: origin address
3. `durationMS`: capturing the total duration of the flow in milliseconds

Additional logs were added to capture when a flow was migrated or when cloudflared sent off a registration response retry.

Closes TUN-8701
2024-11-12 10:54:37 -08:00
Devin Carr 1f3e3045ad TUN-8701: Add metrics and adjust logs for datagram v3
Closes TUN-8701
2024-11-07 11:02:55 -08:00
Devin Carr 952622a965 TUN-8709: Add session migration for datagram v3
When a registration response from cloudflared gets lost on it's way back to the edge, the edge service will retry and send another registration request. Since cloudflared already has bound the local UDP socket for the provided request id, we want to re-send the registration response.

There are three types of retries that the edge will send:

1. A retry from the same QUIC connection index; cloudflared will just respond back with a registration response and reset the idle timer for the session.
2. A retry from a different QUIC connection index; cloudflared will need to migrate the current session connection to this new QUIC connection and reset the idle timer for the session.
3. A retry to a different cloudflared connector; cloudflared will eventually time the session out since no further packets will arrive to the session at the original connector.

Closes TUN-8709
2024-11-06 12:06:07 -08:00
Gonçalo Garcia 3d33f559b1 TUN-8641: Expose methods to simplify V3 Datagram parsing on the edge 2024-11-04 15:23:36 -08:00
Devin Carr 5891c0d955 TUN-8700: Add datagram v3 muxer
The datagram muxer will wrap a QUIC Connection datagram read-writer operations to unmarshal datagrams from the connection to the origin with the session manager. Incoming datagram session registration operations will create new UDP sockets for sessions to proxy UDP packets between the edge and the origin. The muxer is also responsible for marshalling UDP packets and operations into datagrams for communication over the QUIC connection towards the edge.

Closes TUN-8700
2024-11-04 11:20:35 -08:00
Devin Carr 6a6c890700 TUN-8667: Add datagram v3 session manager
New session manager leverages similar functionality that was previously
provided with datagram v2, with the distinct difference that the sessions
are registered via QUIC Datagrams and unregistered via timeouts only; the
sessions will no longer attempt to unregister sessions remotely with the
edge service.

The Session Manager is shared across all QUIC connections that cloudflared
uses to connect to the edge (typically 4). This will help cloudflared be
able to monitor all sessions across the connections and help correlate
in the future if sessions migrate across connections.

The UDP payload size is still limited to 1280 bytes across all OS's. Any
UDP packet that provides a payload size of greater than 1280 will cause
cloudflared to report (as it currently does) a log error and drop the packet.

Closes TUN-8667
2024-10-31 14:05:15 -07:00
Devin Carr abb3466c31 TUN-8638: Add datagram v3 serializers and deserializers
Closes TUN-8638
2024-10-16 12:05:55 -07:00
chungthuang 0b62d45738 TUN-8456: Update quic-go to 0.45 and collect mtu and congestion control metrics 2024-06-17 15:28:56 +00:00
chungthuang a16532dbbb TUN-8451: Log QUIC flow control frames and transport parameters received 2024-06-12 19:23:39 +00:00
Devin Carr eb2e4349e8 TUN-8415: Refactor capnp rpc into a single module
Combines the tunnelrpc and quic/schema capnp files into the same module.

To help reduce future issues with capnp id generation, capnpids are
provided in the capnp files from the existing capnp struct ids generated
in the go files.

Reduces the overall interface of the Capnp methods to the rest of
the code by providing an interface that will handle the quic protocol
selection.

Introduces a new `rpc-timeout` config that will allow all of the
SessionManager and ConfigurationManager RPC requests to have a timeout.
The timeout for these values is set to 5 seconds as non of these operations
for the managers should take a long time to complete.

Removed the RPC-specific logger as it never provided good debugging value
as the RPC method names were not visible in the logs.
2024-05-17 11:22:07 -07:00
João "Pisco" Fernandes da6fac4133 TUN-8297: Improve write timeout logging on safe_stream.go
## Summary:
In order to properly monitor what is happening with the new write timeouts that we introduced
in TUN-8244 we need proper logging. Right now we were logging write timeouts when the safe
stream was being closed which didn't make sense because it was miss leading, so this commit
prevents that by adding a flag that allows us to know whether we are closing the stream or not.
2024-03-13 13:30:45 +00:00
João "Pisco" Fernandes 4f7165530c TUN-8275: Skip write timeout log on "no network activity"
## Summary
To avoid having to verbose logs we need to only log when an
actual issue occurred. Therefore, we will be skipping any error
logging if the write timeout is caused by no network activity
which just means that nothing is being sent through the stream.
2024-03-06 16:05:48 +00:00
chungthuang 34a876e4e7 TUN-8243: Collect metrics on the number of QUIC frames sent/received
This commit also removed the server metrics that is no longer used
2024-02-19 10:09:14 +00:00
João "Pisco" Fernandes 76badfa01b TUN-8236: Add write timeout to quic and tcp connections
## Summary
To prevent bad eyeballs and severs to be able to exhaust the quic
control flows we are adding the possibility of having a timeout
for a write operation to be acknowledged. This will prevent hanging
connections from exhausting the quic control flows, creating a DDoS.
2024-02-15 17:54:52 +00:00
chungthuang 8e69f41833 TUN-7934: Update quic-go to a version that queues datagrams for better throughput and drops large datagram
Remove TestUnregisterUdpSession
2024-01-03 13:01:01 +00:00
Chung-Ting 8068cdebb6 TUN-8006: Update quic-go to latest upstream 2023-12-04 17:09:40 +00:00
Sudarsan Reddy 1abd22ef0a TUN-7480: Added a timeout for unregisterUDP.
I deliberately kept this as an unregistertimeout because that was the
intent. In the future we could change this to a UDPConnConfig if we want
to pass multiple values here.

The idea of this PR is simply to add a configurable unregister UDP
timeout.
2023-06-20 06:20:09 +00:00
João Oliveirinha 20e36c5bf3 TUN-7468: Increase the limit of incoming streams 2023-06-19 10:41:56 +00:00
Devin Carr 9426b60308 TUN-7227: Migrate to devincarr/quic-go
The lucas-clemente/quic-go package moved namespaces and our branch
went stale, this new fork provides support for the new quic-go repo
and applies the max datagram frame size change.

Until the max datagram frame size support gets upstreamed into quic-go,
this can be used to unblock go 1.20 support as the old
lucas-clemente/quic-go will not get go 1.20 support.
2023-05-10 19:44:15 +00:00
João Oliveirinha 0be1ed5284 TUN-7398: Add support for quic safe stream to set deadline 2023-04-27 19:49:56 +01:00
João Oliveirinha 7ef9bb89d3 TUN-7000: Reduce metric cardinality of closedConnections metric by removing error as tag 2022-12-07 11:09:16 +00:00
cthuang 225c344ceb TUN-6855: Add DatagramV2Type for IP packet with trace and tracing spans 2022-10-17 19:45:01 +01:00
cthuang be0305ec58 TUN-6741: ICMP proxy tries to listen on specific IPv4 & IPv6 when possible
If it cannot determine the correct interface IP, it will fallback to all interfaces.
This commit also introduces the icmpv4-src and icmpv6-src flags
2022-09-26 11:37:08 +01:00
Devin Carr f5f3e6a453 TUN-6689: Utilize new RegisterUDPSession to begin tracing 2022-09-13 14:56:08 +00:00
Devin Carr e380333520 TUN-6688: Update RegisterUdpSession capnproto to include trace context 2022-09-08 21:50:58 +00:00
Chung-Ting Huang 3e0ff3a771 TUN-6531: Implement ICMP proxy for Windows using IcmpSendEcho 2022-09-07 19:18:06 +00:00
cthuang faa86ffeca TUN-6737: Fix datagramV2Type should be declared in its own block so it starts at 0 2022-09-05 15:09:53 +01:00
Nuno Diegues 7ca5f7569a TUN-6726: Fix maxDatagramPayloadSize for Windows QUIC datagrams 2022-09-01 21:32:59 +00:00
João Oliveirinha e131125558 TUN-6699: Add metric for packet too big dropped 2022-08-26 16:02:43 +00:00
cthuang 59f5b0df83 TUN-6530: Implement ICMPv4 proxy
This proxy uses unprivileged datagram-oriented endpoint and is shared by all quic connections
2022-08-24 17:33:03 +01:00
cthuang d2bc15e224 TUN-6667: DatagramMuxerV2 provides a method to receive RawPacket 2022-08-24 14:56:08 +01:00
cthuang bad2e8e812 TUN-6666: Define packet package
This package defines IP and ICMP packet, decoders, encoder and flow
2022-08-24 11:36:57 +01:00
cthuang 278df5478a TUN-6584: Define QUIC datagram v2 format to support proxying IP packets 2022-08-12 08:06:56 +00:00
João Oliveirinha 9de4e88ca6 TUN-6646: Add support to SafeStreamCloser to close only write side of stream 2022-08-10 20:57:30 +00:00
cthuang f3ba506880 TUN-6499: Remove log that is per datagram 2022-07-05 18:06:37 +01:00
Igor Postelnik 3da1c25471 TUN-6381: Write error data on QUIC stream when we fail to talk to the origin; separate logging for protocol errors vs. origin errors. 2022-06-15 13:06:45 +00:00
Nuno Diegues 475939a77f TUN-6191: Update quic-go to v0.27.1 and with custom patch to allow keep alive period to be configurable
The idle period is set to 5sec.

We now also ping every second since last activity.
This makes the quic.Connection less prone to being closed with
no network activity, since we send multiple pings per idle
period, and thus a single packet loss cannot cause the problem.
2022-06-07 12:25:18 +01:00
cthuang baed5f4eea TUN-6308: Add debug logs to see if packets are sent/received from edge 2022-05-30 14:01:40 +01:00
Igor Postelnik 2c480a72db TUN-6280: Don't wrap qlog connection tracer for gatethering QUIC metrics since we're not writing qlog files. 2022-05-24 16:59:03 -05:00
cthuang e56c4532ce TUN-5697: Listen for UpdateConfiguration RPC in quic transport 2022-02-18 09:12:39 +00:00
cthuang d07d24e5a2 TUN-5695: Define RPC method to update configuration 2022-02-03 15:05:46 +00:00
Nuno Diegues ed2bac026d TUN-5621: Correctly manage QUIC stream closing
Until this PR, we were naively closing the quic.Stream whenever
the callstack for handling the request (HTTP or TCP) finished.
However, our proxy handler may still be reading or writing from
the quic.Stream at that point, because we return the callstack if
either side finishes, but not necessarily both.

This is a problem for quic-go library because quic.Stream#Close
cannot be called concurrently with quic.Stream#Write

Furthermore, we also noticed that quic.Stream#Close does nothing
to do receiving stream (since, underneath, quic.Stream has 2 streams,
1 for each direction), thus leaking memory, as explained in:
https://github.com/lucas-clemente/quic-go/issues/3322

This PR addresses both problems by wrapping the quic.Stream that
is passed down to the proxying logic and handle all these concerns.
2022-02-01 22:01:57 +00:00
cthuang 6fa58aadba TUN-5623: Configure quic max datagram frame size to 1350 bytes for none Windows platforms 2022-01-11 14:55:43 +00:00
Igor Postelnik 8445b88d3c TUN-5593: Read full packet from UDP connection, even if it exceeds MTU of the transport. When packet length is greater than the MTU of the transport, we will silently drop packets (for now). 2021-12-22 17:18:22 -06:00
cthuang 7a55208c61 TUN-5590: QUIC datagram max user payload is 1217 bytes 2021-12-22 19:53:58 +00:00
cthuang ebae7a7024 TUN-5494: Send a RPC with terminate reason to edge if the session is closed locally 2021-12-21 09:52:39 +00:00
cthuang 8f46065ab5 TUN-5570: Only log RPC server events at error level to reduce noise 2021-12-20 16:50:24 +00:00
cthuang 73a265f2fc TUN-5488: Close session after it's idle for a period defined by registerUdpSession RPC 2021-12-09 11:55:34 +00:00
cthuang b73c588254 TUN-5422: Define RPC to unregister session 2021-12-06 16:37:09 +00:00
cthuang eea3d11e40 TUN-5301: Separate datagram multiplex and session management logic from quic connection logic 2021-12-06 16:37:09 +00:00