1m_tcp_syn_queue_drops

Alerts
1

1m_tcp_syn_queue_drops

OS: Linux

This alert presents the average number of SYN requests that were dropped due to the TCP SYN queue
being full over the last sixty seconds. Receiving this means that the TCP SYN queue is full and that the
system is dropping incoming TCP SYN requests.

The alert is raised to warning when this average exceeds 1 per minute. The alert is raised to critical
when the value exceeds an average of 5 dropped SYN packets per sixty seconds.

What is the SYN queue?

The SYN Queue stores inbound SYN packets (specifically: struct inet_request_sock). It’s
responsible for sending out SYN+ACK packets and retrying them on timeout.
After transmitting the SYN+ACK, the SYN Queue waits for an ACK packet from the client - the last
packet in the three-way-handshake. All received ACK packets must first be matched against the
fully established connection table, and only then against data in the relevant SYN Queue. On
SYN Queue match, the kernel removes the item from the SYN Queue, happily creates a fully fledged
connection (specifically: struct inet_sock), and adds it to the Accept Queue. 1


This alert has a high probability of indicating a SYN flood.

A SYN flood is a form of denial-of-service attack in which an attacker rapidly initiates a
connection to a server without finalizing the connection. The server has to spend resources
waiting for half-opened connections, which can consume enough resources to make the system
unresponsive to legitimate traffic. 2


SYN Cookies

SYN cookies are a technique used to resist IP address spoofing attacks. The technique’s primary
inventor, Daniel J. Bernstein, defines SYN cookies as “particular choices of initial TCP sequence
numbers by TCP servers.” In particular, the use of SYN cookies allows a server to avoid dropping
connections when the SYN queue fills up. Instead of storing additional connections, a SYN queue
entry is encoded into the sequence number sent in the SYN+ACK response. If the server then
receives a subsequent ACK response from the client with the incremented sequence number, the
server is able to reconstruct the SYN queue entry using information encoded in the TCP
sequence number and proceed as usual with the connection. 3


References and sources
  1. SYN packet handling
  2. SYN Floods
  3. SYN Cookies
  4. ip-sysctl.txt
  5. Transmission Control Protocol

Troubleshooting Section

Enable SYN Cookies

If you can determine that the traffic is legitimate, consider enabling SYN cookies. To enable SYN cookies:

  1. Open the /etc/sysctl.conf file and look for the entry “net.ipv4.tcp_syncookies”.
  2. Set the value to 1. net.ipv4.tcp_syncookies=1 (if the value doesn’t exist, append it to the file)
  3. Save your changes and run;
    root@netdata~ #sysctl -p
    to apply the changes.

Netdata strongly suggests knowing exactly what you are configuring before making system changes.