How Network Visibility Uncovered a DDoS Attack—Before Security Tools Could React

Distributed Denial of Service (DDoS) attacks have been around for decades despite the use of specialized prevention tools, architectures, and best practices. DDoS attacks prove especially hard to detect because they leverage legitimate requests without resorting to malware or other threats that traditional inspection tools usually catch.

What you’ll learn

• How DDoS SYN flood attacks work
• Why security tools struggle to stop DDoS
• How visibility intelligence can spot DDoS attacks underway before sending traffic to security tools

As a general rule, a network visibility platform would not help much, either. Network taps and packet brokers would capture and send traffic to firewalls, IPSs, DDoS mitigation and other security monitoring tools whose job it is to detect threats, but not necessarily detect threats on their own—unless they apply advanced intelligence.

In one recent instance, a Keysight visibility solution deteted a DDoS attack before other cyber defenses caught on. We’ll explain here in detail how AppStack and SecureStack software running inside our Vision network packet brokers uncovered an a TCP SYN flood and helped prevent a DDoS attack during a customer proof of concept (PoC) demonstration.

But first…

What are TCP SYN flood DDoS attacks?

DDoS attacks disrupt the normal flow of traffic through a company’s server, service, or network by overwhelming the target with a flood of Internet requiests. DDoS attacks often involve multiple systems and compromised devices that have become part of botnets of devices that unwittingly work together to generate massive volumes of requests. The target system either detects the attack early or becomes unable to process legitimate requests which leads to slowdowns or full-blown outages.

SYN flood DDoS attacks exploit the three-way TCP handshakes that take place at the network level. A high volume of SYN requests floods a target server pretending to initiate a connection but never actually completes the handshake. The server keeps allocating resources for each half-open connection and eventually exhausts its TCP connections’ table capacity and causing legitimate connections to be denied.

Visibility ‘stacks’ the deck against DDoS

The visibility solution that flagged a DDoS attack include a Keysight network packet broker—the devices that collect, filter, prepare, and send traffic to monitoring tools for analysis—running two types of specialized intelligence:

Keysight’s AppStack feature suite provides deep visibility into application traffic by combining Deep Packet Inspection (DPI) with real-time monitoring of user experience, application performance, and protocol metrics. AppStack integrates with Keysight’s network and cloud

packet brokers to allow third-party tools and sensors to perform efficient traffic inspection and find problems early.

By delivering actionable insights into network behavior and performance bottlenecks, AppStack supports faster issue resolution, improved operational efficiency, and greater overall network reliability.

Keysight SecureStack optimizes threat detection and response by enabling advanced Transport Layer Security (TLS) decryption—onboard a network packet broker (NPB) instead of a security tool or standalone decryption device. With close to 90% of all traffic now traversing networks encrypted, organizations need to see inside all packets to identify threats hidden within encrypted traffic.

Offloading decryption to the visibility platform—specifically the network packet broker that does the main filtering—reduces the load on security tools. Keysight’s SecureStack suite efficiently decrypts SSL/TLS traffic at scale so security tools can focus on analyzing the encrypted data itself, without allocating processing resources to decrypt traffic first.

SecureStack integrates seamlessly with security appliances, providing decrypted traffic to tools like intrusion detection systems (IDS), data loss prevention (DLP) solutions, firewalls, and DDoS mitigation solutions.

Together, AppStack and SecureStack help security tools detect and prevent cyber threats, but as noted earlier, their primary job is not to detect or stop threats independently. In this instance, Keysight’s advanced analytics and statistics capabilities exposed DDoS early, without the assistance of security tools.

Customer setup

The PoC setup included Keysight iBypass switches connected to Keysight Vision ONE network packet brokers (NPBs). The Vision NPBs featured Visibility Application Modules running the Inline Decryption netservice, and were connected to an inline Web Application Firewall (WAF) as shown here:

Customer PoC setup

Because PoCs are temporary deployments, the team did not deploy perimeter defenses or dedicated anti-DDoS capabilities. The WAF did support inspection of cleartext (HTTP) and decrypted traffic, but did not inspect HTTPS which meant cyberattacks could hit the servers or Inline Decryption capabilities on the Keysight packet broker directly.

In addition to SecureStack for Inline Decryption, Keysight’s AppStack suite was configured to collect traffic from encrypted ports going in and out of SecureStack and to analyze encrypted traffic.

AppStack collecting encrypted traffic

At the start of the PoC, graphs and statistics showed normal traffic and decryption activity. The dashboard for Inline Decryption showed the decrypted TLS connections as expected:

Inline Decryption graph showing the decrypted TLS connections for the first 3 days

Then, on the third day of the PoC, several Inline Decryption counters detected abnormalties. For one thing, TLS sessions did not increase as the number of active TCP connections (currentTcpSessions) climbed to nearly 150K within minutes. Since 90% of traffic routinely gets encrypted—and therefore should have consisted of HTTPS—the fact that so many more unencrypted TCP sessions were occurring compared to encrypted TLS sessions, sent up a red flag.

Analyzing the counters of the three-way TCP handshakes showed the numbers of TCP SYN requests and SYN-ACK responses to be about equal but that there were far less ACK responses in the TCP handshakes. The handshakeAckRcv counter—which indicates how many handshake ACKs get received from the real client—showed ~185,000 requests compared to ~20,000,000 SYN requests from the real clients (synRcv) and ~22,000,000 SYN-ACK replies from the real servers (synAckRcv).

Drill-down in number of TCP packets from PCAP file

Analysis confirmed that far too many TCP handshakes had been initiated but never completed. Visual inspection of the lines in the PCAP identified the behavior as a TCP SYN flood attack:

PCAP file exposing a TCP SYN flood (Sensitive info, such as IP address, has been blurred)

How it works

1. First, a client on the internet sends a TCP SYN request to the server’s IP address in the customer’s DMZ
2. The server replies immediately with a SYN-ACK
3. The client never responds with the final ACK, making the server keep the half-session open in the session table
4. Almost every millisecond, another client with a different IP, often from a different geographical region, starts a session which it never acknowledges, repeating the cycle.

Because the servers and SecureStack could handle a much higher connection rate and number of concurrent connections than what the attack produced, the attack was contained before it could inflict any damage or cause a service disruption. Keysight’s Inline Decryption also times out idle connections after 2 minutes, which enhances protection against DDoS.

It takes a village to stop DDoS

Responsibility for cybersecurity no longer rests solely with the security team. Along with the 30+ specialized security tools many enterprises operate, users, devices, applications, and other elements of IT all need to contribute and do their part to secure company assets and privileged data.

Intelligent network visibility augments a traditional security monitoring infrastructure by optimizing utilization and offloading processing-intensive functions like decryption. And, as shown by this particular use case, adding insight, intelligence, and context to detect attacks before it even gets to monitoring tools.

Keysight AppStack and SecureStack deliver powerful visibility intelligence that enhances your application monitoring and security inspection capabilities without adding to your processing burden or requiring any special configuration.

To leverage advanced visibility statistics and dashboards to spot abnormal traffic indicating a DDoS and other cyberattacks early on, visit AppStack and SecureStack to learn more.