Five Essentials for Best of Breed Next Gen Firewalls
Beyond basic network firewall testing scenarios, the specialized firewall testing tools needed to accurately assess next-generation firewall (NGFW) security effective remain out of reach to any but the largest IT department budgets. Therefore, most organizations look to independent hands-on test results from respected research laboratories such as NSS Labs. NSS Labs uses a very specific testing methodology that is run on each of the NGFWs being tested. Their Next-Generation Firewall Product Analysis Report provides detailed information on how a specific firewall scored when tested in these key essential areas:
- Security Effectiveness
- Performance
- Stability and Reliability
- Management and Configuration
- Total Cost of Ownership
Security Effectiveness
Security effectiveness verifies that the firewall being tested is capable of enforcing the security policy effectively. Security effectiveness tests include:
|
Firewall Policy enforcement |
Incremental tests that build configuration from simple to complex real world policy consisting of many addresses, policies, applications, inspection engines, protection from DoS attacks, IP spoofing. |
|
Application Control |
Firewall is tested to see if it can correctly determine application regardless of ports/protocols used and enforce appropriate application policy granularity. |
|
User/Group ID aware policies |
Correctly determine user/group from deep packet inspection and enforce policy with user awareness. |
|
Intrusion Prevention |
Correctly block malicious traffic “out of the box” using the default policy (for this test no IPS tuning is allowed). |
| Evasion | Decode/Block basic obfuscated exploits and provide accurate alert based on the actual attack not be fooled by the evasion technique itself. |
How did SonicWall next-generation firewalls do? Passed all criteria. Noteworthy SonicWall results included a 97.9 percent exploit block rate. No NGFW tested achieved 100 percent exploit block rate due to constantly changing NSS Labs test suite. However, over the last three years SonicWall has consistently been rated in the leaders quadrant and has demonstrated consistent improved block rate year over year.
Performance
Measures how well a given NGFW performs when subjected to various traffic conditions. No two networks will have the exact same characteristics but this test does provide metrics to gauge if a given NGFW is appropriate in a given environment.
| Raw Packet Processing Performance (UDP packets of various sizes are tested) | Measures raw packet processing capability of each of the NGFWs in-line port pairs, packet forwarding rate is measured for highest performance /lowest latency. |
| Latency (packet loss/average latency) | Determine the effect the NGFW has on traffic passing through it under various loads. Traffic passes through all port pairs simultaneously. |
| Maximum Capacity ( generates TCP session based connections and HTTP transactions) | Stress the inspection engine with Multi-Gigabit “Real World” traffic generated to determine expected user response times, max connections per second, concurrent open connections, application transaction per second on a backdrop of a heavily utilized network. |
| HTTP Capacity ““ No Transaction Delay (uses HTTP GET request) | How much HTTP traffic can be passed of varying packet sizes and various connection per second loads. |
| Application average response time ““ HTTP (across all in-line port pairs simultaneously) | Measures average HTTP latency using various packet sizes at 90 percent of max load. |
| HTTP Capacity with Transaction Delay | Same as above except introduces 5 second server response delay, forces a high number of open connections. |
| Real World Traffic (generates protocol mix usually seen by industry verticals, i.e. Financial, education, Data Center, Mobile Carrier, etc”¦ ) | Same as previous test, excepts adds additional protocols and real content. |
Stability and Reliability
These tests measure how well a next-generation firewall passes legitimate traffic while under attack. To pass, the NGFW must be able to block and alert on 100 percent of the attacks previously blocked while remaining operational.
| Blocking under Extended Attack | Measures consistency of Blocking. Sends continuous policy violations at 100Mbps over 8 hours. |
| Passing Legitimate Traffic Under Extended Attack | Same as previous test except legitimate traffic is sent in addition. NGFW must pass all legitimate traffic. |
| Behavior of State Engine Under Load ( | Can the NGFW preserve state across large number of connections over extended time. Must not exhaust resources allocated to state tables or “˜leak’ connections through after theoretical max concurrent connection is reached. |
| Protocol Fuzzing and Mutation | Sends random, unexpected, or invalid data to the NGFW, verifies NGFW remains operational and detects/blocks exploit throughout the test. |
| Power Fail | Power is turned off while passing traffic, NGFW should fail closed after power is cut. |
| Persistence of Data | Measures if NGFW retains policy, configuration, log data when restored from power failure. |
Total Cost of Ownership and Value
Measures overall costs over of deployment, maintenance and upkeep over the useful life of the product.
| Product Purchase | Cost of acquisition |
| Product Maintenance | Fees paid to vendor (hardware maintenance, subscription services, etc”¦) |
| Installation | Time required to make the NGFW operational out of the box. |
| Upkeep | Time required to apply vendor supplied firmware, updates, patches. |
