Netflow (Network Data Flow Detection Protocol)
With the software system upgrade and the maturity of the vulnerability repair scheme, the virus attack mode that directly invades the host for damage is gradually reduced, and then turns to malicious consumption of limited network resources, causing network congestion, thus destroying the ability of the system to provide external services. In response to such attacks, the industry has proposed a method of detecting network data Flow to judge network anomalies and attacks. By detecting network data flow information in real time, network managers can check the status of the whole network in real time by matching the historical pattern (to judge whether it is normal) or the abnormal pattern (to judge whether it is attacked). Detect possible bottlenecks in network performance, and automatically handle or alarm display to ensure efficient and reliable network operation.
Netflow technology was first invented by Darren Kerr and Barry Bruins of Cisco in 1996 and registered as a US patent in May of the same year. Netflow technology is first used in network equipment to accelerate data exchange, and can realize the measurement and statistics of high-speed forwarded IP data flow. After years of technological evolution, the original function of Netflow for data exchange acceleration has been gradually replaced by dedicated ASIC chips in network devices, while the function of measuring and statistics of IP data flow through network devices is still retained. It has become the most recognized industry standard for IP/MPLS traffic analysis, statistics and billing in the Internet field. Netflow technology can analyze and measure the detailed behavior pattern of IP/MPLS network traffic, and provide detailed statistics of network operation.
The Netflow system consists of three main parts: the Exporter, the Collector, and the Analysis reporting system.
Exporter: monitors network data
Collector: Used to collect network data exported from Exporter
Analysis: Used to analyze the network data collected from the Collector and generate reports
By analyzing the information collected by Netflow, network administrators can know the source, destination, network service type of packets, and the cause of network congestion. It may not provide a complete record of network traffic like tcpdump does, but when put together it is much easier to manage and read.
NetFlow network data output from routers and switches consists of expired data flows and detailed traffic statistics. These data flows contain the IP address associated with the source and destination of the packet, as well as the Protocol and Port used by the end-to-end session. The traffic statistics include the data flow timestamp, source and destination IP addresses, source and destination port numbers, input and output interface numbers, next hop IP addresses, total bytes in the flow, number of packets in the flow, and time stamps of the first and last packets in the flow. And front mask, packet number, etc.
Netflow V9 is a new flexible and extensible Netflow data output format with template-based statistics output. Easy to add data fields that need to be output and support a variety of new functions, such as: Multicase Netflow, MPLS Aware Netflow, BGP Next Hop V9, Netflow for IPv6, and so on.
In 2003, Netflow V9 was also selected as the IPFIX (IP Flow Information Export) standard by the IETF from five candidates.
IPFIX (Network Traffic Monitoring)
Flow-based technology is widely used in network field, it has great value in QoS policy setting, application deployment and capacity planning. However, network administrators lack a standard format for output data streams. IPFIX (IP Flow Information Export, IP data flow information output) is a standard protocol for measuring flow information in networks published by IETF.
The format defined by IPFIX is based on the Cisco Netflow V9 data output format, which standardizes the statistics and output standards of IP data flows. It is a protocol for analyzing data flow characteristics and output data in a template-based format. Therefore, it has strong scalability. If traffic monitoring requirements change, network administrators can modify the corresponding configurations without upgrading network device software or management tools. Network administrators can easily extract and view important traffic statistics stored in these network devices.
For a more complete output, IPFIX uses seven key domains of network devices by default to represent network traffic per share:
1. Source IP address
2. Destination IP address
3. TCP/UDP source port
4. TCP/UDP destination port
5. Layer 3 protocol type
6. The Type of service (Type of service) byte
7. Enter a logical interface
If all seven key domains in different IP packets match, the IP packets are regarded as belonging to the same traffic. By recording the characteristics of the traffic on the network, such as the traffic duration and average packet length, you can learn about the current network application, optimize the network, detect the security, and charge the traffic.
IPFIX networking architecture
To sum up, IPFIX is based on the concept of Flow. A Flow refers to packets from the same subinterface with the same source and destination IP address, protocol type, source and destination port number, and ToS. The packets are usually 5-tuples. IPFIX records statistics about the stream, including the timestamp, the number of packets, and the total number of bytes. IPFIX consists of three devices: Exporter, Collector, and Analyzer. The relationships among the three devices are as follows:
Export analyzes network flows, extracts qualified flow statistics, and sends the statistics to Collector.
The Collector parses Export data packets and collects statistics in the database for analysis by the Analyser.
The Analyser extracts statistics from the Collector, performs subsequent processing, and displays the statistics as a GUI for various services.
IPFIX application scenarios
Usage-based Accounting
Traffic billing in network operators is generally based on the upload and download traffic of each user. Because IPFIX can be accurate to the destination IP address, protocol port and other fields, the future traffic charging can be segmented based on the characteristics of the application service. Of course, the protocol also explains that IPFIX packet statistics are "sampled". In many applications (such as the backbone layer), the more detailed the data flow statistics are, the better. Due to the performance of network devices, the sampling rate cannot be too small, so it is not necessary to provide completely accurate and reliable traffic billing. However, at the network operator level, the billing unit is generally more than 100 megabits, and the sampling accuracy of IPFIX can meet the relevant needs.
Traffic Profiling, Traffic Engineering
The record output of IPFIX Exporter, IPFIX Collector can output very rich Traffic record information in the form of various charts, this is the concept of Traffic Profiling.
However, just the record of information, can not take advantage of the powerful function of IPFIX, IETF also launched the concept of Traffic Engineering: in the actual operation of the network, often planned load balancing and redundant backup, but the various protocols are generally according to the predetermined route of the network planning, or protocol principles are adjusted. If IPFIX is used to monitor the traffic on the network and a large amount of data is found in a certain period of time, the network administrator can be reported to adjust the traffic, so that more network bandwidth can be allocated to related applications to reduce the uneven load. In addition, you can bind configuration rules, such as route adjustment, bandwidth allocation, and security policies, to the operations on the IPFIX Collector to automatically adjust network traffic.
Attack/Intrusion Detection Attack/Intrusion detection
IPFIX can detect network attacks based on traffic characteristics. For example, typical IP scanning, port scanning, DDOS attacks. The sampling standard IPFIX protocol can also use a "signature database" upgrade to block the latest network attacks, just like the general host-side virus protection.
QoS Monitoring (Network Quality of Service Monitoring)
Typical QoS parameters are:
Packet loss condition: loss [RFC2680]
one-way delay: One-way delay [RFC2679]
round-trip delay: round-trip delay [RFC2681]
delay variation [RFC3393]
Previous technologies are difficult to monitor the above information in real time, but IPFIX's various custom fields and monitoring intervals can easily monitor the above values of various messages.
Here's an expanded table that provides more details about the differences between NetFlow and IPFIX:
