Centos7 ipset命令介绍及使用(Introduction and use of centos7 ipset command)

ipset介绍

  iptables是在linux内核里配置防火墙规则的用户空间工具,它实际上是netfilter框架的一部分。可能因为iptables是netfilter框架里最常见的部分,所以这个框架通常被称为iptables,iptables是linux从2.4版本引入的防火墙解决方案。ipset是iptables的扩展,它允许你创建匹配整个地址sets(地址集合) 的规则。而不像普通的iptables链是线性的存储和过滤,ip集合存储在带索引的数据结构中,这种结构即使集合比较大也可以进行高效的查找。除了一些常用的情况,比如阻止一些危险主机访问本机,从而减少系统资源占用或网络拥塞,ipset也具备一些新防火墙设计方法,并简化了配置。官网:http://ipset.netfilter.org/

1.ipset安装(centos7.x默认已经安装ipset,无需再进行安装)

yum install ipset

2.创建一个ipset

ipset create xxx hash:ip (hash:ip指的是单个ip,xxx是ipset名称)

ipset默认可以存储65536个元素,使用maxelem指定数量

ipset create blacklist hash:ip maxelem 1000000    #黑名单
ipset create whitelist hash:ip maxelem 1000000    #白名单

查看当前服务器已创建的ipset

ipset list

3.加入一个名单ip

ipset add blacklist 10.20.30.12

4.去除名单ip

ipset del blacklist 10.20.30.12

5.创建防火墙规则

iptables -I INPUT -m set --match-set blacklist src -p tcp -j DROP
iptables -I INPUT -m set --match-set whitelist src -p tcp -j DROP
service iptables save

6.将ipset规则保存到文件

ipset save blacklist -f blacklist.txt
ipset save whitelist -f whitelist.txt

7.删除ipset

ipset destroy blacklist
ipset destroy whitelist

8.导入ipset规则

ipset restore -f blacklist.txt
ipset restore -f whitelist.txt

ipset的一个优势是集合可以动态的修改,即使ipset的iptables规则目前已经启动,新加的入ipset的ip也生效。

ipset使用:使用 ipset 封大量ip

  Linux使用iptables封IP,是常用的应对网络攻击的方法,但要封禁成千上万个IP,如果添加成千上万条规则,对机器性能影响较大,使用ipset能解决这个问题。iptables 包含几个表,每个表由链组成。默认的是 filter 表,最常用的也是 filter 表,另一个比较常用的是nat表,封IP就是在 filter 表的 INPUT 链添加规则。在进行规则匹配时,是从规则列表中从头到尾一条一条进行匹配。这像是在链表中搜索指定节点费力。ipset 提供了把这个 O(n) 的操作变成 O(1) 的方法:就是把要处理的 IP 放进一个集合,对这个集合设置一条 iptables 规则。像 iptable 一样,IP sets 是 Linux 内核中的东西,ipset 这个命令是对它进行操作的一个工具。

简单的流程可以用这几条命令概括使用 ipset 和 iptables 进行 IP 封禁的流程

ipset create vader hash:ip
iptables -I INPUT -m set --match-set vader src -j DROP
ipset add vader 4.5.6.7
ipset add vader 1.2.3.4
ipset add vader ...
ipset list vader # 查看 vader 集合的内容

下面分别对各条命令进行描述。1、创建一个集合

ipset create vader hash:ip

这条命令创建了名为 vader 的集合,以 hash 方式存储,存储内容是 IP 地址。

2、添加 iptables 规则

iptables -I INPUT -m set --match-set vader src -j DROP

如果源地址(src)属于 vader 这个集合,就进行 DROP 操作。这条命令中,vader 是作为黑名单的,如果要把某个集合作为白名单,添加一个 ‘!’ 符号就可以。

iptables -I INPUT -m set ! --match-set yoda src -j DROP

到现在虽然创建了集合,添加了过滤规则,但是现在集合还是空的,需要往集合里加内容。

3、找出“坏” IP找出要封禁的 IP,这是封禁过程中重要的步骤,不过不是这里的重点。简要说明一下两种方法思路。

netstat -ntu | tail -n +3 | awk '{print $5}' | sort | uniq -c | sort -nr

直接通过 netstat 的信息,把与本地相关的各种状态的 IP 都计数,排序列出来。

或者从 nginx 或者其他 web server 的日志里找请求数太多的 IP

awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -nr

后半部分,排序,去重,再按次数进行逆向排序的操作,跟上面命令是一样的。

找出“坏” IP,往之前创建的集合里添加就可以了。

ipset add vader 4.5.6.7

有多少“坏” IP,就添加多少 IP,因为针对这些封禁的 IP 只需要一条 iptables 规则,而这些 IP 是以 hash 方式存储,所以封禁大量的 IP 也不会影响性能,这也是 ipset 存在的最大目的。

ipset 更多的用法

1、存储类型前面例子中的 vader 这个集合是以 hash 方式存储 IP 地址,也就是以 IP 地址为 hash 的键。除了 IP 地址,还可以是网络段,端口号(支持指定 TCP/UDP 协议),mac 地址,网络接口名称,或者上述各种类型的组合。

比如指定 hash:ip,port就是 IP 地址和端口号共同作为 hash 的键。查看 ipset 的帮助文档可以看到它支持的所有类型。

下面以两个例子说明。

HASH:NET
ipset create r2d2 hash:net
ipset add r2d2 1.2.3.0/24
ipset add r2d2 1.2.3.0/30 nomatch
ipset add r2d2 6.7.8.9
ipset test r2d2 1.2.3.2
hash:net 指定了可以往 r2d2 这个集合里添加 IP 段或 IP 地址。

第三条命令里的 nomatch 的作用简单来说是把 1.2.3.0/30 从 1.2.3.0/24 这一范围相对更大的段里“剥离”了出来,也就是说执行完 ipset add r2d2 1.2.3.0/24 只后1.2.3.0/24 这一段 IP 是属于 r2d2 集合的,执行了 ipset add r2d2 1.2.3.0/30 nomatch 之后,1.2.3.0/24 里 1.2.3.0/30 这部分,就不属于 r2d2 集合了。执行 ipset test r2d2 1.2.3.2 就会得到结果 1.2.3.2 is NOT in set r2d2.

HASH:IP,PORT
ipset create c-3po hash:ip,port
ipset add c-3po 3.4.5.6,80
ipset add c-3po 5.6.7.8,udp:53
ipset add c-3po 1.2.3.4,80-86

第二条命令添加的是 IP 地址为 3.4.5.6,端口号是 80 的项。没有注明协议,默认就是 TCP,下面一条命令则是指明了是 UDP 的 53 端口。最后一条命令指明了一个 IP 地址和一个端口号范围,这也是合法的命令。

自动过期,解封ipset 支持 timeout 参数,这就意味着,如果一个集合是作为黑名单使用,通过 timeout 参数,就可以到期自动从黑名单里删除内容。

ipset create obiwan hash:ip timeout 300
ipset add obiwan 1.2.3.4
ipset add obiwan 6.6.6.6 timeout 60

上面第一条命令创建了名为 obiwan 的集合,后面多加了 timeout 参数,值为 300,往集合里添加条目的默认 timeout 时间就是 300。第三条命令在向集合添加 IP 时指定了一个不同于默认值的 timeout 值 60,那么这一条就会在 60 秒后自动删除。

隔几秒执行一次 ipset list obiwan 可以看到这个集合里条目的 timeout 一直在随着时间变化,标志着它们在多少秒之后会被删除。

如果要重新为某个条目指定 timeout 参数,要使用 -exit 这一选项。

ipset -exist add obiwan 1.2.3.4 timeout 100

这样 1.2.3.4 这一条数据的 timeout 值就变成了 100,如果这里设置 300,那么它的 timeout,也就是存活时间又重新变成 300。

如果在创建集合是没有指定 timeout,那么之后添加条目也就不支持 timeout 参数,执行 add 会收到报错。想要默认条目不会过期(自动删除),又需要添加某些条目时加上 timeout 参数,可以在创建集合时指定 timeout 为 0。

ipset create luke hash:ip
ipset add luke 5.5.5.5 timeout 100

得到报错信息 kernel error received: Unknown error -1

hashsize, maxelem 这两个参数分别指定了创建集合时初始的 hash 大小,和最大存储的条目数量。

ipset create yoda hash:ip,port hashsize 4096 maxelem 1000000
ipset add yoda 3.4.5.6,3306

这样创建了名为 yoda 的集合,初始 hash 大小是 4096,如果满了,这个 hash 会自动扩容为之前的两倍。最大能存储的数量是 100000 个。

如果没有指定,hashsize 的默认值是 1024,maxelem 的默认值是 65536。

2、另外几条常用命令

ipset del yoda x.x.x.x # 从 yoda 集合中删除内容
ipset list yoda # 查看 yoda 集合内容
ipset list # 查看所有集合的内容
ipset flush yoda # 清空 yoda 集合
ipset flush # 清空所有集合
ipset destroy yoda # 销毁 yoda 集合
ipset destroy # 销毁所有集合
ipset save yoda # 输出 yoda 集合内容到标准输出
ipset save # 输出所有集合内容到标准输出
ipset restore # 根据输入内容恢复集合内容
还有……

如果创建集合是指定的存储内容包含 ip, 例如 hash:ip 或 hash:ip,port ,在添加条目时,可以填 IP 段,但是仍然是以单独一个个 IP 的方式来存。上面所有的例子都是用 hash 的方式进行存储,实际上 ipset 还可以以 bitmap 或者 link 方式存储,用这两种方式创建的集合大小,是固定的。

参考:https://www.cnblogs.com/xiaofeng666/p/10952627.html

————————

ipset介绍

Iptables is a user space tool for configuring firewall rules in the Linux kernel. It is actually a part of the Netfilter framework. Perhaps because iptables is the most common part of the Netfilter framework, this framework is usually called iptables. Iptables is a firewall solution introduced by Linux from version 2.4. Ipset is an extension of iptables, which allows you to create rules that match the entire address sets. Unlike the ordinary iptables chain, which is linear storage and filtering, IP sets are stored in an indexed data structure, which can search efficiently even if the set is relatively large. In addition to some common situations, such as preventing some dangerous hosts from accessing the machine, so as to reduce system resource occupation or network congestion, ipset also has some new firewall design methods and simplify the configuration. Official website: http://ipset.netfilter.org/

1. < strong > ipset installation (ipset is installed by default in CentOS 7. X, and no further installation is required) < / strong >

yum install ipset

2. < strong > create an ipset < / strong >

ipset create xxx hash:ip (hash:ip指的是单个ip,xxx是ipset名称)

Ipset can store 65536 elements by default, and use maxelem to specify the number

ipset create blacklist hash:ip maxelem 1000000    #黑名单
ipset create whitelist hash:ip maxelem 1000000    #白名单

View the ipsets created by the current server

ipset list

3. < strong > join a list IP < / strong >

ipset add blacklist 10.20.30.12

4. < strong > remove list IP < / strong >

ipset del blacklist 10.20.30.12

5. < strong > create firewall rules < / strong >

iptables -I INPUT -m set --match-set blacklist src -p tcp -j DROP
iptables -I INPUT -m set --match-set whitelist src -p tcp -j DROP
service iptables save

6. < strong > save ipset rules to the file < / strong >

ipset save blacklist -f blacklist.txt
ipset save whitelist -f whitelist.txt

7.删除ipset

ipset destroy blacklist
ipset destroy whitelist

8. < strong > Import ipset rules < / strong >

ipset restore -f blacklist.txt
ipset restore -f whitelist.txt

< strong > an advantage of ipset is that the set can be dynamically modified. Even if the iptables rule of ipset has been started, the newly added IP into ipset will also take effect

ipset使用:使用 ipset 封大量ip

Using iptables to seal IP in Linux is a common method to deal with network attacks, but thousands of IP should be blocked. If thousands of rules are added, it will have a great impact on the machine performance. Using ipset can solve this problem. Iptables contains several tables, each consisting of a chain. The default is the filter table. The most commonly used table is the filter table. The other commonly used table is the NAT table. Sealing IP is to add rules to the input chain of the filter table. During rule matching, the rules are matched one by one from the beginning to the end in the rule list. This is like searching for a specified node in a linked list< Strong > ipset provides a method to change this o (n) operation into o (1): put the IP to be processed into a set and set an iptables rule for this set. Like IPtable, IP sets is something in the Linux kernel, and the command ipset is a tool to operate it

These commands can be used to summarize the process of IP blocking using ipset and iptables

ipset create vader hash:ip
iptables -I INPUT -m set --match-set vader src -j DROP
ipset add vader 4.5.6.7
ipset add vader 1.2.3.4
ipset add vader ...
ipset list vader # 查看 vader 集合的内容

Each command is described below< Strong > 1. Create a collection < / strong >

ipset create vader hash:ip

This command creates a collection called Vader and stores it in hash mode. The stored content is IP address.

< strong > 2. Add iptables rules < / strong >

iptables -I INPUT -m set --match-set vader src -j DROP

If the source address (SRC) belongs to the Vader set, the drop operation is performed. In this command, Vader is used as a blacklist. If you want to use a collection as a whitelist, add a ‘!’ Just the symbol.

iptables -I INPUT -m set ! --match-set yoda src -j DROP

Up to now, although the collection has been created and filtering rules have been added, the collection is still empty and content needs to be added to the collection.

< strong > 3. Find out the “bad” IP < / strong > find out the IP to be blocked. This is an important step in the blocking process, but it is not the focus here. Briefly explain the two methods and ideas.

netstat -ntu | tail -n +3 | awk '{print $5}' | sort | uniq -c | sort -nr

Directly count and sort out the IP addresses in various local states through the information of netstat.

Or find the IP with too many requests from the logs of nginx or other web servers

awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -nr

In the second half, the operations of sorting, de duplication, and reverse sorting according to the number of times are the same as the above commands.

Find out the “bad” IP and add it to the previously created set.

ipset add vader 4.5.6.7

You can add as many “bad” IPS as there are. Because only one iptables rule is required for these blocked IPS, and these IPS are stored in hash mode, blocking a large number of IPS will not affect performance, which is also the biggest purpose of ipset.

More usage of ipset

< strong > 1. Storage type < / strong > the Vader set in the previous example stores IP addresses in hash mode, that is, the IP address is the hash key. In addition to the IP address, it can also be a network segment, port number (supporting the specified TCP / UDP protocol), MAC address, network interface name, or a combination of the above types.

For example, if you specify hash: IP, port means that the IP address and port number are used as the key of hash. Check the help documentation for ipset to see all the types it supports.

Two examples are given below.

HASH:NET
ipset create r2d2 hash:net
ipset add r2d2 1.2.3.0/24
ipset add r2d2 1.2.3.0/30 nomatch
ipset add r2d2 6.7.8.9
ipset test r2d2 1.2.3.2
hash:net 指定了可以往 r2d2 这个集合里添加 IP 段或 IP 地址。

The function of nomatch in the third command is simply to “peel” 1.2.3.0/30 from the relatively larger range of 1.2.3.0/24, that is, after executing ipset add R2D2 1.2.3.0/24, the IP of 1.2.3.0/24 belongs to R2D2 set, and after executing ipset add R2D2 1.2.3.0/30 nomatch, the IP of 1.2.3.0/30 in 1.2.3.0/24 belongs to R2D2 set, It doesn’t belong to the R2D2 set. Execute ipset test R2D2 1.2.3.2 to get the result 1.2.3.2 is not in set R2D2

HASH:IP,PORT
ipset create c-3po hash:ip,port
ipset add c-3po 3.4.5.6,80
ipset add c-3po 5.6.7.8,udp:53
ipset add c-3po 1.2.3.4,80-86

The second command adds the item with IP address 3.4.5.6 and port number 80. If the protocol is not specified, the default is TCP. The following command indicates the 53 port of UDP. The last command indicates an IP address and a port number range, which is also a legal command.

Automatic expiration and unsealing ipset supports the timeout parameter, which means that if a collection is used as a blacklist, the content can be automatically deleted from the blacklist through the timeout parameter.

ipset create obiwan hash:ip timeout 300
ipset add obiwan 1.2.3.4
ipset add obiwan 6.6.6.6 timeout 60

The first command above creates a collection named Obiwan, followed by a timeout parameter with a value of 300. The default timeout time for adding entries to the collection is 300. The third command specifies a timeout value 60 different from the default value when adding IP to the collection, and this one will be deleted automatically after 60 seconds.

Execute the ipset list Obiwan every few seconds. You can see that the timeout of the entries in the collection has been changing with time, indicating how many seconds they will be deleted.

If you want to re specify the timeout parameter for an entry, use the – exit option.

ipset -exist add obiwan 1.2.3.4 timeout 100

In this way, the timeout value of 1.2.3.4 becomes 100. If 300 is set here, its timeout, that is, the survival time, becomes 300 again.

If you do not specify timeout when creating a collection, then adding entries later does not support the timeout parameter, and you will receive an error when you execute add. If you want the default entries not to expire (automatically deleted), and you need to add the timeout parameter when adding some entries, you can specify the timeout as 0 when creating the collection.

ipset create luke hash:ip
ipset add luke 5.5.5.5 timeout 100

得到报错信息 kernel error received: Unknown error -1

The hashsize and maxelem parameters specify the initial hash size and the maximum number of stored entries when creating a collection.

ipset create yoda hash:ip,port hashsize 4096 maxelem 1000000
ipset add yoda 3.4.5.6,3306

In this way, a set named Yoda is created. The initial hash size is 4096. If it is full, the hash will be automatically expanded to twice the previous size. The maximum number that can be stored is 100000.

If not specified, the default value of hashsize is 1024 and the default value of maxelem is 65536.

< strong > 2. Several other common commands < / strong >

ipset del yoda x.x.x.x # 从 yoda 集合中删除内容
ipset list yoda # 查看 yoda 集合内容
ipset list # 查看所有集合的内容
ipset flush yoda # 清空 yoda 集合
ipset flush # 清空所有集合
ipset destroy yoda # 销毁 yoda 集合
ipset destroy # 销毁所有集合
ipset save yoda # 输出 yoda 集合内容到标准输出
ipset save # 输出所有集合内容到标准输出
ipset restore # 根据输入内容恢复集合内容
还有……

If the created set is specified and the storage content includes IP, such as hash: IP or hash: IP, port, you can fill in the IP segment when adding entries, but it is still saved as a single IP. All the above examples are stored in hash mode. In fact, ipset can also be stored in bitmap or link mode. The size of the set created in these two ways is fixed.

参考:https://www.cnblogs.com/xiaofeng666/p/10952627.html