Tengine & Nginx性能测试

发布于 2015-09-03 15:54:51 | 603 次阅读 | 评论: 0 | 来源: 网络整理

Tengine & Nginx性能测试

背景

我们在Tengine中实现了 SO_REUSEPORT [1]的支持。为了查看其效果，我们进行了一个简单的测试。我们在同一个局域网中部署了一共四台同等配置的服务器，其中一台同时部署Tengine和 Nginx，分别监听不同的端口，另外三台部署ab，三个ab同时压测，从总并发100逐步递增到1000，分别压测Tengine和Nginx，访问空 gif图片。

三种压测场景：

1. Tengine打开SO_REUSEPORT，reuse_port on。
2. Nginx默认配置。
3. Nginx优化配置，关闭mutex锁，accept_mutex off。

ab压测命令：

  ab -r -n 10000000 -c 100 http://ip:81/empty.gif

测试环境

  Intel(R)Xeon(R)E5-2650v2@2.60GHz 32core
  Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection
  Red Hat Enterprise Linux Server release 5.7 (Tikanga)
  linux-3.17.2.x86_64
  128GB Memroy

软件

  Nginx/1.6.2
  Tengine/2.1.0 (http://tengine.taobao.org)
  ApacheBench/2.3

系统配置

  net.ipv4.tcp_mem = 3097431 4129911 6194862
  net.ipv4.tcp_rmem = 4096 87380 6291456
  net.ipv4.tcp_wmem = 4096 65536 4194304
  net.ipv4.tcp_max_tw_buckets = 262144
  net.ipv4.tcp_tw_recycle = 0
  net.ipv4.tcp_tw_reuse  = 1
  net.ipv4.tcp_syncookies  = 1
  net.ipv4.tcp_fin_timeout = 15
  net.ipv4.ip_local_port_range = 1024 65535
  net.ipv4.tcp_max_syn_backlog = 65535
  net.core.somaxconn  = 65535
  net.core.netdev_max_backlog  = 200000

  Limit                 Soft Limit          Hard Limit            Units
  Max open  files         65535               65535               files

服务器配置

Nginx/1.6.2 配置文件：

  worker_processes auto;
  worker_cpu_affinity
  00000000000000000000000000000001
  00000000000000000000000000000010
  00000000000000000000000000000100
  00000000000000000000000000001000
  00000000000000000000000000010000
  00000000000000000000000000100000
  00000000000000000000000001000000
  00000000000000000000000010000000
  00000000000000000000000100000000
  00000000000000000000001000000000
  00000000000000000000010000000000
  00000000000000000000100000000000
  00000000000000000001000000000000
  00000000000000000010000000000000
  00000000000000000100000000000000
  00000000000000001000000000000000
  00000000000000010000000000000000
  00000000000000100000000000000000
  00000000000001000000000000000000
  00000000000010000000000000000000
  00000000000100000000000000000000
  00000000001000000000000000000000
  00000000010000000000000000000000
  00000000100000000000000000000000
  00000001000000000000000000000000
  00000010000000000000000000000000
  00000100000000000000000000000000
  00001000000000000000000000000000
  00010000000000000000000000000000
  00100000000000000000000000000000
  01000000000000000000000000000000
  10000000000000000000000000000000
  ;
  worker_rlimit_nofile 65535;

  events {
    worker_connections 65535;
    accept_mutex off;
  }

  http {
    include mime.types;
    default_type application/octet-stream;
    access_log logs/access.log;
    keepalive_timeout 0;
    server {
      listen 82 backlog=65535;
      server_name localhost;
      location = /empty.gif {
        empty_gif;
      }
    }
  }

Tengine/2.1.0配置文件：

  worker_processes auto;
  worker_cpu_affinity auto;
  worker_rlimit_nofile 65535;

  events {
    worker_connections 65535;
    reuse_port on;
  }

  http {
    include mime.types;
    default_type application/octet-stream;
    access_log logs/access.log;
    keepalive_timeout 0;
    server {
      listen 81 backlog=65535;
      server_name localhost;
      location = /empty.gif {
        empty_gif;
      }
    }
  }

Tengine和Nginx只有reuse_port和accept_mutex两处配置不同。
Tengine的worker_cpu_affinity等同于Nginx的相应配置。

测试结果

1. Tengine相比Nginx默认配置，提升200%的处理能力。
2. Tengine相比Nginx优化配置，提升60%的处理能力。

[1] SO_REUSEPORT套接字属性：https://lwn.net/Articles/542629/