Guillaume Maury's Ramblings

When running httperf if you get this error:

httperf --timeout=5 --client=0/1 --server=domU-12-31-39-00-A9-F7 --port=80 --uri=/articles.html --rate=500 --send-buffer=4096 --recv-buffer=16384 --num-conns=20000 --num-calls=10
httperf: warning: open file limit > FD_SETSIZE; limiting max. # of open files to FD_SETSIZE
Maximum connect burst length: 50

Total: connections 9789 requests 96039 replies 95959 test-duration 44.312 s

Connection rate: 220.9 conn/s (4.5 ms/conn, <=1022 concurrent connections)
Connection time [ms]: min 6.5 avg 4139.6 max 13432.2 median 3770.5 stddev 1697.6
Connection time [ms]: connect 495.6
Connection length [replies/conn]: 9.998

Request rate: 2167.4 req/s (0.5 ms/req)
Request size [B]: 88.0

Reply rate [replies/s]: min 2054.9 avg 2250.7 max 2559.2 stddev 155.9 (8 samples)
Reply time [ms]: response 242.4 transfer 123.3
Reply size [B]: header 242.0 content 17073.0 footer 0.0 (total 17315.0)
Reply status: 1xx=0 2xx=95959 3xx=0 4xx=0 5xx=0

CPU time [s]: user 1.42 system 17.77 (user 3.2% system 40.1% total 43.3%)
Net I/O: 36803.9 KB/s (301.5*10^6 bps)

Errors: total 10405 client-timo 194 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 10211 addrunavail 0 ftab-full 0 other 0

fd-unavail means that there was the per-process on the limit of open files has been exceded. httperf uses select() and uses a new file descriptor for each connection it opens concurrently. So let’s increase the limit on file descriptors.

• Edit /etc/security/limits.conf and add the line * hard nofile 65535 (or instead of * you can put the username of the user for whom you want to change the limit)
• Edit /usr/include/bits/typesizes.h and change #define __FD_SET_SIZE 1024 to #define __FD_SET_SIZE 65535 (in /usr/include/sys/select.h FD_SETSIZE is defined as __FD_SETSIZE)
• Download and recompile httperf

  wget ftp://ftp.hpl.hp.com/pub/httperf/httperf-0.9.0.tar.gz
  tar xvzf httperf-0.9.0.tar.gz
  cd httperf-0.9.0
  ./configure && make
  sudo make install

I’ve tested those steps on ubuntu

NB: This is one of the reason you should make sure that nginx is using epoll (linux) or kqueue (bsd) instead of select (this is determined at ./configure time but you can force it by using a use epoll in an events block).

I’m currently available for hire on a contract basis.