diff options
author | Ben Pfaff <blp@nicira.com> | 2012-03-15 21:15:38 -0700 |
---|---|---|
committer | Ben Pfaff <blp@nicira.com> | 2012-03-15 21:46:18 -0700 |
commit | d4c4c4c5943feb46aad7f5dad5bcf573933a62a2 (patch) | |
tree | ea2e048a045e28eeafebd1e01868e54cee26436c /lib | |
parent | bcbf843a4b2005c34d15f8cb1625028cef4f501d (diff) |
netlink-socket: Increase Netlink socket receive buffer size.
Open vSwitch userspace can set up flows at a high rate, but it is somewhat
"bursty" in opportunities to set up flows, by which I mean that OVS sets up
a batch of flows, then goes off and does some other work for a while, then
sets up another batch of flows, and so on. The result is that, if a large
number of packets that need flow setups come in all at once, then some of
them can overflow the relatively small kernel-to-user buffers.
This commit increases the kernel-to-user buffers from the default of
approximately 120 kB each to 1 MB each. In one somewhat synthetic test
case that I ran based on an "hping3" that generated a load of about 20,000
new flows per second (including both requests and replies), this reduced
the packets dropped at the kernel-to-user interface from about 30% to none.
I expect that it will similarly improve packet loss in workloads where
flow arrival is not easily predictable.
(This has little effect on workloads generated by "ovs-benchmark rate"
because that benchmark is effectively "self-clocking", that is, a new flow
is triggered only by a reply to a request made earlier, which means that
the number of buffered packets at any given has a known, constant upper
limit.)
Bug #10210.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Diffstat (limited to 'lib')
-rw-r--r-- | lib/netlink-socket.c | 10 |
1 files changed, 9 insertions, 1 deletions
diff --git a/lib/netlink-socket.c b/lib/netlink-socket.c index bc462353..df6f1d8b 100644 --- a/lib/netlink-socket.c +++ b/lib/netlink-socket.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008, 2009, 2010, 2011 Nicira Networks. + * Copyright (c) 2008, 2009, 2010, 2011, 2012 Nicira Networks. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -89,6 +89,7 @@ nl_sock_create(int protocol, struct nl_sock **sockp) struct nl_sock *sock; struct sockaddr_nl local, remote; socklen_t local_size; + int rcvbuf; int retval = 0; if (!max_iovs) { @@ -122,6 +123,13 @@ nl_sock_create(int protocol, struct nl_sock **sockp) sock->protocol = protocol; sock->dump = NULL; + rcvbuf = 1024 * 1024; + if (setsockopt(sock->fd, SOL_SOCKET, SO_RCVBUFFORCE, + &rcvbuf, sizeof rcvbuf)) { + VLOG_WARN_RL(&rl, "setting %d-byte socket receive buffer failed (%s)", + rcvbuf, strerror(errno)); + } + retval = get_socket_rcvbuf(sock->fd); if (retval < 0) { retval = -retval; |