Tags

, ,

Problem

What will happen if clients of TCP/IP stream socket service send data faster than the other client receive and work with it. That is some sort of service has to receive incoming stream and store it until  that service call ‘recv’.

Solution:

TCP provides flow control . The TCP stack (both on the sender and receiver side) will be able to buffer some data, and this is usually done in the OS kernel.

When the receiver buffers fill up, the sender will know about it, and stop sending more data, eventually leading to the sending application blocking(or otherwise not being able to send more data) until space becomes available again.

Shortly described, every TCP packet(segment) sent includes the size of data that can be buffered – the window size. This means the other end at all times know how much data it can send without the receiver throwing it away because the buffers are full. If the window size becomes 0, buffers are full and no more data will be sent (and in case of the sender being blocking, a send() call will block), Theres procedures for probing whether the tcp window is still 0, so sending can resume again when the data has been consumed.

The default maximum Linux TCP buffer sizes are way too small. TCP memory is calculated automatically based on system memory; you can find the actual values by typing the following commands:
$ cat /proc/sys/net/ipv4/tcp_mem

The default and maximum amount for the receive socket memory:
$ cat /proc/sys/net/core/rmem_default
$ cat /proc/sys/net/core/rmem_max

The default and maximum amount for the send socket memory:
$ cat /proc/sys/net/core/wmem_default
$ cat /proc/sys/net/core/wmem_max
Tune values

Set the max OS send buffer size (wmem) and receive buffer size (rmem) to 12 MB for queues on all protocols. In other words set the amount of memory that is allocated for each TCP socket when it is opened or created while transferring files:
[Warning examples may crash your computer] WARNING! The default value of rmem_max and wmem_max is about 128 KB in most Linux distributions, which may be enough for a low-latency general purpose network environment or for apps such as DNS / Web server. However, if the latency is large, the default size might be too small. Please note that the following settings going to increase memory usage on your server.

# echo ‘net.core.wmem_max=12582912’ >> /etc/sysctl.conf
# echo ‘net.core.rmem_max=12582912’ >> /etc/sysctl.conf

You also need to set minimum size, initial size, and maximum size in bytes:
# echo ‘net.ipv4.tcp_rmem= 10240 87380 12582912’ >> /etc/sysctl.conf
# echo ‘net.ipv4.tcp_wmem= 10240 87380 12582912’ >> /etc/sysctl.conf

Now reload the changes:
# sysctl -p

Advertisements