UDP and packets larger than 1500 bytes

pabloski · 2011-01-02 20:57:30

Hi, I'm developing a tftp client and server and I'm trying to dynamically select the size of the datagrams in order to maximize throughput eliminating ip fragmentation.

Practically I have two machines ( one is a desktop with a gigabit ethernet which MTU is setted to 2048 bytes, the other is a laptop with a fast ethernet and MTU of 1500 ). The client installed on the desktop sends udp packets of 2020 bytes and I have setted IP_MTU_DISCOVER to IP_PMTUDISC_DO. This way ( at least the linux manual says so ) I should be able to read the minimum MTU of the devices which are communicating ( the desktop, the laptop and the router).

The manual says that the first host which MTU is lesser than the datagram size will send an ICMP error reply with its MTU value. It seems to work partially and I can read the MTU of the desktop ethernet card throught getsockopt, but It reads only the desktop ethernet MTU and don't do the MTU path discovery.

Also the recipients of the datagrams simply drop them. I'm running wireshark and I simply don't see udp datagrams arriving. Dropping datagrams is good in this case, because 2020 is greater than 1500, but the recipient should send an error message to the sender and this is not the case.

Someone has some advice?

hexanol · 2011-01-02 23:03:44

Because of the lock-step design of TFTP, I don't think you'll achieve any increase in throughput by trying to eliminate IP fragmentation. I believe you might even see a decrease in throughput depending on the MTU/blocksize parameters.

You should test the throughput with an already existing TFTP client/server that support the TFTP blocksize option (RFC2348), like atftp, and look at the throughput achieved with difference blocksize on different MTU.

And anyway, if you want to write a RFC compliant server/client, you'll notice that it's the client who ask for a certain blocksize, and the server can only respond with a blocksize equal or lower to this value.

pabloski · 2011-01-02 23:28:20

Thanks for the reply, however I think I should clarify the current situation. First, I don't have a tftp client or server, I'm stuck at this code

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <string.h>

                            // la dimensione del datagramma è data da MTU - ( ip header + udp header ) e cioè 1500 - ( 20 + 8 ) = 1472
#define DGRAM_SIZE 2020

int main()
{
    int sockfd = socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP);
    if ( sockfd == -1 ) { perror("Socket"); exit(0); }

    int optval = IP_PMTUDISC_DO;
    if (setsockopt(sockfd, IPPROTO_IP, IP_MTU_DISCOVER, &optval, sizeof(optval)) == -1) { perror("MTU Discover"); exit(0); }

    void *buf = malloc(DGRAM_SIZE);
    memset(buf, 0x1f, DGRAM_SIZE);

    struct sockaddr_in svraddr;

    svraddr.sin_family = PF_INET;
    svraddr.sin_port = htons(69);
    svraddr.sin_addr.s_addr = inet_addr("192.168.1.254");

    if ( connect(sockfd, (struct sockaddr*)&svraddr, sizeof(svraddr)) == -1 ) { perror("connect"); exit(0); }

    int dgram_size, dg_len = sizeof(dgram_size);
/*
    if ( getsockopt(sockfd, IPPROTO_IP, IP_MTU, &dgram_size, (socklen_t*)&dg_len) == -1 ) {
            perror("getsockopt"); exit(0);
    }

    printf("Dimensione max fragment ( prima dell'invio ): %d\n", dgram_size);
*/
    if (sendto(sockfd, buf, DGRAM_SIZE, 0, (struct sockaddr*)&svraddr, sizeof(svraddr)) == -1 ) {
        dgram_size = 0;

        if ( getsockopt(sockfd, IPPROTO_IP, IP_MTU, &dgram_size, (socklen_t*)&dg_len) == -1 ) {
            perror("getsockopt"); exit(0);
        }

        perror("Sendto");
        printf("Dimensione max fragment ( dopo l'invio dei dati ): %d\n", dgram_size);
    } else printf("Successo\n");


    return 0;
}

What this program do is opening a datagram socket, setting the DF bit and starts sending udp packets. That thing works like a charm for datagram sizes between 0 and 1472 bytes. It start not working at all for greater datagram sizes. It is correct behaviour for the recipients to drop the packets, but they should send back icmp errors. I simply don't receive the errors. I should clarify that I'm using to computers connected through an ethernet switch.

The problem doesn't lie in the tftp that I'm not even using for now.

pabloski · 2011-01-03 14:54:39

I'm experimenting a bit more with path MTU discovery and I'm using the tracepath utility. What I see is that as soon as I start tracepath the local host send me an icmp error to signal that the message is too long ( the MTU is set to 4096 ). However the signaling mechanism doesn't work at all for the router and/or other computers. I tried to connect 2 PCs through a router and directly through a cross cable, but nothing.

It seems that linux is configured to not send icmp replies to external hosts. Is it possible?

I add that I'm using archlinux on the sender host, without iptables policies and so on. Also I'm using fedora on the receiver machine but I have disables iptables rules and selinux, so they are not interfering.

cactus · 2011-01-03 21:33:45

If your frame is too large, the recipient will not be able to decode it, and will drop it (frame is just too large).
However, if you have a router, or a device with multiple NICs, and a frame is too large to forward, then the device will send back an icmp control message -- because it CAN read the large frame and knows that it cannot send it out the other link as it is. Most of the time it will simply refragment, unless the DF bit is set or the device is configured not to fragment packets.

Note that MTU discovery is for remote links (with links in between) and it not meant for connected collision domain networks.

pabloski · 2011-01-03 21:49:42

cactus wrote:

If your frame is too large, the recipient will not be able to decode it, and will drop it (frame is just too large).
However, if you have a router, or a device with multiple NICs, and a frame is too large to forward, then the device will send back an icmp control message -- because it CAN read the large frame and knows that it cannot send it out the other link as it is. Most of the time it will simply refragment, unless the DF bit is set or the device is configured not to fragment packets.
Note that MTU discovery is for remote links (with links in between) and it not meant for connected collision domain networks.

Great. I will buy a gigabit switch to do the trick.

But I have another question now. When I start my pc, linux sets the MTU to 1500. Is 1500 an hardcoded value for ethernet cards? Or linux is able to negotiate the MTU with the switch/router? For example, if I connect the pc to a router that supports jumbo frames, linux will set my card MTU to 1500 too?

I'm asking because I'm trying to figure out how two network devices ( when connected ) negotiate these parameters.

Last edited by pabloski (2011-01-03 21:55:37)

cactus · 2011-01-03 23:00:30

Great. I will buy a gigabit switch to do the trick.

Either you need it to be a layer3 switch, or it would be better to just put linux on a box with two nics and put it between your two test machines (basically make it a cheapo router/switch).

A standard layer2 switch will not do any layer3 assistery (is that a word?) like 'need to frag' or 'no host', etc.

But I have another question now. When I start my pc, linux sets the MTU to 1500. Is 1500 an hardcoded value for ethernet cards? Or linux is able to negotiate the MTU with the switch/router? For example, if I connect the pc to a router that supports jumbo frames, linux will set my card MTU to 1500 too?
I'm asking because I'm trying to figure out how two network devices ( when connected ) negotiate these parameters.

No. MTU is not auto-negotiated. Just link speed and duplex settings. Either dhcp pushes a mtu setting -- dhcp packets are usually pretty damn small (around 600 bytes??), so even with a default mtu of 1500 you can set it lower for something like adsl and be relatively ok -- or you configure it manually on your systems via a static ip config or a dhcp 'pre-up' scriptlet or something.

1500 might be a driver default..but, it is certainly the ifconfig helper util default for ethernet. I believe in the modprobe.conf config for the driver you can set the mtu (on some linuxen like redhat maybe), but for arch it is probably much easier to just set it in your arch network config.

eth0="eth0 192.168.100.2 netmask 255.255.255.0 broadcast 192.168.100.255 mtu 4000"

Last edited by cactus (2011-01-03 23:06:09)

pabloski · 2011-01-03 23:21:59

Thanks for the advice. I was oriented to buy the dlink dgs 1008d but it is a layer 2 switch. I will opt for a gigabit router.

Arch Linux

#1 2011-01-02 20:57:30

UDP and packets larger than 1500 bytes

#2 2011-01-02 23:03:44

Re: UDP and packets larger than 1500 bytes

#3 2011-01-02 23:28:20

Re: UDP and packets larger than 1500 bytes

#4 2011-01-03 14:54:39

Re: UDP and packets larger than 1500 bytes

#5 2011-01-03 21:33:45

Re: UDP and packets larger than 1500 bytes

#6 2011-01-03 21:49:42

Re: UDP and packets larger than 1500 bytes

#7 2011-01-03 23:00:30

Re: UDP and packets larger than 1500 bytes

#8 2011-01-03 23:21:59

Re: UDP and packets larger than 1500 bytes

Board footer