Issue with XDP_REDIRECT in a namespace environment

185 views Asked by At

I am trying to redirect a packet from one interface to another. I have set up three namespaces named red, blue, and router. Red and blue can communicate with each other only through the router namespace. The router interfaces are veth2 to red and veth3 to blue.

| RED |---veth0----veth2---| ROUTER |---veth3----veth1---| BLUE |

My purpose is to send pings from red to blue through router where my program is attached and I need to redirect because the main purpose would be to implement a NAT64 router.

The nodes currently communicate with each other, the red node can ping the blue node and vice versa, the problem is when my xdp program is attached and needs to redirect to the veth3 interface.

As xpd program, other than my implementation, I tested with BCC examples in skb mode as my NIC does not support native mode, attaching the program to veth2 and a dummy function to veth3. But it didn't work.

To solve the problem I tried using xdpdump since tcpdump doesn't capture anything. xdpdump on veth2 captures packets correctly and on output shows the packet with XDP_REDIRECT verdict. On veth3 it does not receive anything.

I tried other redirect methods, such as XDP_TX and XDP_REDIRECT from veth2 to veth2 and they worked

I tried using other methods, such as bpf_redirect() and map.redirect_map() with the appropriate values but veth3 still receives nothing. Also I noticed that even when inserting non-existing ifindex I do not receive XDP_ABORTED but always XDP_REDIRECT.

Does anyone have any idea what might be causing this problem or how I could solve it?

Thank you in advance for your help.

EDIT1: forwarding is enabled in all namespaces. And xdp-monitor show this:

    Summary                         1 redir/s               0 err,drop/s            0 xmit/s       
  kthread                       0 pkt/s                 0 drop/s                0 sched        
  redirect total                1 redir/s      
    cpu:3                       1 redir/s      
  redirect_err                  0 error/s      
  xdp_exception                 0 hit/s        
  devmap_xmit                   0 xmit/s                0 drop/s                0 drv_err/s          0.00 bulk-avg 

EDIT2: this is my code, I omitted some parts for readability:

...
flags = BPF.XDP_FLAGS_SKB_MODE
...
b = BPF(
    text="""
#include <uapi/linux/bpf.h>
#include <linux/in.h>
#include <linux/if_ether.h>
#include <linux/ip.h>

BPF_DEVMAP(tx_port, 1);
BPF_PERCPU_ARRAY(rxcnt, long, 1);

static inline void swap_src_dst_mac(void *data)
{
    unsigned short *p = data;
    unsigned short dst[3];

    dst[0] = p[0];
    dst[1] = p[1];
    dst[2] = p[2];
    p[0] = p[3];
    p[1] = p[4];
    p[2] = p[5];
    p[3] = dst[0];
    p[4] = dst[1];
    p[5] = dst[2];
}

int xdp_redirect_map(struct xdp_md *ctx) {
    
    // omitted code, simple packet access
    
    
    struct iphdr dst_hdr = {
        .version = 4,
        .ihl = 5,
        .frag_off = bpf_htons(1<<14),
    };
    
    // blue ip address
    dst_hdr.daddr = bpf_htonl(0xc0a80301);
    
    // red ip address
    dst_hdr.saddr = bpf_htonl(0xc0a80201);
    dst_hdr.protocol = iph->protocol;
    dst_hdr.ttl = iph->ttl;
    dst_hdr.tos = iph->tos;
    dst_hdr.tot_len = iph->tot_len;

    struct iphdr *new_iph;

    new_iph = data + sizeof(*eth);
    if (new_iph + 1 > data_end)
        return XDP_DROP;

    *new_iph = dst_hdr; 
    
    struct bpf_fib_lookup fib_params = {0};
    fib_params.family = AF_INET;
    fib_params.tos = new_iph->tos;
    fib_params.l4_protocol = new_iph->protocol;
    fib_params.sport = 0;
    fib_params.dport = 0;
    fib_params.tot_len = new_iph->tot_len;
    fib_params.ipv4_src = new_iph->saddr;
    fib_params.ipv4_dst = new_iph->daddr;

    fib_params.ifindex = ctx->ingress_ifindex;

    
    rc = bpf_fib_lookup(ctx, &fib_params, sizeof(fib_params), 0);
    switch (rc)
        {
        case BPF_FIB_LKUP_RET_SUCCESS: /* lookup successful */
            memcpy(eth->h_dest, fib_params.dmac, ETH_ALEN);
            memcpy(eth->h_source, fib_params.smac, ETH_ALEN);
            int action = tx_port.redirect_map(0, 0);
            return action;
        ...
        // other cases
        ...
        }
    return XDP_PASS;
}

int xdp_dummy(struct xdp_md *ctx) {
    bpf_trace_printk("xdp_dummy\\n");
    return XDP_PASS;
}
""",
    cflags=["-w"],
)

tx_port = b.get_table("tx_port")
tx_port[0] = ct.c_int(out_idx)

in_fn = b.load_func("xdp_redirect_map", BPF.XDP)
out_fn = b.load_func("xdp_dummy", BPF.XDP)

b.attach_xdp(in_if, in_fn, flags)
b.attach_xdp(out_if, out_fn, flags)

...
0

There are 0 answers