Extracting UDP packet payload with eBPF tracepoints using libbpf

Question

I am trying to retrieve the payload of UDP packets using tracepoint netif_receive_skb at the receiver. The goal is to get the packet payload and send it to userspace for further processing.

Method 1: I am navigating to the payload with the following code but, when I print and check the payload, some part of the data at the end is missing.

unsigned char *payload = (unsigned char *)(udph + 1);
bpf_printk("%s", payload);

Method 2: Get the payload data in the custom structure and send it out using event. But I receive the error R8 invalid mem access 'inv'. Here is the code snippet:

If data is non-linear, then skb->data_len is a non-zero value page-19. In my case, skb->data_len is zero which means it is linear and so, in principle, Method 1 should provide the full value. Also, the values of skb->end and skb->tail are less than the values of data and head which I don't understand why (an explanation would be helpful for this case). Thus, I am using skb->data and skb->len to calculate the data_end value.

struct custom_data
{
    __u32 val1;
    __u32 val2;
    __u32 val3;
    __u32 val4;
};


SEC("tp/net/netif_receive_skb")
int net_netif_receive_skb(struct trace_event_raw_consume_skb *args) 
{
    struct sk_buff *skb = (struct sk_buff *)BPF_CORE_READ(args, skbaddr);

    // packet parsing steps

    

    unsigned char *data_end = BPF_CORE_READ(skb, data) + BPF_CORE_READ(skb, len);

    if ((unsigned char *)(udph + 1) > data_end)
        return -1;
    
    struct custom_data *custom_data= (struct custom_data *)(udph + 1);
    
    if ((unsigned char *)(custom_data + 1) > data_end)
        return -1;

    __u32 val1 = custom_data ->val1;
    bpf_printk("port: %d", bpf_htons(port2_real_dst));
    // when the bpf_printk() line is commented out, the code is loaded fine!

Error from the verifier

; unsigned char *data_end = BPF_CORE_READ(skb, data) + BPF_CORE_READ(skb, len);
189: (61) r1 = *(u32 *)(r10 -32)
; unsigned char *data_end = BPF_CORE_READ(skb, data) + BPF_CORE_READ(skb, len);
190: (0f) r8 += r1
; if ((unsigned char *)(udph + 1) > data_end)
191: (3d) if r8 >= r7 goto pc+124

from 191 to 316: frame1: R0_w=invP(id=0) R1_w=invP(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R6_w=invP(id=22) R7=invP(id=21) R8_w=invP(id=0) R9=invP61 R10=fp0 fp-32=mmmmmmmm fp-40=00000000 fp-48=00000000 fp-56=000mmmmm fp-64=mmmmmmmm fp-72=00000000 fp-80=mmmmmmmm fp-88=invP61
; if ((unsigned char *)(telemetry_data + 1) > data_end)
316: (bf) r1 = r7
317: (07) r1 += 28
; if ((unsigned char *)(telemetry_data + 1) > data_end)
318: (3d) if r8 >= r1 goto pc+71

from 318 to 390: frame1: R0=invP(id=0) R1=invP(id=0) R6=invP(id=22) R7=invP(id=21) R8=invP(id=0) R9=invP61 R10=fp0 fp-32=mmmmmmmm fp-40=00000000 fp-48=00000000 fp-56=000mmmmm fp-64=mmmmmmmm fp-72=00000000 fp-80=mmmmmmmm fp-88=invP61
; __u16 port2_real_dst  = telemetry_data->port2_real_dst;
390: (71) r1 = *(u8 *)(r7 +26)
R7 invalid mem access 'inv'

Method 3: Read n bytes from the kernel memory using bpf_probe_read_kernel(data, sizeof(*custom_data), udph + 1); retrieves only part of the data with garbage values.

Am I missing any obvious steps? Any other methods using libbpf library to read udp packet payloads.

I am using kernel version 4.19

Answer 1

The UDP payload can be extracted successfully using bpf_probe_read_kernel(). At the receiver, navigate to the payload address (e.g. - skb_head + l4_offset + sizeof(udp header)) and then read the required number of bytes with the above function. Further processing can be done in the userspace.

Method 1 On sending the unsigned char *payload to userspace, I could see the complete payload as required. Seems bpf_printk() was not printing the whole data.

Method 2 Even on performing bound checks, the verifier complained of invalid mem access

Method 3 The payload offset was not calculated correctly previously. As mentioned above, skb_head + l4_offset + sizeof(udp header) is the way that worked, where:

void *skb_head = BPF_CORE_READ(skb, head);
__u16 l3_offset = BPF_CORE_READ(skb, network_header);
__u16 l4_offset = l3_offset + BPF_CORE_READ_BITFIELD_PROBED(iph, ihl) * 4;
__u16 payload_offset = skb_head + l4_offset + sizeof(struct udphdr);