How to read data from sk_buff using netfilter kernel module?

Question

We are writing a kernel module to add some additional data to the data packets. We add 120 bytes of data at the source in the data section of the skbuff and we are trying to extract that data from the skbuff at the destination.

Following is the kernel module code that we are running on both the machines.

#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/netfilter.h>
#include <linux/netfilter_ipv4.h>
#include <linux/slab.h>
#include <linux/inet.h>
#include <linux/time.h>
#include <linux/ktime.h>

static struct nf_hook_ops nf_out;
static struct nf_hook_ops nf_in;

struct my_struct {
// 24 byte struct
};

unsigned int outgoing_hook_func(unsigned int hooknum, struct sk_buff *skb, const struct              net_device *in, const struct net_device *out, int(*okfn)(struct sk_buff *))
{
int i = 0;
struct my_struct *ptr;
unsigned char *new_data;
int size = 5 * sizeof(struct my_struct);
ptr = kmalloc(size, GFP_ATOMIC);
for(i = 0; i < 5; i++) {
    //Assign some values to the array of mystruct
    }
}       
printk("output_Before: %d\n",skb->len);

new_data = skb_tail_pointer(skb);
SKB_LINEAR_ASSERT(skb);
// Add this additional data only if there is enough room in the data section of the skbuff
if (skb->tail + size < skb->end) {
    skb->tail += size;
        skb->len  += size;
        memcpy(new_data, ptr, size);
}

kfree(ptr);
printk("output_After: %d\n",skb->len);
return NF_ACCEPT;
}

unsigned int incoming_hook_func(unsigned int hooknum, struct sk_buff *skb, const struct net_device *in, const struct net_device *out, int(*okfn)(struct sk_buff *))
{
int i = 0;
struct loc_time_tag *ptr;
int size = 5 * sizeof(struct my_struct);
printk("input_Before: %d\n",skb->len);

// Collect data only from packets that have my_struct appended to them

if (skb->tail + size < skb->end) {
    unsigned int tail_ptr = skb_tail_pointer(skb);
    ptr = (struct my_struct *)(tail_ptr - 5 * sizeof(struct my_struct));
    printk("tail = %d; end =  %d; ptr = %d", skb->tail,skb->end, ptr);

    for(i = 0; i < 5; i++) {
    // Print out the values of ptr
    }

}

printk("input_after: %d\n",skb->len);
return NF_ACCEPT;
}

//Called when module loaded using 'insmod'
int init_module()
{
  nf_out.hook = outgoing_hook_func;               
  nf_out.hooknum = NF_INET_LOCAL_OUT;         
  nf_out.pf = PF_INET;
  nf_out.priority = NF_IP_PRI_FIRST;
  nf_register_hook(&nf_out);

  nf_in.hook = incoming_hook_func;
  nf_in.hooknum = NF_INET_LOCAL_IN;
  nf_in.pf = PF_INET;
  nf_in.priority = NF_IP_PRI_FIRST;
  nf_register_hook(&nf_in);
  return 0;             
}

//Called when module unloaded using 'rmmod'
void cleanup_module()
{
  nf_unregister_hook(&nf_out);                     //cleanup – unregister hook
  nf_unregister_hook(&nf_in);
}

But when we perform simple operation like SSH between source and destination and exmine the wireshark trace on both the machines, following are our observations.

1. If both the incoming_hook_func and outgoing_hook_func are registered and run on both source and destination machines, wireshark shows "ETHERNET FRAME CHECKSUM SEQUENCE INCORRECT" error for packets going out of the source machine. No packet is being seen on the destination machine. I am not sure if the packet didn't leave the source machine or if the destination is unable to read the packet. After a while both source and destination crash. (We assume this is due to some bug in the incoming_hook_func() code)

2. If only the outgoing_hook_func is registered on the source machine and nothing on the destination, same "ETHERNET FRAME CHECKSUM SEQUENCE INCORRECT" is seen on the source trace, this packet is seen as received on the destination, but the destination doesn't reply to that packet. No system crashes are seen in this case.

We would be very grateful if someone can answer the following questions with the given information.

1. Where exactly is Ethernet Frame Check Sequence calculated? Is it done in the kernel or is it done by wireshark? What should we do to overcome the error shown in wireshark. Should we recalculate any checksum after adding our data?

2. Our outgoing_hook_func doesn't crash the system, so we are assuming that our pointer manipulations in that routine are fine. But the incoming_hook_func is leading to system crashes. Can someone please let us know where we are going wrong?

3. In the case 2 above, why is the destination not responding? Is it due to bad frame check sequence or is there any other flaw in our understanding of things.

4. Also, ping tests seem to be working perfectly fine in both the above cases. We are unable to comprehend why that is behaving so.

Thanks in advance

Answer 1

unsigned int tail_ptr = skb_tail_pointer(skb);
ptr = (struct my_struct *)(tail_ptr - 5 * sizeof(struct my_struct));

Replace the above code with the following.

unsigned char* tail_ptr = skb_tail_pointer(skb);
ptr = (struct my_struct *)(tail_ptr - 5 * sizeof(struct my_struct));

This would be right way to move back 5 * 24 bytes from the tail pointer. This might explain the reason for your crashes.