How to offload NAPI poll function to workqueue

Question

I'm working with linux 3.3, Ethernet driver for smsc911x. and I want to move the NAPI poll function to workqueue.

My questions are :
1. How do I pass the NAPI poll function arguments to the work_struct?
2. How do I get the NAPI poll function arguments back from the work_struct? (related to Q.1 above)
3. How can I return the npackets value to the original NAPI poll function caller?

Here are some explanations :

Current NAPI poll function reads recevie FIFO directly which I want to change to do it with DMA controller. For this DMA, I trigger DMA, sleep with wait_event_interruptible, and get woken up by DMA's ISR with wake_up_interruptible. As you know, NAPI poll function is in interrupt context (softirq) so I cannot sleep there for DMA completion. I want to move the NAPI poll function(reading RX FIFO) to waitqueue(process context) usnig a work_struct.

The problem is, NAPI poll function is called by the kernel with two arguments : struct napi_struct *napi and int budget. I want to pass those argument to the work_struct and queue the work_struct to the workqueue (using queue_work function).

the work_struct looks like below. (include/linux/workqueue.h)

struct work_struct { 
        atomic_long_t data;
        struct list_head entry; 
        work_func_t func;
#ifdef CONFIG_LOCKDEP
        struct lockdep_map lockdep_map;
#endif
};

I take that atomic_long_t data is for passing the argument to the work_struct. how can I pass the arguments to the work_struct? I tried this (I added in the structure for device driver struct smsc911x_data a member struct work_struct rx_work for passing the work.) :

struct work_arg { // a new struct for pass the arguments
        struct napi_struct *napi;
        int budget;
};

/* NAPI poll function */
static int smsc911x_poll(struct napi_struct *napi, int budget) {
        struct smsc911x_data *pdata =
                container_of(napi, struct smsc911x_data, napi);
        struct net_device *dev = pdata->dev;
        int npackets = 0;
if (enable_rx_use_dma == 1) {  // when using DMA for FIFO read
        prom_printf("moving it to workqueue\n");
        struct work_arg *p;
        p = kzalloc(sizeof(struct work_arg), GFP_KERNEL);
        p->napi = napi;
        p->budget = budget;
        pdata->rx_work.data = (atomic_long_t) p; // <== THIS LINE
        prom_printf("queue work, with napi = %x, budget = %d\n", napi, budget);
        queue_work(rx_work_workqueue, &pdata->rx_work); // smsc911x_poll_work } else {
        -- original NAP poll function, reads FIFO until it's empty and enables the RX interrupt and 
        -- keeps the number of processed packets to npackets.
        return npackets;
}

For "THIS LINE" above, I'm getting error during compile.

with pdata->rx_work.data = p; , I get error: incompatible types when assigning to type 'atomic_long_t' from type 'struct work_arg *'
with pdata->rx_work.data = (atomic_long_t) p; , I get error: conversion to non-scalar type requested.

Also, in the new work function, How can I extract the original argments? I tried this below which gives me errors.

/* New work function called by the default worker thread */ static int smsc911x_poll_work(struct work_struct *work) {
        struct smsc911x_data *pdata =
                container_of(work, struct smsc911x_data, rx_work);
        struct net_device *dev = pdata->dev;
        int npackets = 0;
        struct napi_struct *napi = (struct work_struct *)work->data.napi;  // <== THIS LINE
        int budget = (struct work_struct *)work->data.budget;  // <== THIS LINE ..
}

From the above 'THIS LINE's, I get erros below.

error: 'atomic_long_t' has no member named 'napi' error: 'atomic_long_t' has no member named 'budget'

and I don't know how to pass the return value to the original NAPI poll functino caller.

I'm not sure if this kind of conversion (from NAPI poll to workqueue) is possible. Sorry for the long questions but any help will be greatly appreciated.

ADD : Because struct smsc911x_data has both struct napi napi; and struct work_struct rx_work; as members, I can easily obtain the struct napi *napi from work_struct *work (an argument of work function) by :

struct smsc911x_data *pdata = container_of(work, struct smsc911x_data, rx_work); struct napi_struct *napi = &pdata.napi;

so maybe I can just pass the int budget through a new member value in struct smsc911x_data. I sill want to know the correct practice for this case.

Answer 1

1. How do I pass the NAPI poll function arguments to the work_struct?

Just create new structure, which embed work_struct and add your arguments into it:

struct my_work {
    struct work_struct base_work;// Embedded work_struct
    struct napi_struct *napi; // Your arguments
    int budget;
};

static int smsc911x_poll(struct napi_struct *napi, int budget) {
    struct my_work* p = kmalloc(sizeof(*p), GFP_ATOMIC /* Flag usable for interrupt context */);
    INIT_WORK(&p->base_work, smsc911x_poll_work); // Initialize underliying structure.
    p->budget = budget; // Initialize your members
    p->napi = napi;
    ...
}

2. How do I get the NAPI poll function arguments back from the work_struct? (related to Q.1 above)

Use container_of:

static int smsc911x_poll_work(struct work_struct *work) {
    struct my_work* p = container_of(work, struct my_work, base_work);
    ...
}

3. How can I return the npackets value to the original NAPI poll function caller?

As I understand from description(see, e.g., http://www.linuxfoundation.org/collaborate/workgroups/networking/napi) this function should process packets which are ready. And this processing should be done within function itself, without deferring to workqueue or similar.

Answer 2

This approach seems very ineffective since you need two interrupts, one when packet is received, and one when DMA tansfer is done.

I think this it the way of working of DMA capable network interfaces: When packet(s) arrive, Socket Buffers are already allocated and mapped to DMA memory buffer, and DMA is armed.

1. Packet is transferred from NIC to Socket Buffer through DMA
2. NIC raises hardware interrupt (when DMA transfer is done).
3. Hardware interrupt handler schedules packet receiving software interrupt (SOFTIRQ)
4. SOFTIRQ does NAPI poll() for further processing.
5. NAPI poll() process packets in DMA buffers and and passes it to upper layers as sk_buff and initializes new DMA buffers. if all packets (quota) are processed, IRQ is enabled and NAPI is told to stop polling.