Post event to nginx event loop from another thread

Question

I have 3rd party library, which is nonblocking and has its own event loop, it accepts pointer of callback function and executes it in same thread. What I want is to post event from this thread to nginx main thread, something like ngx_add_timer but without time option, to safely add event to nginx main event loop.

Answer 1

So very late to the party here, but I found this thread in my research and wanted to post the solution I came up with

Nginx has a mechanism to post from a worker thread - one that is perhaps running another event loop - to the main thread of the nginx worker process. That is 'ngx_post_event' which lets you post an event handler which will be invoked at some point in the future by the main thread.

You have to choose an event queue to post it on, but whatever you're doing, the answer is certainly &ngx_posted_events.

Here we come to the problem (and a solution): if you do this, your event handler will not get invoked in a timely manner because the main nginx worker process thread is waiting on i/o. It won't even deign to look at the posted events queue until it has some 'real' work to do from i/o.

The solution that's working for me currently (and bear in mind this is only on Linux), is to send the main thread a signal which will wake it up from its epoll_wait reverie so it can get to work on the pipeline coming from the other thread.

So here's what worked:

First grab the id of the worker process main thread and hold it in some process-global state:

// In you 'c' source:

static pthread_t nginx_thread;

// In some code that runs once at nginx startup (in my case the module's preconfiguration step)

nginx_thread = pthread_self();

Now when to post your callback you use the ngx_post_event call I mentioned earlier, then send a SIGIO signal to the main thread to wake up the epoll_wait operation

// Post the event and wake up the Nginx epoll event loop

ngx_post_event( event_ptr, &ngx_posted_events );
pthread_kill( nginx_thread, SIGIO );

The SIGIO event is handled in the main Nginx signal handler - and is ignored (well that's what the log says), but crucially, causes the posted events to be processed immediately.

That's it - and it seems to be working so far... please do point out anything stupid I've done.

To complete the story, you'll need the following #includes:

#include <pthread.h>

#include <signal.h>