Nodejs Event Loop

Question

Are there internally two event loops in nodejs architecture?

• libev/libuv
• v8 javascript event loop

On an I/O request does node queue the request to libeio which in turn notifies the availability of data via events using libev and finally those events are handled by v8 event loop using callbacks?

Basically, How are libev and libeio integrated in nodejs architecture?

Are there any documentation available to give a clear picture of nodejs internal architecture?

Answer 1

I have been personally reading the source code of node.js & v8.

I went into a similar problem like you when I tried to understand node.js architecture in order to write native modules.

What I am posting here is my understanding of node.js and this might be a bit off track as well.

1. Libev is the event loop which actually runs internally in node.js to perform simple event loop operations. It's written originally for *nix systems. Libev provides a simple yet optimized event loop for the process to run on. You can read more about libev here.

2. LibEio is a library to perform input output asynchronously. It handles file descriptors, data handlers, sockets etc. You can read more about it here here.

3. LibUv is an abstraction layer on the top of libeio , libev, c-ares ( for DNS ) and iocp (for windows asynchronous-io). LibUv performs, maintains and manages all the io and events in the event pool. ( in case of libeio threadpool ). You should check out Ryan Dahl's tutorial on libUv. That will start making more sense to you about how libUv works itself and then you will understand how node.js works on the top of libuv and v8.

To understand just the javascript event loop you should consider watching these videos

• JS-conference
• JSConf2011 ( has very irritative sfx)
• Understanding event driven programming
• Understanding the node.js event loop

To see how libeio is used with node.js in order to create async modules you should see this example.

Basically what happens inside the node.js is that v8 loop runs and handles all javascript parts as well as C++ modules [ when they are running in a main thread ( as per official documentation node.js itself is single threaded) ]. When outside of the main thread, libev and libeio handle it in the thread pool and libev provide the interaction with the main loop. So from my understanding, node.js has 1 permanent event loop: that's the v8 event loop. To handle C++ async tasks it's using a threadpool [via libeio & libev ].

For example:

eio_custom(Task,FLAG,AfterTask,Eio_REQUEST);

Which appears in all modules is usually calling the function Task in the threadpool. When it's complete, it calls the AfterTask function in the main thread. Whereas Eio_REQUEST is the request handler which can be a structure / object whose motive is to provide communication between the threadpool and main thread.

Answer 2

Looks like some of the entities discussed (eg: libev etc.) have had lost relevance, due to the fact that it has been a while, but I think the question still has great potential.

Let me try to explain the working of event driven model with the help of an abstract example, in an abstract UNIX environment, in Node's context, as of today.

Program's perspective:

• Script engine starts execution of the script.
• Any time a CPU bound operation is encountered, it is executed inline (real machine), in its completeness.
• Any time an I/O bound operation is encountered, the request, and its completion handler are registered with an 'event machinery' (virtual machine)
• Repeat the operations in the same manner above until the script ends. CPU bound operation - execute in-line, I/O bound ones, request to the machinery as above.
• When I/O completes, the listeners are called back.

The event machinery above is called libuv AKA event loop framework. Node leverages this library to implement its event driven programming model.

Node's perspective:

• Have one thread to host the runtime.
• Pick up the user script.
• Compile it into native [ leverage v8 ]
• Load the binary, and jump into the entry point.
• The compiled code executes the CPU bound activities in-line, using programming primitives.
• Many I/O and timer related code have native wraps. For example, network I/O.
• So I/O calls are routed from the script to C++ bridges, with the I/O handle and the completion handler passed as arguments.
• The native code exercises the libuv loop. It acquires the loop, enqueues a low level event which represents the I/O, and a native callback wrapper into the libuv loop structure.
• The native code returns to the script - no I/O is taken place at the moment!
• Items above are repeated many times, until all the non-I/O code are executed, and all the I/O code are registered will the libuv.
• Finally, when there is nothing left in the system to execute, node pass the control to libuv
• libuv gets into action, it picks up all the registered events, queries the operating system to get their operability.
• Those which are are ready for I/O in a non-blocking mode, are picked up, I/O performed, and their callbacks issued. One after the other.
• Those which are not yet ready (for example a socket read, for which the other end point hasn't written anything yet) will continued to be probed with the OS until they are available.
• The loop internally maintains an ever increasing timer. When application requests for a deferred callback(such as setTimeout), this internal timer value is leveraged to compute the right time for firing the callback.

While most of the functionalities are catered to in this manner, some (async versions) of the file operations are carried out with the help of additional threads, well integrated into the libuv. While network I/O operations can wait in expectation of an external event such as the other endpoint responding with data etc. the file operations need some work from node itself. For example, if you open a file and wait for the fd to be ready with data, it won't happen, as no one is reading actually! At the same time, if you read from the file inline in the main thread, it can potentially block other activities in the program, and can make visible problems, as file operations are very slow compared to cpu bound activities. So internal worker threads (configurable through UV_THREADPOOL_SIZE environment variable) are employed to operate on files, while the event driven abstraction works intact, from the program's perspective.

Hope this helps.

Answer 3

In simplest words, Node event loop is cycle or loop on architectural level which help Javascript code to handle asynchronous code.

The event loop have different loop/cycle inside, which are use to handle appropriate job, e.g setTimeouts, setimmediate, file system, network requests, promises and other stuff.

Answer 4

There is one event loop in the NodeJs Architecture.

Node.js Event Loop Model

Node applications run in a single-threaded event-driven model. However, Node implements a thread pool in the background so that work can be performed.

Node.js adds work to an event queue and then has a single thread running an event loop pick it up. The event loop grabs the top item in the event queue, executes it, and then grabs the next item.

When executing code that is longer lived or has blocking I/O, instead of calling the function directly, it adds the function to the event queue along with a callback that will be executed after the function completes. When all events on the Node.js event queue have been executed, the Node.js application terminates.

The event loop starts to encouner problems when our application functions block on I/O.

Node.js uses event callbacks to avoid having to wait for blocking I/O. Therefore, any requests that perform blocking I/O are performed on a different thread in the background.

When an event that blocks I/O is retrieved from the event queue, Node.js retrieves a thread from the thread pool, and executes the function there instead of on the main event loop thread. This prevents the blocking I/O from holding up the rest of the events in the event queue.

Answer 5

The pbkdf2 function has the JavaScript implementation but it actually delegates all the work to be done to the C++ side.

env->SetMethod(target, "pbkdf2", PBKDF2);
  env->SetMethod(target, "generateKeyPairRSA", GenerateKeyPairRSA);
  env->SetMethod(target, "generateKeyPairDSA", GenerateKeyPairDSA);
  env->SetMethod(target, "generateKeyPairEC", GenerateKeyPairEC);
  NODE_DEFINE_CONSTANT(target, OPENSSL_EC_NAMED_CURVE);
  NODE_DEFINE_CONSTANT(target, OPENSSL_EC_EXPLICIT_CURVE);
  NODE_DEFINE_CONSTANT(target, kKeyEncodingPKCS1);
  NODE_DEFINE_CONSTANT(target, kKeyEncodingPKCS8);
  NODE_DEFINE_CONSTANT(target, kKeyEncodingSPKI);
  NODE_DEFINE_CONSTANT(target, kKeyEncodingSEC1);
  NODE_DEFINE_CONSTANT(target, kKeyFormatDER);
  NODE_DEFINE_CONSTANT(target, kKeyFormatPEM);
  NODE_DEFINE_CONSTANT(target, kKeyTypeSecret);
  NODE_DEFINE_CONSTANT(target, kKeyTypePublic);
  NODE_DEFINE_CONSTANT(target, kKeyTypePrivate);
  env->SetMethod(target, "randomBytes", RandomBytes);
  env->SetMethodNoSideEffect(target, "timingSafeEqual", TimingSafeEqual);
  env->SetMethodNoSideEffect(target, "getSSLCiphers", GetSSLCiphers);
  env->SetMethodNoSideEffect(target, "getCiphers", GetCiphers);
  env->SetMethodNoSideEffect(target, "getHashes", GetHashes);
  env->SetMethodNoSideEffect(target, "getCurves", GetCurves);
  env->SetMethod(target, "publicEncrypt",
                 PublicKeyCipher::Cipher<PublicKeyCipher::kPublic,
                                         EVP_PKEY_encrypt_init,
                                         EVP_PKEY_encrypt>);
  env->SetMethod(target, "privateDecrypt",
                 PublicKeyCipher::Cipher<PublicKeyCipher::kPrivate,
                                         EVP_PKEY_decrypt_init,
                                         EVP_PKEY_decrypt>);
  env->SetMethod(target, "privateEncrypt",
                 PublicKeyCipher::Cipher<PublicKeyCipher::kPrivate,
                                         EVP_PKEY_sign_init,
                                         EVP_PKEY_sign>);
  env->SetMethod(target, "publicDecrypt",
                 PublicKeyCipher::Cipher<PublicKeyCipher::kPublic,
                                         EVP_PKEY_verify_recover_init,
                                         EVP_PKEY_verify_recover>);

resource: https://github.com/nodejs/node/blob/master/src/node_crypto.cc

The Libuv module has another responsibility that is relevant for some very particular functions in the standard library.

For some standard library function calls, the Node C++ side and Libuv decide to do expensive calculations outside of the event loop entirely.

Instead they make use of something called a thread pool, the thread pool is a series of four threads that can be used for running computationally expensive tasks such as the pbkdf2 function.

By default Libuv creates 4 threads in this thread pool.

In addition to the threads used in the event loop there are four other threads that can be used to offload expensive calculations that need to occur inside our application.

Many of the functions included in the Node standard library automatically make use of this thread pool. The pbkdf2 function being one of them.

The presence of this thread pool is very significant.

So Node is not truly single threaded, because there are other threads that Node uses for doing some computationally expensive tasks.

If the event pool was responsible for doing the computationally expensive task, then our Node application could do nothing else.

Our CPU runs all the instructions inside a thread one by one.

By using the thread pool we can do other things inside an event loop while calculations are occurring.

Answer 6

As a javascript beginner, I had also the same doubt, does NodeJS contains 2 event loops?. After a long research and discussing with one of V8 contributor, I got the following concepts.

• The event loop is a fundamental abstract concept of the JavaScript programming model. So V8 engine provides a default implementation for event loop, which embedders (browser, node) can replace or extend. You guys can find the V8 default implementation of event loop here
• In NodeJS, there is only one event loop exist, which is provided by the node runtime. The V8 default event loop implementation was replaced with NodeJS event loop implementation

Answer 7

An Introduction to libuv

The node.js project began in 2009 as a JavaScript environment decoupled from the browser. Using Google’s V8 and Marc Lehmann’s libev, node.js combined a model of I/O – evented – with a language that was well suited to the style of programming; due to the way it had been shaped by browsers. As node.js grew in popularity, it was important to make it work on Windows, but libev ran only on Unix. The Windows equivalent of kernel event notification mechanisms like kqueue or (e)poll is IOCP. libuv was an abstraction around libev or IOCP depending on the platform, providing users an API based on libev. In the node-v0.9.0 version of libuv libev was removed.

Also one picture which describe the Event Loop in Node.js by @BusyRich

---

Update 05/09/2017

Per this doc Node.js event loop,

The following diagram shows a simplified overview of the event loop's order of operations.

   ┌───────────────────────┐
┌─>│        timers         │
│  └──────────┬────────────┘
│  ┌──────────┴────────────┐
│  │     I/O callbacks     │
│  └──────────┬────────────┘
│  ┌──────────┴────────────┐
│  │     idle, prepare     │
│  └──────────┬────────────┘      ┌───────────────┐
│  ┌──────────┴────────────┐      │   incoming:   │
│  │         poll          │<─────┤  connections, │
│  └──────────┬────────────┘      │   data, etc.  │
│  ┌──────────┴────────────┐      └───────────────┘
│  │        check          │
│  └──────────┬────────────┘
│  ┌──────────┴────────────┐
└──┤    close callbacks    │
   └───────────────────────┘

note: each box will be referred to as a "phase" of the event loop.

Phases Overview

• timers: this phase executes callbacks scheduled by setTimeout() and setInterval().
• I/O callbacks: executes almost all callbacks with the exception of close callbacks, the ones scheduled by timers, and setImmediate().
• idle, prepare: only used internally.
• poll: retrieve new I/O events; node will block here when appropriate.
• check: setImmediate() callbacks are invoked here.
• close callbacks: e.g. socket.on('close', ...).

Between each run of the event loop, Node.js checks if it is waiting for any asynchronous I/O or timers and shuts down cleanly if there are not any.

Answer 8

There is only one event loop provided by libuv, V8 is just a JS runtime engine.