What are these flags for workqueue means?

Question

While studying workqueue, I came across WorkQueue flags & constants defined in kernel. I have following doubts which i could not understand.

1. What exactly draining & rescuer mean here?

   WQ_DRAINING             = 1 << 6, /* internal: workqueue is draining */
   WQ_RESCUER              = 1 << 7, /* internal: workqueue has rescuer */
   

2. The number of CPUs defined for unbound workqueues is 4. What if I have an octa core processor. How the unbounded wq will be bounded to cpus. How they decided which CPUs to run as they have now 8 cpus not 4 cpus. Is it that, they can run on any of 8 or only 4 specific cpus?

   WQ_MAX_UNBOUND_PER_CPU = 4, /* 4 * #cpus for unbound wq */

Answer 1

WQ_DRAINING

This flag is used to indicate that the kernel is currently flushing the workqueue and new work items cannot be queued on it. Only currently pending or running work items are allowed to so during this phase until the entire workqueue is completely empty.

For details, check out the implementation of drain_workqueue() in kernel/workqueue.c.

---

WQ_RESCUER

This flag is already deprecated in the latest Kernel by this patch and the behaviour is now determined by the WQ_MEM_RECLAIM flag.

As far as the "rescuer" functionality is concerned, here is the relevant section of documentation from kernel/workqueue.c,

Workqueue rescuer thread function. There's one rescuer for each workqueue which has WQ_MEM_RECLAIM set.

Regular work processing on a pool may block trying to create a new worker which uses GFP_KERNEL allocation which has slight chance of developing into deadlock if some works currently on the same queue need to be processed to satisfy the GFP_KERNEL allocation. This is the problem rescuer solves.

When such condition is possible, the pool summons rescuers of all workqueues which have works queued on the pool and let them process those works so that forward progress can be guaranteed.

---

WQ_MAX_UNBOUND_PER_CPU

_{(Contrary to how you have interpreted it, WQ_MAX_UNBOUND_PER_CPU is NOT the number of cpus. It is the number of workqueues that can be associated with a cpu.)}

Workqueues have been traditionally per-cpu i.e. each workqueue was associated with a particular cpu, resulting in better performance due to cache-locality. The kernel scheduler does NOT have a choice but to schedule it always on the cpu it was defined on. On current architectures, this leads to increased power consumption as even a single workqueue can prevent a cpu from idling and being turned off. Hence unbound workqueues have been introduced. The scheduler is free to re-schedule unbound workqueues on any cpu as it sees fit.

The total number of such workqueues is limited to WQ_UNBOUND_MAX_ACTIVE which is defined as num_possible_cpus() * WQ_MAX_UNBOUND_PER_CPU (upto a limit of total workqueues in the system determined by WQ_MAX_ACTIVE).