We have an embedded product with quad Cortex-A53 cores, 4 GB RAM and runs Linux Kernel v5.4 aarch64. Whole Linux currently consumes only 200MB to work, therefore I have plenty of RAM available to use.
I made a kernel module which reads 20MB/sec data from an external device through PCIe bus with chunks of 1MB. I want to push this 1MB data in the interrupt context to a FIFO works in the kernel module to be transferred later to a user space.
I looked for available fifo patterns in the Linux kernel and saw that kfifo_alloc() and bunch of APIs to put and get data already provided. This suits my purpose perfectly. As I understand it supports single producer and single consumer approach without any locking.
Required FIFO size for my application should be around 128MB. I already have 4GB RAM, therefore I thought initially that allocating such memory should have no issue.
But following code returns ENOMEM error unexpectedly
ret = kfifo_alloc(&io_dev.streaming_dma_fifo, 128 * 1024 * 1024, GFP_KERNEL);
if (ret) {
dev_err(&pci_dev->dev, "%s:error kfifo_alloc, ret=%d\n", __func__, ret);
return ret;
}
Then I searched the stackoverflow and saw people use kmalloc() complain about same thing. I think kfifo_alloc() function internally also uses kmalloc().
Then I executed following code as suggested again in this forum:
dev_info(&pci_dev->dev, "KMALLOC_SHILFT_LOW:%d, KMALLOC_SHILFT_HIGH:%d, KMALLOC_MIN_SIZE:%d, KMALLOC_MAX_SIZE:%lu\n", KMALLOC_SHIFT_LOW, KMALLOC_SHIFT_HIGH, KMALLOC_MIN_SIZE, KMALLOC_MAX_SIZE);
And got these outputs for my system:
KMALLOC_SHILFT_LOW: 7
KMALLOC_SHILFT_HIGH: 13
KMALLOC_MIN_SIZE: 128
KMALLOC_MAX_SIZE: 33554432
What I understand from this output is that max size that I can allocate with kfifo_alloc() is limited to 32MB.
If I try to reduce the size requested for kfifo_alloc() from 128MB to 32MB, it returns no error.
Questions:
My FIFO RAM doesn't have any special requirements such as it should be contiguous, or atomic or DMAable or something else. I didn't understand why kfifo_alloc() fails with 128MB while I can allocate same amount in the user space easily with
malloc(). Is there a way to askkfifo_allocto usemallocor kernel equivalent of it, instead of specialkmalloc?Is there a way to increase KMALLOC_MAX_SIZE defined as 32MB for my system?
What are the consequences of setting KMALLOC_MAX_SIZE as 256MB for example?
What is the most suggested way if my driver requires a FIFO larger than 32MB?