Linux: how to change maximum number of files a process can open?

Question

I have to execute a process on a cluster of machines. Size of cluster is of order 100. So I cannot execute processes manually, I have to execute them by script(which uses ssh, currently I am using python-paramiko for this). Number of tcp sockets these processes open is more than 1024(default limit of linux.) So I need to change that using {ulimit -n 10000}. This makes the changes for that shell session only. And this command works only with root user. So my script is not able to do that. I tried to execute this command

sudo su && ulimit -n 10000 && <commandToExecuteMyProcess>

But this didn't work. The commands after "sudo su" didn't execute at all. They execute only when I logout of the su session. This article shows way to make the changes permanently. But when I open limits.conf, I didn't find anything there. It only has some commented notes.

Please suggest me some way to increase the limit permanently or change it by script for each session.

Answer 1

That's not how it works: sudo su just opens a new shell so you can introduce commands as root, and after you exit that shell it executes the rest of the line as normal user.

Second: your this is a special case because ulimit is not actually a program, but a bash shell built-in command, so it must be used within bash, that is why something like sudo ulimit -n 10000 won't work: sudo can't find that program because it doesn't exist.

So, the only alternative is a bit ugly but works:

 sudo bash -c 'ulimit -n 10000 && <command>'

Everything inside '...' will execute in a bash session of the root user.

Note that you can replace && with ; in this case: that's because it is being executed as root and ulimit -n 10000 will always complete successfully.