I want to deploy a traditional monolithic application in Kubernetes.
Thousands of customers use this application and each customer has its own instance of application. if we have 5 customers we should run 5 separate instances of this application.
The application also calls Kubernetes API for running some jobs.
I want to make sure that everything is isolated, Is it a good idea to create a separate namespace for each customer? Does it cause some performance issues? Is there any better solution for it?
I think you should create multi-tenant cluster.
Such clusters shared by multiple users and/or workloads which are referred to as "tenants". The operators of multi-tenant clusters must isolate tenants from each other to avoid the damage that a compromised. You should know that cluster resources must be fairly allocated among tenants.
When you plan a multi-tenant architecture you should consider the layers of resource isolation in Kubernetes: cluster, namespace, node, pod, and container. You should also consider the security aspects of sharing different types of resources among specific tenants.
Although Kubernetes cannot guarantee perfectly secure isolation between tenants, it does offer features that may be sufficient for specific solutions. For example you can separate each tenant and their Kubernetes resources into their own separate namespaces. Then use policies to enforce tenant isolation. Policies are usually scoped by namespace and can be used to restrict API access, to constrain resource usage, and to restrict what containers are allowed to do.
Read more: multi-tenant-cluster.
However while implementing multi-tenancy with Kubernetes, you need to decide if you need soft multi-tenancy (is focused on minimising accidents and managing the fallout) or hard multi-tenancy(assumes tenants to be malicious and therefore advocates zero trust between them). In any case, you have to answer questions: how to limit their resource usage, how to manage the users/tenants and how to isolate them from each other. There are many tools, for example: loft which can help you to get multi-tenancy with Kubernetes.
See: multi-tenant-loft.
Take a look: best-practices-multitenant.