Can we have more than one worker process for presto?

501 views Asked by At

Suppose in my presto cluster I have two data sources HIVE and Cassandra. i.e. I have one Hive catalog which connect to Hive source and one Cassandra catalog which connect to Cassandra source.Two teams are working separately on each of them.

If both the team submits separate queries in parallel, worker will create tasks for each queries. As in presto for each task worker create a new thread that means tasks of both the queries resides in same JVM(Heap). As both team wants their data to be very secure, they want memory isolation for their tasks(they don't want to take any risk of code injection).

Is there any way, so that all the tasks of each source run in separate JVM(Heap). I meant can we run more than one worker process: each for each data source ? Like some worker group or something.

PS.Not expert in presto.just exploring now.

1

There are 1 answers

2
Dain Sundstrom On BEST ANSWER

There is no way to do that, but the work around is simple; just run two clusters.