Listener to get Job status from Spark Job Server

Question

I am making rest requests to query Spark Job Server to get the status of the job. The code looks like below :

private Future<JobResult> getJobResultFuture(String jobId) {
        ExecutorService executorService = Executors.newSingleThreadExecutor();
        Future<JobResult> resultFuture = executorService.submit(() -> {
            while (true) {
                JobResult jobResponse = getJobResponse(jobController.getStatus(jobId));
                if (jobResponse.isJobComplete()) {
                    return jobResponse;
                }
                Thread.sleep(RESPONSE_RETRY_TIME);
            }
        });
        return resultFuture;
    }

I am making the request to the job server as follows :

public String getStatus(String jobId) {
        WebResource resource = CLIENT.resource(jobServerURL() + "/jobs/" + jobId);
        return resource.type(MediaType.TEXT_PLAIN).get(String.class);
    }

but I don't feel this is the right way to do it, I just need to know how can I add some kind of handler / listener to the web request so that when I can get status back from job server and I can perform some kind of handling like notify user.

is there any library which does this?

Answer 1

What you are doing is, REST API continuously polling for job completion status. What you want is to make it asynchronous. So make your driver send the job completion status back to REST API via some end point. This end point will do whatever it needs based on the job completion status.