Luigi: Task with optional output

Question

Not much to say here. I have a task which most of the time creates a json file. Sometimes though it does not. In order to implement this optional output I created something like this:

class OptionalOutput(luigi.Task):
    
    input_path = luigi.Parameter()
    output_path = luigi.Parameter()
    
    def output(self):
        return luigi.LocalTarget(uri=f'/tmp/{self.output_path}')
    
    def complete(self):
        if not URIUtils.exists(self.input_path):
            return True
        else:
            return super(OptionalOutput,self).complete()
        
    def run(self):
        # Some other code

This does not work. What is the canonical way to create a task with an optional output?

Answer 1

The way we have done it at my company is to make sure the file exists, and just make it empty or empty JSON like {} for these cases.