restarting a route initialized with File component does not poll the existing files again

Question

Thanks to JMX (java console), I try to restart a route with a file component consumer endpoint.

from("file:<some dir>?noop=true")

I am using the wiretap pattern to record the intermediate data transformation through other files endpoint. On first start of the camel application, everything is fine, and all the files already present in the input directory are polled and processed.

But when I try to restart the route thanks to jmx, nothing happens. I try to manually removed .camel directory - created by I guess the default FileIdempotentRepository - before restarting the route, in vain. I also tried to change the kind of IdempotentRepository with a MemoryIdempotentRepository :

from("file:<somedir>?noop=true").idempotentConsumer(header("CamelFileName"), MemoryIdempotentRepository.memoryIdempotentRepository(1000))

Even if I trigger the clear() operation of this MemoryIdempotentRepository before restarting the route in java console, nothing is polled from the input directory after restarting.

If I add a file, it works. Everything behaves like if there is a persistent history of the files already polled once.

I wonder if the use of the option "noop=true" creates an unmanaged idempotent repository I cannot control with jmx.

If true, the file is not moved or deleted in any way. This option is good for readonly data, or for ETL type requirements. If noop=true, Camel will set idempotent=true as well, to avoid consuming the same files over and over again.

Any idea ? (i am using camel-core 2.21)

Answer 1

I found the solution to my issue. I made a bad use of idempotentConsumer; I needed to initialize the endpoint idempotent consumer inside the endpoint URI parameters list.

First, create an entry in a bean registry:

registry.put("myIdempotentRepository", MemoryIdempotentRepository.memoryIdempotentRepository(1000));

Then, refer to this idempotentRepository in the endpoint:

from("file:<somedire>noop=true&initialDelay=10&delay=1000&idempotentRepository=#myIdempotentRepository")

By doing this, GenericFileEndPoint:

• will not create a default idempotentRepository
• will add the idempotentRepository given in options of the endpoint to the services of the camel context. This means that it will be possible to manage it thanks to JMX

I think it would be useful to be allowed to manage the default idempotent repository in the FileEndPoint class of camel-core.