In Java how to read the latest string of constantly generated stream fast?

Question

In Java I have a process constantly generating output. Of course it's placed into some buffer of the out stream (FiFo) until it's processed. But what I need is sometimes read the latest, actual string of the stream as if it was LiFo. The problem is when I need it, I have to read all the previous output generated between my reads, because streams don't have random access - which is very slow.

I use BufferedReader(StreamReader(process.getInputStream()))

The buffer of BufferedReader also poses a little problem. How can I discard all the output I don't need, fast? If possible I wouldn't like to create separate reader-discarder thread.

I tried:

stdInput = new BufferedReader(new
      InputStreamReader(process.getInputStream()), 1000);

then when I need to read the Output:

            stdInput.skip(iS.available() + 1000); //get the generated up
            //till now sequence length and discard it
            stdInput.readLine(); //to 'flush' the BufferedReader buffer
            s = stdInput.readLine(); //to read the latest string

this way is very slow and takes undetermined time

Answer 1

Since you haven't posted the full code, it may be useful for you to try a few things and see what performs best. Overall, I'm not sure how much improvement you will see.

1. Remove BufferedReader wrapper and benchmark. It seems that your InputStream is memory backed. Hence reads maybe cheap. If it is then buffered reader may slow you down. If reads are expensive, then you should keep it.

2. Try Channels.newChannel(InputStream in) and use ReadableByteChannel.read(ByteBUffer dst) and benchmark. Again, it depends on your InputStream, but a Channel may show performance benefits.

Overall I'd recommend going the multithreaded approach with an ExecutorService and Callable class doing the reading and processing into a memory buffer.

Answer 2

My suggestion (I don't know your implementation): https://docs.oracle.com/javase/7/docs/api/java/io/BufferedReader.html#mark(int)

BufferedReader has a mark method:

public void mark(int readAheadLimit)
          throws IOException

Marks the present position in the stream. Subsequent calls to reset() will attempt to reposition the stream to this point.

If you know the size of your data and you can utilize that size to set the position to the part of the stream in which you care about.