I am not sure if it is possible to basically gzip the stream of log lines as they arrive at the logback appender, rather than compressing the file when we log-rotate. Is that at all possible, and if so, how to achieve that and is there a lot of benefit of compressing "on the fly" rather than the whole file?
Logback - compressing log lines before writing to file
458 views Asked by Bober02 At
1
There are 1 answers
Related Questions in JAVA
- Swagger: How do you add ApiModelProperty for 3rd party code?
- Why `additionalProperties` is the way to represent Dictionary/Map in Swagger/OpenAPI 2.0
- Allow swagger query param to be array of strings or integers
- How to refer to an external JSON file containing response examples in Swagger?
- How to generate JSON examples from OpenAPI/Swagger model definition?
- Swagger/OpenAPI - use $ref to pass a reusable defined parameter
- Provide alternate (international) spelling for defined Swagger route
- Unable to use tags in swagger documentation
- Hello,I am using swagger 3.0.0.The operation oneOf is not working here too?Where is the mistaken?
- How to document a response comprised of a list of resources using OpenAPI
Related Questions in LOGGING
- Swagger: How do you add ApiModelProperty for 3rd party code?
- Why `additionalProperties` is the way to represent Dictionary/Map in Swagger/OpenAPI 2.0
- Allow swagger query param to be array of strings or integers
- How to refer to an external JSON file containing response examples in Swagger?
- How to generate JSON examples from OpenAPI/Swagger model definition?
- Swagger/OpenAPI - use $ref to pass a reusable defined parameter
- Provide alternate (international) spelling for defined Swagger route
- Unable to use tags in swagger documentation
- Hello,I am using swagger 3.0.0.The operation oneOf is not working here too?Where is the mistaken?
- How to document a response comprised of a list of resources using OpenAPI
Related Questions in COMPRESSION
- Swagger: How do you add ApiModelProperty for 3rd party code?
- Why `additionalProperties` is the way to represent Dictionary/Map in Swagger/OpenAPI 2.0
- Allow swagger query param to be array of strings or integers
- How to refer to an external JSON file containing response examples in Swagger?
- How to generate JSON examples from OpenAPI/Swagger model definition?
- Swagger/OpenAPI - use $ref to pass a reusable defined parameter
- Provide alternate (international) spelling for defined Swagger route
- Unable to use tags in swagger documentation
- Hello,I am using swagger 3.0.0.The operation oneOf is not working here too?Where is the mistaken?
- How to document a response comprised of a list of resources using OpenAPI
Related Questions in GZIP
- Swagger: How do you add ApiModelProperty for 3rd party code?
- Why `additionalProperties` is the way to represent Dictionary/Map in Swagger/OpenAPI 2.0
- Allow swagger query param to be array of strings or integers
- How to refer to an external JSON file containing response examples in Swagger?
- How to generate JSON examples from OpenAPI/Swagger model definition?
- Swagger/OpenAPI - use $ref to pass a reusable defined parameter
- Provide alternate (international) spelling for defined Swagger route
- Unable to use tags in swagger documentation
- Hello,I am using swagger 3.0.0.The operation oneOf is not working here too?Where is the mistaken?
- How to document a response comprised of a list of resources using OpenAPI
Related Questions in LOGBACK-CLASSIC
- Swagger: How do you add ApiModelProperty for 3rd party code?
- Why `additionalProperties` is the way to represent Dictionary/Map in Swagger/OpenAPI 2.0
- Allow swagger query param to be array of strings or integers
- How to refer to an external JSON file containing response examples in Swagger?
- How to generate JSON examples from OpenAPI/Swagger model definition?
- Swagger/OpenAPI - use $ref to pass a reusable defined parameter
- Provide alternate (international) spelling for defined Swagger route
- Unable to use tags in swagger documentation
- Hello,I am using swagger 3.0.0.The operation oneOf is not working here too?Where is the mistaken?
- How to document a response comprised of a list of resources using OpenAPI
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Popular Tags
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Sure. You can simply keep a gzip compression process open and feed it lines as they come in. That would significantly reduce the space required by the log file, and would not take any more CPU resources on average, since you were going to eventually compress it anyways.
The downside is that at any point in time the compressed log file will not yet contain many of the already supplied log lines, since there is a latency and burstiness to the compression process. Many lines will need to be accumulated before a compressed block is emitted. Second, the compressed file will not be a valid gzip file until it is closed. You would still be able to decompress what's there, but it will not have the trailer with a check value. If the process is killed or the machine crashes, you are left with an invalid gzip file that doesn't have the most recent several log lines. Of course, the most recent log lines may be exactly the ones that you're most interested in, to find out what the heck happened.
All of those downsides can be cured with a specialized approach for this application, which is implemented in gzlog.h/gzlog.c. gzlog assures that after each line is written, the gzipped log file is complete and valid, and contains that log line. Furthermore, it can reconstruct the gzip file with the last provided log line even if the gzlog process itself is interrupted in the middle of adding a log line.