Storm bolt always logs to worker.log

554 views Asked by At

I have a storm bolt that is responsible for extracting the contents of an email. I would like to log some details about the email in a log file of my choosing instead of in worker.log. The bolt does log to worker.log for errors etc. and I want it to continue to do so.

I've updated the cluster.xml to include the new logger

<configuration monitorInterval="60" shutdownHook="disable">
        <properties>
            <property name="pattern">%d{yyyy-MM-dd HH:mm:ss.SSS} %c{1.} %t [%p] %msg%n</property>
        </properties>
        <appenders>
            <RollingFile name="A1" immediateFlush="false"
                fileName="${sys:storm.log.dir}/${sys:logfile.name}"
                filePattern="${sys:storm.log.dir}/${sys:logfile.name}.%i.gz">
                <PatternLayout>
                    <pattern>${pattern}</pattern>
                </PatternLayout>
                <Policies>
                    <SizeBasedTriggeringPolicy size="100 MB"/>
                </Policies>
                <DefaultRolloverStrategy max="9"/>
            </RollingFile>
            <RollingFile name="TEST-SIZE" immediateFlush="false"
                fileName="${sys:storm.log.dir}/test_debug.log"
                filePattern="${sys:storm.log.dir}/test_debug.log.%i.gz">
            <PatternLayout>
                <pattern>${pattern}</pattern>
            </PatternLayout>
                <Policies>
                    <SizeBasedTriggeringPolicy size="100 MB"/>
                </Policies>
                <DefaultRolloverStrategy max="9"/>
            </RollingFile>
            <Syslog name="syslog" format="RFC5424" charset="UTF-8" host="localhost" port="514"
                protocol="UDP" appName="[${sys:daemon.name}]" mdcId="mdc" includeMDC="true"
                facility="LOCAL5" enterpriseNumber="18060" newLine="true" exceptionPattern="%rEx{full}"
                messageId="[${sys:user.name}:S0]" id="storm" immediateFlush="true" immediateFail="true"/>
    </appenders>
    <loggers>
        <Logger name="TestSizeLogger" level="info" additivity="false">
            <AppenderRef ref="TEST-SIZE"/>
            <AppenderRef ref="syslog"/>
        </Logger>
        <root level="info"> <!-- We log everything -->
            <appender-ref ref="A1"/>
            <appender-ref ref="syslog"/>
        </root>
    </logger>

In my Java code I then create two logger instances but they both log to worker.log

static Logger logger = LoggerFactory.getLogger(ExtractorBolt.class);
static Logger testLogger = LoggerFactory.getLogger("TestSizeLogger");

I would appreciate any advice in regards to how I can get my testLogger to log to my new log file.

1

There are 1 answers

0
Stig Rohde Døssing On

cluster.xml is for configuring the logging for Nimbus, the supervisor and other daemons. You should edit the log4j2/worker.xml file to configure worker logging.