application.cfc - conditionally turn on session and/or client management?

Question

I want to reduce the overhead of spider/crawler traffic. I'm not expecting to catch all of it, but if I can catch 90% of it then it's a win.

What's the best way to conditionally turn on/off session or client management in application.cfc? I'm thinking something along the lines of this, but I'm not sure if the CGI scope is always defined and initialized when application.cfc is instantiated.

this.sessionManagement = !isSpiderRequest();

and:

private boolean function isSpiderRequest() {

    if (REFindNoCase("googlebot|msnbot|crawler|crawling|spider|wget|curl|baidu|robot|slurp|Gigabot|ia_archiver|libwww-perl|lwp-trivial|Mediapartners-Google", CGI.HTTP_USER_AGENT))
        return(true);

    return(false);
}

Answer 1

We set the session timeout to 10 seconds for bots. They don't get errors, but don't consume any (much) memory.

<!--- SET UP THE APP --->
<cfscript>
    THIS.Name = "ASDF";
    THIS.ApplicationTimeout = createTimeSpan( 0, 0, 60, 0 );
    THIS.SetClientCookies = true;
    THIS.Datasource = "ASDF";
    THIS.SessionManagement = true;

    // TEST WHETHER USER IS A BOT
    THIS.IsBot = THIS.checkUserAgent();

    // VISITOR IS A BOT ~ SET FAST TIMEOUT
    if (THIS.IsBot == true) {
        //abort;
        THIS.SessionTimeout = createTimeSpan( 0, 0, 0, 10 );
    // VISITOR IS A NOT A BOT ~ SET SLOW TIMEOUT
    } else {
        THIS.SessionTimeout = createTimeSpan( 0, 0, 60, 10 );
    }

</cfscript>

<cffunction name="checkUserAgent">
    <cfscript>
        // QUERY THE CURRENT LIST OF BOT WORDS
        LOCAL.BotWordList = THIS.getBotWords();
        // GET THE VISITORS CURRENT USER AGENT IN LOWER CASE
        LOCAL.ThisAgent = trim(lCase(CGI.HTTP_USER_AGENT));

        // look at the user agent to see if the browser 
        // browser's user agent contains a banned word
        // return true or false

    </cfscript>
</cffunction>

Alternative answer... I haven't tried this, but I don't see why it won't work.

<!--- SET UP THE APP --->
<cfscript>
    THIS.Name = "ASDF";
    THIS.ApplicationTimeout = createTimeSpan( 0, 0, 60, 0 );
    THIS.Datasource = "ASDF";
    THIS.SetClientCookies = true;

    // TEST WHETHER USER IS A BOT
    THIS.IsBot = THIS.checkUserAgent();

    // VISITOR IS A BOT ~ SET FAST TIMEOUT
    if (THIS.IsBot == true) {
        THIS.SessionManagement = false;
    // VISITOR IS NOT A BOT
    } else {
        THIS.SessionManagement = true;
        THIS.SessionTimeout = createTimeSpan( 0, 0, 60, 10 );
    }

</cfscript>