Intermittent ConcurrentModificationException when iterating over Grails parameter map

833 views Asked by At

I occasionally see the ConcurrentModificationException below in my grails application's log in CI. I think it only happens when the app is under a lot of load (we fire 8 concurrent Geb test shards at the same tomcat instance).

2015-06-14 13:44:16,702 [http-bio-8080-exec-53] ERROR errors.GrailsExceptionResolver  - ConcurrentModificationException occurred when processing request: [POST] /myapp/task/getSchedules
Stacktrace follows:
java.util.ConcurrentModificationException
    at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(LinkedHashMap.java:394)
    at java.util.LinkedHashMap$EntryIterator.next(LinkedHashMap.java:413)
    at java.util.LinkedHashMap$EntryIterator.next(LinkedHashMap.java:412)
    at com.myapp.controller.ControllerSearchService.getMostRecentPrefixedParams(ControllerSearchService.groovy:83)
    at com.myapp.controller.ControllerSearchService.getMostRecentFilterParams(ControllerSearchService.groovy:65)
    at com.myapp.controller.ControllerSearchService.advancedSearch(ControllerSearchService.groovy:239)
    at com.myapp.aspect.ServiceMethodTimingAspect.traceServiceMethodCall(ServiceMethodTimingAspect.java:20)
    at GrailsMelodyGrailsPlugin$_closure4_closure16_closure17.doCall(GrailsMelodyGrailsPlugin.groovy:184)
    at com.myapp.task.TaskController.getSchedules(TaskController.groovy:287)
    at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:195)
    at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
    at net.bull.javamelody.JspWrapper.invoke(JspWrapper.java:150)
    at net.bull.javamelody.JdbcWrapper$DelegatingInvocationHandler.invoke(JdbcWrapper.java:285)
    at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:198)
    at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:176)
    at grails.plugin.springsecurity.web.filter.GrailsAnonymousAuthenticationFilter.doFilter(GrailsAnonymousAuthenticationFilter.java:53)
    at com.myapp.organisation.security.RestAuthenticationFilter.doFilter(RestAuthenticationFilter.groovy:160)
    at grails.plugin.springsecurity.web.authentication.RequestHolderAuthenticationFilter.doFilter(RequestHolderAuthenticationFilter.java:49)
    at grails.plugin.springsecurity.web.authentication.logout.MutableLogoutFilter.doFilter(MutableLogoutFilter.java:82)
    at com.odobo.grails.plugin.springsecurity.rest.RestLogoutFilter.doFilter(RestLogoutFilter.groovy:63)
    at org.grails.jaxrs.web.JaxrsFilter.doFilterInternal(JaxrsFilter.java:46)
    at com.brandseye.cors.CorsFilter.doFilter(CorsFilter.java:82)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

The code in the getMostRecentPrefixedParams method looks like this:

private def getMostRecentPrefixedParams(params, session, prefix, sessionParamName) {
    synchronized (params) {
        for (String key : params) {     // line 83 - exception happens here
            if (key.startsWith(prefix)) {
                if (session != null) {
                    clearTags(params)
                }
                return params
            }
        }
    }

    if (session == null) {
        return params
    }

    return session."${sessionParamName}" == null ? params : session."${sessionParamName}"
}

I added the synchronized block in an attempt to stop this from happening, but clearly something else is going on.

The params argument passed into the getMostRecentPrefixedParams method just comes from a controller (the implicit params controller property) - as far as I'm aware, only a single request thread should have access to that map during the entire request/response cycle, but even if that wasn't the case and another thread somehow had access to that map, I would have thought the synchronized block would have prevented the ConcurrentModificationException.

We're using Grails 2.3.7.

0

There are 0 answers