This is an extreme longshot, but once in a while there is an engineer on here that knows what's going on under the hood.
First, I know that you can almost always rewrite some code to avoid QoQ for much better performance. However, what I'm trying to debug is a potential reason that code that uses QoQ would perform so poorly under certain conditions to develop guidelines for future projects.
1) Is there something about the ColdFusion QoQ Java implementation that would cause access to the QoQ system to be single threaded? (I saw post on Railo about this)
2) According to FusionReactor, the stack trace has the request always in the evaluate (90% chance) or execution phase (10% chance) of the QoQ -- never any other part of the logic surrounding it. That's with it running in a loop with a query param.
We have several different features that make use of QoQ in a loop over tens of thousands of iterations to generate a report, regardless of that being best practice or not. The only processes that slow down are the ones that use QoQ. They become 10 times as long to complete unless they are kicked off nearly immediately after a server reboot. FusionReactor always has the thread state at evaluating the QoQ or processing the QoQ. Of note: the Heap and CPU are just fine holding steady at under 20% during the entire duration. All memory spaces look good -- code cache, perm, etc.
I know that running QoQ in a loop is not the way to get it done and should be changed, but I'm just looking for answers. Is code locked? Is it an available thread issue? Is there something about evaluating the statement? Why would it be fine right after a restart, but quickly degrade within hours? That's what bugs me -- I'd be fine with it taking forever always, it's the oddball pattern that has me on stack overflow.
ColdFusion 10 if it helps.