Ruby profiler stack level too deep error

Question

It seems like I always get this error on one of my scripts:

/Users/amosng/.rvm/gems/ruby-1.9.3-p194/gems/ruby-prof-0.11.2/lib/ruby-prof/profile.rb:25: stack level too deep (SystemStackError)

Has anyone encountered this error before? What could be causing it, and what can I be doing to prevent it from happening?

I run my ruby-prof scripts using the command

ruby-prof --printer=graph --file=profile.txt scraper.rb -- "fall 2012"

Edit I'm on Mac OS X, if that matters. Doing ulimit -s 64000 doesn't seem to help much, unfortunately. Here is what ulimit -a gives:

$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 256
pipe size            (512 bytes, -p) 1
stack size              (kbytes, -s) 64000
cpu time               (seconds, -t) unlimited
max user processes              (-u) 709
virtual memory          (kbytes, -v) unlimited

Edit 2

Andrew Grimm's solution worked just fine to prevent ruby-prof from crashing, but the profiler seems to have problems of its own, because I see percentages like 679.50% of total time taken for a process...

Answer 1

One workaround would be to turn tail call optimization on.

The following is an example of something that works with TCO on, but doesn't work when TCO is off.

RubyVM::InstructionSequence.compile_option = {
  :tailcall_optimization => true,
  :trace_instruction => false
}

def countUpTo(current, final)
  puts current
  return nil if current == final
  countUpTo(current+1, final)
end

countUpTo(1, 10_000)

Answer 2

Stack level too deep usually means an infinite loop. If you look at the ruby-prof code where the error happens you will see that it's a method that detects recursion in the call stack.

Try looking into the code where you are using recursion (how many places in your code can you be using recursion?) and see if there is a condition that would cause it to never bottom-out?

It could also mean that your system stack just isn't big enough to handle what you are trying to do. Maybe you are processing a large data set recursively? You can check your stack size (unixy systems):

$ ulimit -a

and increase the stack size:

$ ulimit -s 16384

You can also consider adjusting your algorithm. See this stack overflow quesion

I hope I'm not just re-hashing an existing question...

Answer 3

Having percentages go over 100% in Ruby-prof has been a known bug, but should be fixed now.