issue running "afl-fuzz -Q" with binary libraries

Question

I am trying to use afl-fuzz to find security vulnerabilities in Android native libraries (ex. libssl.so, libhttp.so, etc).

It's my understanding that I am supposed to use QEMU since afl by itself is not sufficient to find bugs without the source code.

When attempting to run the following:

#> afl-fuzz -Q -i input -o output -- ./libssl.so

I run into the following issue:

[-] The current memory limit (200 MB) is too restrictive, causing an OOM fault
    in the dynamic linker.

Which I managed to solve by setting the memory restrictions to "none". Then, I ran into this error:

[-] Hmm, looks like the target binary terminated before we could complete
    a handshake with the injected code. Perhaps there is a horrible bug in
    the fuzzer.

I just started to use afl recently so I am not quite familiar with how to use it especially with binary libraries. Any help with this will be greatly appreciated!

Answer 1

First of all it am not sure you really need to run AFL in the -Q mode for fuzzing open-source libraries. You should better recompile it with afl-gcc to get the instrumentation installed in the binary. Then, it will be much easier (and quicker) to fuzz the whole application.

In fact, you run in -Q mode only when you have a binary only program to fuzz (I mean, you do not have sources of it).

Then, afl is basically working by creating partially random inputs for the fuzzed program. I do not exactly understand what you expect to do with a library as it has absolutely no inputs (it plugs to another program but that's all).

So, you may consider reading a bit more about fuzzing and sotfware I/O, it would probably the best way to go before trying to do more complex things.