Have followed the instructions to create a fuzzer using libfuzzer with ASAN support. I point it at a corpus and it will generally run for a few days and then I'll get a message:
Segmentation fault (core dumped)
INFO: exiting: 139 time: XXXXXXs
I cannot locate a core dump file anywhere on my system.
Some troubleshooting that I've applied:
- I have coded real memory errors into my application and when run as a fuzzer, it DOES produce a crash file.
- I have run other binaries with deliberate memory errors and they DO produce core dump files in the working directory.
My environment is Ubuntu 20.04.2 LTS (GNU/Linux 5.4.0-66-generic x86_64). The compiler is clang++-11.
I have posted a snip of the output immediately prior to the Segmentation fault below:
#1253299: cov: 54386 ft: 261993 corp: 3483 exec/s 0 oom/timeout/crash: 0/0/0 time: 540397s job: 6782 dft_time: 0
INFO: log from the inner process:
INFO: Seed: 614118371
INFO: Loaded 1 modules (196616 inline 8-bit counters): 196616 [0x3f96900, 0x3fc6908),
INFO: Loaded 1 PC tables (196616 PCs): 196616 [0x33cd138,0x36cd1b8),
INFO: 0 files found in /tmp/libFuzzerTemp.FuzzWithFork42425.dir/C6782
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 514888 bytes
INFO: seed corpus: files: 59 min: 30838b max: 514888b total: 10611661b rss: 53Mb
#8 pulse cov: 38701 ft: 64071 corp: 5/471Kb exec/s: 0 rss: 260Mb
#16 pulse cov: 43230 ft: 102964 corp: 12/1418Kb exec/s: 0 rss: 519Mb
Slowest unit: 11 s:
artifact_prefix='./'; Test unit written to ./slow-unit-befe4808dffe50d5a0458238314af61206e5bd28
#32 pulse cov: 44227 ft: 121374 corp: 28/3756Kb exec/s: 0 rss: 559Mb
Slowest unit: 15 s:
artifact_prefix='./'; Test unit written to ./slow-unit-01babca845e1d73257d1f0bb436b072147eddab1
#64 pulse cov: 45082 ft: 155650 corp: 58/9860Kb exec/s: 0 rss: 598Mb
#64 INITED cov: 45082 ft: 155650 corp: 59/10362Kb exec/s: 0 rss: 598Mb
NEW_PC: [REMOVED]
#65 NEW cov: 45083 ft: 155793 corp: 60/10464Kb lim: 514888 exec/s: 0 rss: 598Mb L: 103906/514888 MS: 1 CrossOver-
#66 NEW cov: 45083 ft: 156037 corp: 61/10610Kb lim: 514888 exec/s: 0 rss: 598Mb L: 149586/514888 MS: 1 ChangeASCIIInt-
#67 NEW cov: 45083 ft: 156064 corp: 62/10964Kb lim: 514888 exec/s: 0 rss: 598Mb L: 362655/514888 MS: 1 ChangeBinInt-
#68 NEW cov: 45083 ft: 156078 corp: 63/11110Kb lim: 514888 exec/s: 0 rss: 598Mb L: 149587/514888 MS: 1 InsertByte-
#69 NEW cov: 45083 ft: 156128 corp: 64/11278Kb lim: 514888 exec/s: 0 rss: 598Mb L: 171552/514888 MS: 1 ShuffleBytes-
#70 NEW cov: 45083 ft: 156174 corp: 65/11531Kb lim: 514888 exec/s: 0 rss: 598Mb L: 259501/514888 MS: 1 CopyPart-
#72 NEW cov: 45083 ft: 156409 corp: 66/11699Kb lim: 514888 exec/s: 0 rss: 598Mb L: 171552/514888 MS: 2 ChangeBinInt-CopyPart-
#73 NEW cov: 45083 ft: 156549 corp: 67/11910Kb lim: 514888 exec/s: 0 rss: 598Mb L: 216339/514888 MS: 1 InsertRepeatedBytes-
#74 NEW cov: 45083 ft: 156554 corp: 68/12021Kb lim: 514888 exec/s: 0 rss: 598Mb L: 113500/514888 MS: 1 EraseBytes-
#76 NEW cov: 45083 ft: 156559 corp: 69/12274Kb lim: 514888 exec/s: 0 rss: 598Mb L: 259535/514888 MS: 2 InsertRepeatedBytes-ShuffleBytes-
NEW_PC: [REMOVED]
NEW_PC: [REMOVED]
NEW_PC: [REMOVED]
NEW_PC: [REMOVED]
#79 NEW cov: 45087 ft: 157020 corp: 70/12777Kb lim: 514888 exec/s: 0 rss: 598Mb L: 514888/514888 MS: 3 ChangeByte-ChangeByte-CrossOver-
#81 NEW cov: 45087 ft: 157024 corp: 71/12923Kb lim: 514888 exec/s: 0 rss: 598Mb L: 149587/514888 MS: 2 CopyPart-InsertByte-
NEW_PC: [REMOVED]
NEW_PC: [REMOVED]
NEW_PC: [REMOVED]
NEW_PC: [REMOVED]
#92 NEW cov: 45091 ft: 157249 corp: 72/13006Kb lim: 514888 exec/s: 0 rss: 598Mb L: 84803/514888 MS: 1 EraseBytes-
#93 NEW cov: 45091 ft: 157356 corp: 73/13332Kb lim: 514888 exec/s: 0 rss: 598Mb L: 334030/514888 MS: 1 CopyPart-
#99 NEW cov: 45091 ft: 157365 corp: 74/13543Kb lim: 514888 exec/s: 0 rss: 598Mb L: 216315/514888 MS: 1 InsertByte-
#100 NEW cov: 45091 ft: 157377 corp: 75/13690Kb lim: 514888 exec/s: 0 rss: 598Mb L: 149586/514888 MS: 1 ChangeBinInt-
#101 NEW cov: 45091 ft: 157384 corp: 76/14044Kb lim: 514888 exec/s: 0 rss: 598Mb L: 362655/514888 MS: 1 ChangeBit-
#103 NEW cov: 45091 ft: 157443 corp: 77/14211Kb lim: 514888 exec/s: 0 rss: 598Mb L: 171517/514888 MS: 2 ChangeByte-InsertRepeatedBytes-
#106 NEW cov: 45091 ft: 157446 corp: 78/14537Kb lim: 514888 exec/s: 0 rss: 598Mb L: 334084/514888 MS: 3 ChangeBit-ChangeBinInt-InsertRepeatedBytes-
NEW_PC: [REMOVED]
#107 NEW cov: 45092 ft: 157996 corp: 79/14705Kb lim: 514888 exec/s: 0 rss: 598Mb L: 171476/514888 MS: 1 ChangeByte-
#108 NEW cov: 45092 ft: 157999 corp: 80/14872Kb lim: 514888 exec/s: 0 rss: 598Mb L: 171476/514888 MS: 1 CMP- DE: "Metad"-
#116 NEW cov: 45092 ft: 158007 corp: 81/15079Kb lim: 514888 exec/s: 0 rss: 598Mb L: 211785/514888 MS: 3 ShuffleBytes-ChangeBit-CrossOver-
#117 NEW cov: 45092 ft: 158019 corp: 82/15416Kb lim: 514888 exec/s: 0 rss: 598Mb L: 344825/514888 MS: 1 EraseBytes-
#123 NEW cov: 45092 ft: 158036 corp: 83/15909Kb lim: 514888 exec/s: 0 rss: 598Mb L: 504648/514888 MS: 1 CopyPart-
#128 pulse cov: 45092 ft: 158055 corp: 83/15909Kb lim: 514888 exec/s: 0 rss: 598Mb
#128 NEW cov: 45092 ft: 158055 corp: 84/16162Kb lim: 514888 exec/s: 0 rss: 598Mb L: 259620/514888 MS: 5 CMP-InsertRepeatedBytes-ChangeBit-ChangeBinInt-CopyPart- DE: "\x91._\x01\x00\x00\x00\x00"-
#130 NEW cov: 45092 ft: 158073 corp: 85/16Mb lim: 514888 exec/s: 0 rss: 598Mb L: 259620/514888 MS: 2 ChangeBinInt-CopyPart-
#137 NEW cov: 45092 ft: 158137 corp: 86/16Mb lim: 514888 exec/s: 0 rss: 598Mb L: 171484/514888 MS: 2 ChangeByte-PersAutoDict- DE: "\x91._\x01\x00\x00\x00\x00"-
#138 NEW cov: 45092 ft: 158138 corp: 87/16Mb lim: 514888 exec/s: 0 rss: 598Mb L: 338523/514888 MS: 1 CopyPart-
#144 NEW cov: 45092 ft: 158142 corp: 88/16Mb lim: 514888 exec/s: 0 rss: 598Mb L: 171476/514888 MS: 1 ChangeASCIIInt-
#147 NEW cov: 45092 ft: 158167 corp: 89/16Mb lim: 514888 exec/s: 0 rss: 598Mb L: 103906/514888 MS: 3 ChangeASCIIInt-ChangeBinInt-CopyPart-
#148 NEW cov: 45092 ft: 158386 corp: 90/16Mb lim: 514888 exec/s: 0 rss: 604Mb L: 149586/514888 MS: 1 CopyPart-
#150 NEW cov: 45092 ft: 158388 corp: 91/17Mb lim: 514888 exec/s: 0 rss: 604Mb L: 119131/514888 MS: 2 ShuffleBytes-EraseBytes-
#152 NEW cov: 45092 ft: 158404 corp: 92/17Mb lim: 514888 exec/s: 0 rss: 604Mb L: 167390/514888 MS: 2 ChangeBit-EraseBytes-
#153 NEW cov: 45092 ft: 158427 corp: 93/17Mb lim: 514888 exec/s: 0 rss: 605Mb L: 149586/514888 MS: 1 ChangeBit-
#154 NEW cov: 45092 ft: 158428 corp: 94/17Mb lim: 514888 exec/s: 0 rss: 605Mb L: 149586/514888 MS: 1 ChangeBinInt-
#160 NEW cov: 45092 ft: 158488 corp: 95/17Mb lim: 514888 exec/s: 0 rss: 605Mb L: 107319/514888 MS: 1 CrossOver-
#162 NEW cov: 45092 ft: 158490 corp: 96/17Mb lim: 514888 exec/s: 0 rss: 605Mb L: 171553/514888 MS: 2 ChangeByte-InsertByte-
NEW_PC: [REMOVED]
NEW_PC: [REMOVED]
NEW_PC: [REMOVED]
NEW_PC: [REMOVED]
NEW_PC: [REMOVED]
NEW_PC: [REMOVED]
#164 NEW cov: 45098 ft: 158729 corp: 97/17Mb lim: 514888 exec/s: 0 rss: 605Mb L: 84806/514888 MS: 2 CMP-CMP- DE: "Si"-"CCF"-
#165 NEW cov: 45098 ft: 158800 corp: 98/18Mb lim: 514888 exec/s: 0 rss: 605Mb L: 514888/514888 MS: 1 CrossOver-
#166 NEW cov: 45098 ft: 158815 corp: 99/18Mb lim: 514888 exec/s: 0 rss: 605Mb L: 108441/514888 MS: 1 EraseBytes-
#167 NEW cov: 45098 ft: 158833 corp: 100/18Mb lim: 514888 exec/s: 0 rss: 605Mb L: 443842/514888 MS: 1 InsertRepeatedBytes-
#168 NEW cov: 45098 ft: 158835 corp: 101/18Mb lim: 514888 exec/s: 0 rss: 605Mb L: 149590/514888 MS: 1 CMP- DE: "\x00\x00\x00#"-
#169 NEW cov: 45098 ft: 158847 corp: 102/19Mb lim: 514888 exec/s: 0 rss: 605Mb L: 116574/514888 MS: 1 ChangeBit-
#175 NEW cov: 45098 ft: 158860 corp: 103/19Mb lim: 514888 exec/s: 0 rss: 607Mb L: 149586/514888 MS: 1 ChangeBinInt-
#177 NEW cov: 45098 ft: 158866 corp: 104/19Mb lim: 514888 exec/s: 0 rss: 607Mb L: 171476/514888 MS: 2 ChangeBinInt-CopyPart-
#178 NEW cov: 45098 ft: 158876 corp: 105/19Mb lim: 514888 exec/s: 0 rss: 607Mb L: 149682/514888 MS: 1 InsertRepeatedBytes-
#180 NEW cov: 45098 ft: 158946 corp: 106/19Mb lim: 514888 exec/s: 0 rss: 607Mb L: 65397/514888 MS: 2 ChangeASCIIInt-EraseBytes-
#182 NEW cov: 45098 ft: 159002 corp: 107/19Mb lim: 514888 exec/s: 0 rss: 607Mb L: 263946/514888 MS: 2 ChangeASCIIInt-CopyPart-
NEW_PC: [REMOVED]
NEW_PC: [REMOVED]
NEW_PC: [REMOVED]
#184 NEW cov: 45101 ft: 159017 corp: 108/20Mb lim: 514888 exec/s: 0 rss: 607Mb L: 365913/514888 MS: 2 PersAutoDict-CrossOver- DE: "CCF"-
#188 NEW cov: 45101 ft: 159033 corp: 109/20Mb lim: 514888 exec/s: 0 rss: 607Mb L: 76954/514888 MS: 4 ChangeByte-InsertByte-InsertByte-EraseBytes-
#191 NEW cov: 45101 ft: 159055 corp: 110/20Mb lim: 514888 exec/s: 0 rss: 610Mb L: 514888/514888 MS: 3 CMP-PersAutoDict-CrossOver- DE: "\x94,\x01\x00\x00\x00\x00\x00"-"Si"-
#192 NEW cov: 45101 ft: 159081 corp: 111/21Mb lim: 514888 exec/s: 0 rss: 611Mb L: 514888/514888 MS: 1 CopyPart-
#194 NEW cov: 45101 ft: 159118 corp: 112/21Mb lim: 514888 exec/s: 0 rss: 611Mb L: 118409/514888 MS: 2 InsertRepeatedBytes-EraseBytes-
#195 NEW cov: 45101 ft: 159130 corp: 113/21Mb lim: 514888 exec/s: 0 rss: 611Mb L: 334084/514888 MS: 1 CopyPart-
#196 NEW cov: 45101 ft: 159132 corp: 114/22Mb lim: 514888 exec/s: 0 rss: 611Mb L: 362686/514888 MS: 1 InsertRepeatedBytes-
#200 NEW cov: 45101 ft: 159134 corp: 115/22Mb lim: 514888 exec/s: 0 rss: 611Mb L: 125415/514888 MS: 4 ChangeBit-ChangeASCIIInt-ShuffleBytes-EraseBytes-
#201 NEW cov: 45101 ft: 159139 corp: 116/22Mb lim: 514888 exec/s: 0 rss: 611Mb L: 149586/514888 MS: 1 ShuffleBytes-
#207 NEW cov: 45101 ft: 159140 corp: 117/22Mb lim: 514888 exec/s: 0 rss: 611Mb L: 149682/514888 MS: 1 CopyPart-
#208 NEW cov: 45101 ft: 159183 corp: 118/22Mb lim: 514888 exec/s: 0 rss: 611Mb L: 423326/514888 MS: 1 CrossOver-
Segmentation fault (core dumped)
INFO: exiting: 139 time: 540567s
Any ideas? Is the fuzzer itself crashing as it is manipulating the input? As stated, there is no core dump file so have little to go on here.