Crashes

[Akiira]

All three programs have crashed, I'll post the output for one of the crashes at the end of this. It was optimized and without debug symbols so the backtrace is not very helpful. The crash error was this: "*** glibc detected *** ./research1Comp524: double free or corruption (!prev): "

From what I have read there are two likely causes for this. Calling delete or free on something twice or overrunning an array.

The backtrace in gdb lends credence to the overrunning theory. idk if you have noticed it but somtimes the backtrace has ?? symbols or weird address for functions or just far fewer functions then there should be in the backtrace. This is generally cause by the stack getting corrupted, which can be caused by buffer overflows.

I just pushed a commit to help catch additional delete calls. I'm running it in debug mode on my home comp but it could take away to crash, so I still don't know if it helped. If they have not crashed by tomorrow morning I will incororate the change into the version we are running on fermat.

We have several options open to us in terms of getting the data we need and finishing the program

1. Keep restarting crashed tests but save the data it wrote before crashing and start it running at the appropriate spot.
2. Continue debugging and hope we find it, possibly use Valgrind to help find the memory problems. (The only reason I haven't used valgrind yet is that it is linux only.) If we had time this is what I would want to do since I have been wanting to learn about Valgrind.
3. Fill program with asserts around every array access to ensure no over running the array bounds.
4. Convert all arrays to the standard library arrays and all pointers to smart pointers. (I imagine this would definitely solve the problem or make the error easy to find.)
5. Some combination of the above.

At a certain point though, we may have to ask is an 'A' or 'A-' worth what seems to be an exponential amount of work-hours.

Let me know what you think.

           Generation # 400 CoverageRatio: 0.745614
            Generation # 500 CoverageRatio: 0.785088

*** glibc detected *** ./research1Comp524: double free or corruption (!prev): 0x0000000106f97aa0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7db26)[0x7f27b3755b26]
./research1Comp524[0x4129f9]
./research1Comp524[0x413ce7]
./research1Comp524[0x413f97]
./research1Comp524[0x415b1b]
./research1Comp524[0x402708]
./research1Comp524[0x402ce1]
./research1Comp524[0x4019ea]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed)[0x7f27b36f976d]
./research1Comp524[0x401f91]
======= Memory map: ========
00400000-00420000 r-xp 00000000 00:1c 45293714 /home/rvh5220/git/Comp-524-program/Comp 524 Project/src/research1Comp524
0061f000-00620000 r--p 0001f000 00:1c 45293714 /home/rvh5220/git/Comp-524-program/Comp 524 Project/src/research1Comp524
00620000-00621000 rw-p 00020000 00:1c 45293714 /home/rvh5220/git/Comp-524-program/Comp 524 Project/src/research1Comp524
00621000-00622000 rw-p 00000000 00:00 0
01a71000-10935b000 rw-p 00000000 00:00 0 [heap]
7f27b36d8000-7f27b388c000 r-xp 00000000 08:01 34869821 /lib/x86_64-linux-gnu/libc-2.15.so
7f27b388c000-7f27b3a8b000 ---p 001b4000 08:01 34869821 /lib/x86_64-linux-gnu/libc-2.15.so
7f27b3a8b000-7f27b3a8f000 r--p 001b3000 08:01 34869821 /lib/x86_64-linux-gnu/libc-2.15.so
7f27b3a8f000-7f27b3a91000 rw-p 001b7000 08:01 34869821 /lib/x86_64-linux-gnu/libc-2.15.so
7f27b3a91000-7f27b3a96000 rw-p 00000000 00:00 0
7f27b3a96000-7f27b3aac000 r-xp 00000000 08:01 34865174 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f27b3aac000-7f27b3cab000 ---p 00016000 08:01 34865174 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f27b3cab000-7f27b3cac000 r--p 00015000 08:01 34865174 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f27b3cac000-7f27b3cad000 rw-p 00016000 08:01 34865174 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f27b3cad000-7f27b3da8000 r-xp 00000000 08:01 34869772 /lib/x86_64-linux-gnu/libm-2.15.so
7f27b3da8000-7f27b3fa7000 ---p 000fb000 08:01 34869772 /lib/x86_64-linux-gnu/libm-2.15.so
7f27b3fa7000-7f27b3fa8000 r--p 000fa000 08:01 34869772 /lib/x86_64-linux-gnu/libm-2.15.so
7f27b3fa8000-7f27b3fa9000 rw-p 000fb000 08:01 34869772 /lib/x86_64-linux-gnu/libm-2.15.so
7f27b3fa9000-7f27b4096000 r-xp 00000000 08:01 39716397 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.20
7f27b4096000-7f27b4295000 ---p 000ed000 08:01 39716397 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.20
7f27b4295000-7f27b429e000 r--p 000ec000 08:01 39716397 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.20
7f27b429e000-7f27b42a0000 rw-p 000f5000 08:01 39716397 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.20
7f27b42a0000-7f27b42b5000 rw-p 00000000 00:00 0
7f27b42b5000-7f27b42d7000 r-xp 00000000 08:01 34869812 /lib/x86_64-linux-gnu/ld-2.15.so
7f27b4438000-7f27b44bd000 rw-p 00000000 00:00 0
7f27b44d4000-7f27b44d7000 rw-p 00000000 00:00 0
7f27b44d7000-7f27b44d8000 r--p 00022000 08:01 34869812 /lib/x86_64-linux-gnu/ld-2.15.so
7f27b44d8000-7f27b44da000 rw-p 00023000 08:01 34869812 /lib/x86_64-linux-gnu/ld-2.15.so
7ffc1bcf1000-7ffc1bd12000 rw-p 00000000 00:00 0 [stack]
7ffc1bd2a000-7ffc1bd2c000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]

[Akiira]

The ones i was running at home that tried to prevent any double deletes both crashed as well.

[Akiira]

On windows, the program still gives the follow error message:

warning: HEAP[Comp 524 Project.exe]:
warning: Heap block at 00C95360 modified at 00C954F8 past requested size of 190

the bt at this point is:
#0 0x7789082d in ntdll!RtlpNtEnumerateSubKey () from C:\Windows\system32\ntdll.dll
#1 0x77875a22 in ntdll!RtlImageRvaToVa () from C:\Windows\system32\ntdll.dll
#2 0x77852d02 in ntdll!RtlUlonglongByteSwap () from C:\Windows\system32\ntdll.dll
#3 0x00000000 in ?? ()

after calling step twice the bt is this:
#0 0x77852d02 in ntdll!RtlUlonglongByteSwap () from C:\Windows\system32\ntdll.dll
#1 0x00c95360 in ?? ()
#2 0x77891887 in ntdll!RtlpNtEnumerateSubKey () from C:\Windows\system32\ntdll.dll
#3 0x7784ae8a in ntdll!RtlUlonglongByteSwap () from C:\Windows\system32\ntdll.dll
#4 0x00890000 in ?? ()
#5 0x777f2bf6 in ntdll!RtlInterlockedFlushSList () from C:\Windows\system32\ntdll.dll
#6 0x761998cd in msvcrt!free () from C:\Windows\syswow64\msvcrt.dll
#7 0x00890000 in ?? ()
#8 0x00409a98 in Range::~Range (this=0x71c468, __in_chrg=) at ../src/Range.cpp:75
#9 0x00409acb in Range::~Range (this=0x71c468, __in_chrg=) at ../src/Range.cpp:78
#10 0x0040ab6e in RangeSet::deleteRange (this=0x899240, index=4) at ../src/RangeSet.cpp:261
#11 0x0040aa38 in RangeSet::splitRange (this=0x899240, index=4) at ../src/RangeSet.cpp:244
#12 0x0040a882 in RangeSet::adaptRangesBasedOnUsefulness (this=0x899240) at ../src/RangeSet.cpp:217
#13 0x0040bb59 in _fu320___ZSt4cout () at ../src/Simulation.cpp:91
#14 0x00402562 in _fu6___ZSt4cout () at ../src/Comp 524 Project.cpp:210
#15 0x00401b09 in testCutPointsToMutationProb () at ../src/Comp 524 Project.cpp:125
#16 0x004019b8 in runTestsOnAllGraphs () at ../src/Comp 524 Project.cpp:110
#17 0x004016f3 in main () at ../src/Comp 524 Project.cpp:62

[ambarket]

I'm not sure what to say. It seems the range code is most likely the
problem although I guess it could just be bringing out problems else where
though. It seems like its probably the usage buckets. I'm thinking maybe if
we just cut out the usage buckets and replace them with a simple integer
again that might fix it.
On Apr 27, 2015 4:10 PM, "Akiira" [email protected] wrote:

On windows, the program still gives the follow error message:

warning: HEAP[Comp 524 Project.exe]:
warning: Heap block at 00C95360 modified at 00C954F8 past requested size
of 190

the bt at this point is:
#0 0x7789082d in ntdll!RtlpNtEnumerateSubKey () from
C:\Windows\system32\ntdll.dll
#1 #1 0x77875a22 in
ntdll!RtlImageRvaToVa () from C:\Windows\system32\ntdll.dll
#2 #2 0x77852d02 in
ntdll!RtlUlonglongByteSwap () from C:\Windows\system32\ntdll.dll
#3 #3 0x00000000 in ??
()

after calling step twice the bt is this:
#0 0x77852d02 in ntdll!RtlUlonglongByteSwap () from
C:\Windows\system32\ntdll.dll
#1 #1 0x00c95360 in ??
()
#2 #2 0x77891887 in
ntdll!RtlpNtEnumerateSubKey () from C:\Windows\system32\ntdll.dll
#3 #3 0x7784ae8a in
ntdll!RtlUlonglongByteSwap () from C:\Windows\system32\ntdll.dll
#4 #4 0x00890000 in ??
()
#5 #5 0x777f2bf6 in
ntdll!RtlInterlockedFlushSList () from C:\Windows\system32\ntdll.dll
#6 #6 0x761998cd in
msvcrt!free () from C:\Windows\syswow64\msvcrt.dll
#7 #7 0x00890000 in ??
()
#8 #8 0x00409a98 in
Range::~Range (this=0x71c468, __in_chrg=) at ../src/Range.cpp:75
#9 #9 0x00409acb in
Range::~Range (this=0x71c468, __in_chrg=) at ../src/Range.cpp:78
#10 #10 0x0040ab6e in
RangeSet::deleteRange (this=0x899240, index=4) at ../src/RangeSet.cpp:261
#11 #11 0x0040aa38 in
RangeSet::splitRange (this=0x899240, index=4) at ../src/RangeSet.cpp:244
#12 #12 0x0040a882 in
RangeSet::adaptRangesBasedOnUsefulness (this=0x899240) at
../src/RangeSet.cpp:217
#13 #13 0x0040bb59 in
_fu320___ZSt4cout () at ../src/Simulation.cpp:91
#14 #14 0x00402562 in
_fu6___ZSt4cout () at ../src/Comp 524 Project.cpp:210
#15 #15 0x00401b09 in
testCutPointsToMutationProb () at ../src/Comp 524 Project.cpp:125
#16 #16 0x004019b8 in
runTestsOnAllGraphs () at ../src/Comp 524 Project.cpp:110
#17 #17 0x004016f3 in
main () at ../src/Comp 524 Project.cpp:62

—
Reply to this email directly or view it on GitHub
#19 (comment)
.

[Akiira]

I'm not sure if the range set is the problem, I think we may have had some heap corruption errors for a while now but I don't remember if it was before or after range set. I think I may try commenting it out or looking at an older commit to see if it still breaks.

I have to fix my 512 program first though, the demo today went very bad...

I will add more after class.

[ambarket]

I'm using valgrind now and it's finding some things. I'm still getting the
feel for how to use it and I can't determine why some of the errors are
memory leaks. I'll post as I figure out more.
On Apr 27, 2015 7:25 PM, "Akiira" [email protected] wrote:

I'm not sure if the range set is the problem, I think we may have had some
heap corruption errors for a while now but I don't remember if it was
before or after range set. I think I may try commenting it out or looking
at an older commit to see if it still breaks.

I have to fix my 512 program first though, the demo today went very bad...

I will add more after class.

—
Reply to this email directly or view it on GitHub
#19 (comment)
.

[ambarket]

I think I fixed the big problem, as suspected we were in fact overrunning the usesArray in ranges. See the commit I just made for details.

Edit: This has now been merged into master

[ambarket]

Here's the errors I'm still getting with the TestCase() constructor. See my latest commit in the tyingToFixThings branch to see what I've already tried.

==27540== HEAP SUMMARY:
==27540== in use at exit: 2,396 bytes in 98 blocks
==27540== total heap usage: 10,415,430 allocs, 10,415,332 frees, 250,275,798 bytes allocated
==27540==
==27540== 96 (56 direct, 40 indirect) bytes in 1 blocks are definitely lost in loss record 22 of 30
==27540== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27540== by 0x40D3B7: Population::crossover(TestCase const&, TestCase const&, TestCase_&, TestCase_&, int) (Population.cpp:268)
==27540== by 0x4142DB: Simulation::testCaseCrossoverAndMutation(Organism_) (Simulation.cpp:124)
==27540== by 0x4140F6: Simulation::run(int, int, double) (Simulation.cpp:64)
==27540== by 0x4019AC: shortTest() (Comp 524 Project.cpp:65)
==27540== by 0x401AA0: main (Comp 524 Project.cpp:78)
==27540==
==27540== 192 (112 direct, 80 indirect) bytes in 2 blocks are definitely lost in loss record 26 of 30
==27540== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27540== by 0x40D3D3: Population::crossover(TestCase const&, TestCase const&, TestCase_&, TestCase_&, int) (Population.cpp:269)
==27540== by 0x4142DB: Simulation::testCaseCrossoverAndMutation(Organism_) (Simulation.cpp:124)
==27540== by 0x4140F6: Simulation::run(int, int, double) (Simulation.cpp:64)
==27540== by 0x4019AC: shortTest() (Comp 524 Project.cpp:65)
==27540== by 0x401AA0: main (Comp 524 Project.cpp:78)
==27540==
==27540== 192 (112 direct, 80 indirect) bytes in 2 blocks are definitely lost in loss record 27 of 30
==27540== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27540== by 0x4109BF: RangeSet::getNewTestCase() (RangeSet.cpp:70)
==27540== by 0x40BB16: Organism::mutate(double) (Organism.cpp:52)
==27540== by 0x414581: Simulation::testSuiteCrossoverAndMutation() (Simulation.cpp:165)
==27540== by 0x4140BB: Simulation::run(int, int, double) (Simulation.cpp:59)
==27540== by 0x4019AC: shortTest() (Comp 524 Project.cpp:65)
==27540== by 0x401AA0: main (Comp 524 Project.cpp:78)
==27540==
==27540== 192 (112 direct, 80 indirect) bytes in 2 blocks are definitely lost in loss record 28 of 30
==27540== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27540== by 0x4109BF: RangeSet::getNewTestCase() (RangeSet.cpp:70)
==27540== by 0x40BB16: Organism::mutate(double) (Organism.cpp:52)
==27540== by 0x414568: Simulation::testSuiteCrossoverAndMutation() (Simulation.cpp:164)
==27540== by 0x4140BB: Simulation::run(int, int, double) (Simulation.cpp:59)
==27540== by 0x4019AC: shortTest() (Comp 524 Project.cpp:65)
==27540== by 0x401AA0: main (Comp 524 Project.cpp:78)
==27540==
==27540== 384 (224 direct, 160 indirect) bytes in 4 blocks are definitely lost in loss record 29 of 30
==27540== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27540== by 0x4146D1: Simulation::localOptFromZero(TestCase_) (Simulation.cpp:200)
==27540== by 0x414683: Simulation::callRandomLocalOpt(Organism_) (Simulation.cpp:188)
==27540== by 0x4145D8: Simulation::tryLocalOptimization(Organism_) (Simulation.cpp:172)
==27540== by 0x41415A: Simulation::run(int, int, double) (Simulation.cpp:67)
==27540== by 0x4019AC: shortTest() (Comp 524 Project.cpp:65)
==27540== by 0x401AA0: main (Comp 524 Project.cpp:78)
==27540==
==27540== 1,152 (672 direct, 480 indirect) bytes in 12 blocks are definitely lost in loss record 30 of 30
==27540== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27540== by 0x41486A: Simulation::localOptFromGivenParams(TestCase_) (Simulation.cpp:228)
==27540== by 0x41466E: Simulation::callRandomLocalOpt(Organism_) (Simulation.cpp:185)
==27540== by 0x4145D8: Simulation::tryLocalOptimization(Organism_) (Simulation.cpp:172)
==27540== by 0x41415A: Simulation::run(int, int, double) (Simulation.cpp:67)
==27540== by 0x4019AC: shortTest() (Comp 524 Project.cpp:65)
==27540== by 0x401AA0: main (Comp 524 Project.cpp:78)
==27540==
==27540== LEAK SUMMARY:
==27540== definitely lost: 1,288 bytes in 23 blocks
==27540== indirectly lost: 920 bytes in 69 blocks
==27540== possibly lost: 0 bytes in 0 blocks
==27540== still reachable: 188 bytes in 6 blocks
==27540== suppressed: 0 bytes in 0 blocks
==27540== Reachable blocks (those to which a pointer was found) are not shown.
==27540== To see them, rerun with: --leak-check=full --show-leak-kinds=all

[Akiira]

Valgrind is pretty cool, I'll have to keep it in mind next time im debugging c++.