Segmentation fault during ctest in local build

[JiaoMaWHU]

When compiling mspass locally based on this guide, I was able to finish

cmake ..
make
make install

and also installing the python package

pip3 install ./ -v

However, when I ran ctest (this command is included in the cmake workflow file), the following error happened,

ctest --rerun-failed --output-on-failure                      
Test project /Users/jiaoma/gitCode/mspass/cxx/build
    Start 6: test_bundle
1/1 Test #6: test_bundle ......................***Exception: SegFault  0.07 sec
Starting test_bundle program
Input ensemble summary
net sta chan loc npts fill_value
AA ABC HHN 00 1
AA DEF HHN 00 2
AA BBLONG HHN 00 3
AA ABC HHN EMPTY 4
AA DEF HHN EMPTY 5
AA BBLONG HHN EMPTY 6
XY ABC HHN 00 7
XY DEF HHN 00 8
XY BBLONG HHN 00 9
XY ABC HHN EMPTY 10
XY DEF HHN EMPTY 11
XY BBLONG HHN EMPTY 12
AA ABC HHZ 00 13
AA DEF HHZ 00 14
AA BBLONG HHZ 00 15
AA ABC HHZ EMPTY 16
AA DEF HHZ EMPTY 17
AA BBLONG HHZ EMPTY 18
XY ABC HHZ 00 19
XY DEF HHZ 00 20
XY BBLONG HHZ 00 21
XY ABC HHZ EMPTY 22
XY DEF HHZ EMPTY 23
XY BBLONG HHZ EMPTY 24
AA ABC HHE 00 25
AA DEF HHE 00 26
AA BBLONG HHE 00 27
AA ABC HHE EMPTY 28
AA DEF HHE EMPTY 29
AA BBLONG HHE EMPTY 30
XY ABC HHE 00 31
XY DEF HHE 00 32
XY BBLONG HHE 00 33
XY ABC HHE EMPTY 34
XY DEF HHE EMPTY 35
XY BBLONG HHE EMPTY 36
Running best case test for bundle_seed_data
net sta chan loc npts fill_values
XY DEF 00 20 8 32 
XY DEF EMPTY 23 11 35 
XY BBLONG 00 21 9 33 
XY BBLONG EMPTY 24 12 36 
XY ABC 00 19 7 31 
XY ABC EMPTY 22 10 34 
AA DEF 00 14 2 26 
AA DEF EMPTY 17 5 29 
AA BBLONG 00 15 3 27 
AA BBLONG EMPTY 18 6 30 
AA ABC 00 13 1 25 
AA ABC EMPTY 16 4 28 
Trying an ensemble with some net values undefined
net sta chan loc npts fill_values
XY DEF 00 20 8 32 
XY DEF EMPTY 23 11 35 
XY BBLONG 00 21 9 33 
XY BBLONG EMPTY 24 12 36 
XY ABC 00 19 7 31 
XY ABC EMPTY 22 10 34 
Undefined DEF 00 14 2 26 
Undefined DEF EMPTY 17 5 29 
Undefined BBLONG 00 15 3 27 
Undefined BBLONG EMPTY 18 6 30 
Undefined ABC 00 13 1 25 
Undefined ABC EMPTY 16 4 28 
Trying similar test with some loc values undefined
net sta chan loc npts fill_values
XY DEF EMPTY 23 11 35 
XY DEF Undefined 20 8 32 
XY BBLONG EMPTY 24 12 36 
XY BBLONG Undefined 21 9 33 
XY ABC EMPTY 22 10 34 
XY ABC Undefined 19 7 31 
AA DEF EMPTY 17 5 29 
AA DEF Undefined 14 2 26 
AA BBLONG EMPTY 18 6 30 
AA BBLONG Undefined 15 3 27 
AA ABC EMPTY 16 4 28 
AA ABC Undefined 13 1 25 
Test handling of pure duplicates
net sta chan loc npts fill_values
XY DEF 00 20 8 32 
XY DEF EMPTY 23 11 35 
XY BBLONG 00 33 9 21 
XY BBLONG EMPTY 24 12 36 
XY ABC 00 19 7 31 
XY ABC EMPTY 22 10 34 
AA DEF 00 14 2 26 
AA DEF EMPTY 29 5 17 
AA BBLONG 00 15 3 27 
AA BBLONG EMPTY 18 6 30 
AA ABC 00 13 1 25 
AA ABC EMPTY 16 4 28 
Test handling of incomplete bundles


0% tests passed, 1 tests failed out of 1

Total Test time (real) =   0.08 sec

The following tests FAILED:
          6 - test_bundle (SEGFAULT)
Errors while running CTest

Is this expected? Would it be related to the memory leak issue we discussed? 🤔

[pavlis]

Unlikely to be a memory leak issue. A SEG fault nearly always indicates a stray index or pointer in some loop. I'll have a look at this. I'm going to be fixing a working on a different issue to address #511 and it may need to edit the C test suite at that time anyway

[JiaoMaWHU]

Unlikely to be a memory leak issue. A SEG fault nearly always indicates a stray index or pointer in some loop. I'll have a look at this. I'm going to be fixing a working on a different issue to address #511 and it may need to edit the C test suite at that time anyway

thank you!

[pavlis]

Hmm - I'm unable to reproduce that error. I didn't, however, run it through ctest but ran a local version I had just compiled in debug mode. Did that to address #511 already so thought I could find this potential bug easily that way. It is conceivable that building it in debug mode worked around what caused this problem. I've seen that more than once in the past. There could also be system or compiler difference - one of the motivations for using a docker container.

For the record here are relevant configurations for the system I tried this on:

(base) pavlis@pavlis-ubuntu:~/src/mspass/cxx/build/test/bundle$ gcc --version
gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

(base) pavlis@pavlis-ubuntu:~/src/mspass/cxx/build/test/bundle$ uname -a
Linux pavlis-ubuntu 6.5.0-18-generic #18~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Feb  7 11:40:03 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

I have no idea how to proceed to try to resolve this other than see if it crops up again.

[JiaoMaWHU]

thanks for working on this. It probably is related to my dev environment then. I will skip it for now and circle back when it's actually blocking something.