Build errors not displayed automatically when using colorgcc

[dornhege]

System Info

• Operating System: Trusty
• Python Version: 2.7.6
• Version of catkin_tools: 0.4.2
• ROS Distro: indigo

Build / Run Issue

Whenever there is a compiler error (g++) with the current version of catkin tools it is not displayed, i.e., I get:

Starting  >>> test_pkg                                                                                 
_______________________________________________________________________________________________________
Errors     << test_pkg:make ...rosws/test/logs/test_pkg/build.make.002.log                 
make[2]: *** [CMakeFiles/test_pkg_node.dir/src/main.cpp.o] Error 1
make[1]: *** [CMakeFiles/test_pkg_node.dir/all] Error 2
make: *** [all] Error 2
cd ...rosws/test/build/test_pkg; catkin build --get-env test_pkg | catkin env -si  /usr/bin/make --jobserver-fds=6,7 -j; cd -
.......................................................................................................

The listed log file correctly contains the build errors.

rosws/test/src/test_pkg/src/main.cpp:5:2: error: #error Test
 #error Test
  ^
make[2]: *** [CMakeFiles/test_pkg_node.dir/src/main.cpp.o] Error 1
make[1]: *** [CMakeFiles/test_pkg_node.dir/all] Error 2
make: *** [all] Error 2

catkin build -v also shows the errors directly together with all the other verbose-ness.

I believe that not showing errors or warnings is not the intended behavior and these should show in the build output.

[jbohren]

@dornhege Can you give a little more info about your shell and terminal emulator?

This works for me, building the following package on 14.04 / 0.4.2:
https://github.com/jbohren/test_pkg

Asciinema here:
https://asciinema.org/a/2bocgugi2qcuzy6b6r3ai8032

[jbohren]

@dornhege Can you also post your CMake and GCC versions?

[dornhege]

I have gnome-terminal (3.6.2), with bash 4.3.11. CMake 2.8.12.2 and G++ 4.8.4 (ran through colorgcc).

The test_pkg that I used is in principle identical and I would have expected your output with the compiler errors displayed and not only make.

[jbohren]

I didn't even think to ask about colorgcc, but that's your problem. colorgcc combines stderr and stdout into stdout, and catkin build's output controller only prints stderr unless it's in verbose mode.

[dornhege]

That's good to know. Is there any chance to fix that (even for now locally in a hacky way)?

[jbohren]

That's good to know. Is there any chance to fix that (even for now locally in a hacky way)?

Other not using colorgcc or operating in verbose mode? We could add an option to catkin build which dumps all of stdout and stderr if a package fails, but not very soon.

[jbohren]

@dornhege Whatever, I implemented it anyway. Try #362

catkin build --verbose-errors

[wjwwood]

Hmm I didn't know that. In my opinion I think the entire build out for the stage (make or cmake) should be printed when there is an error. It's often the case that more context is required when trying to figure out the cause of the issue. On the other hand I don't think there is a significant benefit to limiting the output when there is an error.

Another thing to consider is that using colorgcc will probably prevent catkin build from capturing your warnings from gcc. It might be better for us to have a use colorgcc option in catkin build so we can apply the output of gcc to it after capturing the streams separately.

[jbohren]

Hmm I didn't know that. In my opinion I think the entire build out for the stage (make or cmake) should be printed when there is an error. It's often the case that more context is required when trying to figure out the cause of the issue. On the other hand I don't think there is a significant benefit to limiting the output when there is an error.

I'm fine either way.

Another thing to consider is that using colorgcc will probably prevent catkin build from capturing your warnings from gcc. It might be better for us to have a use colorgcc option in catkin build so we can apply the output of gcc to it after capturing the streams separately.

The problem is that some people just symlink colorgcc to gcc. I don't think we should be trying to call it exactly. Instead, we should probably just grep stdout for the words error and warning.

[wjwwood]

The problem is that some people just symlink colorgcc to gcc. I don't think we should be trying to call it exactly. Instead, we should probably just grep stdout for the words error and warning.

Yeah, but then it's hard include the right amount of context and there's always the risk of false positives.

[wjwwood]

We could potentially set the value of TERM when using catkin_tools to something that will prevent colorgcc from colorizing (and therefore combining stderr and stdout):

https://github.com/colorgcc/colorgcc/blob/master/colorgccrc.txt#L39

From https://github.com/colorgcc/colorgcc/blob/master/README:

colorgcc will only emit color codes if:

(1) Its STDOUT is a tty and
(2) the value of $TERM is not listed in the "nocolor" option.

If colorgcc colorizes the output, the compiler's STDERR will be
combined with STDOUT. Otherwise, colorgcc just passes the output from
the compiler through without modification.

Of course that means that there will be no colorizing the output of gcc when used with catkin_tools.

[wjwwood]

Also, I think starting in gcc 4.9 colorized output is built into gcc, so this might be only a temporary issue:

https://gcc.gnu.org/gcc-4.9/changes.html (look for -fdiagnostics-color=always)

[jbohren]

We could potentially set the value of TERM when using catkin_tools to something that will prevent colorgcc from colorizing (and therefore combining stderr and stdout):

I think that will probably break all colors, since other tools also rely on this variable.

Also, I think starting in gcc 4.9 colorized output is built into gcc, so this might be only a temporary issue:

Ah, that's nice.

[wjwwood]

I think that will probably break all colors, since other tools also rely on this variable.

You're probably right... Though at least some tools only care about whether or not stdout is a tty and don't consider TERM (though perhaps they should).

[dornhege]

Thanks, #362 works!

The symlink setup is actually, what I have. Removing colorization in general is not a good option for me.

I've tried warnings, and yes, they don't appear. For now, I added the same output dump if "warn" appears in the output. This works, but of course isn't perfect. I'm not sure if you want to add something like that, but warnings don't even appear in the summary (i.e. Warnings: None).

Regarding gcc 4.9: I'm not sure if that will be coming for trusty, which might need to be considered.

[jbohren]

For now, I added the same output dump if "warn" appears in the output.

Yeah, I think this is the best way to go. We can search for narrower patterns like [Ww]arning: to avoid false positives, and also make an option such as --warn-patterns. It might also be good to have a user-supplied set of patterns / filters / events which can go into .config/catkin.