bash 4.4: using $PS0

[d630]

As far I can see, we may avoid the DEBUG trap in version 4.4:

       PS0    
             The value of this parameter is expanded  (see  PROMPTING
              below) and displayed by interactive shells after reading
              a command and before the command is executed.

PS0 is expanded before DEBUG

[rcaloras]

@d630 thanks for opening! Was thinking about this when I saw http://superuser.com/a/1052132 a little while back. Probably makes sense to add some version detection and then hook into the appropriate interface.

[d630]

Ok, I have updated my repo to use PS0 only.

Note:
PSO is meant to be expanded; there is no PROMPT_COMMAND like thing for it. So we have to write something like PSO='$(preexec)'. Then there are the following problems:

• BASH_COMMAND is unusable;
• you can not set any further strings in PSO via preexec (you can set PS1 in precmd);
• the command substitution removes the trailing newline.

[rcaloras]

@d630 awesome. Giving it a look.

Could you elaborate on your second bullet?

• you can not set any further strings in PSO via preexec (you can set PS1 in precmd);

Do you mean you can't alter PS0 itself while it's invoked?

[d630]

That's possible:

foo () { printf '%s\n\r' FOO; }
PSO='\h\n\T$(foo)'

PS0='$(printf "%s\n%s\n%s\n\r" \h \T FOO)'

But not:

foo () { printf '%s\n%s\n%s\n\r' \\h \\T FOO; }
PS0='$(foo)'

PSO='$(PS0=FOO)'

[d630]

Also possible:

foo () { printf '%s\n\r' "${paa[u]}" "${paa[V]}" "${pa[0]}" "$PROMPT_w" "$1"; }
PS0='$(declare -A paa=([u]="\u" [V]="\V"); pa=("\w") ;PROMPT_w="\w" foo "\H")'

[d630]

I would go with something like:

foo ()
{ 
        for i in "${!BPX_PROMPT[@]}"
        do 
            printf '%s -> %s\n\r' "$i" "${BPX_PROMPT[$i]}"
        done
}

PS0='$(
        builtin unset -v BPX_PROMPT;
        builtin unset -f typeset;
        builtin unalias typeset 2>/dev/null;
        typeset -A BPX_PROMPT=(
                [h]="\h"
                [H]="\H"
                [j]="\j"
                [l]="\l"
                [s]="\s"
                [u]="\u"
                [v]="\v"
                [V]="\V"
                [w]="\w"
                [W]="\W"
                [\\!]="\!"
                [#]="\#"
                [\$]="\$"
                [d]="\d"
                [t]="\t"
                [T]="\T"
                [\\@]="\@"
                [A]="\A"

                [unixtime]="\D{%s}"
        );
        foo
)'

What is then:

! -> 1445
# -> 14
$ -> $
unixtime -> 1475483154
@ -> 09:25 AM
A -> 09:25
H -> ICH3
T -> 09:25:54
V -> 4.4.0
W -> ~
d -> Mon Oct 03
h -> ICH3
j -> 0
l -> 0
s -> bash
t -> 09:25:54
u -> user1
v -> 4.4
w -> ~

edit:

In bash 4.4 we can use "parameter transformation". Most of these prompt strings can then be expanded within the command substitution in PS0.

v='\T \W \#'
echo "${v@P}"

> 08:09:59 ~ 70

[raimue]

I doubt PS0 can ever be used for an implementation of preexec(). PS0 is treated as a prompt string, not a list of commands. The only way to inject commands is by substitution with $().

However, command substitutions run in their own subshell, therefore no variables in the original shell can be modified from this context. This would be different behavior than running commands in the trap handler as it is now, from where the state of the shell can be modified. Within PS0, commands can only print data for stdout, or at most write to file descriptors.

[d630]

Thanks for pointing it out again. I think, it's obviously a preprompt, not preexecution thing!

PS0 will be set before the hole command line is beeing executed, which may also be a command list; it's expanded after the list from the line has been read. That is, you have only got access to the hole line (when you have enabled the command history feature). The DEBUG trap applies to every command pipeline from the list, just before their executions.

Even though you might have a similar thing like PROMPT_COMMAND for it, it wouldn't be useful without having access to the commands from the list, which will be executing. So, you also need a second thing like BASH_COMMAND, that indicates, which command is beeing read, right?!

Unfortunately yes, PS0 cannot replace the DEBUG trap.

[rcaloras]

Ahh bummer, was really hoping to move to something more reliable than the DEBUG trap due to #25 Hopes dashed for now :/

[d630]

I have been thinking about the preexec hook for quite a while (again). Maybe I am wrong, but in the end I found, that PS0 cannot replace the DEBUG trap as key to emulate the zsh prexec itself, (since you need command substitution for functions in it) but with the help of Readline key sequences you can! I have just now updated my bpx repo. Check it out and let me know, what do you think!

[tycho-kirchner]

Haven't checked further but one idea could be to send a signal from the PS0-subshell.

test.sh:

counter=0

trap_handler(){
    counter=$((counter+1))
    echo "hi from trap_handler: $counter: $(history 1)" >&2

}


trap trap_handler SIGRTMIN

PS0='$(echo "sending signal... "; kill -SIGRTMIN  $$; )'

Terminal session:

$ source test.sh
$ echo ok
sending signal... hi from trap_handler: 1:  4118  echo ok
ok
$ echo two
sending signal... hi from trap_handler: 2:  4119  echo two
two
$

In the bash manual I don't see a dealbreaker:

If Bash is waiting for a command to complete and receives a signal for which a trap has been set, the trap will not be executed until the command completes. When Bash is waiting for an asynchronous command via the wait builtin, the reception of a signal for which a trap has been set will cause the wait builtin to return immediately with an exit status greater than 128, immediately after which the trap is executed.

[akinomyoga]

The next version of Bash, 5.3 (still under development), supports function substitutions (and value substitutions) of the form ${ command; } (and ${| command; }), which are variants of the command substitution but executed in the main shell.

I guess PS0 can be combined to the value substitutions ${| command; } to provide a more robust preexec in Bash >= 5.3.

[d630]

ah, interesting. similiar to funsubs and valsubs in mksh, right?

[akinomyoga]

Right. Chet indeed mentioned mksh:

https://lists.gnu.org/archive/html/bug-bash/2023-05/msg00078.html

By the way, I wrote above that a value substitution can be used, but I noticed that the shell variable REPLY is rewritten and there is no way to prevent it. I now think function substitution should be used. Something like this:

PS0+='${ __bp_preexec_invoke_from_ps0 >/dev/tty; }'

One possible caveat of this approach is that preexec is executed before the PS0 content is output to the terminal, though I'm not sure if that is really an issue.

[spacelama]

A possible caution: I came across this suggestion and implemented it in my own PROMPT_COMMAND handling (rather than integrating bash-preexec into my own overly complex prompt setup) sometime earlier than 2022, and it only just occurred to me that it was behind some annoying behaviour I've been experiencing:

debian stable bash: 5.2.15(1)-release

        trap 'precmd_hook' SIGRTMIN
        PS0='$(
               kill -SIGRTMIN  $$
            )'

precmd_hook may have had some backgrounded functions within it, or may have called scripts that backgrounded things themselves. disown or double forking was appropriately used. I'm pretty sure the same behaviour was exhibited even if there was nothing ever background invoked.

But job1 & job2 & job3 & would result in them getting job numbers %3, %6, %9, and jobs gave quite ugly output (I worked around it by aliasing jobs=command jobs -r ; command jobs -s:

# we get this as an ugly jobs output after a simple, non-backgrounded
# command exits (backgrounded jobs are worse):
#: 56829,14; sleep 1
#0-0-20:30:16, Mon Dec 23 tconnors@dirac:~/.bashrc.d [master ↑·7|✚ 66…29] (bash)                                       
#: 56830,15; jobs
#[1]   Done                    sleep 1
#[2]   Exit 1                  sleep 1 | sleep 1 | sleep 1 | sleep 1

This was perhaps because the signal handler had executed something in the context of a (backgrounded?) subshell even though it was behaviour as if it was entirely operating within the context of the parent shell. I never solved the issue and ported my bash pre- and post- exec stuff over to using bash-preexec instead, problem free (except for #25).

I'd just suggest testing this before going to any effort to porting over to a signal handler and PS0 triggering, as neat as the idea is.

[akinomyoga]

it only just occurred to me that it was behind some annoying behaviour I've been experiencing:

[...]

But job1 & job2 & job3 & would result in them getting job numbers %3, %6, %9, and jobs gave quite ugly output (I worked around it by aliasing jobs=command jobs -r ; command jobs -s:

Could you provide a concrete setup that reproduces that annoying behavior?

I guess this is one of the behaviors that will be fixed in the next release of Bash 5.3. See the following example. With Bash 5.2,

$ bash-5.2 --norc
$ trap 'for i in 1 2 3; do /bin/true; /bin/true; sleep 1 & done; jobs' RTMIN
$ kill -RTMIN $$
[1]   Done                    kill -RTMIN $$
[2]   Done                    kill -RTMIN $$
[3]   Running                 kill -RTMIN $$ &
[4]   Done                    kill -RTMIN $$
[5]   Done                    kill -RTMIN $$
[6]-  Running                 kill -RTMIN $$ &
[7]   Done                    kill -RTMIN $$
[8]   Done                    kill -RTMIN $$
[9]+  Running                 kill -RTMIN $$ &
$
[3]   Done                    kill -RTMIN $$
[6]-  Done                    kill -RTMIN $$
[9]+  Done                    kill -RTMIN $$
$

This behavior will be fixed in Bash 5.3 as follows:

$ bash-dev --norc
$ trap 'for i in 1 2 3; do /bin/true; /bin/true; sleep 1 & done; jobs' RTMIN
$ kill -RTMIN $$
[1]   Running                    kill -RTMIN $$ &
[2]-  Running                    kill -RTMIN $$ &
[3]+  Running                    kill -RTMIN $$ &
$
[1]   Done                       kill -RTMIN $$
[2]-  Done                       kill -RTMIN $$
[3]+  Done                       kill -RTMIN $$
$

I guess this would be related to what you've described, but I'm not totally sure.

This was perhaps because the signal handler had executed something in the context of a (backgrounded?) subshell even though it was behaviour as if it was entirely operating within the context of the parent shell.

The Bash behavior was based on the Bash maintainer's interpretation of the POSIX wording. The foreground jobs also get its job numbers, but the foreground ones are usually immediately removed from the job table at its termination. However, inside trap handlers, the foreground jobs have never been removed from the job table until the trap handler completes.

I first reported this to Bash with patches in 2022 [1], but it was only partially fixed [2]. I asked the reasoning in [3], but Chet, the Bash maintainer, seemed to have believed that the POSIX standard requires this. I gave up then. This year, I reported another case [4], and Chet still seemed to think the POSIX required this strange behavior. Then, POSIX people chimed in, and this triggered a big discussion about the POSIX interpretation [4,5], and the behavior was finally fixed.

• [1] https://lists.gnu.org/archive/html/bug-bash/2022-07/msg00117.html
• [2] https://lists.gnu.org/archive/html/bug-bash/2022-09/msg00067.html
• [3] https://lists.gnu.org/archive/html/bug-bash/2022-10/threads.html#00003
• [4] https://lists.gnu.org/archive/html/bug-bash/2024-04/threads.html#00180
• [5] https://lists.gnu.org/archive/html/bug-bash/2024-05/threads.html#00000

[spacelama]

Could you provide a concrete setup that reproduces that annoying behavior?

OK, seem to have whittled it down to something pretty "minimal" (ok, smaller than the 30K of files that makes up my .bashrc.d/*). Doesn't exactly reproduce what I was seeing in my full setup, but seems pretty close:

export HISTCONTROL=
set +o histexpand
set -b
trap 'precmd_hook' SIGRTMIN
# function background_processing_job() { ( echo background_processing_job | GREP_COLORS="mt=01;36" grep --color=yes . 1>&2 & ) ; }
# function background_processing_job() { echo background_processing_job | GREP_COLORS="mt=01;36" grep --color=yes . 1>&2 ; }
mkdir -p ~/bin
echo '#!/bin/sh
#(
echo background_processing_job | GREP_COLORS="mt=01;36" grep --color=yes . 1>&2 # &
#)
' > ~/bin/background_processing_job
chmod 755 ~/bin/background_processing_job
function precmd_hook() { background_processing_job ; CMD_START_SECONDS=$SECONDS ; }
function setprompt () { PS1="Here is a prompt: $PWD > " ; }
handleprompt() { 
retcode=$?; 
setprompt; 
return $retcode; 
}
PROMPT_COMMAND='handleprompt'
PS0='$( kill -SIGRTMIN  $$ )'

Followed by for actual testing:

Here is a prompt: /home/tconnors > sleep 1
background_processing_job
Here is a prompt: /home/tconnors > jobs
background_processing_job
[1]   Done                    sleep 1
Here is a prompt: /home/tconnors > sleep 2 &
background_processing_job
[2] 7768
Here is a prompt: /home/tconnors > [2]+  Done                    sleep 2
jobs
background_processing_job
[1]   Done                    sleep 2

etc
(oh that's exciting, isn't it? sleep 2 & got a job number of 2 but jobs after the job was already finished said an unbackgrounded sleep 2 ended with a job number of 1. Cute)

You can try with ~/bin/background_processing_job as a shell script (with or without backgrounding, with or without double forking), or as a shell function (ditto), so slightly worse scenario than yours above.

I have not tested bash 5.3

[akinomyoga]

The provided setup still seems redundant. The minimal reproducer is this:

$ trap '/bin/true' RTMIN; PS0='$(kill -RTMIN  $$)'
$ /bin/true
$ jobs
[1]   Done                    /bin/true
$ /bin/true &
[2] 40067
$ jobs
[2]+  Done                    /bin/true
[3]   Done                    /bin/true

I confirmed that this is fixed in Bash 5.3-beta.