How can I prevent soft interrupts stacking up until schedule queue is full?

[rkompass]

In an attempt to do de-bouncing without asyncio I encountered the following problem:

Soft interrupts elicited by contact bounce of a switch are scrambling the schedule queue, despite the attempt to de-activate the interrupt within the ISR. Code:

from machine import Pin
from micropython import schedule

def pr_imm(pin):
    print(f'{pin} press. Value: {pin()}')

def isr(pin):
    pin.irq(handler=None, trigger=0)
    schedule(pr_imm, pin) # !!!! can give: RuntimeError: schedule queue full

p = Pin(16, Pin.IN, Pin.PULL_UP)

p.irq(handler=isr, trigger=Pin.IRQ_FALLING | Pin.IRQ_RISING, hard=False)
print('started')

This often gives RuntimeError: schedule queue full , once I connect the gpio pin to ground.
Observed on MicroPython v1.22.0-preview.226.g0960d64d3 on 2023-12-06; Raspberry Pi Pico with RP2040.

I decided for a soft interrupt, because I thought this would be less intrusive. Now I see the opposite.
Is this a design flaw (an issue)?
Is a hard interrupt ISR guaranteed to be run just once (including de-activation of the interrupt) before getting called again, possibly pre-empting itself? My current understanding is: it isn't. Therefore I opted for a soft interrupt, only to see that the behaviour is worse in this respect.

[peterhinch]

There are good reasons why using interrupts to debounce mechanical contacts is risky. There is no limit to the rate at which transitions can occur. Further you can get invalid logic levels for periods of time. This can trigger unexpected hardware behaviour. It is instructive to look at the signal on an oscilloscope: they can look pretty vile.

That said, given that you are using hard=False, there is no need to use schedule. The essence of soft IRQ's is that the ISR is run using schedule. Maybe this is putting extra demands on the queue. FWIW my encoder driver uses IRQ's to handle mechanical contacts. It uses hard IRQ's if available, otherwise it falls back to soft. I have not encountered any problems on platforms that only support soft IRQ's.

[GitHubsSilverBullet]

p.irq(handler=isr, trigger=Pin.IRQ_FALLING | Pin.IRQ_RISING, hard=False)

That line already prepares to schedule delayed calls which will run once a full Python line has been completed.
Problem is, it initially does that using hardware interrupts. So a bouncing contact quite likely triggers a whole bunch of hardware IRQs within a few µs, resulting in the schedule stack overflow.

Have you tried using ›hard=True‹ and disable IRQ there?
Not that it would be the best practice…

[GitHubsSilverBullet]

#!/usr/bin/python3
# -*- coding: UTF-8 -*-
# vim:fileencoding=UTF-8:ts=4

from time import sleep_ms
from machine import Pin, PWM, WDT
from micropython import schedule

def pr_imm(pin):
    print(f'{pin} press. Value: {pin()}')

def isr(pin):
    pin.irq(handler=None, trigger=0)
    schedule(pr_imm, pin) # !!!! can give: RuntimeError: schedule queue full

p_in = Pin(16, Pin.IN, Pin.PULL_UP)  # bridge pins 16 and 17
p_out = Pin(17, Pin.OUT)

pwm = PWM(p_out)
pwm.freq(12_500_000)
pwm.duty_u16(0)

wdt = WDT(timeout=1000)

p_in.irq(handler=isr, trigger=Pin.IRQ_FALLING | Pin.IRQ_RISING, hard=True)
print('Start…')
pwm.duty_u16(2**15)
while True:
    sleep_ms(10)

Start…
Pin(GPIO16, mode=IN, pull=PULL_UP) press. Value: 1

Only one IRQ, even @ 12.5MHz input. And that would be a truly bad switch
Works as intended on a MicroPython v1.20.0.

[robert-hh]

Slightly related I made a test once for the pattern of transitions when there is a slow slope at a digital input. For the test a triangle signal was applied to a pin, and a small loop would read that pin and echo the value on another pin. Done with a MIMXRT1020. The green trace is the input signal, the yellow tells what is seen by the code. The lower window is a zoom of the upper one. Since this was done by python code, the bandwidth of that test is limited.

The PWM signal as trigger is surely clean with fast transitions. A switch will have more of the slow transient behavior in addition to bouncing.

[GitHubsSilverBullet]

Agreed, from a signal quality point of view that looks pretty fugly.
An IRQ input deserves a Schmitt-Trigger. Luckily the RP2040 has'em built in.
Tried with 62.5MHz (a level change each 8ns) … AND … reenabling IRQs in the schedule routine. Runs splendid…
irq.py.zip

[rkompass]

Thank you all very much for help and ideas.

I modified your @GitHubsSilverBullet (+my) example like so:

from time import sleep_ms
from machine import Pin, PWM
from micropython import schedule

cnt = 0

def pr_imm(pin):
    global cnt
    cnt += 1
    pin.irq(handler=isr, trigger=Pin.IRQ_FALLING | Pin.IRQ_RISING, hard=True)  # reenable IRQ

def isr(pin):
    pin.irq(handler=None, trigger=0)  # disable IRQ
    schedule(pr_imm, pin)

p_in = Pin(16, Pin.IN, Pin.PULL_UP)  # bridge pins 16 and 17
p_out = Pin(17, Pin.OUT)

pwm = PWM(p_out)
pwm.freq(50_000_000)  # highest value at irq hard=False without schedule queue full: 15_000
pwm.duty_u16(0)

print('Start…')
p_in.irq(handler=isr, trigger=Pin.IRQ_FALLING | Pin.IRQ_RISING, hard=True)
pwm.duty_u16(2**15)
for _ in range(100):
    sleep_ms(20)
    print(cnt)

to learn now that even at very high rates there is no overlap of interrupts if the IRQ option is hard=True.
If the interrupt is not reenabled in pr_imm() then the output stays at 1, even at highest pwm rates. So the disabling of the IRQ before doing the schedule works as desired.
As you told me with IRQ option hard=False there is an internal scheduling of interrupts and here the maximum rate of interrupts observed is 15 kHz. This probably depends on the size/processing load of the ISR which I did not test. Above that rate the RuntimeError: schedule queue full appears at the start of processing.
This makes perfect sense now in agreement with your explanations.

I see now that I can schedule a periodic polling of the button to be debounced with a hard ISR that deactivates itself, without fear of congesting the queues.
Later at long non-pressed button this will be disabled and the ISR will be reactivated.
This functionality is almost finished already. I was just in the testing phase and observed the problem described here.

[peterhinch]

I do think this is doing things the hard way. It's much easier with asyncio and a polling driver.

[rkompass]

Yes, agreed. Your driver is nice. I just stubbornly wanted to see whether my approach was viable at all.
An now it is. It can be seen as filling a little niche, if someone needs de-bouncing and at the same time does not want (for what reason ever) to use asyncio.
And, of course, it is a polling driver too. It uses a single first interrupt only to start the polling, which again is stopped after a longer button release.

Driver (debounce.py):

# Micropython standalone button debounce function
#
# Author: Raul Kompass
# License: The MIT license
# Motivated by Peter Hinch's aswitch.py and Kevin Köck's rework of that.
#
# This is a study and attempt to provide the same functionality without usage of the
#   asyncio framework.
# As further memory savings were tried:
# - Usage of a pin interrupt for the first button press and single timer-induced polling for
#   de-bouncing later.
# - Usage of a function closure for the ISR and polling routine (instead of a full class)
#
# Memory usage could be reduced by this by ca. 50 % (from ~3.2K to ~1.6K) -
#   which is only effective if you do not use asyncio elsewhere in your framework/setup.
# 
# Created 12/2023

from machine import Pin, Timer
from micropython import schedule

# debounce function - only function to be seen from outside
# pin        .. a pin instance
# cb_first   .. (optional) function to be scheduled when the button is pressed the first time
# cb_long    .. (optional) function to be scheduled when the button is pressed a long time
# cb_rel     .. (optional) function to be scheduled when the button is released after a long-time press
# cb_dbl     .. (optional) function to be scheduled when the button is double-pressed
#                       -- all these functions are called with the pin object as single argument --
# chk_period .. period (in ms) of pin value polling
# np         .. 3-tuple of periods corresponding to de-bouncing (i.e. reliable value), short and long pressing
#
def db_init(pin, cb_first=None, cb_long=None, cb_rel=None, cb_dbl=None, chk_period = 5, np=(8,10,80)):
    tim = Timer(-1)  # Soft timer
    v0 = pin()
    val = 0
    v, cnt, ldbl, llng, done, rdone = 0, -1, 0, 0, 0, 0
    def isr(pin):
        pin.irq(handler=None, trigger=0)
        tim.init(mode=Timer.ONE_SHOT, period=chk_period, callback=poll_ref)
    def poll(t):          # this is timer callback
        nonlocal v, val, cnt, ldbl, llng, done, rdone # these vars are kept with the closure
        v = pin() ^ v0
        if cnt < 0:
            schedule(cb_first, pin)
        if v == val:  # no change
            cnt += 1
            if cnt == np[0] and cb_dbl: # count up ldbl, if no bouncing any more
                ldbl += 1
            if not v and cnt == np[0] and llng and cb_rel and not rdone: 
                rdone = 1               # short release: if previous long: --> release action
                schedule(cb_rel, pin)
            if not v and cnt == np[2]:  # long release: reset, switch back to interrupt mode
                cnt, ldbl, llng, done, rdone = -1, 0, 0, 0, 0
                pin.irq(handler=isr, trigger=Pin.IRQ_FALLING | Pin.IRQ_RISING, hard=True)
                return                  #    which implies no new timer oneshot         
            if v and cnt == np[1] and cb_dbl and ldbl >= 3 and not done:    # long press:
                done = 1                # if double then done: no long press later
                schedule(cb_dbl, pin)
            if v and cnt == np[2] and not done:
                llng = 1
                if cb_long:  # .. and long duration
                    done = 1
                    schedule(cb_long, pin)
        else:  # change --> register value, reset count
            val, cnt = v, 0
        tim.init(mode=Timer.ONE_SHOT, period=chk_period, callback=poll_ref)

    poll_ref = poll
    pin.irq(handler=isr, trigger=Pin.IRQ_FALLING | Pin.IRQ_RISING, hard=True)

I still use a lot of schedule()'s to keep the execution of the action functions away from the driver, which does not feel perfect and indicates the need of something like asyncio (:-)()
OTOH 50% memory is saved.

Test script:

# Micropython standalone button debounce function
#
# Test script

from debounce import db_init
import gc
from machine import Pin

gc.collect(); mfree0 = gc.mem_free()

p4 = Pin(4, Pin.IN, Pin.PULL_UP)

def pr_imm(pin):
    print(f'{pin} first press. Value: {pin()}')
def pr_rel(pin):
    print(f'{pin} released. Value: {pin()}')
def pr_dbl(pin):
    print(f'{pin} double pressed. Value: {pin()}')
def pr_settl(pin):
    print(f'{pin} long pressed. Value: {pin()}')
    gc.collect(); mfree1 = gc.mem_free()
    print('Free mem:', mfree1, '. Mem used:', mfree0-mfree1)

db_init(p4, cb_first=pr_imm, cb_long=pr_settl, cb_rel=pr_rel, cb_dbl=pr_dbl)
gc.collect(); mfree1 = gc.mem_free()
print('Free mem:', mfree1, '. debounce used:', mfree0-mfree1)

[peterhinch]

Congratulations on getting this working.

As a very general comment, after 40 years of using cooperative scheduling and callbacks I have belatedly come to the conclusion that callbacks are no longer the best API for drivers. The current MicroPython asyncio provides efficient Event objects and asynchronous iterators. If your drivers use these in their API, users can employ them to run callbacks, schedule tasks, or for any other purpose. Such an API removes issues associated with callbacks such as passing args to them and retrieving return values.

[rkompass]

To add another observation:
The esp8266 suffers the same problem like the other boards: If soft interrupts are triggered very fast, then
the RuntimeError: schedule queue full occurs, even if the interrupt is disabled in the ISR immediately.
I tested this just now with a pwm signal of 25 kHz (generated by another board, the esp8266 only can generate PWM up to 1 kHz).
The schedule queue apparently is only 4 entries long.
With the esp8266 this rare problem cannot be avoided by hard interrupt because there is no hard interrupt.

[MRam03]

[peterhinch]

Please don't post messages like this. It appears to have been written by a generative AI, contains points irrelevant to MicroPython, and seems intended to promote an external organisation.

Basically, spam. I will delete any further instances.

[GitHubsSilverBullet]

Basically, spam. I will delete any further instances.

Please reconsider and delete that one as well. It is of no use at all.

[robert-hh]

Done.