MicroPython CANPico: Is the received CAN messages delayed by the GC?

[beyonlo]

Hello.

Congratulations for the great project supporting CANBUS on Rp2040 using MicroPython!

I would like to use MicroPython on the rp2040/RPico (CANPico) for realtime communication using CANPico firmware, and I would like to know what is the behaviour if CAN received message when Garbage Collector (GC) is running. Will the received/read message delayed by the GC, or the CAN will interrupt the GC to handle the CAN message?

Thank you very much!

[kentindell]

Frame are received by interrupt handler, and the garbage collector is run with interrupts enabled, so they will continue to be received during GC. There is a very large (128 frame) receive FIFO filled by the ISR, so the recv() call can be locked out for some time (several milliseconds) without frames being dropped.

The XIP flash of the RP2040 does cause interrupts to be locked out for some time. You can read more about the priority inversion effects of the XIP in my blog post here:

https://kentindell.github.io/2021/03/05/pico-priority-inversion/

The ISR runs from RAM to minimize the effects of XIP flash. And the MCP2517FD / MCP2518FD CAN controller itself is configured to use a large receive FIFO so even if the ISR is delayed by XIP of lower priority code, frames shouldn't be dropped.

Finally, hardware CAN ID filters can be used to discard unwanted CAN frames so that they don't fill up the buffers.

[beyonlo]

Hello @kentindell

I took a time to read that docs and others to be sure that I will not say something wrong, but maybe still can happen! And want to buy some CANPico boards to do the tests :)

Frame are received by interrupt handler, and the garbage collector is run with interrupts enabled, so they will continue to be received during GC. There is a very large (128 frame) receive FIFO filled by the ISR, so the recv() call can be locked out for some time (several milliseconds) without frames being dropped.

1. Sorry if I not understand very well, but this is still not clear to me, if we are talking about the same thing. I'm talking about if GC will delayed the CAN frame to came to the user source code (mon() - method below - that will to decide what to do, like as act a relay), not about ISR FIFO or the CAN controller FIFO.
   Example: If a frame came to the CAN Transceiver, and at this moment the GC are running, the mon() will be delayed by the GC or the GC will be interrupted to call the mon() method and after that has no more frames, GC will resume/restarted?

I get this mon() example from the CANPico MicroPython SDK Reference Manual page 17:

# Simple CAN bus monitor
def mon(self):
    while True:
         frames = self.c.recv()
         for frame in frames:
              print(frame)
              
from canpico import *
c = CAN()
cp = CANPico(c)
cp.mon()              

2. The Pyboard running MicroPython has a function called CAN.rxcallback() that is a callback when a CAN frame came -just to the user do not need to do a pooling checking if there are new frames. I suppose that the mon() is the equivalent as that callback, am I right? If not, are there something similar to that on the CANPico API?

3. I see in the CANPico Hardware Reference Manual (page 4) that there is two more pins (in addition to 4 pins SPI): the IRQ and SOF. I see that they are very useful for debug/check everything on the scope, but I would like to know if this two pins are necessary for the CANPico firmware/driver runs? I mean, If I to decide to create my own board using RP2040 to use with the CANPico, is possible to use just the SPI pins, without to use that two more pins? I'm afraid that CANPico firmware/driver can not works if do not have that IRQ and SOF pins.

Thank you in advance!

[kentindell]

Yes, it's possible that GC delays the Python code. Also the time taken to print to the REPL console is large.

But the drivers created a very large receive buffer so that if the GC time is not long then frames should not be dropped. There are also status indicators that can be included in the FIFO to indicate if frames have been dropped (and how many).

The IRQ pin from the CAN controller is necessary for the drivers to operate. If you want to debug using a logic analyzer then you can use the trigger pin included for this purpose.

The SOF pin is not necessary: the drivers do not use the pin.

[beyonlo]

Hello @kentindell

Understood! That print(frame) on the mon() method was just an example, in the real world that will to do something else in real-time, like as act a Digital Output, write in the DAC, and so on - so, without waiting the large time used by the print().

I understand very well about the large FIFO on the ISR and the CAN Controller - that is great to frames do not be dropped! I was just thinking in a scenario where has few frames, so the dropped frames is no the point, but if that frames can be handle on Python code with less than 200 us, or even sub-millisecond precision. Unfortunately for this scenario is no possible because rp2040 running in the default clock (125Mhz) the gc.collect() time is around 4200 us. Changing to maximum clock (270Mhz) the gc.collect() time is around 2000 us. So in the best scenario the CAN frame will be delayed by 2000 us, and that time unfortunately is not acceptable for some applications.

However, let me please suggest you a work around that maybe can works, but just you can tell me if is possible to implement on the CANPico. I imagine that what I will write you already know, but I just want to trying to figure out if exist some alternative. If I'm not wrong, the rp2040 support the hardware external interrupt. When external IRQ is configured as hard=True, that IRQ will interrupt the GC (if it is running), so is possible to handle on Python code that interrupt, and at this moment should be possible read the CAN frame and execute what need to do without delayed by GC. After that Python code on IRQ is finished, the GC is resumed. If this make sense for you, maybe that pin IRQ that already exists on the CANPico can be used for this purpose as well.

If it's not too much to ask, I'd appreciate it if you could answer my question number 2 above.

Thank you very much!

[kentindell]

I see what you're trying to do now. Yes, GC is way too slow.

You could insert a callback into the current ISR so that it will call into your code with the parameters of the received CAN frame. It could possibly even run on the second CPU core, while the ISR ran to completion (perhaps discarding the frame if the callback returned a certain value).

You could code this callback either in Python or in C. If you want to use Python then it has to be carefully coded not use dynamic memory allocation, of course, along the lines you outlined. It could even use the Python threading support to start a worker thread on the second CPU core.

It would require a custom firmware build but if the idea works out we could add the support in to the main firmware.

[beyonlo]

Hi @kentindell

This feature will be amazing for the real-time execution task via CAN :) Maybe less than 200 us?

My knowledge about C is very basic and I never builded by myself a MicroPython for the RPico - I always get the ready binary from MicroPython website. But I want to learn how to build because my intention is to apply the CANPico MicroPython patch to the original MicroPython. Anyway, unfortunately I have no knowledge to do modifications on the CANPico firmware to check if that idea really works.

perhaps discarding the frame if the callback returned a certain value

I think this idea is excellent, because that can help to take out CAN frames from the FIFO and do not need to process the same CAN frame!

If you really think to implement this feature, my suggestion is it to be configurable by a Python set_something() if user want, yes or no, this feature running.

Let me know if you would like to open a new issue with subject "Feature request..." for this new feature - I can do that :)

Thank you very much for your attention! This is a great project!