Add Python version of gen_northbound_callbacks #3

choppsv1 · 2025-06-30T19:46:53Z

Summary

add a Python rewrite of the gen_northbound_callbacks tool

Testing

make check (fails: No rule to make target 'check')

https://chatgpt.com/codex/tasks/task_b_6862e4071614832fb7331a77b928b854

Signed-off-by: Christian Hopps <chopps@labn.net>

A crash is detected on an invalid memory access to the 0x0 address zone. > #0 __pthread_kill_implementation (no_tid=0, signo=11, threadid=130889386464320) > at ./nptl/pthread_kill.c:44 > #1 __pthread_kill_internal (signo=11, threadid=130889386464320) at ./nptl/pthread_kill.c:78 > #2 __GI___pthread_kill (threadid=130889386464320, signo=signo@entry=11) at ./nptl/pthread_kill.c:89 > #3 0x0000770b0f042476 in __GI_raise (sig=11) at ../sysdeps/posix/raise.c:26 > #4 0x0000770b0f507846 in core_handler (signo=11, siginfo=0x7ffd4f7ec9f0, context=0x7ffd4f7ec8c0) > at /build/make-pkg/output/_packages/cp-routing/src/lib/sigevent.c:262 > #5 <signal handler called> > FRRouting#6 __memmove_evex_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:339 > FRRouting#7 0x0000770b0f50bb54 in sockunion_set (su=0x7ffd4f7ed7b0, family=2, addr=0x0, bytes=4) > at /build/make-pkg/output/_packages/cp-routing/src/lib/sockunion.c:500 > FRRouting#8 0x00005f75d5430817 in nhrp_cie_pull (zb=0x5f75f262c4d0, hdr=0x5f75f2627dd8, nbma=0x7ffd4f7ed6d0, > proto=0x7ffd4f7ed7b0) at /build/make-pkg/output/_packages/cp-routing/src/nhrpd/nhrp_packet.c:180 > FRRouting#9 0x00005f75d5434652 in nhrp_peer_forward (p=0x5f75f2605f30, pp=0x7ffd4f7ed8c0) > at /build/make-pkg/output/_packages/cp-routing/src/nhrpd/nhrp_peer.c:1050 > FRRouting#10 0x00005f75d54356cb in nhrp_peer_recv (p=0x5f75f2605f30, zb=0x5f75f2627da0) > at /build/make-pkg/output/_packages/cp-routing/src/nhrpd/nhrp_peer.c:1341 > FRRouting#11 0x00005f75d5430d8e in nhrp_packet_recvraw (t=0x7ffd4f7ede80) > at /build/make-pkg/output/_packages/cp-routing/src/nhrpd/nhrp_packet.c:332 > FRRouting#12 0x0000770b0f521188 in thread_call (thread=0x7ffd4f7ede80) > at /build/make-pkg/output/_packages/cp-routing/src/lib/thread.c:1825 > FRRouting#13 0x0000770b0f4b7737 in frr_run (master=0x5f75f2440570) > at /build/make-pkg/output/_packages/cp-routing/src/lib/libfrr.c:1155 > FRRouting#14 0x00005f75d542d2b4 in main (argc=3, argv=0x7ffd4f7ee0b8) > at /build/make-pkg/output/_packages/cp-routing/src/nhrpd/nhrp_main.c:317 The incoming nhrp packet is too short, and the call to sockunion_set() uses a 0x0 memory zone, because the whole nhrp packet has been parsed, and the zbuf length used was 0. Fix this by detecting the zbuf remaining length before calling sockunion_set. Signed-off-by: Philippe Guibert <philippe.guibert@6wind.com> (cherry picked from commit 30e479e)

issue detected by Address Sanitizer Error : Address Sanitizer Error detected in /tmp_topotests/bgp_listen_l3vrf.test_bgp_listen_l3vrf/r1.asan.bgpd.6703 ================================================================= ==6703==ERROR: LeakSanitizer: detected memory leaks Direct leak of 56 byte(s) in 1 object(s) allocated from: #0 0x7f34c28b83b7 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:77 #1 0x7f34c241b45a in qcalloc lib/memory.c:111 #2 0x7f34c247b1da in prefix_new lib/prefix.c:1192 #3 0x55e0992e2041 in peer_group_listen_range_add bgpd/bgpd.c:3258 #4 0x55e099282694 in bgp_listen_range bgpd/bgp_vty.c:4848 #5 0x7f34c2397bc0 in cmd_execute_command_real lib/command.c:1011 FRRouting#6 0x7f34c2397edf in cmd_execute_command lib/command.c:1070 FRRouting#7 0x7f34c239840b in cmd_execute lib/command.c:1236 FRRouting#8 0x7f34c24e204e in vty_command lib/vty.c:626 FRRouting#9 0x7f34c24e259b in vty_execute lib/vty.c:1389 FRRouting#10 0x7f34c24e5f97 in vtysh_read lib/vty.c:2408 FRRouting#11 0x7f34c24d2958 in event_call lib/event.c:2005 FRRouting#12 0x7f34c23fc4e0 in frr_run lib/libfrr.c:1247 FRRouting#13 0x55e0990949ff in main bgpd/bgp_main.c:565 FRRouting#14 0x7f34c1e2c249 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 SUMMARY: AddressSanitizer: 56 byte(s) leaked in 1 allocation(s). *********************************************************************************** Signed-off-by: Francois Dumontet <francois.dumontet@6wind.com>

Memory leak happens when modifying srv6 configuration. Some sid notifications events will flush a valid srv6 context. > Direct leak of 736 byte(s) in 2 object(s) allocated from: > #0 0x7c112c0fd340 in calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:77 > #1 0x7c112bc3595e in qcalloc lib/memory.c:111 > #2 0x7c112bc4d37e in srv6_locator_alloc lib/srv6.c:227 > #3 0x5f9fa87e7acf in bgp_zebra_srv6_sid_notify bgpd/bgp_zebra.c:3662 > #4 0x7c112bd32d12 in zclient_read lib/zclient.c:4804 > #5 0x7c112bcfaa90 in event_call lib/event.c:2005 > FRRouting#6 0x7c112bc118a7 in frr_run lib/libfrr.c:1252 > FRRouting#7 0x5f9fa85674b4 in main bgpd/bgp_main.c:565 > FRRouting#8 0x7c112b42a1c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 > FRRouting#9 0x7c112b42a28a in __libc_start_main_impl ../csu/libc-start.c:360 > FRRouting#10 0x5f9fa856cbd4 in _start (/usr/lib/frr/bgpd+0x2d5bd4) (BuildId: 16288c5292cf235ab5251a93b7dbae5874c3f4bc) > > Indirect leak of 80 byte(s) in 2 object(s) allocated from: > #0 0x7c112c0fd340 in calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:77 > #1 0x7c112bc3595e in qcalloc lib/memory.c:111 > #2 0x7c112bc4d396 in srv6_locator_alloc lib/srv6.c:229 > #3 0x5f9fa87e7acf in bgp_zebra_srv6_sid_notify bgpd/bgp_zebra.c:3662 > #4 0x7c112bd32d12 in zclient_read lib/zclient.c:4804 > #5 0x7c112bcfaa90 in event_call lib/event.c:2005 > FRRouting#6 0x7c112bc118a7 in frr_run lib/libfrr.c:1252 > FRRouting#7 0x5f9fa85674b4 in main bgpd/bgp_main.c:565 > FRRouting#8 0x7c112b42a1c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 > FRRouting#9 0x7c112b42a28a in __libc_start_main_impl ../csu/libc-start.c:360 > FRRouting#10 0x5f9fa856cbd4 in _start (/usr/lib/frr/bgpd+0x2d5bd4) (BuildId: 16288c5292cf235ab5251a93b7dbae5874c3f4bc) Fixes: 7a2e64e ("bgpd: Receive SRv6 SIDs notification from zebra") Signed-off-by: Philippe Guibert <philippe.guibert@6wind.com>

When SRv6 locator is modified for configuration, a memory leak is observed. > ==26714==ERROR: LeakSanitizer: detected memory leaks > > Direct leak of 1104 byte(s) in 3 object(s) allocated from: > #0 0x7fb232cb83b7 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:77 > #1 0x7fb232822b79 in qcalloc lib/memory.c:111 > #2 0x7fb23283a8b6 in srv6_locator_alloc lib/srv6.c:227 > #3 0x56347cdd4b57 in bgp_zebra_srv6_sid_notify bgpd/bgp_zebra.c:3661 > #4 0x7fb23290d03e in zclient_read lib/zclient.c:4804 > #5 0x7fb2328da6a0 in event_call lib/event.c:2005 > FRRouting#6 0x7fb232800791 in frr_run lib/libfrr.c:1252 > FRRouting#7 0x56347cb929ff in main bgpd/bgp_main.c:565 > FRRouting#8 0x7fb23222c249 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 Signed-off-by: Philippe Guibert <philippe.guibert@6wind.com>

The `match->rule_str` may is NULL, like: ``` ip prefix-list plist1 deny any route-map rm1 deny 10 match evpn default-route ``` The stack: ``` #0 __strcmp_sse42 () at ../sysdeps/x86_64/multiarch/strcmp-sse4_2.S:173 #1 0x00007ffff7e5a7ea in route_map_pentry_process_dependency ( bucket=0x5555561fb270, data=0x7fffffff96e0) at ../lib/routemap.c:2466 #2 0x00007ffff7de983d in hash_iterate (hash=0x555556208e50, func=0x7ffff7e5a6f3 <route_map_pentry_process_dependency>, arg=0x7fffffff96e0) at ../lib/hash.c:252 #3 0x00007ffff7e5a99d in route_map_notify_pentry_dependencies ( affected_name=0x5555561fb720 "plist1", pentry=0x555556201040, event=RMAP_EVENT_PLIST_ADDED) at ../lib/routemap.c:2513 #4 0x00007ffff7e4a275 in prefix_list_entry_update_finish (ple=0x555556201040) at ../lib/plist.c:697 #5 0x00007ffff7de38c9 in lib_prefix_list_entry_apply_finish (args=0x7fffffff97b0) at ../lib/filter_nb.c:1233 FRRouting#6 0x00007ffff7e3228a in nb_callback_apply_finish (context=0x555556204970, nb_node=0x555555b51860, dnode=0x5555561e47b0, errmsg=0x7fffffff9d00 "", errmsg_len=8192) at ../lib/northbound.c:1772 ``` Signed-off-by: anlan_cs <anlan_cs@126.com>

The `match->rule_str` may is NULL, like: ``` ip prefix-list plist1 deny any route-map rm1 deny 10 match evpn default-route ``` The stack: ``` #0 __strcmp_sse42 () at ../sysdeps/x86_64/multiarch/strcmp-sse4_2.S:173 #1 0x00007ffff7e5a7ea in route_map_pentry_process_dependency ( bucket=0x5555561fb270, data=0x7fffffff96e0) at ../lib/routemap.c:2466 #2 0x00007ffff7de983d in hash_iterate (hash=0x555556208e50, func=0x7ffff7e5a6f3 <route_map_pentry_process_dependency>, arg=0x7fffffff96e0) at ../lib/hash.c:252 #3 0x00007ffff7e5a99d in route_map_notify_pentry_dependencies ( affected_name=0x5555561fb720 "plist1", pentry=0x555556201040, event=RMAP_EVENT_PLIST_ADDED) at ../lib/routemap.c:2513 #4 0x00007ffff7e4a275 in prefix_list_entry_update_finish (ple=0x555556201040) at ../lib/plist.c:697 #5 0x00007ffff7de38c9 in lib_prefix_list_entry_apply_finish (args=0x7fffffff97b0) at ../lib/filter_nb.c:1233 FRRouting#6 0x00007ffff7e3228a in nb_callback_apply_finish (context=0x555556204970, nb_node=0x555555b51860, dnode=0x5555561e47b0, errmsg=0x7fffffff9d00 "", errmsg_len=8192) at ../lib/northbound.c:1772 ``` Signed-off-by: anlan_cs <anlan_cs@126.com> (cherry picked from commit fa67f51)

Problem 1: 1. when s_client->gr_instance_count > 0 the code removed info from gr_info_queue and returned without freeing it. Fix: We now free info on that early return, so that leak is closed. Problem 2. During shutdown of zebra, stale clients are scheduled for deletion in META_QUEUE_GR. But before the META_QUEUE_GR is processed, zebra shuts down as a result there's a leak Fix: Implemented synchronous free on shutdown path. Leak in both cases: Indirect leak of 72 byte(s) in 1 object(s) allocated from: #0 0x7f48922b83b7 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:77 #1 0x7f4891e23c0d in qcalloc lib/memory.c:111 #2 0x55602360e3ac in zebra_gr_client_info_create zebra/zebra_gr.c:101 #3 0x55602360e3ac in zread_client_capabilities zebra/zebra_gr.c:359 #4 0x5560235f2ead in zserv_handle_commands zebra/zapi_msg.c:4226 #5 0x556023719ef1 in zserv_process_messages zebra/zserv.c:561 FRRouting#6 0x7f4891edbc17 in event_call lib/event.c:2009 FRRouting#7 0x7f4891e017d9 in frr_run lib/libfrr.c:1252 FRRouting#8 0x5560235a63eb in main zebra/main.c:552 FRRouting#9 0x7f489190c249 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 Signed-off-by: Pooja Jagadeesh Doijode <pdoijode@nvidia.com>

bgp_flowspec.test_bgp_flowspec_topo started to fail (crash) after this. Let's revert it for now. It's freed a bit above already: hash_release(bpm->entry_hash, bpme); bgp_pbr_match_entry_free(bpme); ERROR: AddressSanitizer: attempting to call malloc_usable_size() for pointer which is not owned: 0x60e00009f8a0 #0 0x7f27d6cb7f04 in __interceptor_malloc_usable_size ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:119 #1 0x7f27d6c264f6 in __sanitizer::BufferedStackTrace::Unwind(unsigned long, unsigned long, void*, bool, unsigned int) ../../../../src/libsanitizer/sanitizer_common/sanitizer_stacktrace.h:131 #2 0x7f27d6c264f6 in __asan::asan_malloc_usable_size(void const*, unsigned long, unsigned long) ../../../../src/libsanitizer/asan/asan_allocator.cpp:1058 #3 0x7f27d68254df in mt_count_free lib/memory.c:83 #4 0x7f27d68254df in qfree lib/memory.c:135 #5 0x5637d57b04a2 in bgp_pbr_match_entry_free bgpd/bgp_pbr.c:977 FRRouting#6 0x5637d57b04a2 in bgp_pbr_flush_entry bgpd/bgp_pbr.c:1737 FRRouting#7 0x5637d57b40be in bgp_pbr_policyroute_remove_from_zebra_unit bgpd/bgp_pbr.c:1980 FRRouting#8 0x5637d57bb7c0 in bgp_pbr_policyroute_remove_from_zebra bgpd/bgp_pbr.c:2144 FRRouting#9 0x5637d57bb7c0 in bgp_pbr_handle_entry bgpd/bgp_pbr.c:2781 FRRouting#10 0x5637d57bb7c0 in bgp_pbr_update_entry bgpd/bgp_pbr.c:2905 FRRouting#11 0x5637d58d23e1 in bgp_zebra_withdraw_actual bgpd/bgp_zebra.c:1733 FRRouting#12 0x5637d57ccc9e in bgp_cleanup_table bgpd/bgp_route.c:7300 FRRouting#13 0x5637d57e27d2 in bgp_cleanup_routes bgpd/bgp_route.c:7318 FRRouting#14 0x5637d5911b91 in bgp_delete bgpd/bgpd.c:4370 FRRouting#15 0x5637d56961b4 in bgp_exit bgpd/bgp_main.c:212 FRRouting#16 0x5637d56961b4 in sigint bgpd/bgp_main.c:162 FRRouting#17 0x7f27d68af501 in frr_sigevent_process lib/sigevent.c:117 FRRouting#18 0x7f27d68db77a in event_fetch lib/event.c:1742 FRRouting#19 0x7f27d68027e4 in frr_run lib/libfrr.c:1251 FRRouting#20 0x5637d5697c55 in main bgpd/bgp_main.c:569 FRRouting#21 0x7f27d630c249 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 FRRouting#22 0x7f27d630c304 in __libc_start_main_impl ../csu/libc-start.c:360 FRRouting#23 0x5637d5695ac0 in _start (/usr/lib/frr/bgpd+0x2cfac0) 0x60e00009f8a0 is located 0 bytes inside of 160-byte region [0x60e00009f8a0,0x60e00009f940) freed by thread T0 here: #0 0x7f27d6cb76a8 in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:52 #1 0x7f27d6825500 in qfree lib/memory.c:136 #2 0x5637d57b0366 in bgp_pbr_match_entry_free bgpd/bgp_pbr.c:977 #3 0x5637d57b0366 in bgp_pbr_flush_entry bgpd/bgp_pbr.c:1715 #4 0x5637d57b40be in bgp_pbr_policyroute_remove_from_zebra_unit bgpd/bgp_pbr.c:1980 #5 0x5637d57bb7c0 in bgp_pbr_policyroute_remove_from_zebra bgpd/bgp_pbr.c:2144 FRRouting#6 0x5637d57bb7c0 in bgp_pbr_handle_entry bgpd/bgp_pbr.c:2781 FRRouting#7 0x5637d57bb7c0 in bgp_pbr_update_entry bgpd/bgp_pbr.c:2905 FRRouting#8 0x5637d58d23e1 in bgp_zebra_withdraw_actual bgpd/bgp_zebra.c:1733 FRRouting#9 0x5637d57ccc9e in bgp_cleanup_table bgpd/bgp_route.c:7300 FRRouting#10 0x5637d57e27d2 in bgp_cleanup_routes bgpd/bgp_route.c:7318 FRRouting#11 0x5637d5911b91 in bgp_delete bgpd/bgpd.c:4370 FRRouting#12 0x5637d56961b4 in bgp_exit bgpd/bgp_main.c:212 FRRouting#13 0x5637d56961b4 in sigint bgpd/bgp_main.c:162 FRRouting#14 0x7f27d68af501 in frr_sigevent_process lib/sigevent.c:117 FRRouting#15 0x7f27d68db77a in event_fetch lib/event.c:1742 FRRouting#16 0x7f27d68027e4 in frr_run lib/libfrr.c:1251 FRRouting#17 0x5637d5697c55 in main bgpd/bgp_main.c:569 FRRouting#18 0x7f27d630c249 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 This reverts commit d0df550. Signed-off-by: Donatas Abraitis <donatas@opensourcerouting.org>

This commit addresses a leak where temporary memory allocated earlier by the `prefix_copy` function for AF_FLOWSPEC prefixes was not being freed. To ensure proper memory management, we now release this temporary memory by calling `prefix_flowspec_ptr_free`. The ASan leak log for reference: ``` *********************************************************************************** Address Sanitizer Error detected in bgp_flowspec.test_bgp_flowspec_topo/r1.asan.bgpd.11539 ================================================================= ==11539==ERROR: LeakSanitizer: detected memory leaks Direct leak of 56 byte(s) in 2 object(s) allocated from: #0 0x7feaa956ad28 in __interceptor_calloc (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xded28) #1 0x7feaa8f670da in qcalloc lib/memory.c:105 #2 0x7feaa8fac1d4 in prefix_copy lib/prefix.c:346 #3 0x7feaa8ff43e8 in route_node_get lib/table.c:274 #4 0x56247cc798c0 in bgp_node_get bgpd/bgp_table.h:236 #5 0x56247cc798c0 in bgp_afi_node_get bgpd/bgp_route.c:145 FRRouting#6 0x56247cc92622 in bgp_update bgpd/bgp_route.c:4188 FRRouting#7 0x56247ce55b21 in bgp_nlri_parse_flowspec bgpd/bgp_flowspec.c:193 FRRouting#8 0x56247cc4cdd8 in bgp_nlri_parse bgpd/bgp_packet.c:350 FRRouting#9 0x56247cc4f37c in bgp_update_receive bgpd/bgp_packet.c:2153 FRRouting#10 0x56247cc591e2 in bgp_process_packet bgpd/bgp_packet.c:3214 FRRouting#11 0x7feaa9005b99 in event_call lib/event.c:1979 FRRouting#12 0x7feaa8f4a379 in frr_run lib/libfrr.c:1213 FRRouting#13 0x56247cb51b21 in main bgpd/bgp_main.c:510 FRRouting#14 0x7feaa7f8dc86 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21c86) SUMMARY: AddressSanitizer: 56 byte(s) leaked in 2 allocation(s). *********************************************************************************** ``` Signed-off-by: Keelan Cannoo <keelan.cannoo@icloud.com> (cherry picked from commit a7fe30e) Conflicts: bgpd/bgp_table.c lib/prefix.c lib/prefix.h lib/table.c Signed-off-by: Louis Scalbert <louis.scalbert@6wind.com> Signed-off-by: Philippe Guibert <philippe.guibert@6wind.com>

Upon examining this Indirect leak: Indirect leak of 160 byte(s) in 4 object(s) allocated from: #0 0x7fe4f40b83b7 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:77 #1 0x7fe4f3c24c1d in qcalloc lib/memory.c:111 #2 0x7fe4f3c03441 in list_new lib/linklist.c:49 #3 0x564c81d076f9 in ospf_spf_vertex_copy ospfd/ospf_spf.c:335 #4 0x564c81d0bff2 in ospf_spf_copy ospfd/ospf_spf.c:378 #5 0x564c81d158e8 in ospf_ti_lfa_generate_p_space ospfd/ospf_ti_lfa.c:787 FRRouting#6 0x564c81d162f5 in ospf_ti_lfa_generate_p_spaces ospfd/ospf_ti_lfa.c:923 FRRouting#7 0x564c81d16532 in ospf_ti_lfa_compute ospfd/ospf_ti_lfa.c:1101 FRRouting#8 0x564c81d0e942 in ospf_spf_calculate_area ospfd/ospf_spf.c:1811 FRRouting#9 0x564c81d0eaa6 in ospf_spf_calculate_areas ospfd/ospf_spf.c:1840 FRRouting#10 0x564c81d0eda2 in ospf_spf_calculate_schedule_worker ospfd/ospf_spf.c:1871 FRRouting#11 0x7fe4f3cdd7c3 in event_call lib/event.c:2009 FRRouting#12 0x7fe4f3c027e9 in frr_run lib/libfrr.c:1252 FRRouting#13 0x564c81c95191 in main ospfd/ospf_main.c:307 FRRouting#14 0x7fe4f370c249 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 It was noticed that the vertex has another list that is not being cleanedup. Let's allow this to happen. Signed-off-by: Donald Sharp <sharpd@nvidia.com>

We can do this now in gdb: (rr) walk_bgp_table table Walking BGP table at 0x55bd95ec5b70 AFI: 3, SAFI: 5 Version: 0 (Two-level table: RD -> Routes) === RD: 10.0.0.21:2 === === Dest #1: 0x55bd961a0130 === Prefix: [5]:[0]:[32]:10.1.1.1 dest->flags: 0x1 PROCESS_SCHEDULED --- Path #1 --- bgp_path_info: 0x55bd961a04b0 peer: 0x55bd95ebdfd0 (Static announcement) type: 10, sub_type: 1 (STATIC) flags: 0x80010 VALID UNSORTED uptime: 764569, lock: 1 attr: 0x55bd961a0380 (nexthop: 120.0.0.3) extra: 0x55bd960ac720 [has labels] [has evpn] next: 0x0, prev: 0x0 === RD: 10.0.0.33:1 === === Dest #2: 0x55bd95eb41e0 === Prefix: [5]:[0]:[32]:10.1.1.1 dest->flags: 0x0 --- Path #1 --- bgp_path_info: 0x55bd95ea2a20 peer: 0x55bd95ed56a0 (10.0.0.18) type: 10, sub_type: 0 (NORMAL) flags: 0x418 SELECTED VALID COUNTED uptime: 764568, lock: 2 attr: 0x55bd956aa3b0 (nexthop: 120.0.0.1) extra: 0x55bd960a5d60 [has labels] [has evpn] next: 0x0, prev: 0x0 === Dest #3: 0x55bd960aa4b0 === Prefix: [5]:[0]:[128]:10:0:0:0:0:0:0:1 dest->flags: 0x0 --- Path #1 --- bgp_path_info: 0x55bd960ad190 peer: 0x55bd95ed56a0 (10.0.0.18) type: 10, sub_type: 0 (NORMAL) flags: 0x418 SELECTED VALID COUNTED uptime: 764569, lock: 2 attr: 0x55bd960ad2e0 (nexthop: 120.0.0.1) extra: 0x55bd960aa540 [has labels] [has evpn] next: 0x0, prev: 0x0 === RD: 10.0.0.37:2 === === Dest #4: 0x55bd960ad930 === Prefix: [5]:[0]:[32]:20.1.1.1 dest->flags: 0x0 --- Path #1 --- bgp_path_info: 0x55bd960a97b0 peer: 0x55bd95ed56a0 (10.0.0.18) type: 10, sub_type: 0 (NORMAL) flags: 0x418 SELECTED VALID COUNTED uptime: 764568, lock: 2 attr: 0x55bd960a93b0 (nexthop: 120.0.0.1) extra: 0x55bd960a6b30 [has labels] [has evpn] next: 0x0, prev: 0x0 --Type <RET> for more, q to quit, c to continue without paging-- === RD: 10.0.0.41:3 === === Dest #5: 0x55bd960a9c30 === Prefix: [5]:[0]:[32]:30.1.1.1 dest->flags: 0x0 --- Path #1 --- bgp_path_info: 0x55bd960a9e10 peer: 0x55bd95ed56a0 (10.0.0.18) type: 10, sub_type: 0 (NORMAL) flags: 0x418 SELECTED VALID COUNTED uptime: 764568, lock: 2 attr: 0x55bd960a9cc0 (nexthop: 120.0.0.1) extra: 0x55bd960a9eb0 [has labels] [has evpn] next: 0x0, prev: 0x0 === Summary === Total destinations with paths: 5 Total paths: 5 Or: (rr) walk_bgp_table table Walking BGP table at 0x55bd95ee53b0 AFI: 2, SAFI: 1 Version: 1 === Dest #1: 0x55bd960ad4a0 === Prefix: IPv6:10:0:0:0:0:0:0:1/128 dest->flags: 0x1 PROCESS_SCHEDULED --- Path #1 --- bgp_path_info: 0x55bd960a5eb0 peer: 0x55bd95ef92c0 (fd00:0:0:5::2) type: 10, sub_type: 0 (NORMAL) flags: 0x80400 COUNTED UNSORTED uptime: 764569, lock: 1 attr: 0x55bd9619fb20 (nexthop: 0.0.0.0) extra: 0x55bd95ef31d0 next: 0x55bd960abe30, prev: 0x0 --- Path #2 --- bgp_path_info: 0x55bd960abe30 peer: 0x55bd95ed56a0 (10.0.0.18) type: 10, sub_type: 5 (IMPORTED) flags: 0x4018 SELECTED VALID ANNC_NH_SELF uptime: 764569, lock: 1 attr: 0x55bd960ad530 (nexthop: 120.0.0.1) extra: 0x55bd960abed0 [has labels] [has vrfleak] next: 0x0, prev: 0x55bd960a5eb0 === Summary === Total destinations with paths: 1 Total paths: 2 People might find this useful. Signed-off-by: Donald Sharp <sharpd@nvidia.com>

On one interface without any mld/pim/igmp configuration, set the command: `ip igmp require-router-alert` or `ipv6 mld require-router-alert`. It will crash for empty `pim_ifp`. ``` #0 0x000055cd72861d40 in lib_interface_gmp_require_router_alert_modify (args=0x7ffec1894e70) at ../pimd/pim_nb_config.c:4768 #1 0x00007f5cdcda137b in nb_callback_modify (context=0x55cd74647a10, nb_node=0x55cd7441c970, event=NB_EV_APPLY, dnode=0x55cd74646350, resource=0x55cd746470c8, errmsg=0x7ffec1895460 "", errmsg_len=8192) at ../lib/northbound.c:1598 #2 0x00007f5cdcda20b7 in nb_callback_configuration (context=0x55cd74647a10, event=NB_EV_APPLY, change=0x55cd74647090, errmsg=0x7ffec1895460 "", errmsg_len=8192) at ../lib/northbound.c:1962 #3 0x00007f5cdcda261f in nb_transaction_process (event=NB_EV_APPLY, transaction=0x55cd74647a10, errmsg=0x7ffec1895460 "", errmsg_len=8192) at ../lib/northbound.c:2091 #4 0x00007f5cdcda0cee in nb_candidate_commit_apply (transaction=0x55cd74647a10, save_transaction=true, transaction_id=0x0, errmsg=0x7ffec1895460 "", errmsg_len=8192) at ../lib/northbound.c:1409 #5 0x00007f5cdcda0e76 in nb_candidate_commit (context=..., candidate=0x55cd7439d960, save_transaction=true, comment=0x0, transaction_id=0x0, errmsg=0x7ffec1895460 "", errmsg_len=8192) at ../lib/northbound.c:1449 FRRouting#6 0x00007f5cdcda78aa in nb_cli_classic_commit (vty=0x55cd74639b60) at ../lib/northbound_cli.c:57 FRRouting#7 0x00007f5cdcda7ea5 in nb_cli_apply_changes_internal (vty=0x55cd74639b60, xpath_base=0x7ffec18994f0 "/frr-interface:lib/interface[name='xx']/frr-gmp:gmp/address-family[address-family='frr-routing:ipv4']", clear_pending=false) at ../lib/northbound_cli.c:195 FRRouting#8 0x00007f5cdcda8196 in _nb_cli_apply_changes (vty=0x55cd74639b60, xpath_base=0x7ffec1899940 "./frr-gmp:gmp/address-family[address-family='frr-routing:ipv4']", clear_pending=false) at ../lib/northbound_cli.c:251 ``` Signed-off-by: anlan_cs <anlan_cs@126.com>

On one interface without any mld/pim/igmp configuration, set the command: `ip igmp require-router-alert` or `ipv6 mld require-router-alert`. It will crash for empty `pim_ifp`. ``` #0 0x000055cd72861d40 in lib_interface_gmp_require_router_alert_modify (args=0x7ffec1894e70) at ../pimd/pim_nb_config.c:4768 #1 0x00007f5cdcda137b in nb_callback_modify (context=0x55cd74647a10, nb_node=0x55cd7441c970, event=NB_EV_APPLY, dnode=0x55cd74646350, resource=0x55cd746470c8, errmsg=0x7ffec1895460 "", errmsg_len=8192) at ../lib/northbound.c:1598 #2 0x00007f5cdcda20b7 in nb_callback_configuration (context=0x55cd74647a10, event=NB_EV_APPLY, change=0x55cd74647090, errmsg=0x7ffec1895460 "", errmsg_len=8192) at ../lib/northbound.c:1962 #3 0x00007f5cdcda261f in nb_transaction_process (event=NB_EV_APPLY, transaction=0x55cd74647a10, errmsg=0x7ffec1895460 "", errmsg_len=8192) at ../lib/northbound.c:2091 #4 0x00007f5cdcda0cee in nb_candidate_commit_apply (transaction=0x55cd74647a10, save_transaction=true, transaction_id=0x0, errmsg=0x7ffec1895460 "", errmsg_len=8192) at ../lib/northbound.c:1409 #5 0x00007f5cdcda0e76 in nb_candidate_commit (context=..., candidate=0x55cd7439d960, save_transaction=true, comment=0x0, transaction_id=0x0, errmsg=0x7ffec1895460 "", errmsg_len=8192) at ../lib/northbound.c:1449 FRRouting#6 0x00007f5cdcda78aa in nb_cli_classic_commit (vty=0x55cd74639b60) at ../lib/northbound_cli.c:57 FRRouting#7 0x00007f5cdcda7ea5 in nb_cli_apply_changes_internal (vty=0x55cd74639b60, xpath_base=0x7ffec18994f0 "/frr-interface:lib/interface[name='xx']/frr-gmp:gmp/address-family[address-family='frr-routing:ipv4']", clear_pending=false) at ../lib/northbound_cli.c:195 FRRouting#8 0x00007f5cdcda8196 in _nb_cli_apply_changes (vty=0x55cd74639b60, xpath_base=0x7ffec1899940 "./frr-gmp:gmp/address-family[address-family='frr-routing:ipv4']", clear_pending=false) at ../lib/northbound_cli.c:251 ``` Signed-off-by: anlan_cs <anlan_cs@126.com> (cherry picked from commit 7491c07)

Error: ERROR: AddressSanitizer: heap-use-after-free on address 0x6070000ef8a0 at pc 0x555df66ba094 bp 0x7ffc13d67240 sp 0x7ffc13d67238 READ of size 4 at 0x6070000ef8a0 thread T0 #0 0x555df66ba093 in zebra_gr_delete_stale_route_table_afi zebra/zebra_gr.c:514 #1 0x7fd33d6db06e in event_call lib/event.c:2013 #2 0x7fd33d5fffa1 in frr_run lib/libfrr.c:1257 #3 0x555df66531ec in main zebra/main.c:552 #4 0x7fd33d10c249 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 #5 0x7fd33d10c304 in __libc_start_main_impl ../csu/libc-start.c:360 FRRouting#6 0x555df6626b40 in _start (/usr/lib/frr/zebra+0x1a1b40) 0x6070000ef8a0 is located 0 bytes inside of 72-byte region [0x6070000ef8a0,0x6070000ef8e8) freed by thread T0 here: #0 0x7fd33dab76a8 in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:52 #1 0x7fd33d622cd5 in qfree lib/memory.c:136 #2 0x555df66b9e5f in zebra_gr_client_info_delete zebra/zebra_gr.c:130 #3 0x555df66bc66f in zread_client_capabilities zebra/zebra_gr.c:355 #4 0x555df66a025c in zserv_handle_commands zebra/zapi_msg.c:4228 #5 0x555df67cde33 in zserv_process_messages zebra/zserv.c:565 FRRouting#6 0x7fd33d6db06e in event_call lib/event.c:2013 FRRouting#7 0x7fd33d5fffa1 in frr_run lib/libfrr.c:1257 FRRouting#8 0x555df66531ec in main zebra/main.c:552 FRRouting#9 0x7fd33d10c249 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 previously allocated by thread T0 here: #0 0x7fd33dab83b7 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:77 #1 0x7fd33d6223e2 in qcalloc lib/memory.c:111 #2 0x555df66bbace in zebra_gr_client_info_create zebra/zebra_gr.c:101 #3 0x555df66bbace in zread_client_capabilities zebra/zebra_gr.c:360 #4 0x555df66a025c in zserv_handle_commands zebra/zapi_msg.c:4228 #5 0x555df67cde33 in zserv_process_messages zebra/zserv.c:565 FRRouting#6 0x7fd33d6db06e in event_call lib/event.c:2013 FRRouting#7 0x7fd33d5fffa1 in frr_run lib/libfrr.c:1257 FRRouting#8 0x555df66531ec in main zebra/main.c:552 FRRouting#9 0x7fd33d10c249 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 Signed-off-by: Pooja Jagadeesh Doijode <pdoijode@nvidia.com>

The following crash happens, when moving from level-2 to level-1 an isis flex-algorithm configuration > warning: 44 ./nptl/pthread_kill.c: No such file or directory > [Current thread is 1 (Thread 0x7108d4cb2980 (LWP 1023))] > (gdb) bt > #0 __pthread_kill_implementation (no_tid=0, signo=11, > threadid=<optimized out>) at ./nptl/pthread_kill.c:44 > #1 __pthread_kill_internal (signo=11, threadid=<optimized out>) > at ./nptl/pthread_kill.c:78 > #2 __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=11) > at ./nptl/pthread_kill.c:89 > #3 0x00007108d3e4527e in __GI_raise (sig=11) at ../sysdeps/posix/raise.c:26 > #4 0x00007108d4b44926 in core_handler (signo=11, siginfo=0x7ffe7c10fb30, > context=0x7ffe7c10fa00) > at /build/make-pkg/output/_packages/cp-routing/src/lib/sigevent.c:248 > #5 <signal handler called> > FRRouting#6 0x00005b5d803bf07b in isis_spf_invalidate_routes (tree=0x0) > at /build/make-pkg/output/_packages/cp-routing/src/isisd/isis_spf.c:2118 > FRRouting#7 0x00005b5d803fb23e in isis_area_invalidate_routes (area=0x5b5db8d5be40, > levels=1) > at /build/make-pkg/output/_packages/cp-routing/src/isisd/isisd.c:3152 > FRRouting#8 0x00005b5d803bf280 in isis_run_spf_cb (thread=0x7ffe7c110180) > at /build/make-pkg/output/_packages/cp-routing/src/isisd/isis_spf.c:2165 > FRRouting#9 0x00007108d4b5ff7f in event_call (thread=0x7ffe7c110180) > at /build/make-pkg/output/_packages/cp-routing/src/lib/event.c:2011 > FRRouting#10 0x00007108d4adb761 in frr_run (master=0x5b5db7f7ca40) > at /build/make-pkg/output/_packages/cp-routing/src/lib/libfrr.c:1219 > FRRouting#11 0x00005b5d8038333a in main (argc=5, argv=0x7ffe7c1103d8, > --Type <RET> for more, q to quit, c to continue without paging-- > envp=0x7ffe7c110408) > at /build/make-pkg/output/_packages/cp-routing/src/isisd/isis_main.c:360 > (gdb) Fix this by adding protection before invalidating all routes. Signed-off-by: Philippe Guibert <philippe.guibert@6wind.com>

In bgp_evpn_mh_finish(), accessing es->es_evi_list after calling bgp_evpn_es_local_info_clear() causes use-after-free when the ES gets freed. Fix by checking and cleaning ES-EVIs before clearing local info. AddressSanitizer error in topotest `test_bgp_evpn_ead_evi_routes.py`, test `check_daemon`, router `tor2` ERROR: AddressSanitizer: heap-use-after-free on address 0xffff817d9f28 at pc 0xaaaae8a31974 bp 0xffffc069e410 sp 0xffffc069e400 READ of size 8 at 0xffff817d9f28 thread T0 #0 0xaaaae8a31970 in bgp_evpn_mh_finish bgpd/bgp_evpn_mh.c:5093 #1 0xaaaae89b66bc in bgp_exit bgpd/bgp_main.c:193 #2 0xaaaae89b66bc in sigint bgpd/bgp_main.c:141 #3 0xffff862f6440 in frr_sigevent_process lib/sigevent.c:117 #4 0xffff8632095c in event_fetch_inner_loop lib/event.c:2427 #5 0xffff8632095c in event_fetch lib/event.c:2559 FRRouting#6 0xffff86258100 in frr_run lib/libfrr.c:1257 FRRouting#7 0xaaaae89af55c in main bgpd/bgp_main.c:549 FRRouting#8 0xffff85c773fc in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 FRRouting#9 0xffff85c774d4 in __libc_start_main_impl ../csu/libc-start.c:392 FRRouting#10 0xaaaae89b4dac in _start (/usr/lib/frr/bgpd+0x2e4dac) 0xffff817d9f28 is located 168 bytes inside of 248-byte region [0xffff817d9e80,0xffff817d9f78) freed by thread T0 here: #0 0xffff8669f0d8 in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:127 #1 0xaaaae8a316f0 in bgp_evpn_es_local_info_clear bgpd/bgp_evpn_mh.c:2058 #2 0xaaaae8a316f0 in bgp_evpn_mh_finish bgpd/bgp_evpn_mh.c:5088 #3 0xaaaae89b66bc in bgp_exit bgpd/bgp_main.c:193 #4 0xaaaae89b66bc in sigint bgpd/bgp_main.c:141 #5 0xffff862f6440 in frr_sigevent_process lib/sigevent.c:117 FRRouting#6 0xffff8632095c in event_fetch_inner_loop lib/event.c:2427 FRRouting#7 0xffff8632095c in event_fetch lib/event.c:2559 FRRouting#8 0xffff86258100 in frr_run lib/libfrr.c:1257 FRRouting#9 0xaaaae89af55c in main bgpd/bgp_main.c:549 FRRouting#10 0xffff85c773fc in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 FRRouting#11 0xffff85c774d4 in __libc_start_main_impl ../csu/libc-start.c:392 FRRouting#12 0xaaaae89b4dac in _start (/usr/lib/frr/bgpd+0x2e4dac) previously allocated by thread T0 here: #0 0xffff8669f5f4 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:154 #1 0xffff86277c30 in qcalloc lib/memory.c:111 #2 0xaaaae8a19ba8 in bgp_evpn_es_new bgpd/bgp_evpn_mh.c:1925 #3 0xaaaae8a2a4d4 in bgp_evpn_local_es_add bgpd/bgp_evpn_mh.c:2420 #4 0xaaaae8bc7080 in bgp_zebra_process_local_es_add bgpd/bgp_zebra.c:3272 #5 0xffff863511c4 in zclient_read lib/zclient.c:4870 FRRouting#6 0xffff863212e4 in event_call lib/event.c:2730 FRRouting#7 0xffff862580f4 in frr_run lib/libfrr.c:1258 FRRouting#8 0xaaaae89af55c in main bgpd/bgp_main.c:549 FRRouting#9 0xffff85c773fc in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 FRRouting#10 0xffff85c774d4 in __libc_start_main_impl ../csu/libc-start.c:392 FRRouting#11 0xaaaae89b4dac in _start (/usr/lib/frr/bgpd+0x2e4dac) SUMMARY: AddressSanitizer: heap-use-after-free bgpd/bgp_evpn_mh.c:5093 in bgp_evpn_mh_finish Signed-off-by: Rajasekar Raja <rajasekarr@nvidia.com>

Changes: - check for zif->brslave_info.br_if, before access - If ES config is pushed when bond is not associated with bridge throw warning in zebra_evpn_es_setup_evis, when bond gets linked process zebra_evpn_es_setup_evis and send update to bgp. BT: ==268631==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000050 (pc 0xaaaac9734754 bp 0xffffe9ae0370 sp 0xffffe9ae00b0 T0) ==268631==The signal is caused by a READ memory access. ==268631==Hint: address points to the zero page. #0 0xaaaac9734754 in zebra_evpn_acc_vl_find zebra/zebra_evpn_mh.c:554 #1 0xaaaac9734754 in zebra_evpn_es_setup_evis zebra/zebra_evpn_mh.c:2127 #2 0xaaaac9734754 in zebra_evpn_es_local_info_set zebra/zebra_evpn_mh.c:2350 #3 0xaaaac9734754 in zebra_evpn_local_es_update zebra/zebra_evpn_mh.c:2514 #4 0xaaaac968df28 in lib_interface_zebra_evpn_mh_type_3_local_discriminator_modify zebra/zebra_nb_config.c:2491 #5 0xffff8ab9d1f0 in nb_callback_modify lib/northbound.c:1600 FRRouting#6 0xffff8ab9d1f0 in nb_callback_configuration lib/northbound.c:1964 FRRouting#7 0xffff8ab9def4 in nb_transaction_process lib/northbound.c:2093 FRRouting#8 0xffff8ab9def4 in nb_candidate_commit_apply lib/northbound.c:1411 FRRouting#9 0xffff8ab70b54 in mgmt_be_txn_proc_cfgapply lib/mgmt_be_client.c:614 FRRouting#10 0xffff8ab73de8 in be_client_handle_cfg_apply lib/mgmt_be_client.c:645 FRRouting#11 0xffff8ab73de8 in be_client_handle_native_msg lib/mgmt_be_client.c:1001 FRRouting#12 0xffff8ab73de8 in mgmt_be_client_process_msg lib/mgmt_be_client.c:1040 FRRouting#13 0xffff8ab73de8 in mgmt_be_client_process_msg lib/mgmt_be_client.c:1027 FRRouting#14 0xffff8ab7b9b0 in mgmt_msg_procbufs lib/mgmt_msg.c:199 FRRouting#15 0xffff8ab7bb40 in msg_conn_proc_msgs lib/mgmt_msg.c:520 FRRouting#16 0xffff8ac1a894 in event_call lib/event.c:2730 FRRouting#17 0xffff8ab4f4d4 in frr_run lib/libfrr.c:1258 FRRouting#18 0xaaaac95a9698 in main zebra/main.c:580 FRRouting#19 0xffff8a5b73fc in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 FRRouting#20 0xffff8a5b74d4 in __libc_start_main_impl ../csu/libc-start.c:392 FRRouting#21 0xaaaac95accac in _start (/usr/lib/frr/zebra+0x1accac) AddressSanitizer can not provide additional info. SUMMARY: AddressSanitizer: SEGV zebra/zebra_evpn_mh.c:554 in zebra_evpn_acc_vl_find Signed-off-by: Ashwini Reddy <ashred@nvidia.com> Signed-off-by: Rajasekar Raja <rajasekarr@nvidia.com>

Free the newly allocated VLAN bitmap and restore the old one when no bridge VLAN info is available. ASAN Leak: ==315014==ERROR: LeakSanitizer: detected memory leaks Direct leak of 516 byte(s) in 1 object(s) allocated from: #0 0xffffa65af5f4 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:154 #1 0xffffa617f040 in qcalloc lib/memory.c:111 #2 0xaaaad9853130 in interface_bridge_vlan_update zebra/interface.c:1920 #3 0xaaaad9853130 in interface_bridge_handling zebra/interface.c:1967 #4 0xaaaad9853130 in zebra_if_dplane_ifp_handling zebra/interface.c:2063 #5 0xaaaad9853130 in zebra_if_dplane_result zebra/interface.c:2380 FRRouting#6 0xaaaad9961664 in rib_process_dplane_results zebra/zebra_rib.c:5043 FRRouting#7 0xffffa622a894 in event_call lib/event.c:2730 FRRouting#8 0xffffa615f4d4 in frr_run lib/libfrr.c:1258 FRRouting#9 0xaaaad9839918 in main zebra/main.c:580 FRRouting#10 0xffffa5bc73fc in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 FRRouting#11 0xffffa5bc74d4 in __libc_start_main_impl ../csu/libc-start.c:392 FRRouting#12 0xaaaad983cf2c in _start (/usr/lib/frr/zebra+0x1acf2c) SUMMARY: AddressSanitizer: 516 byte(s) leaked in 1 allocation(s). Signed-off-by: Rajasekar Raja <rajasekarr@nvidia.com>

Add Python version of gen_northbound_callbacks

0f2cbac

choppsv1 added the codex label Jun 30, 2025 — with ChatGPT Codex Connector

simple fix, needs a lot more work.

a0eab76

Signed-off-by: Christian Hopps <chopps@labn.net>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Python version of gen_northbound_callbacks #3

Add Python version of gen_northbound_callbacks #3

Uh oh!

choppsv1 commented Jun 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add Python version of gen_northbound_callbacks #3

Are you sure you want to change the base?

Add Python version of gen_northbound_callbacks #3

Uh oh!

Conversation

choppsv1 commented Jun 30, 2025

Summary

Testing

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants