diff --git a/docs/porting_guide.rst b/docs/porting_guide.rst index 633e129..06fa8d8 100644 --- a/docs/porting_guide.rst +++ b/docs/porting_guide.rst @@ -4,6 +4,201 @@ Porting GuideLine ================= +******** +Hardware +******** + +The porting of OpenAMP to a new :doc:`multicore system <../openamp/overview>` requires +configuring the hardware on each core so it aligns with the OpenAMP architecture. + +This setup typically includes defining a shared memory region for +:ref:`RPMsg` based +:ref:`Interprocessor Communications (IPC)`, with or without interrupts for +asynchronous inter-core notification and defining the firmware execution flow either independently +or with :ref:`Remoteproc` and associated +:ref:`Resource Table`. + +Memory and interrupt assignments are critical design choices for any port. For a broader overview, refer to +:doc:`../protocol_details/system_considerations`. + + +Shared Memory +============= + +Shared memory forms the :ref:`physical layer` for +:doc:`RPMsg <../docs/rpmsg_design>` protocol. +The specific memory type and layout are implementation dependent, but should be a dedicated +SRAM or DDR region accessible by both cores, with caching disabled. + +Memory requirements are generally modest because :doc:`RPMsg <../docs/rpmsg_design>` is a +control‑oriented protocol rather than a high‑bandwidth streaming channel. For example, using +the Linux RPMsg packet size of 512 bytes, a 64kB shared memory region can hold roughly 128 +messages — sufficient for most applications. Larger or smaller allocations can be chosen based +on system needs. + +If the :ref:`Resource Table` is not embedded in the remote firmware image, +additional shared memory may be required for a dynamic table. + +:ref:`Remoteproc` can also use shared memory for optional +trace buffers. + + +Memory Protection +----------------- + +Because this is shared memory, appropriate hardware memory protection should be configured +on both processors. + +Depending on the memory type, this may involve configuring the Memory Management Unit (MMU), +Memory Protection Unit (MPU), or Input-Output Memory Management Unit (IOMMU) to enforce +correct access permissions. + +On systems running an advanced OS — such as Linux on the main +processor — these protections may be applied through OS mechanisms like the device tree or +via the :ref:`Remoteproc` :ref:`Resource Table`. + + +Notification +------------ + +:doc:`RPMsg <../docs/rpmsg_design>` uses :ref:`ring buffers` in shared +memory, so either processor can poll for incoming messages. However, asynchronous notification +via interrupts is recommended. + +Most heterogeneous SoCs include a built‑in inter‑core interrupt mechanism, often called a +mailbox. These implementations typically combine shared memory with interrupt signaling and +may be managed through an Inter‑Processor Communication Controller (IPCC). + +If no dedicated hardware is available — or it is reserved for other purposes — +software‑generated interrupts can be used instead. + + +*************** +Porting Options +*************** + +OpenAMP consists of two major components: :ref:`Remoteproc` +and :doc:`RPMsg <../docs/rpmsg_design>`. These can be ported +independently or together, at either the driver level or the device level. + +Driver level ports integrate with an operating system’s existing frameworks, +while device level ports implement the functionality directly on bare metal without +leveraging OS‑provided drivers. + +`libmetal `_ provides the +:ref:`hardware abstraction layer` for both. + +The main porting approaches include: + +- :ref:`Remoteproc` on the remote processor only, + with the main processor using an existing + :ref:`Remoteproc` implementation (e.g., Linux Remoteproc) + and no IPC. + +- :doc:`RPMsg <../docs/rpmsg_design>` on the remote processor only, with the main processor + using an existing :doc:`RPMsg <../docs/rpmsg_design>` stack (e.g., Linux RPMsg) and no + remote firmware management. + +- Custom device‑level implementation of :ref:`Remoteproc` + and/or :doc:`RPMsg <../docs/rpmsg_design>` for both processors. + + +.. _driver-lcm-remoteproc: + +Driver Lifecycle Management via Remoteproc +========================================== + +Some systems do not require IPC or use an alternative IPC mechanism. In these cases, only +:ref:`Remoteproc` may be ported (or reused, as on Linux) +on both the main and remote processors. + +The main processor uses driver level :ref:`Remoteproc` to +load, start, stop, and manage remote firmware. +This approach is useful when the remote firmware must be externally controlled or when +multiple firmware images may be deployed depending on runtime needs. +This configuration is common in custom or bare‑metal remote environments. + +- Pros: Full remote firmware management +- Cons: No IPC. Larger software footprint + + +.. _driver-lcm-remoteproc-rpmsg: + +Driver Lifecycle Management via Remoteproc with IPC +=================================================== + +This setup is the most complete and includes driver based lifecycle management and IPC. +In these cases, both :ref:`Remoteproc` and +:doc:`RPMsg <../docs/rpmsg_design>` need to be ported (or reused, as on Linux) on both +the main and remote processors. + +The main processor uses driver level :ref:`Remoteproc` and +:doc:`RPMsg <../docs/rpmsg_design>`. + +This approach is useful when full IPC and remote firmware control is required. + +This configuration is common in custom or bare‑metal remote environments. + +- Pros: Full IPC and remote firmware management +- Cons: Largest software footprint + + +.. _driver-rpmsg: + +Driver to Remote IPC via RPMsg +============================== + +If the remote firmware is static and starts at boot, or if another framework manages +firmware loading, only :doc:`RPMsg <../docs/rpmsg_design>` needs to be ported. +In this model, the remote processor runs its firmware autonomously, and the main processor +interacts with it solely through the :doc:`RPMsg <../docs/rpmsg_design>` communication channel, +without any involvement in firmware lifecycle control. This approach suits systems where the +remote environment is minimal or bare‑metal, and where the primary requirement is efficient +message‑based IPC rather than external management of the remote core. + +- Pros: Lightweight. Provides IPC. +- Cons: No remote firmware management. + + +.. _device-lcm-remoteproc: + +Device Level Custom Remoteproc and RPMsg +======================================== + +In highly customized or bare‑metal only environments, a port of +:ref:`Remoteproc` and :doc:`RPMsg <../docs/rpmsg_design>` may +be required without any driver‑level abstraction. +In this case, the full :ref:`Remoteproc` and +:doc:`RPMsg <../docs/rpmsg_design>` mechanism must be implemented +directly on both the main and remote processors, ensuring that each core provides the necessary +firmware lifecycle management, messaging, shared‑memory handling, and notification logic without +relying on OS‑level drivers or frameworks. + +- Pros: Lightweight. +- Cons: Highly custom and less portable. + + +.. _device-rpmsg: + +Device Level Custom RPMsg +========================= + +In a simpler bare‑metal only environments, a port of only :doc:`RPMsg <../docs/rpmsg_design>` may +be required without any driver‑level abstraction. +In this case, only the :doc:`RPMsg <../docs/rpmsg_design>` mechanism need be implemented +directly on both the main and remote processors, ensuring that each core provides the necessary +IPC and notification logic without relying on OS‑level drivers or frameworks. + +- Pros: Very lightweight. +- Cons: Highly custom and less portable. + + +.. _hardware-abstraction: + +******************** +Hardware Abstraction +******************** + The `OpenAMP Framework `_ uses `libmetal `_ to provide abstractions that allows for porting of the OpenAMP Framework to various software environments (operating systems and bare metal @@ -107,13 +302,16 @@ compiler to GNU gcc, you may need to implement the atomic operations defined in .. _port-remoteproc-driver: + *********************************** Platform Specific Remoteproc Driver *********************************** -An OpenAMP port could need a platform specific remoteproc driver to use remoteproc -life cycle management (LCM) APIs. The remoteproc driver platform specific functions are defined -in `lib/include/openamp/remoteproc.h `_ and provided through the :openamp_doc_link:`remoteproc_ops data structure `. +An OpenAMP port could need a platform specific :ref:`Remoteproc` +driver to use :ref:`Remoteproc` life cycle management (LCM) APIs. +The :ref:`Remoteproc` driver platform specific functions are defined in +`lib/include/openamp/remoteproc.h `_ +and provided through the :openamp_doc_link:`remoteproc_ops data structure `. The remoteproc LCM APIs use these platform specific implementation of init, remove, mmap, handle_rsc, config, start, stop, shutdown and notify. These functions are passed to remoteproc @@ -127,243 +325,80 @@ by the other APIs. .. _port-remoteproc: -********************************************************************** -Platform Specific Porting to Use Remoteproc to Manage Remote Processor -********************************************************************** -With the platform specific :ref:`remoteproc driver functions` -implemented by the port, the user can use remoteproc APIs to run application on a remote processor. +Use Remoteproc to Manage Remote Processor +========================================= -.. doxygenfunction:: remoteproc_init - :project: openamp_doc_embed +With the :ref:`remoteproc driver functions` required +by the framework ported, the user can call the :ref:`Remoteproc` +APIs to run an application on a remote processor, as described in the +:ref:`Remote User APIs` section of the Remoteproc design. -.. doxygenfunction:: remoteproc_remove +The following code snippets from the +`Load FW System Reference Example `_ +demonstrate the use of the Remote User APIs. -.. doxygenfunction:: remoteproc_mmap +Remoteproc Init +--------------- -.. doxygenfunction:: remoteproc_config +From `platform_info.c `_ -.. doxygenfunction:: remoteproc_start +.. literalinclude:: ../openamp-system-reference/examples/legacy_apps/examples/load_fw/platform_info.c + :language: c + :lines: 16-31 -.. doxygenfunction:: remoteproc_stop -.. doxygenfunction:: remoteproc_shutdown +Lifecycle APIs +-------------- +From `load_fw.c `_ -The following code snippet is an example execution. +.. literalinclude:: ../openamp-system-reference/examples/legacy_apps/examples/load_fw/load_fw.c + :language: c + :lines: 21-57 -.. code-block:: c - - #include - - /* User defined remoteproc operations */ - extern struct remoteproc_ops rproc_ops; - - /* User defined image store operations, such as open the image file, read - * image from storage, and close the image file. - */ - - extern struct image_store_ops img_store_ops; - /* Pointer to keep the image store information. It will be passed to user - * defined image store operations by the remoteproc loading application - * function. Its structure is defined by user. - */ - void *img_store_info; +.. _port-rpmsg: - struct remoteproc rproc; +*********************** +Platform Specific RPMsg +*********************** - void main(void) - { - /* Instantiate the remoteproc instance */ - remoteproc_init(&rproc, &rproc_ops, &private_data); +In OpenAMP, :doc:`RPMsg <../docs/rpmsg_design>` uses +`VirtIO `_ to manage shared buffers. +The OpenAMP library provides a +`Remoteproc VirtIO `_ +backend implementation, and a +`VirtIO and RPMsg `_ +implementation. - /* Optional, required, if user needs to configure the remote before - * loading applications. - */ - remoteproc_config(&rproc, &platform_config); +You can also implement your own VirtIO backend using the +`OpenAMP VirtIO `_ and +`RPMsg `_ components provided by OpenAMP. - /* Load Application. It only supports ELF for now. */ - remoteproc_load(&rproc, img_path, img_store_info, &img_store_ops, NULL); +If you choose to create your own backend, you can use the +`Remoteproc VirtIO `_ +backend as a reference. - /* Start the processor to run the application. */ - remoteproc_start(&rproc); +For an example of setting up Remoteproc and Virtio for a remote device refer to +`zynqmp platform_info.c `_ - /* ... */ +***************** +Examples of Ports +***************** - /* Optional. Stop the processor, but the processor is not powered - * down. - */ - remoteproc_stop(&rproc); - /* Shutdown the processor. The processor is supposed to be powered - * down. - */ - remoteproc_shutdown(&rproc); +Example of Driver based Remoteproc Virtio +========================================= - /* Destroy the remoteproc instance */ - remoteproc_remove(&rproc); - } +Most of the :doc:`../demos/index` are examples of example of :ref:`driver-lcm-remoteproc-rpmsg`, +using Virtio through a :ref:`Resource Table` with the most advanced being the +`Zephyr RPMsg Multi Service Demo `_ +detailed in :doc:`../demos/rpmsg_multi_services`. -.. _port-rpmsg: -************************************** -Platform Specific Porting to Use RPMsg -************************************** +Example of Device based RPMsg only with Virtio +============================================== -RPMsg in OpenAMP implementation uses `VirtIO `_ -to manage the shared buffers. OpenAMP library provides -`remoteproc VirtIO backend implementation `_. -You don't have to use remoteproc backend. You can implement your VirtIO backend with the VirtIO -and RPMsg implementation in OpenAMP. If you want to implement your own VirtIO backend, you can -refer to the -`remoteproc VirtIO backend implementation < https://github.com/OpenAMP/open-amp/blob/main/lib/remoteproc/remoteproc_virtio.c>`_. - -Here are the steps to use OpenAMP for RPMsg communication: - - -.. code-block:: c - - #include - #include - #include - - /* User defined remoteproc operations for communication */ - sturct remoteproc rproc_ops = { - .init = local_rproc_init; - .mmap = local_rproc_mmap; - .notify = local_rproc_notify; - .remove = local_rproc_remove; - }; - - /* Remoteproc instance. If you don't use Remoteproc VirtIO backend, - * you don't need to define the remoteproc instance. - */ - struct remoteproc rproc; - - /* RPMsg VirtIO device instance. */ - struct rpmsg_virtio_device rpmsg_vdev; - - /* RPMsg device */ - struct rpmsg_device *rpmsg_dev; - - /* Resource Table. Resource table is used by remoteproc to describe - * the shared resources such as vdev(VirtIO device) and other shared memory. - * Resource table resources definition is in the remoteproc.h. - * Examples of the resource table can be found in the OpenAMP repo: - * - apps/machine/zynqmp/rsc_table.c - * - apps/machine/zynqmp_r5/rsc_table.c - * - apps/machine/zynq7/rsc_table.c - */ - void *rsc_table = &resource_table; - - /* Size of the resource table */ - int rsc_size = sizeof(resource_table); - - /* Shared memory metal I/O region. It will be used by OpenAMP library - * to access the memory. You can have more than one shared memory regions - * in your application. - */ - struct metal_io_region *shm_io; - - /* VirtIO device */ - struct virtio_device *vdev; - - /* RPMsg shared buffers pool */ - struct rpmsg_virtio_shm_pool shpool; - - /* Shared buffers */ - void *shbuf; - - /* RPMsg endpoint */ - struct rpmsg_endpoint ept; - - /* User defined RPMsg name service callback. This callback is called - * when there is no registered RPMsg endpoint is found for this name - * service. User can create RPMsg endpoint in this callback. */ - void ns_bind_cb(struct rpmsg_device *rdev, const char *name, uint32_t dest); - - /* User defined RPMsg endpoint received message callback */ - void rpmsg_ept_cb(struct rpmsg_endpoint *ept, void *data, size_t len, - uint32_t src, void *priv); - - /* User defined RPMsg name service unbind request callback */ - void ns_unbind_cb(struct rpmsg_device *rdev, const char *name, uint32_t dest); - - void main(void) - { - /* Instantiate remoteproc instance */ - remoteproc_init(&rproc, &rproc_ops); - - /* Mmap shared memories so that they can be used */ - remoteproc_mmap(&rproc, &physical_address, NULL, size, - , &shm_io); - - /* Parse resource table to remoteproc */ - remoteproc_set_rsc_table(&rproc, rsc_table, rsc_size); - - /* Create VirtIO device from remoteproc. - * VirtIO device main controller will initiate the VirtIO rings, and assign - * shared buffers. If you running the application as VirtIO device, you - * set the role as VIRTIO_DEV_DEVICE. - * If you don't use remoteproc, you will need to define your own VirtIO - * device. - */ - vdev = remoteproc_create_virtio(&rproc, 0, VIRTIO_DEV_DRIVER, NULL); - - /* This step is only required if you are VirtIO device main controller. - * Initialize the shared buffers pool. - */ - shbuf = metal_io_phys_to_virt(shm_io, SHARED_BUF_PA); - rpmsg_virtio_init_shm_pool(&shpool, shbuf, SHARED_BUFF_SIZE); - - /* Initialize RPMsg VirtIO device with the VirtIO device */ - /* If it is VirtIO device, it will not return until the main - * controller side sets the VirtIO device DRIVER OK status bit. - */ - rpmsg_init_vdev(&rpmsg_vdev, vdev, ns_bind_cb, io, shm_io, &shpool); - - /* Get RPMsg device from RPMsg VirtIO device */ - rpmsg_dev = rpmsg_virtio_get_rpmsg_device(&rpmsg_vdev); - - /* Create RPMsg endpoint. */ - rpmsg_create_ept(&ept, rdev, RPMSG_SERVICE_NAME, RPMSG_ADDR_ANY, - rpmsg_ept_cb, ns_unbind_cb); - - /* If it is VirtIO device main controller, it sends the first message */ - while (!is_rpmsg_ept_read(&ept)) { - /* check if the endpoint has binded. - * If not, wait for notification. If local endpoint hasn't - * been bound with the remote endpoint, it will fail to - * send the message to the remote. - */ - /* If you prefer to use interrupt, you can wait for - * interrupt here, and call the VirtIO notified function - * in the interrupt handling task. - */ - rproc_virtio_notified(vdev, RSC_NOTIFY_ID_ANY); - } - /* Send RPMsg */ - rpmsg_send(&ept, data, size); - - do { - /* If you prefer to use interrupt, you can wait for - * interrupt here, and call the VirtIO notified function - * in the interrupt handling task. - * If vdev is notified, the endpoint callback will be - * called. - */ - rproc_virtio_notified(vdev, RSC_NOTIFY_ID_ANY); - } while(!ns_unbind_cb_is_called && !user_decided_to_end_communication); - - /* End of communication, destroy the endpoint */ - rpmsg_destroy_ept(&ept); - - rpmsg_deinit_vdev(&rpmsg_vdev); - - remoteproc_remove_virtio(&rproc, vdev); - - remoteproc_remove(&rproc); - } - -. \ No newline at end of file +For an example of :ref:`device-rpmsg` without Remoteproc and a Resource Table, refer to the +`Zephyr IPC OpenAMP Demo `_ diff --git a/openamp/glossary.rst b/openamp/glossary.rst index 02b03bc..189f1d5 100644 --- a/openamp/glossary.rst +++ b/openamp/glossary.rst @@ -8,12 +8,20 @@ Glossary AMP, `Asymmentric Multiprocessing `_ API, Application Interface + DDR (RAM), Double Data Rate (Random Access Memory) GPL, `GNU General Public License `_ + HAL, Hardware Abstraction Layer + IOMMU, Input-Output Memory Management Unit IPC, `Inter-Processor Communications `_ + IPCC, Inter-Processor Communication Controller LCM, Life-Cycle Management + MMU, Memory Management Unit + MPU, Memory Protection Unit + RAM, Random Access Memory RPC, :ref:`Remote Procedure Calls (RPC)` RTOS, Real Time Operating System RPMsg, `Remote Processor Messaging `_ SMP, `Symmetric Multiprocessing (SMP) `_ SoC, System on Chip + SRAM, Static RAM Virtio, Virtual Input Output diff --git a/protocol_details/resource_tbl.rst b/protocol_details/resource_tbl.rst index 6861b98..d1e1a15 100644 --- a/protocol_details/resource_tbl.rst +++ b/protocol_details/resource_tbl.rst @@ -14,6 +14,7 @@ at the offset location of each entry. Directly following the header are the reso themselves, each of which has a 32 bit type. These in remote context will likely be memory carveouts for locations of parts of the remote system and virtio device definitions. + +--------------+--------------------------------------------------------------------------------+ | Item | Description | +==============+================================================================================+ @@ -30,14 +31,21 @@ for locations of parts of the remote system and virtio device definitions. | | can correctly extract the information. | +--------------+--------------------------------------------------------------------------------+ -The fw_resource_type's are listed in the -`remoteproc header `_. +The resource table is effectively a list of resource definitions, with each entry detailed by the +corresponding :ref:`resource structure`, and the Remoteproc framework +is responsible for configuring the relevant drivers and/or bare metal hardware. + +The possible resource types are defined in the +`Remoteproc Header `_ +and detailed in the :ref:`resource-types` section below. + Compatibility with Linux kernel ------------------------------- -The resource table should maintain compatibility with that of -`remoteproc `_. +With Linux being a key full stack operating system in the embedded devices space, the resource +table should maintain compatibility with that of +`Linux Remoteproc `_. :: @@ -45,3 +53,42 @@ The resource table should maintain compatibility with that of Related documentation can be found under the Linux kernel's remoteproc documentation: `Binary Firmware Structure `_. + + +.. _resource-types: + +Resource Types +************** + +Each of the resources in the :ref:`resource-table` is defined by a type (fw_resource_type) and +its corresponding :ref:`structure`. + +.. doxygenenum:: fw_resource_type + :project: openamp_doc_embed + + +.. _resource-structure: + +Resource Structure Definitions +------------------------------ + +The following structures correspond to each of the :ref:`resource-types` (fw_resource_type). + +.. doxygenstruct:: fw_rsc_carveout + :project: openamp_doc_embed + +.. doxygenstruct:: fw_rsc_devmem + :project: openamp_doc_embed + +.. doxygenstruct:: fw_rsc_trace + :project: openamp_doc_embed + +.. doxygenstruct:: fw_rsc_vdev + :project: openamp_doc_embed + + +In addition to the predefined :ref:`resource-types` vendors can define their +own with types between RSC_VENDOR_START and RSC_VENDOR_END. + +.. doxygenstruct:: fw_rsc_vendor + :project: openamp_doc_embed