From 2f51c16a873b3ee955e94a023c22d6c031d39b1d Mon Sep 17 00:00:00 2001 From: nick Date: Mon, 26 Oct 2026 09:25:50 +0800 Subject: [PATCH 01/22] refactor(consul)!: adopt typed plugin construction Replace Consul creation macros with typed bases, a private owning factory and explicit DLL entry points. Preserve interface registration, disabled updates and request behavior, and add construction/lifecycle checks with bilingual migration guidance. BREAKING CHANGE: Consul implementation and plugin class layouts change. Rebuild Consul and its consumers together; all existing public types and logical header paths remain available. --- docs/guides/plugins.md | 2 + docs/guides/plugins_CN.md | 2 + docs/project-layout.md | 14 ++++ docs/project-layout_CN.md | 14 ++++ docs/reference/build-options.md | 2 +- docs/reference/build-options_CN.md | 2 +- src/plugins/README.md | 4 +- src/plugins/README_CN.md | 4 +- src/plugins/consul/CMakeLists.txt | 18 +----- src/plugins/consul/consul_factory.hpp | 13 ++++ src/plugins/consul/consul_module_impl.cpp | 5 ++ src/plugins/consul/consul_module_impl.hpp | 5 +- src/plugins/consul/consul_plugin.cpp | 23 +++++-- src/plugins/consul/consul_plugin.hpp | 12 +++- tests/consul/CMakeLists.txt | 36 +++++++++++ tests/consul/construction.cpp | 53 +++++++++++++++ tests/consul/entry.cpp | 45 +++++++++++++ tests/consul/entry_factory.cpp | 79 +++++++++++++++++++++++ tests/consul/exports.cpp | 31 +++++++++ tests/core/CMakeLists.txt | 4 ++ 20 files changed, 337 insertions(+), 31 deletions(-) create mode 100644 src/plugins/consul/consul_factory.hpp create mode 100644 tests/consul/CMakeLists.txt create mode 100644 tests/consul/construction.cpp create mode 100644 tests/consul/entry.cpp create mode 100644 tests/consul/entry_factory.cpp create mode 100644 tests/consul/exports.cpp diff --git a/docs/guides/plugins.md b/docs/guides/plugins.md index 0403f25b5..9c9e8f5cf 100644 --- a/docs/guides/plugins.md +++ b/docs/guides/plugins.md @@ -211,3 +211,5 @@ ABI stability. Lifecycle phases, draining and library unload rules are unchanged Net constructs its HTTP client through a private factory returning `std::unique_ptr` and registers it with updates enabled. Applications consume `` and typed manager lookup; the old `http_client_module_impl.hpp` is removed from the public/install interfaces. Rebuild affected consumers together; see [Net migration](../reference/network.md#api-and-private-construction). Kernel also uses typed construction while retaining its public `*_impl` and `kernel_plugin` types and header paths. Its private factory header is not installed. Rebuild Kernel and consumers together because those public class layouts change; the [Kernel migration table](../reference/kernel.md#typed-kernel-construction) records unchanged interface keys, update selection and registration order. + +Consul follows the same construction pattern while retaining all three public headers and types. Its private factory is not installed; the module keeps its interface RTTI key and disabled updates. Rebuild Consul and consumers together because its public implementation/plugin layouts change. See [Consul construction migration](../project-layout.md#typed-consul-construction) for test entry points and dependency conditions. diff --git a/docs/guides/plugins_CN.md b/docs/guides/plugins_CN.md index 1458fc47d..39ba62f2a 100644 --- a/docs/guides/plugins_CN.md +++ b/docs/guides/plugins_CN.md @@ -198,3 +198,5 @@ module_update::enabled)` 启用 update；`disabled` 则不参与 update，即使 Net 通过私有工厂返回 `std::unique_ptr`，并启用该模块的 update 注册。应用使用 `` 和管理器的类型化查找；旧 `http_client_module_impl.hpp` 从公开／安装接口移除。受影响的消费者需要一起重建，详见 [Net 迁移](../reference/network_CN.md#api-与私有创建)。 Kernel 同样使用类型化构造，并保留公开 `*_impl`、`kernel_plugin` 类型与头路径；私有工厂头不安装。公开类布局发生变化，Kernel 与消费者需要一起重建。[Kernel 迁移表](../reference/kernel_CN.md#kernel-类型化构造)记录保持不变的接口键、update 选择与注册顺序。 + +Consul 使用相同构造方式，保留全部三个公开头与类型。私有工厂不安装；模块保留接口 RTTI 键和禁用 update 的选择。公开实现／插件布局发生变化，Consul 与消费者必须一起重建。测试入口与依赖条件见 [Consul 构造迁移](../project-layout_CN.md#consul-类型化构造)。 diff --git a/docs/project-layout.md b/docs/project-layout.md index 6ce9b31b3..79cff408b 100644 --- a/docs/project-layout.md +++ b/docs/project-layout.md @@ -134,3 +134,17 @@ cmake --build etcd-client-headers --target check-etcd-client-headers --parallel ``` Header mapping does not establish Consul, Kafka or OSS SDK runtime compatibility. Physic and `physics_scene` still require the unsupported legacy PhysX 3 integration and retain their explicit configuration failure; no passing PhysX build is claimed. + +## Typed Consul construction + +**Breaking ABI/layout change:** `consul_module_impl` now inherits `module_base` and `consul_plugin` inherits `plugin_base`. Rebuild Consul and consumers together. All three public headers, their `` paths, public types and request methods remain available through `ark::consul`; the interface itself is unchanged. The C++ plugin boundary does not promise compiler-independent ABI compatibility. + +The private `consul_factory.hpp` returns `std::unique_ptr` and is excluded from public mappings and installation. Explicit `DllEntryPlugin` and `DllExitPlugin` retain the existing platform exports and logger setup. Installation registers the single `consul_module` interface RTTI key with `module_update::disabled`; uninstallation removes and destroys that module. The implementation still inherits the default update method. Configuration storage, HTTP-client lookup, TOML loading, request serialization and endpoints are unchanged. + +With `ARK_BUILD_CORE_TESTS=ON`, `ARK_BUILD_PROTO=ON` and `ARK_BUILD_CONFIG=ON`, `check-consul-construction` runs the actual factory/assembly doctest cases and a controlled-factory DSO test of the actual entry/exit source. These cover public bases, factory metadata, interface registration, duplicate rejection, disabled updates, configuration handoff, ownership and repeated stop. They require Protobuf, toml++ and the normal core/arknet dependencies, but no runtime plugin or Consul service. For an existing build configured with those options: + +```sh +cmake --build build --target check-consul-construction --parallel 2 +``` + +`ARK_BUILD_PLUGIN_CONSUL=ON` also requires `ARK_BUILD_APPLICATIONS=ON` and remains off by default. It links the generated protocol target `ark::proto` directly. With the core tests enabled, `check-consul-plugin` additionally inspects the actual Consul DSO exports in a separate process; `check-core` includes the applicable Consul checks. The controlled factory does not exercise the production HTTP client or a Consul server. These construction checks do not establish live Consul interoperability, protocol acceptance or arbitrary hot-update safety. diff --git a/docs/project-layout_CN.md b/docs/project-layout_CN.md index 82a8ee230..a251e8d4e 100644 --- a/docs/project-layout_CN.md +++ b/docs/project-layout_CN.md @@ -124,3 +124,17 @@ cmake --build etcd-client-headers --target check-etcd-client-headers --parallel ``` 头文件映射检查不代表 Consul、Kafka、OSS SDK 的运行时兼容性已经验证。Physic 与 `physics_scene` 仍依赖未受支持的旧版 PhysX 3 集成，保留明确的配置失败；此处没有宣称 PhysX 构建通过。 + +## Consul 类型化构造 + +**ABI／布局不兼容变更：** `consul_module_impl` 现在继承 `module_base`，`consul_plugin` 继承 `plugin_base`。Consul 与消费者必须一起重建。原有三个公开头、`` 路径、公开类型与请求方法继续通过 `ark::consul` 提供，接口本身不变。C++ 插件边界不承诺跨编译器的 ABI 兼容性。 + +私有 `consul_factory.hpp` 返回 `std::unique_ptr`，不映射、不安装。显式 `DllEntryPlugin` 与 `DllExitPlugin` 保留原有平台导出和日志初始化。安装时按 `consul_module` 接口 RTTI 键注册唯一模块，显式使用 `module_update::disabled`；卸载时移除并销毁该模块。实现仍继承默认 update 方法。配置存储、HTTP 客户端查找、TOML 加载、请求序列化及端点保持不变。 + +开启 `ARK_BUILD_CORE_TESTS=ON`、`ARK_BUILD_PROTO=ON` 与 `ARK_BUILD_CONFIG=ON` 后，`check-consul-construction` 运行真实工厂／装配的 doctest 用例，以及使用受控工厂与真实入口／出口源码的 DSO 测试。检查覆盖公开基类、工厂元数据、接口注册、重复注册拒绝、禁用 update、配置传递、所有权与重复 stop。依赖 Protobuf、toml++ 及通常的 core／arknet 依赖，无需运行时插件或 Consul 服务。对于已按上述选项配置的构建： + +```sh +cmake --build build --target check-consul-construction --parallel 2 +``` + +`ARK_BUILD_PLUGIN_CONSUL=ON` 还要求 `ARK_BUILD_APPLICATIONS=ON`，默认保持关闭。插件直接链接生成协议目标 `ark::proto`。开启核心测试后，`check-consul-plugin` 还在独立进程中检查真实 Consul DSO 的导出符号；`check-core` 包含当前条件下可用的 Consul 检查。受控工厂不执行生产 HTTP 客户端或访问 Consul 服务。这些构造检查不代表真实 Consul 互通、协议验收或任意热更新安全。 diff --git a/docs/reference/build-options.md b/docs/reference/build-options.md index bf0875f98..52cd11ef9 100644 --- a/docs/reference/build-options.md +++ b/docs/reference/build-options.md @@ -39,7 +39,7 @@ cmake -S . -B build -DARK_BUILD_APPLICATIONS=ON \ cmake --build build --target ark_mysql_plugin ark_redis_plugin ``` -All official plugin public headers are copied from explicit lists at configuration time into each build tree's generated plugin header mapping. Each `ark::` API target carries that include root even with runtime plugins disabled. MySQL, Redis, Utility, Net, Bus and HTTP exclude private assembly and factories. Kernel retains all 38 existing headers, including `kernel_plugin.hpp`; Consul, Etcd Client, Kafka, OSS SDK and Physic retain all 22 existing public headers, including implementation and assembly headers. Registration keys, public type visibility and optional plugin policies are retained; [Kernel typed construction](kernel.md#typed-kernel-construction) changes public class layouts and requires rebuilding consumers. Its private factory header is excluded from mapping and installation; header mapping does not validate optional driver integrations. Each public header has one co-located source; its generated include mapping belongs to the selected build tree, rather than a second source layout. +All official plugin public headers are copied from explicit lists at configuration time into each build tree's generated plugin header mapping. Each `ark::` API target carries that include root even with runtime plugins disabled. MySQL, Redis, Utility, Net, Bus and HTTP exclude private assembly and factories. Kernel retains all 38 existing headers, including `kernel_plugin.hpp`; Consul, Etcd Client, Kafka, OSS SDK and Physic retain all 22 existing public headers, including implementation and assembly headers. Registration keys, public type visibility and optional plugin policies are retained; [Kernel typed construction](kernel.md#typed-kernel-construction) and [Consul typed construction](../project-layout.md#typed-consul-construction) change public class layouts and require rebuilding consumers. Their private factory headers are excluded from mapping and installation; header mapping does not validate optional driver integrations. Each public header has one co-located source; its generated include mapping belongs to the selected build tree, rather than a second source layout. Link a consumer to `ark::mysql` and/or `ark::redis` for the public headers and usage requirements. These public interface targets do not load a runtime library. Deploy the selected libraries with the launcher and add their stems to the application's manifest: diff --git a/docs/reference/build-options_CN.md b/docs/reference/build-options_CN.md index da133ef1b..447194a30 100644 --- a/docs/reference/build-options_CN.md +++ b/docs/reference/build-options_CN.md @@ -39,7 +39,7 @@ cmake -S . -B build -DARK_BUILD_APPLICATIONS=ON \ cmake --build build --target ark_mysql_plugin ark_redis_plugin ``` -全部官方插件的公开头通过显式清单在配置阶段复制到各构建树生成的插件公开头映射。各 `ark::` API target 传递该目录，插件关闭时仍可使用。MySQL、Redis、Utility、Net、Bus、HTTP 的私有装配头和工厂不进入映射；Kernel 保留包括 `kernel_plugin.hpp` 在内的原有 38 个头；Consul、Etcd Client、Kafka、OSS SDK、Physic 保留包括实现头与装配头在内的原有 22 个公开头。注册键、公开类型可见性和可选插件策略继续保留；[Kernel 类型化构造](kernel_CN.md#kernel-类型化构造)改变公开类布局，需要重建消费者。其私有工厂头不映射、不安装；头文件映射不代表可选驱动集成已经验证。每个公共头仅有一份共置源码；生成的头文件映射属于选定构建树，不构成第二棵源码目录。 +全部官方插件的公开头通过显式清单在配置阶段复制到各构建树生成的插件公开头映射。各 `ark::` API target 传递该目录，插件关闭时仍可使用。MySQL、Redis、Utility、Net、Bus、HTTP 的私有装配头和工厂不进入映射；Kernel 保留包括 `kernel_plugin.hpp` 在内的原有 38 个头；Consul、Etcd Client、Kafka、OSS SDK、Physic 保留包括实现头与装配头在内的原有 22 个公开头。注册键、公开类型可见性和可选插件策略继续保留；[Kernel 类型化构造](kernel_CN.md#kernel-类型化构造)与 [Consul 类型化构造](../project-layout_CN.md#consul-类型化构造)改变公开类布局，需要重建消费者。两者的私有工厂头均不映射、不安装；头文件映射不代表可选驱动集成已经验证。每个公共头仅有一份共置源码；生成的头文件映射属于选定构建树，不构成第二棵源码目录。 业务插件链接 `ark::mysql` 和/或 `ark::redis`，以获取公开头文件和编译要求；公共接口目标不会加载运行时动态库。将所需动态库与启动器一起部署，并在应用清单中填写库名称： diff --git a/src/plugins/README.md b/src/plugins/README.md index 07bbf6720..66e429e98 100755 --- a/src/plugins/README.md +++ b/src/plugins/README.md @@ -26,4 +26,6 @@ See [project layout and public plugin interfaces](../../docs/project-layout.md) Kernel co-locates its 38 existing headers with its cpp files in `src/plugins/kernel/`. `ark::kernel` preserves every `` path, including `data_impl.hpp`, `data_list_impl.hpp` and `kernel_plugin.hpp`. The retained public implementation/plugin types use typed construction with private owning factories and explicit entry points. `kernel_factory.hpp` is neither mapped nor installed. The public class layouts change, so rebuild Kernel and consumers together; registration keys, update selection and lifecycle order are preserved. See [Kernel construction migration](../../docs/reference/kernel.md#typed-kernel-construction) for compatibility boundaries, consumption checks and Protobuf requirements. -The five remaining plugin families retain 22 byte-identical headers. Mapping checks do not validate Consul, Kafka or OSS SDK runtime integrations. Physic retains the unsupported PhysX 3 configuration failure; see [optional legacy dependencies](../../docs/dependencies.md#optional-legacy-integrations). +Consul retains its three public headers and implementation/plugin types while using typed construction, a private owning factory and explicit entry points. Rebuild Consul and consumers together because the public class layouts change. The single interface registration key, disabled updates and request behavior are preserved. See [Consul construction migration](../../docs/project-layout.md#typed-consul-construction) for build conditions and service-free construction checks. + +The other four legacy plugin families retain 19 byte-identical headers. Mapping checks do not validate Consul, Kafka or OSS SDK runtime integrations. Physic retains the unsupported PhysX 3 configuration failure; see [optional legacy dependencies](../../docs/dependencies.md#optional-legacy-integrations). diff --git a/src/plugins/README_CN.md b/src/plugins/README_CN.md index 188679dc2..82c42abd5 100644 --- a/src/plugins/README_CN.md +++ b/src/plugins/README_CN.md @@ -26,4 +26,6 @@ CMake target 和构建命令见[项目布局与公共插件接口](../../docs/pr Kernel 的原有 38 个头与 cpp 共置于 `src/plugins/kernel/`，通过 `ark::kernel` 保留全部 `` 路径，包括 `data_impl.hpp`、`data_list_impl.hpp` 和 `kernel_plugin.hpp`。保留的公开实现类型与插件类型使用类型化构造、私有拥有型工厂和显式入口。`kernel_factory.hpp` 不映射、不安装。公开类布局发生变化，Kernel 与消费者必须一起重建；注册键、update 选择和生命周期顺序保持不变。兼容边界、消费检查及 Protobuf 条件见 [Kernel 构造迁移](../../docs/reference/kernel_CN.md#kernel-类型化构造)。 -其余五类插件保留原有 22 个头的全部字节。映射检查不代表 Consul、Kafka、OSS SDK 的运行时集成已经验证。Physic 保留未受支持的 PhysX 3 配置失败；参见[可选旧集成依赖](../../docs/dependencies_CN.md)。 +Consul 保留原有三个公开头及实现／插件类型，同时使用类型化构造、私有拥有型工厂和显式入口。公开类布局发生变化，Consul 与消费者必须一起重建。唯一接口注册键、禁用 update 与请求行为保持不变。构建条件及无需服务的构造检查见 [Consul 构造迁移](../../docs/project-layout_CN.md#consul-类型化构造)。 + +其他四类旧插件保留原有 19 个头的全部字节。映射检查不代表 Consul、Kafka、OSS SDK 的运行时集成已经验证。Physic 保留未受支持的 PhysX 3 配置失败；参见[可选旧集成依赖](../../docs/dependencies_CN.md)。 diff --git a/src/plugins/consul/CMakeLists.txt b/src/plugins/consul/CMakeLists.txt index df78a2209..20b422335 100755 --- a/src/plugins/consul/CMakeLists.txt +++ b/src/plugins/consul/CMakeLists.txt @@ -1,17 +1 @@ -if(WIN32) - BUILD_PLUGIN_MACRO(ark_consul_plugin - debug AFProto_d.lib - optimized AFProto.lib - ${PROTOBUF_LIBRARIES}) -elseif(APPLE) - BUILD_PLUGIN_MACRO(ark_consul_plugin - debug ${BIN_OUTPUT_DIR}/AFProto_d.a - optimized ${BIN_OUTPUT_DIR}/AFProto.a - ${PROTOBUF_LIBRARIES}) -else(UNIX) - BUILD_PLUGIN_MACRO(ark_consul_plugin - debug :AFProto_d.a - optimized :AFProto.a - ${PROTOBUF_LIBRARIES}) -endif() -set_property(TARGET ark_consul_plugin APPEND PROPERTY LINK_LIBRARIES ark::config tomlplusplus::tomlplusplus) +BUILD_PLUGIN_MACRO(ark_consul_plugin ark::proto ark::config tomlplusplus::tomlplusplus) diff --git a/src/plugins/consul/consul_factory.hpp b/src/plugins/consul/consul_factory.hpp new file mode 100644 index 000000000..d65b95c48 --- /dev/null +++ b/src/plugins/consul/consul_factory.hpp @@ -0,0 +1,13 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#pragma once + +#include + +#include "ark/consul/consul_module.hpp" + +// Plugin-private construction; deliberately absent from public header mappings. +namespace ark::detail { + +std::unique_ptr make_consul_module(); + +} // namespace ark::detail diff --git a/src/plugins/consul/consul_module_impl.cpp b/src/plugins/consul/consul_module_impl.cpp index df4cc4b1a..65f8f07fb 100644 --- a/src/plugins/consul/consul_module_impl.cpp +++ b/src/plugins/consul/consul_module_impl.cpp @@ -3,9 +3,14 @@ #include "../detail/plugin_configuration.hpp" #include "ark/consul/consul_module_impl.hpp" #include "ark/consul/consul_plugin.hpp" +#include "consul_factory.hpp" namespace ark { +std::unique_ptr detail::make_consul_module() { + return std::make_unique(); +} + namespace { std::future> failed_request() { diff --git a/src/plugins/consul/consul_module_impl.hpp b/src/plugins/consul/consul_module_impl.hpp index b2cb20c0b..be2adec8c 100644 --- a/src/plugins/consul/consul_module_impl.hpp +++ b/src/plugins/consul/consul_module_impl.hpp @@ -1,14 +1,13 @@ #pragma once #include "ark/consul/consul_module.hpp" +#include "ark/module_base.hpp" #include "ark/net/http_client_module.hpp" -#include "ark/plugin_manager.hpp" namespace ark { -class consul_module_impl final : public consul_module +class consul_module_impl final : public module_base { - ARK_DECLARE_MODULE_FUNCTIONS public: bool init() override; diff --git a/src/plugins/consul/consul_plugin.cpp b/src/plugins/consul/consul_plugin.cpp index 1327a4d31..c6feb77ce 100644 --- a/src/plugins/consul/consul_plugin.cpp +++ b/src/plugins/consul/consul_plugin.cpp @@ -1,16 +1,31 @@ -#include "ark/consul/consul_module_impl.hpp" +#include + #include "ark/consul/consul_plugin.hpp" +#include "consul_factory.hpp" namespace ark { -ARK_DECLARE_PLUGIN_DLL_FUNCTION(consul_plugin) +ARK_EXPORT_FUNC void DllEntryPlugin(plugin_manager* manager, const std::string& name, logger* log) { + logger::init(log); + manager->register_plugin(name); +} + +ARK_EXPORT_FUNC void DllExitPlugin(plugin_manager* manager) { + manager->unregister_plugin(); +} void consul_plugin::install() { - ARK_REGISTER_MODULE(consul_module, consul_module_impl); + if (!register_module(detail::make_consul_module(), module_update::disabled)) + { + throw std::runtime_error("Consul module registration rejected"); + } } void consul_plugin::uninstall() { - ARK_UNREGISTER_MODULE(consul_module, consul_module_impl); + if (!unregister_module()) + { + throw std::runtime_error("Consul module unregistration rejected"); + } } } // namespace ark diff --git a/src/plugins/consul/consul_plugin.hpp b/src/plugins/consul/consul_plugin.hpp index d8b93aa2b..2aadec022 100755 --- a/src/plugins/consul/consul_plugin.hpp +++ b/src/plugins/consul/consul_plugin.hpp @@ -1,8 +1,14 @@ -#include "ark/plugin.hpp" -#include "ark/plugin_manager.hpp" +#pragma once + +#include "ark/plugin_base.hpp" namespace ark { -ARK_DECLARE_PLUGIN(consul_plugin) +class consul_plugin final : public plugin_base +{ +public: + void install() override; + void uninstall() override; +}; } // namespace ark diff --git a/tests/consul/CMakeLists.txt b/tests/consul/CMakeLists.txt new file mode 100644 index 000000000..fc45c3e15 --- /dev/null +++ b/tests/consul/CMakeLists.txt @@ -0,0 +1,36 @@ +# Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +set(consul_source "${PROJECT_SOURCE_DIR}/src/plugins/consul") + +# Compile real assembly/factory locally too: private C++ symbols are not DLL exports. +add_executable(ark_consul_construction_tests ../core/main.cpp construction.cpp + "${consul_source}/consul_plugin.cpp" "${consul_source}/consul_module_impl.cpp") +set_property(TARGET ark_consul_construction_tests PROPERTY LINK_LIBRARIES + ark::consul ark::proto ark::config tomlplusplus::tomlplusplus) +target_include_directories(ark_consul_construction_tests PRIVATE "${consul_source}") + +# A controlled module exercises the actual entry/exit functions without HTTP or Consul services. +add_library(ark_consul_entry_probe SHARED entry_factory.cpp "${consul_source}/consul_plugin.cpp") +set_property(TARGET ark_consul_entry_probe PROPERTY LINK_LIBRARIES ark::consul ark::proto) +target_include_directories(ark_consul_entry_probe PRIVATE "${consul_source}") +set_target_properties(ark_consul_entry_probe PROPERTIES PREFIX "") +add_executable(ark_consul_entry_tests entry.cpp) +set_property(TARGET ark_consul_entry_tests PROPERTY LINK_LIBRARIES ark::consul ark::proto) + +add_custom_target(check-consul-construction + COMMAND $ + COMMAND $ "$/" "ark_consul_entry_probe" + DEPENDS ark_consul_construction_tests ark_consul_entry_tests ark_consul_entry_probe + USES_TERMINAL VERBATIM) + +if(TARGET ark_consul_plugin) + # Keep real DSO export inspection separate from the host linking static AFProto. + add_executable(ark_consul_exports_tests exports.cpp) + set_property(TARGET ark_consul_exports_tests PROPERTY LINK_LIBRARIES ark::core) + add_custom_target(check-consul-plugin + COMMAND $ "$/" "ark_consul_plugin" + DEPENDS ark_consul_exports_tests ark_consul_plugin check-consul-construction + USES_TERMINAL VERBATIM) + add_dependencies(check-core check-consul-plugin) +else() + add_dependencies(check-core check-consul-construction) +endif() diff --git a/tests/consul/construction.cpp b/tests/consul/construction.cpp new file mode 100644 index 000000000..c466d65d7 --- /dev/null +++ b/tests/consul/construction.cpp @@ -0,0 +1,53 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include +#include +#include + +#include "../network/doctest.h" + +#include +#include +#include "consul_factory.hpp" + +#if __has_include() +#error "Consul public mappings must exclude the private factory" +#endif + +static_assert(std::is_base_of_v, ark::consul_module_impl>); +static_assert(std::is_base_of_v, ark::consul_plugin>); +static_assert(std::is_same_v); + +TEST_CASE("Consul real factory retains its public implementation and unpublished metadata") { + auto candidate = ark::detail::make_consul_module(); + REQUIRE(candidate != nullptr); + CHECK(typeid(*candidate) == typeid(ark::consul_module_impl)); + CHECK(candidate->get_plugin_manager() == nullptr); + CHECK(candidate->get_name().empty()); +} + +TEST_CASE("Consul assembly retains the interface key, disabled updates and explicit ownership") { + ark::plugin_manager manager; + ark::consul_plugin owner; + owner.set_plugin_manager(&manager); + CHECK(owner.get_plugin_version() == 0); + CHECK(owner.get_plugin_name() == typeid(ark::consul_plugin).name()); + owner.set_plugin_conf("consul-construction.toml"); + CHECK(ark::consul_plugin::get_plugin_conf() == "consul-construction.toml"); + owner.install(); + auto* instance = manager.find_module(); + REQUIRE(instance != nullptr); + CHECK(typeid(*instance) == typeid(ark::consul_module_impl)); + CHECK(instance->get_plugin_manager() == &manager); + CHECK(instance->get_name() == typeid(ark::consul_module).name()); + // Insertion succeeds only when assembly left the module out of the update set. + CHECK(manager.add_update_module(instance)); + manager.remove_update_module(instance->get_name()); + CHECK_THROWS_AS(owner.install(), std::runtime_error); + CHECK(manager.find_module() == instance); + owner.uninstall(); + // Reinstallation proves the public registration key and ownership were released. + CHECK_NOTHROW(owner.install()); + CHECK_NOTHROW(owner.uninstall()); + CHECK(manager.update(1)); + CHECK_THROWS_AS(owner.uninstall(), std::runtime_error); +} diff --git a/tests/consul/entry.cpp b/tests/consul/entry.cpp new file mode 100644 index 000000000..597dd1244 --- /dev/null +++ b/tests/consul/entry.cpp @@ -0,0 +1,45 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include +#include +#include + +#include +#include +#include + +namespace { + +void require(bool condition, const char* message) { + if (!condition) + { + std::cerr << message << std::endl; + std::_Exit(1); + } +} + +} // namespace + +int main(int argc, char* argv[]) { + require(argc == 3, "Expected Consul entry-probe directory and library stem"); + ark::logger log; + ark::logger::init(&log); + log.set_log_func([](spdlog::level::level_enum, const std::string&) {}); + ark::dynamic_library library(argv[2]); + require(library.load(argv[1]), "Cannot load Consul entry-probe library"); + auto events = reinterpret_cast(library.get_symbol("consul_entry_events")); + require(events && events(0) == 0 && events(1) == 0, "Entry-probe state is not fresh"); + require(library.get_symbol("DllEntryPlugin") && library.get_symbol("DllExitPlugin"), + "Consul explicit entry/exit exports are missing"); + ark::plugin_manager manager; + require(manager.start(0, argv[1], "conf/", {{argv[2], "entry-probe.toml"}}), "Consul explicit entry failed"); + auto* instance = manager.find_module(); + require(instance && instance->get_plugin_manager() == &manager && + instance->get_name() == typeid(ark::consul_module).name(), + "Consul explicit entry lost interface registration metadata"); + require(events(0) == 1 && events(1) == 0, "Consul entry did not transfer factory ownership"); + require(manager.update(1) && events(2) == 0, "Consul entry changed disabled update selection"); + require(manager.stop(2) && events(1) == 1, "Consul explicit exit did not destroy its module exactly once"); + require(manager.stop(3) && events(1) == 1, "Repeated stop invoked Consul exit twice"); + require(library.unload(), "Cannot release Consul observation handle"); + std::cout << "Consul real assembly entry/exit, configuration, interface metadata and ownership passed\n"; +} diff --git a/tests/consul/entry_factory.cpp b/tests/consul/entry_factory.cpp new file mode 100644 index 000000000..d9f92b51b --- /dev/null +++ b/tests/consul/entry_factory.cpp @@ -0,0 +1,79 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include + +#include +#include +#include "consul_factory.hpp" + +namespace { + +unsigned created{}; +unsigned destroyed{}; +unsigned updated{}; + +class entry_consul final : public ark::module_base +{ +public: + entry_consul() { + ++created; + } + + ~entry_consul() override { + ++destroyed; + } + + bool init() override { + return ark::consul_plugin::get_plugin_conf() == "conf/entry-probe.toml"; + } + + bool update() override { + ++updated; + return true; + } + + std::future> register_service(const ark::consul::service_data&) override { + return {}; + } + + std::future> deregister_service(const std::string&) override { + return {}; + } + + std::future> get_health_services(const std::string&, const std::string&) override { + return {}; + } + + std::future> get_key_value(const std::string&) override { + return {}; + } + + std::future> set_key_value(const std::string&, const std::string&) override { + return {}; + } + + std::future> del_key_value(const std::string&) override { + return {}; + } +}; + +} // namespace + +namespace ark::detail { + +std::unique_ptr make_consul_module() { + return std::make_unique(); +} + +} // namespace ark::detail + +ARK_EXPORT_FUNC unsigned consul_entry_events(unsigned event) { + switch (event) + { + case 0: + return created; + case 1: + return destroyed; + default: + return updated; + } +} diff --git a/tests/consul/exports.cpp b/tests/consul/exports.cpp new file mode 100644 index 000000000..a5cbd76dc --- /dev/null +++ b/tests/consul/exports.cpp @@ -0,0 +1,31 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include +#include +#include + +#include + +namespace { + +void require(bool condition, const char* message) { + if (!condition) + { + std::cerr << message << std::endl; + std::_Exit(1); + } +} + +} // namespace + +int main(int argc, char* argv[]) { + require(argc == 3, "Expected plugin directory and library stem"); + ark::logger log; + ark::logger::init(&log); + log.set_log_func([](spdlog::level::level_enum, const std::string&) {}); + ark::dynamic_library library(argv[2]); + require(library.load(argv[1]), "Cannot load actual Consul shared library"); + require(library.get_symbol("DllEntryPlugin") != nullptr, "Missing explicit DllEntryPlugin export"); + require(library.get_symbol("DllExitPlugin") != nullptr, "Missing explicit DllExitPlugin export"); + require(library.unload(), "Cannot unload Consul shared library"); + std::cout << argv[2] << ": actual DSO load, explicit entry/exit exports and unload passed\n"; +} diff --git a/tests/core/CMakeLists.txt b/tests/core/CMakeLists.txt index e027b5fe2..ae1f7a9e2 100644 --- a/tests/core/CMakeLists.txt +++ b/tests/core/CMakeLists.txt @@ -222,6 +222,10 @@ if(ARK_BUILD_PROTO) add_dependencies(check-core check-kernel-initialization) endif() +if(ARK_BUILD_PROTO AND ARK_BUILD_CONFIG) + add_subdirectory(../consul "${CMAKE_CURRENT_BINARY_DIR}/consul") +endif() + if(ARK_BUILD_PROTO AND TARGET ark_kernel_plugin AND TARGET ark_utility_plugin) add_executable(ark_kernel_plugin_tests ../app/kernel_plugin.cpp) set_property(TARGET ark_kernel_plugin_tests PROPERTY LINK_LIBRARIES ark::kernel) From fc9035a0d68d3b52d19377e8777435d4471067d8 Mon Sep 17 00:00:00 2001 From: nick Date: Mon, 26 Oct 2026 09:30:13 +0800 Subject: [PATCH 02/22] fix(hash)!: standardize Murmur digests and retain legacy MSVC access Read hash blocks and write digest bytes without alignment or aliasing assumptions. Use C++20 rotations and provide explicit legacy MSVC methods for persisted-key migration. BREAKING CHANGE: default MurmurHash3 now uses canonical little-endian blocks and output. Old MSVC IDs, shard mappings and persisted keys require an explicit migration or the matching legacy MSVC method. --- docs/reference/utility.md | 76 ++++++++++ docs/reference/utility_CN.md | 57 ++++++++ src/core/hash/murmur_hash.hpp | 168 +++++++++++++++-------- tests/core/CMakeLists.txt | 2 +- tests/core/README.md | 10 ++ tests/core/README_CN.md | 8 ++ tests/core/murmur_hash.cpp | 251 ++++++++++++++++++++++++++++++++++ 7 files changed, 514 insertions(+), 58 deletions(-) create mode 100644 tests/core/murmur_hash.cpp diff --git a/docs/reference/utility.md b/docs/reference/utility.md index f468f5e16..22f3f5146 100644 --- a/docs/reference/utility.md +++ b/docs/reference/utility.md @@ -64,6 +64,82 @@ Public headers now have their single source in `src/plugins/utility/`. CMake cop Range selection retains the existing modulo mapping, seed sequence and draw count. Each call with different endpoints consumes one draw; reversing the endpoints produces the same sequence from the same seed. The unsigned overloads retain their existing behavior. +## Core Murmur hashes + +`` provides MurmurHash2 and the x86-32, x86-128 and +x64-128 MurmurHash3 variants through `ark::core`. Supply a nonnegative `int` +length, a readable input buffer (also for an empty input), and writable output +storage of 4 bytes for 32-bit variants or 16 bytes for 128-bit variants. Input +and output may have any byte alignment; word loads and stores use `std::memcpy`. +MurmurHash3 uses C++20 `std::rotl` with the original rotation counts. Seeds, +tail mixing, result word order and the existing public signatures are unchanged; +`murmur_hash3_x86_128` remains an instance method, and the other variants remain +static. MurmurHash2 retains its fixed seed zero. + +### Digest compatibility and legacy access + +**Breaking change:** default MurmurHash3 now reads full blocks as little endian +and writes canonical little-endian digest bytes on every supported platform. +This retains existing little-endian GCC/Clang results and changes the former +ordinary MSVC full-block policy. The former platform detection did not select +MSVC's `_M_IX86`, `_M_X64` or `_M_ARM64` macros +([Microsoft predefined macros](https://learn.microsoft.com/en-us/cpp/preprocessor/predefined-macros)); +it fell through to an unconditional byte swap on a little-endian host. +The public `BYTESWAP32` and `BYTESWAP64` macros remain available, but no longer +control these hash methods. Callers must rebuild to use the new header behavior. + +| Method family | Default result | Previous ordinary MSVC result | +| --- | --- | --- | +| MurmurHash3 x86-32 | `murmur_hash3_x86_32` | `murmur_hash3_x86_32_legacy_msvc` | +| MurmurHash3 x86-128 | `murmur_hash3_x86_128` (instance method) | `murmur_hash3_x86_128_legacy_msvc` (instance method) | +| MurmurHash3 x64-128 | `murmur_hash3_x64_128` | `murmur_hash3_x64_128_legacy_msvc` | +| MurmurHash2 | `murmur_hash2`; unchanged on little-endian platforms | Same method; no separate legacy route is needed | + +The three `_legacy_msvc` methods reproduce the old MSVC result on any supported +platform: full blocks are read as big endian, tails are assembled as little +endian, and output words are serialized as little endian. Pass the original +variant, exact input bytes, length and seed. The policies agree for inputs shorter +than a full block (4 bytes for x86-32, 16 bytes for either 128-bit variant). +For bytes `00 01 02 03` and seed zero, x86-32 produces numeric word `0xf4c0ec39` +by default and `0x2cb313f3` through the legacy entry. A previous big-endian +consumer must also account for the change from native-endian output to canonical +little-endian bytes; `_legacy_msvc` specifically reproduces old MSVC behavior. + +### Migrate stored keys and routing + +Identify each dataset's hash variant, seed, input encoding and previous compiler +policy before upgrading. Keep that policy with the data or in an explicit schema +version. A digest alone cannot reconstruct the original key. IDs, shard mappings, +cache keys and persisted indexes derived from old MSVC hashes require an explicit +migration; replacing the function and silently recomputing routing is unsafe. + +For lookup keys, use a staged **dual-read / new-write** rollout: write new records +with the standard policy and a version tag, read that key first, and fall back to +the matching `_legacy_msvc` key for older records. Compare the original logical +key when resolving a hash hit or collision. Backfill or move old entries using +transactional/atomic application rules, define duplicate handling, and retain +legacy reads until inventory and rollback requirements permit their removal. +Old readers must understand new records before new-only writes begin. The library +does not migrate application records automatically. + +For consistent hashing or shard ownership, keep all routers on one explicit ring +version while planning the transition. Compute the new ring from authoritative +node identities, compare placement, copy/rebalance affected data, and coordinate +the routing epoch and ownership cutover across participants. Keep the legacy +ring and legacy hash entry until that transfer is accepted. Mixed old/new rings +during an ordinary rolling binary update can route the same key differently; +a dual lookup alone does not migrate ownership. + +`check-core` includes the [upstream SMHasher verification values](https://github.com/aappleby/smhasher/blob/master/src/main.cpp#L67-L70) +with its [prefix/seed procedure](https://github.com/aappleby/smhasher/blob/master/src/KeysetTest.cpp#L11-L38), +plus separate scalar-reference fixtures for the legacy MSVC policy. Cases cover +all seven public methods, every MurmurHash3 tail length, full-width seeds, +high-bit input bytes and rotation paths, MurmurHash2 boundary vectors, and +exact-size byte buffers at offsets 0 through 7. AddressSanitizer can detect +allocation-end overreads; UndefinedBehaviorSanitizer can detect invalid alignment. +The fixtures establish hash behavior; application data migration needs its own +acceptance evidence. + ## Verification `check-typed-examples` loads the actual Utility DSO with `timers`, `module_lifecycle` and `entity_operations`. It verifies all four interface RTTI keys and manager ownership, GUID generation/parsing, zlib/Zstandard binary round trips, a timer triggered by owner-thread time advancement, and normal/idempotent shutdown. `check-app-presets` exercises the shipped launcher preset; `check-core` includes both when applications, examples, Kernel and Utility are enabled. The independent `tests/consumer` project's `check-sdk` compiles public/helper paths with runtime plugins disabled and rejects visible implementation or assembly headers. These are correctness checks, not performance or arbitrary callback-lifetime acceptance. diff --git a/docs/reference/utility_CN.md b/docs/reference/utility_CN.md index 933c8f645..f9262cc25 100644 --- a/docs/reference/utility_CN.md +++ b/docs/reference/utility_CN.md @@ -58,6 +58,63 @@ zlib、LZ4 和 Zstandard 普通调用不再共享可写输出缓冲区。Zstanda 范围选择保留现有的取模映射、种子序列和生成次数。端点不同的每次调用消耗一个随机数；使用相同种子并交换端点，得到相同序列。无符号重载的行为保持不变。 +## Core Murmur 哈希 + +`` 通过 `ark::core` 提供 MurmurHash2，以及 MurmurHash3 +的 x86-32、x86-128 和 x64-128 变体。调用方须提供非负 `int` 长度、可读输入缓冲区 +（空输入也需要有效缓冲区），以及可写输出存储：32 位变体需要 4 字节，128 位变体需要 +16 字节。输入和输出允许任意字节对齐，字读取和写入使用 `std::memcpy`。 +MurmurHash3 使用 C++20 `std::rotl`，旋转次数保持不变。种子、尾部混合、结果字顺序和 +原有公开签名不变；`murmur_hash3_x86_128` 仍为实例方法，其余变体仍为静态方法。 +MurmurHash2 保留固定种子零。 + +### 摘要兼容性与旧结果入口 + +**不兼容变更：** 默认 MurmurHash3 现在在所有支持的平台上按小端方式读取完整块， +并输出规范的小端摘要字节。这保留原有小端 GCC/Clang 结果，改变普通 MSVC 的旧版完整块策略。 +旧版平台检测未选择 MSVC 的 `_M_IX86`、`_M_X64` 或 `_M_ARM64` 宏 +（见 [Microsoft 预定义宏](https://learn.microsoft.com/en-us/cpp/preprocessor/predefined-macros)）， +导致小端主机进入无条件字节交换回退分支。公开宏 `BYTESWAP32` 和 `BYTESWAP64` +仍然保留，但不再控制这些哈希方法。调用方须重新编译，才能使用新的头文件行为。 + +| 方法族 | 默认结果 | 普通 MSVC 的旧结果 | +| --- | --- | --- | +| MurmurHash3 x86-32 | `murmur_hash3_x86_32` | `murmur_hash3_x86_32_legacy_msvc` | +| MurmurHash3 x86-128 | `murmur_hash3_x86_128`（实例方法） | `murmur_hash3_x86_128_legacy_msvc`（实例方法） | +| MurmurHash3 x64-128 | `murmur_hash3_x64_128` | `murmur_hash3_x64_128_legacy_msvc` | +| MurmurHash2 | `murmur_hash2`；在小端平台上保持不变 | 使用同一方法，无需独立旧版入口 | + +三个 `_legacy_msvc` 方法在所有支持的平台上复现旧 MSVC 结果：完整块按大端方式读取， +尾部按小端方式组装，输出字按小端方式序列化。调用时须使用原有变体、准确输入字节、 +长度和种子。输入短于完整块时，两种策略结果相同：x86-32 为不足 4 字节， +两个 128 位变体为不足 16 字节。对于字节 `00 01 02 03` 和种子零，x86-32 默认产生 +数值字 `0xf4c0ec39`，旧版入口产生 `0x2cb313f3`。旧版大端使用方还须处理输出从 +原生字节序变为规范小端字节序的变化；`_legacy_msvc` 专用于复现旧 MSVC 行为。 + +### 迁移已存储键与路由 + +升级前，先确认每个数据集的哈希变体、种子、输入编码和旧编译器策略，并在数据中或 +显式 schema 版本中记录该策略。单独的摘要无法还原原始键。由旧 MSVC 哈希派生的 ID、 +分片映射、缓存键和持久化索引需要显式迁移，不能替换函数后直接重算路由。 + +对于查找键，分阶段采用**双读、新写**：新记录使用标准策略并携带版本标记；读取时 +先查新键，未命中再查匹配的 `_legacy_msvc` 旧键。解析命中或哈希冲突时，核对原始逻辑键。 +按应用的事务或原子操作规则回填或移动旧记录，明确重复记录的处理方式；确认存量清理和 +回滚要求后，再移除旧读路径。开始只写新格式前，旧读程序须能理解新记录。 +库本身不会自动迁移应用记录。 + +对于一致性哈希或分片归属，规划迁移时保持所有路由器使用同一个显式环版本。 +从权威节点标识构建新环，比较分配结果，复制或重平衡受影响数据，再协调各参与方的 +路由 epoch 与归属切换。数据转移验收前，保留旧环和旧哈希入口。普通二进制滚动升级中 +混用新旧环，可能把同一键路由到不同位置；仅增加双查不会完成归属迁移。 + +`check-core` 纳入[上游 SMHasher 验证值](https://github.com/aappleby/smhasher/blob/master/src/main.cpp#L67-L70) +及其[前缀和种子验证过程](https://github.com/aappleby/smhasher/blob/master/src/KeysetTest.cpp#L11-L38)， +并为旧 MSVC 策略保留独立的标量参考固定值。用例覆盖全部七个公开方法、MurmurHash3 的 +全部尾部长度、完整宽度种子、高位字节和旋转路径、MurmurHash2 块边界固定值，以及 +偏移 0 到 7 的精确分配字节缓冲区。AddressSanitizer 可检查分配末端越界读取， +UndefinedBehaviorSanitizer 可检查无效对齐。固定值验证哈希行为；应用数据迁移仍须独立验收。 + ## 验证 `check-typed-examples` 与 timers、module_lifecycle、entity_operations 一起加载真实 Utility DSO，验证四个接口的 RTTI 注册键与管理器归属、GUID 生成与解析、zlib/Zstandard 二进制往返、由所属线程时间推进触发的定时器，以及正常和重复停机。`check-app-presets` 使用仓库启动器预设；应用、示例、Kernel、Utility 开启时，`check-core` 包含这两组检查。独立 `tests/consumer` 项目的 `check-sdk` 在关闭运行时插件时编译公开接口与辅助头路径，并拒绝可见的实现或装配头。这些检查验证正确性，不代表性能或任意回调生命周期验收。 diff --git a/src/core/hash/murmur_hash.hpp b/src/core/hash/murmur_hash.hpp index 128e42bcf..20fbdc7fd 100644 --- a/src/core/hash/murmur_hash.hpp +++ b/src/core/hash/murmur_hash.hpp @@ -22,6 +22,12 @@ #pragma once +#include +#include +#include +#include +#include + #include "ark/macros.hpp" #include "ark/platform.hpp" @@ -31,6 +37,33 @@ class murmur_hash { public: static void murmur_hash3_x86_32(const void* key, int len, uint32_t seed, void* out) { + murmur_hash3_x86_32_impl(key, len, seed, out); + } + + void murmur_hash3_x86_128(const void* key, const int len, uint32_t seed, void* out) { + murmur_hash3_x86_128_impl(key, len, seed, out); + } + + static void murmur_hash3_x64_128(const void* key, const int len, const uint32_t seed, void* out) { + murmur_hash3_x64_128_impl(key, len, seed, out); + } + + // Migration-only access to hashes produced by the former MSVC block policy. + static void murmur_hash3_x86_32_legacy_msvc(const void* key, int len, uint32_t seed, void* out) { + murmur_hash3_x86_32_impl(key, len, seed, out); + } + + void murmur_hash3_x86_128_legacy_msvc(const void* key, const int len, uint32_t seed, void* out) { + murmur_hash3_x86_128_impl(key, len, seed, out); + } + + static void murmur_hash3_x64_128_legacy_msvc(const void* key, const int len, const uint32_t seed, void* out) { + murmur_hash3_x64_128_impl(key, len, seed, out); + } + +private: + template + static void murmur_hash3_x86_32_impl(const void* key, int len, uint32_t seed, void* out) { const uint8_t* data = (const uint8_t*)key; const int nblocks = len / 4; @@ -42,18 +75,18 @@ class murmur_hash //---------- // body - const uint32_t* blocks = (const uint32_t*)(data + nblocks * 4); + const uint8_t* blocks = data + nblocks * 4; for (int i = -nblocks; i; i++) { - uint32_t k1 = getblock(blocks, i); + uint32_t k1 = getblock(blocks, i); k1 *= c1; - k1 = rotl32(k1, 15); + k1 = std::rotl(k1, 15); k1 *= c2; h1 ^= k1; - h1 = rotl32(h1, 13); + h1 = std::rotl(h1, 13); h1 = h1 * 5 + 0xe6546b64; } @@ -73,7 +106,7 @@ class murmur_hash case 1: k1 ^= tail[0]; k1 *= c1; - k1 = rotl32(k1, 15); + k1 = std::rotl(k1, 15); k1 *= c2; h1 ^= k1; }; @@ -85,12 +118,13 @@ class murmur_hash h1 = fmix(h1); - *(uint32_t*)out = h1; + store_word(out, h1); } //----------------------------------------------------------------------------- - void murmur_hash3_x86_128(const void* key, const int len, uint32_t seed, void* out) { + template + static void murmur_hash3_x86_128_impl(const void* key, const int len, uint32_t seed, void* out) { const uint8_t* data = (const uint8_t*)key; const int nblocks = len / 16; @@ -107,48 +141,48 @@ class murmur_hash //---------- // body - const uint32_t* blocks = (const uint32_t*)(data + nblocks * 16); + const uint8_t* blocks = data + nblocks * 16; for (int i = -nblocks; i; i++) { - uint32_t k1 = getblock(blocks, i * 4 + 0); - uint32_t k2 = getblock(blocks, i * 4 + 1); - uint32_t k3 = getblock(blocks, i * 4 + 2); - uint32_t k4 = getblock(blocks, i * 4 + 3); + uint32_t k1 = getblock(blocks, i * 4 + 0); + uint32_t k2 = getblock(blocks, i * 4 + 1); + uint32_t k3 = getblock(blocks, i * 4 + 2); + uint32_t k4 = getblock(blocks, i * 4 + 3); k1 *= c1; - k1 = rotl32(k1, 15); + k1 = std::rotl(k1, 15); k1 *= c2; h1 ^= k1; - h1 = rotl32(h1, 19); + h1 = std::rotl(h1, 19); h1 += h2; h1 = h1 * 5 + 0x561ccd1b; k2 *= c2; - k2 = rotl32(k2, 16); + k2 = std::rotl(k2, 16); k2 *= c3; h2 ^= k2; - h2 = rotl32(h2, 17); + h2 = std::rotl(h2, 17); h2 += h3; h2 = h2 * 5 + 0x0bcaa747; k3 *= c3; - k3 = rotl32(k3, 17); + k3 = std::rotl(k3, 17); k3 *= c4; h3 ^= k3; - h3 = rotl32(h3, 15); + h3 = std::rotl(h3, 15); h3 += h4; h3 = h3 * 5 + 0x96cd1c35; k4 *= c4; - k4 = rotl32(k4, 18); + k4 = std::rotl(k4, 18); k4 *= c1; h4 ^= k4; - h4 = rotl32(h4, 13); + h4 = std::rotl(h4, 13); h4 += h1; h4 = h4 * 5 + 0x32ac3b17; } @@ -172,7 +206,7 @@ class murmur_hash case 13: k4 ^= tail[12] << 0; k4 *= c4; - k4 = rotl32(k4, 18); + k4 = std::rotl(k4, 18); k4 *= c1; h4 ^= k4; @@ -185,7 +219,7 @@ class murmur_hash case 9: k3 ^= tail[8] << 0; k3 *= c3; - k3 = rotl32(k3, 17); + k3 = std::rotl(k3, 17); k3 *= c4; h3 ^= k3; @@ -198,7 +232,7 @@ class murmur_hash case 5: k2 ^= tail[4] << 0; k2 *= c2; - k2 = rotl32(k2, 16); + k2 = std::rotl(k2, 16); k2 *= c3; h2 ^= k2; @@ -211,7 +245,7 @@ class murmur_hash case 1: k1 ^= tail[0] << 0; k1 *= c1; - k1 = rotl32(k1, 15); + k1 = std::rotl(k1, 15); k1 *= c2; h1 ^= k1; }; @@ -243,13 +277,12 @@ class murmur_hash h3 += h1; h4 += h1; - ((uint32_t*)out)[0] = h1; - ((uint32_t*)out)[1] = h2; - ((uint32_t*)out)[2] = h3; - ((uint32_t*)out)[3] = h4; + const uint32_t result[]{h1, h2, h3, h4}; + store_words(out, result); } - static void murmur_hash3_x64_128(const void* key, const int len, const uint32_t seed, void* out) { + template + static void murmur_hash3_x64_128_impl(const void* key, const int len, const uint32_t seed, void* out) { const uint8_t* data = (const uint8_t*)key; const int nblocks = len / 16; @@ -262,28 +295,28 @@ class murmur_hash //---------- // body - const uint64_t* blocks = (const uint64_t*)(data); + const uint8_t* blocks = data; for (int i = 0; i < nblocks; i++) { - uint64_t k1 = getblock(blocks, i * 2 + 0); - uint64_t k2 = getblock(blocks, i * 2 + 1); + uint64_t k1 = getblock(blocks, i * 2 + 0); + uint64_t k2 = getblock(blocks, i * 2 + 1); k1 *= c1; - k1 = rotl64(k1, 31); + k1 = std::rotl(k1, 31); k1 *= c2; h1 ^= k1; - h1 = rotl64(h1, 27); + h1 = std::rotl(h1, 27); h1 += h2; h1 = h1 * 5 + 0x52dce729; k2 *= c2; - k2 = rotl64(k2, 33); + k2 = std::rotl(k2, 33); k2 *= c1; h2 ^= k2; - h2 = rotl64(h2, 31); + h2 = std::rotl(h2, 31); h2 += h1; h2 = h2 * 5 + 0x38495ab5; } @@ -313,7 +346,7 @@ class murmur_hash case 9: k2 ^= uint64_t(tail[8]) << 0; k2 *= c2; - k2 = rotl64(k2, 33); + k2 = std::rotl(k2, 33); k2 *= c1; h2 ^= k2; @@ -334,7 +367,7 @@ class murmur_hash case 1: k1 ^= uint64_t(tail[0]) << 0; k1 *= c1; - k1 = rotl64(k1, 31); + k1 = std::rotl(k1, 31); k1 *= c2; h1 ^= k1; }; @@ -354,10 +387,11 @@ class murmur_hash h1 += h2; h2 += h1; - ((uint64_t*)out)[0] = h1; - ((uint64_t*)out)[1] = h2; + const uint64_t result[]{h1, h2}; + store_words(out, result); } +public: static void murmur_hash2(const void* key, int len, void* out) { uint32_t h, k; @@ -398,32 +432,52 @@ class murmur_hash h *= 0x5bd1e995; h ^= h >> 15; - *(uint32_t*)out = h; + store_word(out, h); } private: - inline static uint32_t rotl32(uint32_t x, int8_t r) { -#ifdef ARK_PLATFORM_WIN - return _rotl(x, r); -#else - return (x << r) | (x >> (32 - r)); -#endif + template + static word_type byte_swap(word_type value) { + static_assert(std::is_unsigned_v); + word_type result = 0; + for (std::size_t byte = 0; byte < sizeof(word_type); ++byte) + { + result = (result << 8) | (value & 0xff); + value >>= 8; + } + return result; } - inline static uint64_t rotl64(uint64_t x, int8_t r) { -#ifdef ARK_PLATFORM_WIN - return _rotl64(x, r); -#else - return (x << r) | (x >> (64 - r)); -#endif + template + static word_type getblock(const uint8_t* p, int i) { + // Retain byte-addressed arithmetic even for the backwards x86 block walk. + word_type value; + std::memcpy(&value, p + i * static_cast(sizeof(word_type)), sizeof(value)); + // Standard blocks are little endian; old MSVC blocks were byte-swapped + // on a little-endian host. Legacy tails and output remain little endian. + if constexpr ((legacy_msvc && std::endian::native == std::endian::little) || + (!legacy_msvc && std::endian::native == std::endian::big)) + { + value = byte_swap(value); + } + return value; } - static uint32_t getblock(const uint32_t* p, int i) { - return BYTESWAP32(p[i]); + template + static void store_word(void* out, word_type value) { + if constexpr (std::endian::native == std::endian::big) + { + value = byte_swap(value); + } + std::memcpy(out, &value, sizeof(value)); } - static uint64_t getblock(const uint64_t* p, int i) { - return BYTESWAP64(p[i]); + template + static void store_words(void* out, const word_type (&words)[count]) { + for (std::size_t index = 0; index < count; ++index) + { + store_word(static_cast(out) + index * sizeof(word_type), words[index]); + } } static uint32_t fmix(uint32_t h) { diff --git a/tests/core/CMakeLists.txt b/tests/core/CMakeLists.txt index e027b5fe2..4f68ab9e3 100644 --- a/tests/core/CMakeLists.txt +++ b/tests/core/CMakeLists.txt @@ -6,7 +6,7 @@ target_link_libraries(ark_core_header_checks PRIVATE ark::core) add_executable(ark_core_tests main.cpp strings.cpp containers.cpp templates.cpp callbacks.cpp header_regressions.cpp cxx20_language.cpp concurrency.cpp plugin_manager_startup.cpp data_strings.cpp node_data_storage.cpp - object_vectors.cpp plugin_construction.cpp module_factory.cpp random_generator.cpp xxtea.cpp) + object_vectors.cpp plugin_construction.cpp module_factory.cpp random_generator.cpp xxtea.cpp murmur_hash.cpp) target_link_libraries(ark_core_tests PRIVATE ark::core ark::kernel ark::utility) add_dependencies(ark_core_tests ark_core_header_checks) add_library(ark_construction_header_checks OBJECT headers/module_base.cpp headers/plugin_base.cpp) diff --git a/tests/core/README.md b/tests/core/README.md index 63864e4cd..d57d4a947 100644 --- a/tests/core/README.md +++ b/tests/core/README.md @@ -20,6 +20,16 @@ The public header checks include `` as the first include and include it twice. Compile-time checks preserve its unsigned 32-bit underlying type and every declared error value. + +Murmur regressions include independent upstream SMHasher verification constants, +legacy swapped-block fixtures, all tail lengths, full-width seeds and the original +rotation paths. They also check MurmurHash2 golden words, exact-size input +allocations and input/output offsets 0 through 7 with output guard bytes. +Run the core executable with `--test-case="*Murmur*"` to select these cases. +See [Murmur compatibility](../../docs/reference/utility.md#core-murmur-hashes) +for canonical little-endian output and the explicit legacy MSVC migration path. AddressSanitizer and +UndefinedBehaviorSanitizer builds exercise overread and alignment checks. + With `ARK_BUILD_PROTO=ON`, additional cases verify Protobuf parse failures, actor identity, binary messages, legacy wire payloads and external JSON field names. They also exercise the new `ark::protobuf` descriptor and enum names. diff --git a/tests/core/README_CN.md b/tests/core/README_CN.md index c33ba20d4..12571eea9 100644 --- a/tests/core/README_CN.md +++ b/tests/core/README_CN.md @@ -19,6 +19,14 @@ C++20 语言回归验证六种精确整数类型分类、限定符与引用处 随机数回归将公开头文件放在首个包含位置，并重复包含两次。测试保留固定的原始、有符号及无符号随机序列、最终种子和生成次数，并验证 `INT_MIN`／`INT_MAX` 极值、异号、相邻、逆序及相等端点的有符号半开区间。端点相同时直接返回该值，不消耗随机数。这些用例属于 `ark_core_tests` 和 `check-core`；使用 `-fsanitize=undefined -fno-sanitize-recover=all` 构建时，还会通过 UndefinedBehaviorSanitizer 检查范围运算。参见[随机数工具行为](../../docs/reference/utility_CN.md#core-随机数工具)。 + +Murmur 回归包含独立的上游 SMHasher 验证常量、旧版完整块交换策略固定值、全部尾部长度、 +完整宽度种子和原有旋转路径。用例还验证 MurmurHash2 固定值、精确分配的输入缓冲区、 +输入和输出偏移 0 到 7，以及输出保护字节。运行核心测试程序时，通过 +`--test-case="*Murmur*"` 选择这些用例。规范小端输出和显式旧 MSVC 迁移入口见 +[Murmur 兼容性说明](../../docs/reference/utility_CN.md#core-murmur-哈希)。 +AddressSanitizer 和 UndefinedBehaviorSanitizer 构建可检查越界读取和无效对齐。 + 公共头检查将 `` 放在首个包含位置，并重复包含两次；编译期断言 验证其无符号 32 位底层类型及所有已声明的错误值。 diff --git a/tests/core/murmur_hash.cpp b/tests/core/murmur_hash.cpp new file mode 100644 index 000000000..10dd516c6 --- /dev/null +++ b/tests/core/murmur_hash.cpp @@ -0,0 +1,251 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include + +// Keep the first inclusion separate to check standalone inclusion and the include guard. +#include +#include +#include +#include +#include +#include +#include + +#include "../network/doctest.h" + +#include + +namespace { + +static_assert(std::is_member_function_pointer_v); +static_assert(std::is_member_function_pointer_v); +static_assert(std::is_pointer_v); +static_assert(std::is_pointer_v); + +void hash_x86_128(const void* key, int length, std::uint32_t seed, void* output) { + ark::murmur_hash{}.murmur_hash3_x86_128(key, length, seed, output); +} + +void hash_x86_128_legacy(const void* key, int length, std::uint32_t seed, void* output) { + ark::murmur_hash{}.murmur_hash3_x86_128_legacy_msvc(key, length, seed, output); +} + +void hash_murmur2(const void* key, int length, std::uint32_t, void* output) { + ark::murmur_hash::murmur_hash2(key, length, output); +} + +template +std::uint32_t verification_value(hash_function hash) { + // Austin Appleby's SMHasher VerificationTest: all prefix lengths 0..255, + // seeds 256-length, then hash their concatenated digests with seed zero. + std::array key{}; + std::array hashes{}; + std::array result{}; + for (int length = 0; length < 256; ++length) + { + key[length] = static_cast(length); + hash(key.data(), length, 256 - length, hashes.data() + length * output_size); + } + hash(hashes.data(), static_cast(hashes.size()), 0, result.data()); + return std::uint32_t{result[0]} | (std::uint32_t{result[1]} << 8) | (std::uint32_t{result[2]} << 16) | + (std::uint32_t{result[3]} << 24); +} + +template +std::array little_words(const std::array& bytes) { + std::array words{}; + for (std::size_t word = 0; word < word_count; ++word) + { + for (std::size_t byte = 0; byte < sizeof(word_type); ++byte) + { + words[word] |= static_cast(bytes[word * sizeof(word_type) + byte]) << (byte * 8); + } + } + return words; +} + +template +void check_byte_buffers(hash_function hash) { + alignas(8) std::array key{}; + for (std::size_t index = 0; index < key.size(); ++index) + { + key[index] = static_cast(index * 37 + 0x80); + } + // Cover every tail path and both sides of several full-block boundaries. + for (int length = 0; length <= 257; ++length) + { + if (length > 65 && length != 127 && length != 128 && length != 129 && length != 255 && length != 256 && + length != 257) + { + continue; + } + for (const auto seed : {0u, 1u, 0x80000000u, 0xffffffffu}) + { + alignas(8) std::array expected{}; + hash(key.data(), length, seed, expected.data()); + for (std::size_t offset = 0; offset < 8; ++offset) + { + CAPTURE(length); + CAPTURE(seed); + CAPTURE(offset); + // The readable range ends at the allocation boundary, so a sanitizer + // can detect full-block reads past the exact input length. + auto input = std::make_unique(std::max(1, length + offset)); + std::copy_n(key.begin(), length, input.get() + offset); + alignas(8) std::array output; + output.fill(0xa5); + const auto output_offset = (offset + 3) % 8; + hash(input.get() + offset, length, seed, output.data() + output_offset); + CHECK(std::equal(expected.begin(), expected.end(), output.begin() + output_offset)); + CHECK(std::all_of( + output.begin(), output.begin() + output_offset, [](auto value) { return value == 0xa5; })); + CHECK(std::all_of(output.begin() + output_offset + output_size, output.end(), + [](auto value) { return value == 0xa5; })); + } + } + } +} + +} // namespace + +TEST_CASE("Murmur3 retains upstream verification values and the legacy swapped-block policy") { + // Independent standard values and verification procedure: + // https://github.com/aappleby/smhasher/blob/master/src/main.cpp#L67-L70 + // https://github.com/aappleby/smhasher/blob/master/src/KeysetTest.cpp#L11-L38 + // Swapped-block values were derived from the same public-domain reference + // with only full-block byte order reversed; they are ARK compatibility fixtures. + CHECK(verification_value<4>(ark::murmur_hash::murmur_hash3_x86_32) == 0xb0f57ee3u); + CHECK(verification_value<16>(hash_x86_128) == 0xb3ece62au); + CHECK(verification_value<16>(ark::murmur_hash::murmur_hash3_x64_128) == 0x6384ba69u); + CHECK(verification_value<4>(ark::murmur_hash::murmur_hash3_x86_32_legacy_msvc) == 0xd1ab67cfu); + CHECK(verification_value<16>(hash_x86_128_legacy) == 0x2df8b022u); + CHECK(verification_value<16>(ark::murmur_hash::murmur_hash3_x64_128_legacy_msvc) == 0x501e703bu); +} + +TEST_CASE("Murmur3 rotations preserve full-width seeds and multi-block golden words") { + std::array key{}; + for (std::size_t index = 0; index < key.size(); ++index) + { + key[index] = static_cast(index); + } + + struct golden_case + { + int length; + std::uint32_t seed; + std::uint32_t x86_32; + std::array x86_128; + std::array x64_128; + }; + + // Fixed scalar-reference vectors exercise every production rotation count, + // high-bit seed preservation, exact blocks and a multi-block tail. + const std::array little{{ + {16, 0xffffffffu, 0x9fd6e060u, {0x4f872353u, 0x3ffde8eeu, 0x0c593cefu, 0xd276cbb5u}, + {0x75298e938ac005bfull, 0x26873cd6ed1b3b1eull}}, + {33, 0x12345678u, 0x2a4f25b0u, {0x0101769du, 0x6e0928d4u, 0x5f9361deu, 0xea784760u}, + {0xa94d180f895f3670ull, 0x9635b8a3feb6f62full}}, + {33, 0xffffffffu, 0xa5737017u, {0x53948860u, 0xb573ee1du, 0xb2e1371cu, 0x7d39669eu}, + {0xf14b6deed9373a70ull, 0xf83f262a61312136ull}}, + }}; + const std::array swapped{{ + {16, 0xffffffffu, 0x2d93b016u, {0x84c0695cu, 0x73902949u, 0x6bfdce30u, 0xefc92f7cu}, + {0x703b8fedb7306652ull, 0xb1ce86b7edcd8628ull}}, + {33, 0x12345678u, 0xb255cc88u, {0xaea33583u, 0x1fd73070u, 0x36c553ffu, 0x9f396b6du}, + {0xa34f5dea69959035ull, 0x734c22e1993ef38full}}, + {33, 0xffffffffu, 0x74b4bba8u, {0xc8d61452u, 0x878dd688u, 0xad51e4b6u, 0x5b85edf7u}, + {0xcdcdff4c5a639071ull, 0x8332c8a91ba2cf3dull}}, + }}; + for (std::size_t index = 0; index < little.size(); ++index) + { + CAPTURE(index); + if (index == 2) + { + for (std::size_t byte = 0; byte < key.size(); ++byte) + { + key[byte] = static_cast(byte * 37 + 0x80); + } + } + for (const bool legacy : {false, true}) + { + CAPTURE(legacy); + const auto& expected = legacy ? swapped[index] : little[index]; + std::array actual32{}; + std::array actual86{}; + std::array actual64{}; + const auto hash32 = + legacy ? ark::murmur_hash::murmur_hash3_x86_32_legacy_msvc : ark::murmur_hash::murmur_hash3_x86_32; + const auto hash86 = legacy ? hash_x86_128_legacy : hash_x86_128; + const auto hash64 = + legacy ? ark::murmur_hash::murmur_hash3_x64_128_legacy_msvc : ark::murmur_hash::murmur_hash3_x64_128; + hash32(key.data(), expected.length, expected.seed, actual32.data()); + hash86(key.data(), expected.length, expected.seed, actual86.data()); + hash64(key.data(), expected.length, expected.seed, actual64.data()); + CHECK((little_words(actual32)[0] == expected.x86_32)); + CHECK((little_words(actual86) == expected.x86_128)); + CHECK((little_words(actual64) == expected.x64_128)); + } + } +} + +TEST_CASE("Murmur2 retains seed-zero golden words around block boundaries") { + std::array key{}; + for (std::size_t index = 0; index < key.size(); ++index) + { + key[index] = static_cast(index); + } + const std::array, 10> cases{ + {{0, 0}, {1, 0xe94e6ebdu}, {3, 0xe2558031u}, {4, 0xa4985988u}, {15, 0x8184d621u}, {16, 0x5f3c0743u}, + {17, 0xb1aad377u}, {31, 0x1d7d6ee1u}, {32, 0xf1bc6c23u}, {33, 0x915a6941u}}}; + for (const auto& [length, expected] : cases) + { + CAPTURE(length); + std::array result{}; + ark::murmur_hash::murmur_hash2(key.data(), length, result.data()); + CHECK((little_words(result)[0] == expected)); + } +} + +TEST_CASE("Murmur3 default and legacy policies agree until a complete block is present") { + const std::array key{ + 0x80, 0xff, 0x00, 0x01, 0xa5, 0x5a, 0x81, 0xfe, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09}; + using hash_function = void (*)(const void*, int, std::uint32_t, void*); + + struct policy_case + { + hash_function standard; + hash_function legacy; + int block_size; + }; + + const std::array policies{{ + {ark::murmur_hash::murmur_hash3_x86_32, ark::murmur_hash::murmur_hash3_x86_32_legacy_msvc, 4}, + {hash_x86_128, hash_x86_128_legacy, 16}, + {ark::murmur_hash::murmur_hash3_x64_128, ark::murmur_hash::murmur_hash3_x64_128_legacy_msvc, 16}, + }}; + for (const auto& policy : policies) + { + for (int length = 0; length < policy.block_size; ++length) + { + for (const auto seed : {0u, 0x80000000u, 0xffffffffu}) + { + CAPTURE(length); + CAPTURE(seed); + std::array standard{}; + std::array legacy{}; + policy.standard(key.data(), length, seed, standard.data()); + policy.legacy(key.data(), length, seed, legacy.data()); + CHECK(standard == legacy); + } + } + } +} + +TEST_CASE("All Murmur variants accept byte-aligned inputs and outputs without changing digest bytes") { + check_byte_buffers<4>(ark::murmur_hash::murmur_hash3_x86_32); + check_byte_buffers<16>(hash_x86_128); + check_byte_buffers<16>(ark::murmur_hash::murmur_hash3_x64_128); + check_byte_buffers<4>(hash_murmur2); + check_byte_buffers<4>(ark::murmur_hash::murmur_hash3_x86_32_legacy_msvc); + check_byte_buffers<16>(hash_x86_128_legacy); + check_byte_buffers<16>(ark::murmur_hash::murmur_hash3_x64_128_legacy_msvc); +} From d8b3a61529f5617656ac9dc1e760c297ffc2ac93 Mon Sep 17 00:00:00 2001 From: nick Date: Mon, 26 Oct 2026 09:35:19 +0800 Subject: [PATCH 03/22] test(hash): isolate standalone Murmur header checks --- tests/core/CMakeLists.txt | 2 +- tests/core/headers/murmur_hash.cpp | 18 ++++++++++++++++++ tests/core/murmur_hash.cpp | 3 --- 3 files changed, 19 insertions(+), 4 deletions(-) create mode 100644 tests/core/headers/murmur_hash.cpp diff --git a/tests/core/CMakeLists.txt b/tests/core/CMakeLists.txt index 4f68ab9e3..a327d475d 100644 --- a/tests/core/CMakeLists.txt +++ b/tests/core/CMakeLists.txt @@ -1,7 +1,7 @@ add_library(ark_core_header_checks OBJECT headers/queue.cpp headers/guid128.cpp headers/rw_lock.cpp headers/array_map.cpp headers/bit_value.cpp headers/consistent_hash.cpp headers/list.cpp headers/lock_free_queue.cpp headers/lru_map.cpp headers/map.cpp headers/thread_safe_queue.cpp - headers/object_vector.cpp headers/error_code.cpp headers/logger.cpp) + headers/object_vector.cpp headers/error_code.cpp headers/logger.cpp headers/murmur_hash.cpp) target_link_libraries(ark_core_header_checks PRIVATE ark::core) add_executable(ark_core_tests main.cpp strings.cpp containers.cpp templates.cpp callbacks.cpp diff --git a/tests/core/headers/murmur_hash.cpp b/tests/core/headers/murmur_hash.cpp new file mode 100644 index 000000000..87c28955a --- /dev/null +++ b/tests/core/headers/murmur_hash.cpp @@ -0,0 +1,18 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include + +// Keep the second inclusion separate to retain the include-guard check. +#include + +void murmur_hash_header_check() { + const uint8_t key[16]{}; + uint8_t output[16]{}; + ark::murmur_hash hash; + hash.murmur_hash3_x86_32(key, 16, 0, output); + hash.murmur_hash3_x86_128(key, 16, 0, output); + hash.murmur_hash3_x64_128(key, 16, 0, output); + hash.murmur_hash3_x86_32_legacy_msvc(key, 16, 0, output); + hash.murmur_hash3_x86_128_legacy_msvc(key, 16, 0, output); + hash.murmur_hash3_x64_128_legacy_msvc(key, 16, 0, output); + hash.murmur_hash2(key, 16, output); +} diff --git a/tests/core/murmur_hash.cpp b/tests/core/murmur_hash.cpp index 10dd516c6..a0beb95c9 100644 --- a/tests/core/murmur_hash.cpp +++ b/tests/core/murmur_hash.cpp @@ -1,7 +1,4 @@ // Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. -#include - -// Keep the first inclusion separate to check standalone inclusion and the include guard. #include #include #include From b794cfcdfb3c8f565f143690e17bd6c06f2ce2b2 Mon Sep 17 00:00:00 2001 From: nick Date: Mon, 26 Oct 2026 09:59:50 +0800 Subject: [PATCH 04/22] test(consul): exercise construction across application jobs --- .github/workflows/ci.yml | 1 + docs/reference/build-options.md | 2 ++ docs/reference/build-options_CN.md | 2 ++ tests/ci/package-dependencies.test.mjs | 10 ++++++++++ tests/consul/entry_factory.cpp | 2 +- 5 files changed, 16 insertions(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 36bb77872..de2ec5664 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -285,6 +285,7 @@ jobs: -DARK_BUILD_APPLICATIONS=ON -DARK_BUILD_BENCHMARKS=ON -DARK_BUILD_PLUGIN_HTTP=ON + -DARK_BUILD_PLUGIN_CONSUL=ON -DARK_BUILD_PROTO=ON -DARK_BUILD_TESTS=ON -DARK_BUILD_CORE_TESTS=ON diff --git a/docs/reference/build-options.md b/docs/reference/build-options.md index 52cd11ef9..5b2a5cb37 100644 --- a/docs/reference/build-options.md +++ b/docs/reference/build-options.md @@ -41,6 +41,8 @@ cmake --build build --target ark_mysql_plugin ark_redis_plugin All official plugin public headers are copied from explicit lists at configuration time into each build tree's generated plugin header mapping. Each `ark::` API target carries that include root even with runtime plugins disabled. MySQL, Redis, Utility, Net, Bus and HTTP exclude private assembly and factories. Kernel retains all 38 existing headers, including `kernel_plugin.hpp`; Consul, Etcd Client, Kafka, OSS SDK and Physic retain all 22 existing public headers, including implementation and assembly headers. Registration keys, public type visibility and optional plugin policies are retained; [Kernel typed construction](kernel.md#typed-kernel-construction) and [Consul typed construction](../project-layout.md#typed-consul-construction) change public class layouts and require rebuilding consumers. Their private factory headers are excluded from mapping and installation; header mapping does not validate optional driver integrations. Each public header has one co-located source; its generated include mapping belongs to the selected build tree, rather than a second source layout. +Applications CI enables the optional Consul plugin on Linux, macOS and Windows. Its `check-core` run includes construction, DLL lifecycle and production export checks; these checks require no Consul service and do not establish service interoperability. + Link a consumer to `ark::mysql` and/or `ark::redis` for the public headers and usage requirements. These public interface targets do not load a runtime library. Deploy the selected libraries with the launcher and add their stems to the application's manifest: ```toml diff --git a/docs/reference/build-options_CN.md b/docs/reference/build-options_CN.md index 447194a30..5782fb155 100644 --- a/docs/reference/build-options_CN.md +++ b/docs/reference/build-options_CN.md @@ -41,6 +41,8 @@ cmake --build build --target ark_mysql_plugin ark_redis_plugin 全部官方插件的公开头通过显式清单在配置阶段复制到各构建树生成的插件公开头映射。各 `ark::` API target 传递该目录，插件关闭时仍可使用。MySQL、Redis、Utility、Net、Bus、HTTP 的私有装配头和工厂不进入映射；Kernel 保留包括 `kernel_plugin.hpp` 在内的原有 38 个头；Consul、Etcd Client、Kafka、OSS SDK、Physic 保留包括实现头与装配头在内的原有 22 个公开头。注册键、公开类型可见性和可选插件策略继续保留；[Kernel 类型化构造](kernel_CN.md#kernel-类型化构造)与 [Consul 类型化构造](../project-layout_CN.md#consul-类型化构造)改变公开类布局，需要重建消费者。两者的私有工厂头均不映射、不安装；头文件映射不代表可选驱动集成已经验证。每个公共头仅有一份共置源码；生成的头文件映射属于选定构建树，不构成第二棵源码目录。 +应用 CI 在 Linux、macOS、Windows 启用可选 Consul 插件，`check-core` 包含构造、动态库生命周期与生产库导出检查。这些检查不需要 Consul 服务，也不代表服务互通验收。 + 业务插件链接 `ark::mysql` 和/或 `ark::redis`，以获取公开头文件和编译要求；公共接口目标不会加载运行时动态库。将所需动态库与启动器一起部署，并在应用清单中填写库名称： ```toml diff --git a/tests/ci/package-dependencies.test.mjs b/tests/ci/package-dependencies.test.mjs index c4d44d354..9fee41e8e 100644 --- a/tests/ci/package-dependencies.test.mjs +++ b/tests/ci/package-dependencies.test.mjs @@ -77,3 +77,13 @@ test('unsupported PhysX and simulate integrations fail explicitly instead of sea assert.doesNotMatch(source, /3rdparty|find_package\(PhysX/); } }); + +test('Applications CI enables Consul construction and production export checks without a service', () => { + const applications = read('.github/workflows/ci.yml').split(' applications:\n')[1].split('\n database:')[0]; + assert.match(applications, /-DARK_BUILD_PLUGIN_CONSUL=ON/); + assert.match(applications, /run: cmake --build ci-build --target check-core/); + const checks = read('tests/consul/CMakeLists.txt'); + assert.match(checks, /if\(TARGET ark_consul_plugin\)/); + assert.match(checks, /add_dependencies\(check-core check-consul-plugin\)/); + assert.match(checks, /add_dependencies\(check-core check-consul-construction\)/); +}); diff --git a/tests/consul/entry_factory.cpp b/tests/consul/entry_factory.cpp index d9f92b51b..5fbf51591 100644 --- a/tests/consul/entry_factory.cpp +++ b/tests/consul/entry_factory.cpp @@ -31,7 +31,7 @@ class entry_consul final : public ark::module_base return true; } - std::future> register_service(const ark::consul::service_data&) override { + std::future> register_service(const ::consul::service_data&) override { return {}; } From 87e24d42955f42b0d5452eecbdd4e5d34a066408 Mon Sep 17 00:00:00 2001 From: nick Date: Mon, 26 Oct 2026 09:59:50 +0800 Subject: [PATCH 05/22] docs(hash): distinguish affected routing consumers --- docs/reference/utility.md | 2 ++ docs/reference/utility_CN.md | 2 ++ 2 files changed, 4 insertions(+) diff --git a/docs/reference/utility.md b/docs/reference/utility.md index 22f3f5146..6be8ae7a6 100644 --- a/docs/reference/utility.md +++ b/docs/reference/utility.md @@ -107,6 +107,8 @@ little-endian bytes; `_legacy_msvc` specifically reproduces old MSVC behavior. ### Migrate stored keys and routing +The built-in `ark::consistent_hash` currently uses MurmurHash2, whose results remain unchanged on little-endian platforms. The routing guidance below applies to consumers that derive their ring or shard keys from the affected MurmurHash3 policy. + Identify each dataset's hash variant, seed, input encoding and previous compiler policy before upgrading. Keep that policy with the data or in an explicit schema version. A digest alone cannot reconstruct the original key. IDs, shard mappings, diff --git a/docs/reference/utility_CN.md b/docs/reference/utility_CN.md index f9262cc25..747cec544 100644 --- a/docs/reference/utility_CN.md +++ b/docs/reference/utility_CN.md @@ -93,6 +93,8 @@ MurmurHash2 保留固定种子零。 ### 迁移已存储键与路由 +框架内置 `ark::consistent_hash` 当前使用 MurmurHash2，在小端平台上的结果保持不变。以下路由迁移说明适用于使用受影响 MurmurHash3 策略构建环或分片键的调用方。 + 升级前，先确认每个数据集的哈希变体、种子、输入编码和旧编译器策略，并在数据中或 显式 schema 版本中记录该策略。单独的摘要无法还原原始键。由旧 MSVC 哈希派生的 ID、 分片映射、缓存键和持久化索引需要显式迁移，不能替换函数后直接重算路由。 From 15b09018e51e2caec7a90326920928b5ff57c7f6 Mon Sep 17 00:00:00 2001 From: nick Date: Mon, 26 Oct 2026 10:12:32 +0800 Subject: [PATCH 06/22] docs(consul): publish construction guide in canonical pages --- docs/guides/plugins.md | 2 +- docs/guides/plugins_CN.md | 2 +- docs/project-layout.md | 14 -------------- docs/project-layout_CN.md | 14 -------------- docs/reference/build-options.md | 2 +- docs/reference/build-options_CN.md | 2 +- docs/reference/directory-structure.md | 14 ++++++++++++++ docs/reference/directory-structure_CN.md | 14 ++++++++++++++ src/plugins/README.md | 4 ++-- src/plugins/README_CN.md | 4 ++-- 10 files changed, 36 insertions(+), 36 deletions(-) diff --git a/docs/guides/plugins.md b/docs/guides/plugins.md index 9c9e8f5cf..cd3fbcd5d 100644 --- a/docs/guides/plugins.md +++ b/docs/guides/plugins.md @@ -212,4 +212,4 @@ Net constructs its HTTP client through a private factory returning `std::unique_ Kernel also uses typed construction while retaining its public `*_impl` and `kernel_plugin` types and header paths. Its private factory header is not installed. Rebuild Kernel and consumers together because those public class layouts change; the [Kernel migration table](../reference/kernel.md#typed-kernel-construction) records unchanged interface keys, update selection and registration order. -Consul follows the same construction pattern while retaining all three public headers and types. Its private factory is not installed; the module keeps its interface RTTI key and disabled updates. Rebuild Consul and consumers together because its public implementation/plugin layouts change. See [Consul construction migration](../project-layout.md#typed-consul-construction) for test entry points and dependency conditions. +Consul follows the same construction pattern while retaining all three public headers and types. Its private factory is not installed; the module keeps its interface RTTI key and disabled updates. Rebuild Consul and consumers together because its public implementation/plugin layouts change. See [Consul construction migration](../reference/directory-structure.md#typed-consul-construction) for test entry points and dependency conditions. diff --git a/docs/guides/plugins_CN.md b/docs/guides/plugins_CN.md index 39ba62f2a..b66b700d4 100644 --- a/docs/guides/plugins_CN.md +++ b/docs/guides/plugins_CN.md @@ -199,4 +199,4 @@ Net 通过私有工厂返回 `std::unique_ptr`，并启用 Kernel 同样使用类型化构造，并保留公开 `*_impl`、`kernel_plugin` 类型与头路径；私有工厂头不安装。公开类布局发生变化，Kernel 与消费者需要一起重建。[Kernel 迁移表](../reference/kernel_CN.md#kernel-类型化构造)记录保持不变的接口键、update 选择与注册顺序。 -Consul 使用相同构造方式，保留全部三个公开头与类型。私有工厂不安装；模块保留接口 RTTI 键和禁用 update 的选择。公开实现／插件布局发生变化，Consul 与消费者必须一起重建。测试入口与依赖条件见 [Consul 构造迁移](../project-layout_CN.md#consul-类型化构造)。 +Consul 使用相同构造方式，保留全部三个公开头与类型。私有工厂不安装；模块保留接口 RTTI 键和禁用 update 的选择。公开实现／插件布局发生变化，Consul 与消费者必须一起重建。测试入口与依赖条件见 [Consul 构造迁移](../reference/directory-structure_CN.md#consul-类型化构造)。 diff --git a/docs/project-layout.md b/docs/project-layout.md index 79cff408b..6ce9b31b3 100644 --- a/docs/project-layout.md +++ b/docs/project-layout.md @@ -134,17 +134,3 @@ cmake --build etcd-client-headers --target check-etcd-client-headers --parallel ``` Header mapping does not establish Consul, Kafka or OSS SDK runtime compatibility. Physic and `physics_scene` still require the unsupported legacy PhysX 3 integration and retain their explicit configuration failure; no passing PhysX build is claimed. - -## Typed Consul construction - -**Breaking ABI/layout change:** `consul_module_impl` now inherits `module_base` and `consul_plugin` inherits `plugin_base`. Rebuild Consul and consumers together. All three public headers, their `` paths, public types and request methods remain available through `ark::consul`; the interface itself is unchanged. The C++ plugin boundary does not promise compiler-independent ABI compatibility. - -The private `consul_factory.hpp` returns `std::unique_ptr` and is excluded from public mappings and installation. Explicit `DllEntryPlugin` and `DllExitPlugin` retain the existing platform exports and logger setup. Installation registers the single `consul_module` interface RTTI key with `module_update::disabled`; uninstallation removes and destroys that module. The implementation still inherits the default update method. Configuration storage, HTTP-client lookup, TOML loading, request serialization and endpoints are unchanged. - -With `ARK_BUILD_CORE_TESTS=ON`, `ARK_BUILD_PROTO=ON` and `ARK_BUILD_CONFIG=ON`, `check-consul-construction` runs the actual factory/assembly doctest cases and a controlled-factory DSO test of the actual entry/exit source. These cover public bases, factory metadata, interface registration, duplicate rejection, disabled updates, configuration handoff, ownership and repeated stop. They require Protobuf, toml++ and the normal core/arknet dependencies, but no runtime plugin or Consul service. For an existing build configured with those options: - -```sh -cmake --build build --target check-consul-construction --parallel 2 -``` - -`ARK_BUILD_PLUGIN_CONSUL=ON` also requires `ARK_BUILD_APPLICATIONS=ON` and remains off by default. It links the generated protocol target `ark::proto` directly. With the core tests enabled, `check-consul-plugin` additionally inspects the actual Consul DSO exports in a separate process; `check-core` includes the applicable Consul checks. The controlled factory does not exercise the production HTTP client or a Consul server. These construction checks do not establish live Consul interoperability, protocol acceptance or arbitrary hot-update safety. diff --git a/docs/project-layout_CN.md b/docs/project-layout_CN.md index a251e8d4e..82a8ee230 100644 --- a/docs/project-layout_CN.md +++ b/docs/project-layout_CN.md @@ -124,17 +124,3 @@ cmake --build etcd-client-headers --target check-etcd-client-headers --parallel ``` 头文件映射检查不代表 Consul、Kafka、OSS SDK 的运行时兼容性已经验证。Physic 与 `physics_scene` 仍依赖未受支持的旧版 PhysX 3 集成，保留明确的配置失败；此处没有宣称 PhysX 构建通过。 - -## Consul 类型化构造 - -**ABI／布局不兼容变更：** `consul_module_impl` 现在继承 `module_base`，`consul_plugin` 继承 `plugin_base`。Consul 与消费者必须一起重建。原有三个公开头、`` 路径、公开类型与请求方法继续通过 `ark::consul` 提供，接口本身不变。C++ 插件边界不承诺跨编译器的 ABI 兼容性。 - -私有 `consul_factory.hpp` 返回 `std::unique_ptr`，不映射、不安装。显式 `DllEntryPlugin` 与 `DllExitPlugin` 保留原有平台导出和日志初始化。安装时按 `consul_module` 接口 RTTI 键注册唯一模块，显式使用 `module_update::disabled`；卸载时移除并销毁该模块。实现仍继承默认 update 方法。配置存储、HTTP 客户端查找、TOML 加载、请求序列化及端点保持不变。 - -开启 `ARK_BUILD_CORE_TESTS=ON`、`ARK_BUILD_PROTO=ON` 与 `ARK_BUILD_CONFIG=ON` 后，`check-consul-construction` 运行真实工厂／装配的 doctest 用例，以及使用受控工厂与真实入口／出口源码的 DSO 测试。检查覆盖公开基类、工厂元数据、接口注册、重复注册拒绝、禁用 update、配置传递、所有权与重复 stop。依赖 Protobuf、toml++ 及通常的 core／arknet 依赖，无需运行时插件或 Consul 服务。对于已按上述选项配置的构建： - -```sh -cmake --build build --target check-consul-construction --parallel 2 -``` - -`ARK_BUILD_PLUGIN_CONSUL=ON` 还要求 `ARK_BUILD_APPLICATIONS=ON`，默认保持关闭。插件直接链接生成协议目标 `ark::proto`。开启核心测试后，`check-consul-plugin` 还在独立进程中检查真实 Consul DSO 的导出符号；`check-core` 包含当前条件下可用的 Consul 检查。受控工厂不执行生产 HTTP 客户端或访问 Consul 服务。这些构造检查不代表真实 Consul 互通、协议验收或任意热更新安全。 diff --git a/docs/reference/build-options.md b/docs/reference/build-options.md index 5b2a5cb37..f6be18c32 100644 --- a/docs/reference/build-options.md +++ b/docs/reference/build-options.md @@ -39,7 +39,7 @@ cmake -S . -B build -DARK_BUILD_APPLICATIONS=ON \ cmake --build build --target ark_mysql_plugin ark_redis_plugin ``` -All official plugin public headers are copied from explicit lists at configuration time into each build tree's generated plugin header mapping. Each `ark::` API target carries that include root even with runtime plugins disabled. MySQL, Redis, Utility, Net, Bus and HTTP exclude private assembly and factories. Kernel retains all 38 existing headers, including `kernel_plugin.hpp`; Consul, Etcd Client, Kafka, OSS SDK and Physic retain all 22 existing public headers, including implementation and assembly headers. Registration keys, public type visibility and optional plugin policies are retained; [Kernel typed construction](kernel.md#typed-kernel-construction) and [Consul typed construction](../project-layout.md#typed-consul-construction) change public class layouts and require rebuilding consumers. Their private factory headers are excluded from mapping and installation; header mapping does not validate optional driver integrations. Each public header has one co-located source; its generated include mapping belongs to the selected build tree, rather than a second source layout. +All official plugin public headers are copied from explicit lists at configuration time into each build tree's generated plugin header mapping. Each `ark::` API target carries that include root even with runtime plugins disabled. MySQL, Redis, Utility, Net, Bus and HTTP exclude private assembly and factories. Kernel retains all 38 existing headers, including `kernel_plugin.hpp`; Consul, Etcd Client, Kafka, OSS SDK and Physic retain all 22 existing public headers, including implementation and assembly headers. Registration keys, public type visibility and optional plugin policies are retained; [Kernel typed construction](kernel.md#typed-kernel-construction) and [Consul typed construction](directory-structure.md#typed-consul-construction) change public class layouts and require rebuilding consumers. Their private factory headers are excluded from mapping and installation; header mapping does not validate optional driver integrations. Each public header has one co-located source; its generated include mapping belongs to the selected build tree, rather than a second source layout. Applications CI enables the optional Consul plugin on Linux, macOS and Windows. Its `check-core` run includes construction, DLL lifecycle and production export checks; these checks require no Consul service and do not establish service interoperability. diff --git a/docs/reference/build-options_CN.md b/docs/reference/build-options_CN.md index 5782fb155..b57abfb08 100644 --- a/docs/reference/build-options_CN.md +++ b/docs/reference/build-options_CN.md @@ -39,7 +39,7 @@ cmake -S . -B build -DARK_BUILD_APPLICATIONS=ON \ cmake --build build --target ark_mysql_plugin ark_redis_plugin ``` -全部官方插件的公开头通过显式清单在配置阶段复制到各构建树生成的插件公开头映射。各 `ark::` API target 传递该目录，插件关闭时仍可使用。MySQL、Redis、Utility、Net、Bus、HTTP 的私有装配头和工厂不进入映射；Kernel 保留包括 `kernel_plugin.hpp` 在内的原有 38 个头；Consul、Etcd Client、Kafka、OSS SDK、Physic 保留包括实现头与装配头在内的原有 22 个公开头。注册键、公开类型可见性和可选插件策略继续保留；[Kernel 类型化构造](kernel_CN.md#kernel-类型化构造)与 [Consul 类型化构造](../project-layout_CN.md#consul-类型化构造)改变公开类布局，需要重建消费者。两者的私有工厂头均不映射、不安装；头文件映射不代表可选驱动集成已经验证。每个公共头仅有一份共置源码；生成的头文件映射属于选定构建树，不构成第二棵源码目录。 +全部官方插件的公开头通过显式清单在配置阶段复制到各构建树生成的插件公开头映射。各 `ark::` API target 传递该目录，插件关闭时仍可使用。MySQL、Redis、Utility、Net、Bus、HTTP 的私有装配头和工厂不进入映射；Kernel 保留包括 `kernel_plugin.hpp` 在内的原有 38 个头；Consul、Etcd Client、Kafka、OSS SDK、Physic 保留包括实现头与装配头在内的原有 22 个公开头。注册键、公开类型可见性和可选插件策略继续保留；[Kernel 类型化构造](kernel_CN.md#kernel-类型化构造)与 [Consul 类型化构造](directory-structure_CN.md#consul-类型化构造)改变公开类布局，需要重建消费者。两者的私有工厂头均不映射、不安装；头文件映射不代表可选驱动集成已经验证。每个公共头仅有一份共置源码；生成的头文件映射属于选定构建树，不构成第二棵源码目录。 应用 CI 在 Linux、macOS、Windows 启用可选 Consul 插件，`check-core` 包含构造、动态库生命周期与生产库导出检查。这些检查不需要 Consul 服务，也不代表服务互通验收。 diff --git a/docs/reference/directory-structure.md b/docs/reference/directory-structure.md index bbb33359d..890acc45d 100644 --- a/docs/reference/directory-structure.md +++ b/docs/reference/directory-structure.md @@ -166,3 +166,17 @@ With `ARK_BUILD_APPLICATIONS=ON`, CMake builds seven server libraries: Game, Log ## Installed layout [Installation](../get-started/installation.md#install-and-consume-the-cmake-package) preserves logical `` headers under `/include/ark/`. CMake package files use the GNUInstallDirs library directory under `cmake/ark/`; selected launcher and runtime plugins install under `bin/`. Configuration samples and licensing are under `share/ark/`. Consumers use `find_package(ark CONFIG REQUIRED)` and `ark::core` / `ark::` targets; source consumers use `add_subdirectory` with the same target names. Neither requires a generated mapping path. Source/build directories are absent from installed exports; arknet and other dependencies remain external. `tests/install/` contains independent installed consumers and the disposable-producer relocation test. + +## Typed Consul construction + +**Breaking ABI/layout change:** `consul_module_impl` now inherits `module_base` and `consul_plugin` inherits `plugin_base`. Rebuild Consul and consumers together. All three public headers, their `` paths, public types and request methods remain available through `ark::consul`; the interface itself is unchanged. The C++ plugin boundary does not promise compiler-independent ABI compatibility. + +The private `consul_factory.hpp` returns `std::unique_ptr` and is excluded from public mappings and installation. Explicit `DllEntryPlugin` and `DllExitPlugin` retain the existing platform exports and logger setup. Installation registers the single `consul_module` interface RTTI key with `module_update::disabled`; uninstallation removes and destroys that module. The implementation still inherits the default update method. Configuration storage, HTTP-client lookup, TOML loading, request serialization and endpoints are unchanged. + +With `ARK_BUILD_CORE_TESTS=ON`, `ARK_BUILD_PROTO=ON` and `ARK_BUILD_CONFIG=ON`, `check-consul-construction` runs the actual factory/assembly doctest cases and a controlled-factory DSO test of the actual entry/exit source. These cover public bases, factory metadata, interface registration, duplicate rejection, disabled updates, configuration handoff, ownership and repeated stop. They require Protobuf, toml++ and the normal core/arknet dependencies, but no runtime plugin or Consul service. For an existing build configured with those options: + +```sh +cmake --build build --target check-consul-construction --parallel 2 +``` + +`ARK_BUILD_PLUGIN_CONSUL=ON` also requires `ARK_BUILD_APPLICATIONS=ON` and remains off by default. It links the generated protocol target `ark::proto` directly. With the core tests enabled, `check-consul-plugin` additionally inspects the actual Consul DSO exports in a separate process; `check-core` includes the applicable Consul checks. The controlled factory does not exercise the production HTTP client or a Consul server. These construction checks do not establish live Consul interoperability, protocol acceptance or arbitrary hot-update safety. diff --git a/docs/reference/directory-structure_CN.md b/docs/reference/directory-structure_CN.md index d83bc7044..18665fdf1 100644 --- a/docs/reference/directory-structure_CN.md +++ b/docs/reference/directory-structure_CN.md @@ -156,3 +156,17 @@ cmake --build etcd-client-headers --target check-etcd-client-headers --parallel `login/interface/login_net_module.hpp`、`login/include/login_plugin.hpp` 等源码包含路径通过构建树中的 `src/server/compat-include/` 转发头保留，内部源码使用相邻头文件。转发清单保留原有全部 28 个头文件，包括实现与插件声明，但不包含新增的私有工厂头。该兼容映射独立于公开 `` 映射，不新增可安装的服务端 API。具体类的继承链变化后，C++ 消费方需要重新编译。 设置 `ARK_BUILD_APPLICATIONS=ON` 后，CMake 编译 Game、Login、Proxy、World、Master、Dir、Router 七个服务动态库。Simulate 源码采用相同创建方式，但仍不参与默认构建，并保留未受支持的 PhysX 3／MessagePack 集成配置失败。源码共置不代表该集成已经通过验证。定向创建检查及其边界见[服务器架构](../concepts/server-architecture_CN.md#验证服务端创建方式)。 + +## Consul 类型化构造 + +**ABI／布局不兼容变更：** `consul_module_impl` 现在继承 `module_base`，`consul_plugin` 继承 `plugin_base`。Consul 与消费者必须一起重建。原有三个公开头、`` 路径、公开类型与请求方法继续通过 `ark::consul` 提供，接口本身不变。C++ 插件边界不承诺跨编译器的 ABI 兼容性。 + +私有 `consul_factory.hpp` 返回 `std::unique_ptr`，不映射、不安装。显式 `DllEntryPlugin` 与 `DllExitPlugin` 保留原有平台导出和日志初始化。安装时按 `consul_module` 接口 RTTI 键注册唯一模块，显式使用 `module_update::disabled`；卸载时移除并销毁该模块。实现仍继承默认 update 方法。配置存储、HTTP 客户端查找、TOML 加载、请求序列化及端点保持不变。 + +开启 `ARK_BUILD_CORE_TESTS=ON`、`ARK_BUILD_PROTO=ON` 与 `ARK_BUILD_CONFIG=ON` 后，`check-consul-construction` 运行真实工厂／装配的 doctest 用例，以及使用受控工厂与真实入口／出口源码的 DSO 测试。检查覆盖公开基类、工厂元数据、接口注册、重复注册拒绝、禁用 update、配置传递、所有权与重复 stop。依赖 Protobuf、toml++ 及通常的 core／arknet 依赖，无需运行时插件或 Consul 服务。对于已按上述选项配置的构建： + +```sh +cmake --build build --target check-consul-construction --parallel 2 +``` + +`ARK_BUILD_PLUGIN_CONSUL=ON` 还要求 `ARK_BUILD_APPLICATIONS=ON`，默认保持关闭。插件直接链接生成协议目标 `ark::proto`。开启核心测试后，`check-consul-plugin` 还在独立进程中检查真实 Consul DSO 的导出符号；`check-core` 包含当前条件下可用的 Consul 检查。受控工厂不执行生产 HTTP 客户端或访问 Consul 服务。这些构造检查不代表真实 Consul 互通、协议验收或任意热更新安全。 diff --git a/src/plugins/README.md b/src/plugins/README.md index 66e429e98..bb41192b6 100755 --- a/src/plugins/README.md +++ b/src/plugins/README.md @@ -20,12 +20,12 @@ All official plugins co-locate their headers, implementation and assembly under Use `ARK_BUILD_PLUGIN_` to select a plugin. Optional integrations require their own dependencies and verification. [MySQL](../../docs/reference/mysql.md) provides queries, prepared statements and transactions; [Redis](../../docs/reference/redis.md) provides commands and ordered pipelines. Both use named configuration and manager-owned coroutine tasks. See [database plugin activation](../../docs/reference/build-options.md#native-database-plugins) for build and runtime selection. Lua and Actor integrations remain planned work. -See [project layout and public plugin interfaces](../../docs/project-layout.md) for targets and build commands. +See [project layout and public plugin interfaces](../../docs/reference/directory-structure.md) for targets and build commands. [Utility](../../docs/reference/utility.md) documents the four module interfaces, owner-driven timers and the implementation-header migration. Kernel co-locates its 38 existing headers with its cpp files in `src/plugins/kernel/`. `ark::kernel` preserves every `` path, including `data_impl.hpp`, `data_list_impl.hpp` and `kernel_plugin.hpp`. The retained public implementation/plugin types use typed construction with private owning factories and explicit entry points. `kernel_factory.hpp` is neither mapped nor installed. The public class layouts change, so rebuild Kernel and consumers together; registration keys, update selection and lifecycle order are preserved. See [Kernel construction migration](../../docs/reference/kernel.md#typed-kernel-construction) for compatibility boundaries, consumption checks and Protobuf requirements. -Consul retains its three public headers and implementation/plugin types while using typed construction, a private owning factory and explicit entry points. Rebuild Consul and consumers together because the public class layouts change. The single interface registration key, disabled updates and request behavior are preserved. See [Consul construction migration](../../docs/project-layout.md#typed-consul-construction) for build conditions and service-free construction checks. +Consul retains its three public headers and implementation/plugin types while using typed construction, a private owning factory and explicit entry points. Rebuild Consul and consumers together because the public class layouts change. The single interface registration key, disabled updates and request behavior are preserved. See [Consul construction migration](../../docs/reference/directory-structure.md#typed-consul-construction) for build conditions and service-free construction checks. The other four legacy plugin families retain 19 byte-identical headers. Mapping checks do not validate Consul, Kafka or OSS SDK runtime integrations. Physic retains the unsupported PhysX 3 configuration failure; see [optional legacy dependencies](../../docs/dependencies.md#optional-legacy-integrations). diff --git a/src/plugins/README_CN.md b/src/plugins/README_CN.md index 82c42abd5..bed2c6eb9 100644 --- a/src/plugins/README_CN.md +++ b/src/plugins/README_CN.md @@ -20,12 +20,12 @@ 使用 `ARK_BUILD_PLUGIN_<名称大写>` 选择插件。可选插件需要各自的依赖和验证。[MySQL](../../docs/reference/mysql_CN.md) 提供查询、预处理语句和事务；[Redis](../../docs/reference/redis_CN.md) 提供命令和有序流水线。两者均通过具名配置和管理器托管的协程任务使用。构建与运行时选择见[数据库插件启用方式](../../docs/reference/build-options_CN.md#原生数据库插件)。Lua、Actor 的整合仍在规划中。 -CMake target 和构建命令见[项目布局与公共插件接口](../../docs/project-layout_CN.md)。 +CMake target 和构建命令见[项目布局与公共插件接口](../../docs/reference/directory-structure_CN.md)。 [Utility 参考](../../docs/reference/utility_CN.md)说明四个模块接口、所属线程驱动的定时器与实现头迁移。 Kernel 的原有 38 个头与 cpp 共置于 `src/plugins/kernel/`，通过 `ark::kernel` 保留全部 `` 路径，包括 `data_impl.hpp`、`data_list_impl.hpp` 和 `kernel_plugin.hpp`。保留的公开实现类型与插件类型使用类型化构造、私有拥有型工厂和显式入口。`kernel_factory.hpp` 不映射、不安装。公开类布局发生变化，Kernel 与消费者必须一起重建；注册键、update 选择和生命周期顺序保持不变。兼容边界、消费检查及 Protobuf 条件见 [Kernel 构造迁移](../../docs/reference/kernel_CN.md#kernel-类型化构造)。 -Consul 保留原有三个公开头及实现／插件类型，同时使用类型化构造、私有拥有型工厂和显式入口。公开类布局发生变化，Consul 与消费者必须一起重建。唯一接口注册键、禁用 update 与请求行为保持不变。构建条件及无需服务的构造检查见 [Consul 构造迁移](../../docs/project-layout_CN.md#consul-类型化构造)。 +Consul 保留原有三个公开头及实现／插件类型，同时使用类型化构造、私有拥有型工厂和显式入口。公开类布局发生变化，Consul 与消费者必须一起重建。唯一接口注册键、禁用 update 与请求行为保持不变。构建条件及无需服务的构造检查见 [Consul 构造迁移](../../docs/reference/directory-structure_CN.md#consul-类型化构造)。 其他四类旧插件保留原有 19 个头的全部字节。映射检查不代表 Consul、Kafka、OSS SDK 的运行时集成已经验证。Physic 保留未受支持的 PhysX 3 配置失败；参见[可选旧集成依赖](../../docs/dependencies_CN.md)。 From 7f2256fa31fedfb3494954419979bd8a71a658b4 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 26 Oct 2026 02:38:53 +0000 Subject: [PATCH 07/22] build(deps): bump fast-uri from 3.1.8 to 3.1.9 in /docs/site Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.8 to 3.1.9. - [Release notes](https://github.com/fastify/fast-uri/releases) - [Commits](https://github.com/fastify/fast-uri/compare/v3.1.8...v3.1.9) --- updated-dependencies: - dependency-name: fast-uri dependency-version: 3.1.9 dependency-type: indirect ... Signed-off-by: dependabot[bot] --- docs/site/package-lock.json | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/site/package-lock.json b/docs/site/package-lock.json index 37ca23691..cf2c3678b 100644 --- a/docs/site/package-lock.json +++ b/docs/site/package-lock.json @@ -3442,9 +3442,9 @@ } }, "node_modules/fast-uri": { - "version": "3.1.8", - "resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.8.tgz", - "integrity": "sha512-GZMtZUTNRpOVIECoXwLNZS5xUGE+mVNbTB8h/7Rwh2TFWcBQiPzTgyZi05BF9UMZKkLJv8XBRJTlU7zg8+ZfMg==", + "version": "3.1.9", + "resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.9.tgz", + "integrity": "sha512-hS/0Wt3Ya043oKXs0sIxrMpKQITXhMLIrVAKKXi4gozGC9v7YRWVnXIZzjNPTO02M9BdxLCWqK1yRXgNziVyJQ==", "dev": true, "funding": [ { From 3bed67089ac2e8a2cf38f43e55dcf4f4f3c3e9b3 Mon Sep 17 00:00:00 2001 From: nick Date: Mon, 26 Oct 2026 10:52:38 +0800 Subject: [PATCH 08/22] refactor(proto): remove unused codec scaffolding --- docs/reference/directory-structure.md | 2 +- docs/reference/directory-structure_CN.md | 2 +- docs/reference/protobuf.md | 8 ++- docs/reference/protobuf_CN.md | 8 ++- src/proto/codec/codec.hpp | 68 ------------------------ src/proto/codec/protobuf_codec.cpp | 43 --------------- src/proto/codec/protobuf_codec.hpp | 40 -------------- 7 files changed, 16 insertions(+), 155 deletions(-) delete mode 100644 src/proto/codec/codec.hpp delete mode 100644 src/proto/codec/protobuf_codec.cpp delete mode 100644 src/proto/codec/protobuf_codec.hpp diff --git a/docs/reference/directory-structure.md b/docs/reference/directory-structure.md index bbb33359d..146c5f087 100644 --- a/docs/reference/directory-structure.md +++ b/docs/reference/directory-structure.md @@ -13,7 +13,7 @@ Build artifacts belong to each CMake build directory: `cmake -S . -B out/release | `src/core/` | Core headers, implementation and CMake usage requirements | | `src/application/` | Application entry point and shutdown request | | `src/config/` | Application manifest declarations, shared compiled file/TOML reader and private diagnostics | -| `src/proto/` | Protocol schemas and codec implementation | +| `src/proto/` | Protocol schemas and CMake rules for generated Protobuf messages | | `src/server/` | Built-in server applications | | `src/plugins//` | All official plugin headers, implementation and assembly | | `tests/` | Core, network and private database regressions | diff --git a/docs/reference/directory-structure_CN.md b/docs/reference/directory-structure_CN.md index d83bc7044..95098e10c 100644 --- a/docs/reference/directory-structure_CN.md +++ b/docs/reference/directory-structure_CN.md @@ -12,7 +12,7 @@ ark 使用 C++20。核心与官方插件放在本仓库，业务插件放在应 | `src/core/` | 核心头文件、实现与 CMake 使用要求 | | `src/application/` | 应用入口与停服请求 | | `src/config/` | 应用清单声明、共享的文件／TOML 读取实现与私有诊断 | -| `src/proto/` | 协议定义与编解码实现 | +| `src/proto/` | 协议定义与 Protobuf 消息生成的 CMake 规则 | | `src/server/` | 内置服务端应用 | | `src/plugins//` | 全部官方插件头文件、实现与装配 | | `tests/` | 核心、网络与私有数据库回归测试 | diff --git a/docs/reference/protobuf.md b/docs/reference/protobuf.md index 362dd0b45..2b9087650 100644 --- a/docs/reference/protobuf.md +++ b/docs/reference/protobuf.md @@ -1,6 +1,6 @@ # Protobuf messages and compatibility -Protocol definitions live in `src/proto/proto/`; codec implementation lives in `src/proto/codec/`. Source files, messages, fields and enum identifiers use `snake_case`. The former `AFMsg` package is now `ark.protobuf`, exposed in C++ as `ark::protobuf`. The `consul` and `osslog` packages retain their names. +Protocol definitions live in `src/proto/proto/`. Source files, messages, fields and enum identifiers use `snake_case`. The former `AFMsg` package is now `ark.protobuf`, exposed in C++ as `ark::protobuf`. The `consul` and `osslog` packages retain their names. ## Build and include @@ -16,6 +16,12 @@ Generated C++ files remain in the CMake build directory under `src/proto/generat Use Protobuf, Abseil and utf8_range from one compatible installation prefix. In a manual package selection, align `Protobuf_DIR`, `absl_DIR` and `utf8_range_DIR`; a successful link alone does not validate reflection/JSON behavior. +## Message serialization + +Messages use the generated Protobuf API directly. The Bus module's message send paths in `src/plugins/bus/msg_module_impl.cpp` call `SerializeToString` and return `false` if serialization fails. The public `msg_module::decode_protobuf` helper in `` calls `ParseFromString` on the network message body, returning `false` on a parse failure and copying the actor ID on success. See [Bus](bus.md) for the message and transport boundary. + +ark does not provide a pluggable message codec API. The generated messages and these Bus helpers are the existing serialization path. + ## Migration contract | Preserved | Requires consumer updates | diff --git a/docs/reference/protobuf_CN.md b/docs/reference/protobuf_CN.md index bdf233809..c17e15b07 100644 --- a/docs/reference/protobuf_CN.md +++ b/docs/reference/protobuf_CN.md @@ -1,6 +1,6 @@ # Protobuf 消息与兼容边界 -协议定义位于 `src/proto/proto/`，编解码实现位于 `src/proto/codec/`。源码文件、消息、字段与枚举标识符采用 `snake_case`。原 `AFMsg` package 改为 `ark.protobuf`，C++ 命名空间为 `ark::protobuf`；`consul`、`osslog` 保留既有 package。 +协议定义位于 `src/proto/proto/`。源码文件、消息、字段与枚举标识符采用 `snake_case`。原 `AFMsg` package 改为 `ark.protobuf`，C++ 命名空间为 `ark::protobuf`；`consul`、`osslog` 保留既有 package。 ## 构建与包含 @@ -16,6 +16,12 @@ target_link_libraries(my_service PRIVATE ark::core ark::proto) Protobuf、Abseil 与 utf8_range 应来自同一套兼容安装。手工选择软件包时，同步检查 `Protobuf_DIR`、`absl_DIR` 与 `utf8_range_DIR`；链接成功本身不能证明反射和 JSON 行为正确。 +## 消息序列化 + +消息直接使用 Protobuf 生成的 API。Bus 模块在 `src/plugins/bus/msg_module_impl.cpp` 的消息发送流程中调用 `SerializeToString`，序列化失败时返回 `false`。`` 中的公开辅助接口 `msg_module::decode_protobuf` 对网络消息体调用 `ParseFromString`，解析失败时返回 `false`，成功时复制 actor ID。消息与传输的边界见 [Bus](bus_CN.md)。 + +ark 不提供可替换的消息 codec API。现有序列化流程使用生成的消息与上述 Bus 辅助接口。 + ## 命名迁移约定 | 保持不变 | 调用方需要调整 | diff --git a/src/proto/codec/codec.hpp b/src/proto/codec/codec.hpp deleted file mode 100644 index 1df371ec1..000000000 --- a/src/proto/codec/codec.hpp +++ /dev/null @@ -1,68 +0,0 @@ -/* - * This source file is part of ArkNX - * For the latest info, see https://github.com/ArkNX - * - * Copyright (c) 2013-2020 ArkNX authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - * - */ - -#pragma once - -#include "ark/platform.hpp" - -namespace ark { - -/** - * message codec types - */ -enum class codec_type : uint8_t -{ - unknown = 0, /// unknown codec - protobuf = 1, /// protobuffer codes - json = 2, /// json codec - custom = 3, /// custom codec - websocket_protobuf = 4, /// websocket extension protobuffer data codec - websocket_json = 5, /// websocket extension json data codec -}; - -/** - * codec status - */ - -enum class codec_status : uint8_t -{ - ok = 1, /// OK - invalid = 2, /// different codec type - error = 3, /// codec error -}; - -class codec -{ -public: - // AFICodec(ARK_CODEC_TYPE type) - // { - // set_codec_type(type); - // } - - virtual ~codec() = default; - - virtual codec_status encode() = 0; - virtual codec_status decode() = 0; - - virtual codec_type get_codec_type() const = 0; - virtual void set_codec_type(codec_type type) = 0; -}; - -} // namespace ark diff --git a/src/proto/codec/protobuf_codec.cpp b/src/proto/codec/protobuf_codec.cpp deleted file mode 100644 index d2bf79dfc..000000000 --- a/src/proto/codec/protobuf_codec.cpp +++ /dev/null @@ -1,43 +0,0 @@ -/* - * This source file is part of ArkNX - * For the latest info, see https://github.com/ArkNX - * - * Copyright (c) 2013-2020 ArkNX authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - * - */ - -#include "protobuf_codec.hpp" - -namespace ark { - -ark::codec_status protobuf_codec::encode() { - // TODO: - return codec_status::ok; -} - -ark::codec_status protobuf_codec::decode() { - // TODO: - return codec_status::ok; -} - -codec_type protobuf_codec::get_codec_type() const { - return codec_type::protobuf; -} - -void protobuf_codec::set_codec_type(codec_type type) { - type_ = codec_type::protobuf; -} - -} // namespace ark diff --git a/src/proto/codec/protobuf_codec.hpp b/src/proto/codec/protobuf_codec.hpp deleted file mode 100644 index 5b04c118c..000000000 --- a/src/proto/codec/protobuf_codec.hpp +++ /dev/null @@ -1,40 +0,0 @@ -/* - * This source file is part of ArkNX - * For the latest info, see https://github.com/ArkNX - * - * Copyright (c) 2013-2020 ArkNX authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - * - */ - -#pragma once - -#include "codec.hpp" - -namespace ark { - -class protobuf_codec : public codec -{ -public: - codec_status encode() override; - codec_status decode() override; - - codec_type get_codec_type() const override; - void set_codec_type(codec_type type) override; - -private: - codec_type type_{codec_type::unknown}; -}; - -} // namespace ark From 53222f85ba62f57997a5c58c671c9bc8567db9f0 Mon Sep 17 00:00:00 2001 From: nick Date: Mon, 26 Oct 2026 10:53:11 +0800 Subject: [PATCH 09/22] refactor(hash): remove platform header dependencies --- docs/reference/utility.md | 9 +++++++++ docs/reference/utility_CN.md | 6 ++++++ src/core/container/consistent_hash.hpp | 1 + src/core/hash/murmur_hash.hpp | 11 ++++------- tests/core/CMakeLists.txt | 4 +++- tests/core/README.md | 13 +++++++++++++ tests/core/README_CN.md | 11 +++++++++++ tests/core/hash_header/CMakeLists.txt | 16 ++++++++++++++++ 8 files changed, 63 insertions(+), 8 deletions(-) create mode 100644 tests/core/hash_header/CMakeLists.txt diff --git a/docs/reference/utility.md b/docs/reference/utility.md index 6be8ae7a6..1cadd53e8 100644 --- a/docs/reference/utility.md +++ b/docs/reference/utility.md @@ -76,6 +76,15 @@ tail mixing, result word order and the existing public signatures are unchanged; `murmur_hash3_x86_128` remains an instance method, and the other variants remain static. MurmurHash2 retains its fixed seed zero. +The Murmur header itself uses only standard C++20 headers. It does not include +`ark/macros.hpp` or `ark/platform.hpp`; callers using their declarations or macros +must include those headers explicitly. Public macro definitions are unchanged. +This header independence does not change `ark::core` or its CMake package's +dependency requirements. `check-murmur-hash-header` compiles the logical public +header with an isolated include directory and no SDK target dependencies; it is +also included in `check-core`. See [core regressions](../../tests/core/README.md) +for the standalone check commands. + ### Digest compatibility and legacy access **Breaking change:** default MurmurHash3 now reads full blocks as little endian diff --git a/docs/reference/utility_CN.md b/docs/reference/utility_CN.md index 747cec544..18fb89616 100644 --- a/docs/reference/utility_CN.md +++ b/docs/reference/utility_CN.md @@ -68,6 +68,12 @@ MurmurHash3 使用 C++20 `std::rotl`，旋转次数保持不变。种子、尾 原有公开签名不变；`murmur_hash3_x86_128` 仍为实例方法，其余变体仍为静态方法。 MurmurHash2 保留固定种子零。 +Murmur 头文件自身仅使用标准 C++20 头文件，不再包含 `ark/macros.hpp` 或 +`ark/platform.hpp`。调用方若使用其中的声明或宏，须显式包含对应头文件；公开宏定义 +保持不变。头文件独立性不改变 `ark::core` 或其 CMake 包的依赖要求。 +`check-murmur-hash-header` 仅通过隔离的包含目录编译逻辑公开头文件，不依赖 SDK +目标，并纳入 `check-core`。独立检查命令见[核心回归](../../tests/core/README_CN.md)。 + ### 摘要兼容性与旧结果入口 **不兼容变更：** 默认 MurmurHash3 现在在所有支持的平台上按小端方式读取完整块， diff --git a/src/core/container/consistent_hash.hpp b/src/core/container/consistent_hash.hpp index 2932a634e..3c8f8fb6f 100644 --- a/src/core/container/consistent_hash.hpp +++ b/src/core/container/consistent_hash.hpp @@ -23,6 +23,7 @@ #include "ark/dep.hpp" #include "ark/hash/crc.hpp" #include "ark/hash/murmur_hash.hpp" +#include "ark/macros.hpp" namespace ark { diff --git a/src/core/hash/murmur_hash.hpp b/src/core/hash/murmur_hash.hpp index 20fbdc7fd..a7b793c17 100644 --- a/src/core/hash/murmur_hash.hpp +++ b/src/core/hash/murmur_hash.hpp @@ -28,9 +28,6 @@ #include #include -#include "ark/macros.hpp" -#include "ark/platform.hpp" - namespace ark { class murmur_hash @@ -289,8 +286,8 @@ class murmur_hash uint64_t h1 = seed; uint64_t h2 = seed; - const uint64_t c1 = BIG_CONSTANT(0x87c37b91114253d5); - const uint64_t c2 = BIG_CONSTANT(0x4cf5ad432745937f); + const uint64_t c1 = 0x87c37b91114253d5ULL; + const uint64_t c2 = 0x4cf5ad432745937fULL; //---------- // body @@ -492,9 +489,9 @@ class murmur_hash static uint64_t fmix(uint64_t k) { k ^= k >> 33; - k *= BIG_CONSTANT(0xff51afd7ed558ccd); + k *= 0xff51afd7ed558ccdULL; k ^= k >> 33; - k *= BIG_CONSTANT(0xc4ceb9fe1a85ec53); + k *= 0xc4ceb9fe1a85ec53ULL; k ^= k >> 33; return k; diff --git a/tests/core/CMakeLists.txt b/tests/core/CMakeLists.txt index a327d475d..d1d541cce 100644 --- a/tests/core/CMakeLists.txt +++ b/tests/core/CMakeLists.txt @@ -4,11 +4,13 @@ add_library(ark_core_header_checks OBJECT headers/queue.cpp headers/guid128.cpp headers/object_vector.cpp headers/error_code.cpp headers/logger.cpp headers/murmur_hash.cpp) target_link_libraries(ark_core_header_checks PRIVATE ark::core) +add_subdirectory(hash_header) + add_executable(ark_core_tests main.cpp strings.cpp containers.cpp templates.cpp callbacks.cpp header_regressions.cpp cxx20_language.cpp concurrency.cpp plugin_manager_startup.cpp data_strings.cpp node_data_storage.cpp object_vectors.cpp plugin_construction.cpp module_factory.cpp random_generator.cpp xxtea.cpp murmur_hash.cpp) target_link_libraries(ark_core_tests PRIVATE ark::core ark::kernel ark::utility) -add_dependencies(ark_core_tests ark_core_header_checks) +add_dependencies(ark_core_tests ark_core_header_checks ark_murmur_hash_standalone_check) add_library(ark_construction_header_checks OBJECT headers/module_base.cpp headers/plugin_base.cpp) set_property(TARGET ark_construction_header_checks PROPERTY LINK_LIBRARIES ark::core) add_dependencies(ark_core_tests ark_construction_header_checks) diff --git a/tests/core/README.md b/tests/core/README.md index d57d4a947..bfe66a033 100644 --- a/tests/core/README.md +++ b/tests/core/README.md @@ -30,6 +30,19 @@ See [Murmur compatibility](../../docs/reference/utility.md#core-murmur-hashes) for canonical little-endian output and the explicit legacy MSVC migration path. AddressSanitizer and UndefinedBehaviorSanitizer builds exercise overread and alignment checks. +The existing mapped public-header check remains. An additional compile-only check +stages just `` and uses C++20 without project, platform, +spdlog or Boost include paths or SDK target dependencies. It is part of +`check-core` and can also run without configuring the full dependency graph: + +```sh +cmake -S tests/core/hash_header -B hash-header-build -G Ninja +cmake --build hash-header-build --target check-murmur-hash-header +``` + +Header independence does not remove the dependencies required by `ark::core` or +its CMake package. + With `ARK_BUILD_PROTO=ON`, additional cases verify Protobuf parse failures, actor identity, binary messages, legacy wire payloads and external JSON field names. They also exercise the new `ark::protobuf` descriptor and enum names. diff --git a/tests/core/README_CN.md b/tests/core/README_CN.md index 12571eea9..56005e362 100644 --- a/tests/core/README_CN.md +++ b/tests/core/README_CN.md @@ -27,6 +27,17 @@ Murmur 回归包含独立的上游 SMHasher 验证常量、旧版完整块交换 [Murmur 兼容性说明](../../docs/reference/utility_CN.md#core-murmur-哈希)。 AddressSanitizer 和 UndefinedBehaviorSanitizer 构建可检查越界读取和无效对齐。 +现有公开头映射检查继续保留。新增的纯编译检查仅映射 ``， +使用 C++20，不引入项目、平台、spdlog 或 Boost 包含路径，也不依赖 SDK 目标。 +该检查纳入 `check-core`，也可独立运行，无需配置完整依赖图： + +```sh +cmake -S tests/core/hash_header -B hash-header-build -G Ninja +cmake --build hash-header-build --target check-murmur-hash-header +``` + +头文件独立性不改变 `ark::core` 或其 CMake 包所需的依赖。 + 公共头检查将 `` 放在首个包含位置，并重复包含两次；编译期断言 验证其无符号 32 位底层类型及所有已声明的错误值。 diff --git a/tests/core/hash_header/CMakeLists.txt b/tests/core/hash_header/CMakeLists.txt new file mode 100644 index 000000000..f12fbb21b --- /dev/null +++ b/tests/core/hash_header/CMakeLists.txt @@ -0,0 +1,16 @@ +cmake_minimum_required(VERSION 3.21) +project(ark_murmur_hash_header_check LANGUAGES CXX) + +# Only this logical public header is available; no project or third-party SDK. +set(hash_include_dir "${CMAKE_CURRENT_BINARY_DIR}/include") +configure_file("${CMAKE_CURRENT_LIST_DIR}/../../../src/core/hash/murmur_hash.hpp" + "${hash_include_dir}/ark/hash/murmur_hash.hpp" COPYONLY) +add_library(ark_murmur_hash_standalone_check OBJECT ../headers/murmur_hash.cpp) +# Drop the parent project's directory-wide ark::ark link and include paths. +set_target_properties(ark_murmur_hash_standalone_check PROPERTIES + LINK_LIBRARIES "" + INCLUDE_DIRECTORIES "${hash_include_dir}" + CXX_STANDARD 20 + CXX_STANDARD_REQUIRED ON + CXX_EXTENSIONS OFF) +add_custom_target(check-murmur-hash-header DEPENDS ark_murmur_hash_standalone_check) From 2f6cdf404c2f62f5e184ffbe791e08b3432e8b70 Mon Sep 17 00:00:00 2001 From: nick Date: Mon, 26 Oct 2026 12:11:24 +0800 Subject: [PATCH 10/22] refactor(etcd)!: replace plugin creation macros Use typed module and plugin bases with a private interface-returning factory and explicit entry points. Retain every public header, default plugin selection and existing Etcd request implementation. Add factory, ownership and production DSO checks to Applications acceptance. BREAKING CHANGE: etcd_client_module_impl and etcd_client_plugin have changed public class layouts. Rebuild Etcd Client and all consumers together; existing interfaces and logical header paths remain available. --- .github/workflows/ci.yml | 1 + docs/guides/plugins.md | 2 + docs/guides/plugins_CN.md | 2 + docs/reference/build-options.md | 4 +- docs/reference/build-options_CN.md | 4 +- docs/reference/directory-structure.md | 16 +++++ docs/reference/directory-structure_CN.md | 16 +++++ .../etcd_client/etcd_client_factory.hpp | 13 ++++ .../etcd_client/etcd_client_module_impl.cpp | 5 ++ .../etcd_client/etcd_client_module_impl.hpp | 10 ++-- .../etcd_client/etcd_client_plugin.cpp | 23 ++++++-- .../etcd_client/etcd_client_plugin.hpp | 12 +++- tests/ci/install-package.test.mjs | 4 +- tests/ci/package-dependencies.test.mjs | 17 ++++++ tests/consumer/etcd_client/main.cpp | 8 ++- tests/core/CMakeLists.txt | 5 ++ tests/etcd_client/CMakeLists.txt | 35 +++++++++++ tests/etcd_client/construction.cpp | 56 ++++++++++++++++++ tests/etcd_client/entry.cpp | 45 ++++++++++++++ tests/etcd_client/entry_factory.cpp | 59 +++++++++++++++++++ tests/etcd_client/exports.cpp | 31 ++++++++++ 21 files changed, 347 insertions(+), 21 deletions(-) create mode 100644 src/plugins/etcd_client/etcd_client_factory.hpp create mode 100644 tests/etcd_client/CMakeLists.txt create mode 100644 tests/etcd_client/construction.cpp create mode 100644 tests/etcd_client/entry.cpp create mode 100644 tests/etcd_client/entry_factory.cpp create mode 100644 tests/etcd_client/exports.cpp diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index de2ec5664..4730be9fb 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -286,6 +286,7 @@ jobs: -DARK_BUILD_BENCHMARKS=ON -DARK_BUILD_PLUGIN_HTTP=ON -DARK_BUILD_PLUGIN_CONSUL=ON + -DARK_BUILD_PLUGIN_ETCD_CLIENT=ON -DARK_BUILD_PROTO=ON -DARK_BUILD_TESTS=ON -DARK_BUILD_CORE_TESTS=ON diff --git a/docs/guides/plugins.md b/docs/guides/plugins.md index cd3fbcd5d..7afc46cf7 100644 --- a/docs/guides/plugins.md +++ b/docs/guides/plugins.md @@ -213,3 +213,5 @@ Net constructs its HTTP client through a private factory returning `std::unique_ Kernel also uses typed construction while retaining its public `*_impl` and `kernel_plugin` types and header paths. Its private factory header is not installed. Rebuild Kernel and consumers together because those public class layouts change; the [Kernel migration table](../reference/kernel.md#typed-kernel-construction) records unchanged interface keys, update selection and registration order. Consul follows the same construction pattern while retaining all three public headers and types. Its private factory is not installed; the module keeps its interface RTTI key and disabled updates. Rebuild Consul and consumers together because its public implementation/plugin layouts change. See [Consul construction migration](../reference/directory-structure.md#typed-consul-construction) for test entry points and dependency conditions. + +Etcd Client also retains all five public headers and types while using typed bases, a private interface-returning factory and explicit entry/exit functions. Its interface RTTI key and disabled updates are unchanged. Rebuild the plugin and consumers together because the public implementation/plugin layouts change. The [Etcd Client construction migration](../reference/directory-structure.md#typed-etcd-client-construction) describes optional JSON dependencies and tests without an Etcd service. diff --git a/docs/guides/plugins_CN.md b/docs/guides/plugins_CN.md index b66b700d4..ef5d6defb 100644 --- a/docs/guides/plugins_CN.md +++ b/docs/guides/plugins_CN.md @@ -200,3 +200,5 @@ Net 通过私有工厂返回 `std::unique_ptr`，并启用 Kernel 同样使用类型化构造，并保留公开 `*_impl`、`kernel_plugin` 类型与头路径；私有工厂头不安装。公开类布局发生变化，Kernel 与消费者需要一起重建。[Kernel 迁移表](../reference/kernel_CN.md#kernel-类型化构造)记录保持不变的接口键、update 选择与注册顺序。 Consul 使用相同构造方式，保留全部三个公开头与类型。私有工厂不安装；模块保留接口 RTTI 键和禁用 update 的选择。公开实现／插件布局发生变化，Consul 与消费者必须一起重建。测试入口与依赖条件见 [Consul 构造迁移](../reference/directory-structure_CN.md#consul-类型化构造)。 + +Etcd Client 使用类型化基类、返回接口的私有工厂与显式入口／出口，保留全部五个公开头与类型。接口 RTTI 键和禁用 update 的选择不变。公开实现／插件布局发生变化，插件与消费者必须一起重建。可选 JSON 依赖与无需 Etcd 服务的测试见 [Etcd Client 构造迁移](../reference/directory-structure_CN.md#etcd-client-类型化构造)。 diff --git a/docs/reference/build-options.md b/docs/reference/build-options.md index f6be18c32..1825ad741 100644 --- a/docs/reference/build-options.md +++ b/docs/reference/build-options.md @@ -39,9 +39,9 @@ cmake -S . -B build -DARK_BUILD_APPLICATIONS=ON \ cmake --build build --target ark_mysql_plugin ark_redis_plugin ``` -All official plugin public headers are copied from explicit lists at configuration time into each build tree's generated plugin header mapping. Each `ark::` API target carries that include root even with runtime plugins disabled. MySQL, Redis, Utility, Net, Bus and HTTP exclude private assembly and factories. Kernel retains all 38 existing headers, including `kernel_plugin.hpp`; Consul, Etcd Client, Kafka, OSS SDK and Physic retain all 22 existing public headers, including implementation and assembly headers. Registration keys, public type visibility and optional plugin policies are retained; [Kernel typed construction](kernel.md#typed-kernel-construction) and [Consul typed construction](directory-structure.md#typed-consul-construction) change public class layouts and require rebuilding consumers. Their private factory headers are excluded from mapping and installation; header mapping does not validate optional driver integrations. Each public header has one co-located source; its generated include mapping belongs to the selected build tree, rather than a second source layout. +All official plugin public headers are copied from explicit lists at configuration time into each build tree's generated plugin header mapping. Each `ark::` API target carries that include root even with runtime plugins disabled. MySQL, Redis, Utility, Net, Bus and HTTP exclude private assembly and factories. Kernel retains all 38 existing headers, including `kernel_plugin.hpp`; Consul, Etcd Client, Kafka, OSS SDK and Physic retain all 22 existing public headers, including implementation and assembly headers. Registration keys, public type visibility and optional plugin policies are retained; [Kernel typed construction](kernel.md#typed-kernel-construction), [Consul typed construction](directory-structure.md#typed-consul-construction) and [Etcd Client typed construction](directory-structure.md#typed-etcd-client-construction) change public class layouts and require rebuilding consumers. Their private factory headers are excluded from mapping and installation; header mapping does not validate optional driver integrations. Each public header has one co-located source; its generated include mapping belongs to the selected build tree, rather than a second source layout. -Applications CI enables the optional Consul plugin on Linux, macOS and Windows. Its `check-core` run includes construction, DLL lifecycle and production export checks; these checks require no Consul service and do not establish service interoperability. +Applications CI explicitly enables Consul and Etcd Client on Linux, macOS and Windows. Its `check-core` run includes their construction, DLL lifecycle and production export checks; these checks require no Consul or Etcd service and do not establish service interoperability. Etcd Client keeps its existing default `ON` selection and can be disabled with `ARK_BUILD_PLUGIN_ETCD_CLIENT=OFF`. Building it requires the existing `nlohmann_json` package (`etcd-client` vcpkg feature); disabling it does not add that JSON dependency to core tests. Link a consumer to `ark::mysql` and/or `ark::redis` for the public headers and usage requirements. These public interface targets do not load a runtime library. Deploy the selected libraries with the launcher and add their stems to the application's manifest: diff --git a/docs/reference/build-options_CN.md b/docs/reference/build-options_CN.md index b57abfb08..d9d1cd082 100644 --- a/docs/reference/build-options_CN.md +++ b/docs/reference/build-options_CN.md @@ -39,9 +39,9 @@ cmake -S . -B build -DARK_BUILD_APPLICATIONS=ON \ cmake --build build --target ark_mysql_plugin ark_redis_plugin ``` -全部官方插件的公开头通过显式清单在配置阶段复制到各构建树生成的插件公开头映射。各 `ark::` API target 传递该目录，插件关闭时仍可使用。MySQL、Redis、Utility、Net、Bus、HTTP 的私有装配头和工厂不进入映射；Kernel 保留包括 `kernel_plugin.hpp` 在内的原有 38 个头；Consul、Etcd Client、Kafka、OSS SDK、Physic 保留包括实现头与装配头在内的原有 22 个公开头。注册键、公开类型可见性和可选插件策略继续保留；[Kernel 类型化构造](kernel_CN.md#kernel-类型化构造)与 [Consul 类型化构造](directory-structure_CN.md#consul-类型化构造)改变公开类布局，需要重建消费者。两者的私有工厂头均不映射、不安装；头文件映射不代表可选驱动集成已经验证。每个公共头仅有一份共置源码；生成的头文件映射属于选定构建树，不构成第二棵源码目录。 +全部官方插件的公开头通过显式清单在配置阶段复制到各构建树生成的插件公开头映射。各 `ark::` API target 传递该目录，插件关闭时仍可使用。MySQL、Redis、Utility、Net、Bus、HTTP 的私有装配头和工厂不进入映射；Kernel 保留包括 `kernel_plugin.hpp` 在内的原有 38 个头；Consul、Etcd Client、Kafka、OSS SDK、Physic 保留包括实现头与装配头在内的原有 22 个公开头。注册键、公开类型可见性和可选插件策略继续保留；[Kernel 类型化构造](kernel_CN.md#kernel-类型化构造)、[Consul 类型化构造](directory-structure_CN.md#consul-类型化构造)与 [Etcd Client 类型化构造](directory-structure_CN.md#etcd-client-类型化构造)改变公开类布局，需要重建消费者。这些私有工厂头均不映射、不安装；头文件映射不代表可选驱动集成已经验证。每个公共头仅有一份共置源码；生成的头文件映射属于选定构建树，不构成第二棵源码目录。 -应用 CI 在 Linux、macOS、Windows 启用可选 Consul 插件，`check-core` 包含构造、动态库生命周期与生产库导出检查。这些检查不需要 Consul 服务，也不代表服务互通验收。 +应用 CI 在 Linux、macOS、Windows 显式启用 Consul 与 Etcd Client，`check-core` 包含两者的构造、动态库生命周期与生产库导出检查。这些检查不需要 Consul 或 Etcd 服务，也不代表服务互通验收。Etcd Client 保留现有默认 `ON` 选择，可通过 `ARK_BUILD_PLUGIN_ETCD_CLIENT=OFF` 关闭。构建它需要现有 `nlohmann_json` 软件包（vcpkg 的 `etcd-client` feature）；关闭它不会向核心测试增加该 JSON 依赖。 业务插件链接 `ark::mysql` 和/或 `ark::redis`，以获取公开头文件和编译要求；公共接口目标不会加载运行时动态库。将所需动态库与启动器一起部署，并在应用清单中填写库名称： diff --git a/docs/reference/directory-structure.md b/docs/reference/directory-structure.md index 890acc45d..8689ceae9 100644 --- a/docs/reference/directory-structure.md +++ b/docs/reference/directory-structure.md @@ -180,3 +180,19 @@ cmake --build build --target check-consul-construction --parallel 2 ``` `ARK_BUILD_PLUGIN_CONSUL=ON` also requires `ARK_BUILD_APPLICATIONS=ON` and remains off by default. It links the generated protocol target `ark::proto` directly. With the core tests enabled, `check-consul-plugin` additionally inspects the actual Consul DSO exports in a separate process; `check-core` includes the applicable Consul checks. The controlled factory does not exercise the production HTTP client or a Consul server. These construction checks do not establish live Consul interoperability, protocol acceptance or arbitrary hot-update safety. + +## Typed Etcd Client construction + +**Breaking ABI/layout change:** `etcd_client_module_impl` now inherits `module_base` and `etcd_client_plugin` inherits `plugin_base`. Rebuild Etcd Client and consumers together. All five public headers, their `` paths and public types remain available through `ark::etcd_client`; the `etcd_client_module` and `etcd_client` interfaces are unchanged. C++ plugin boundaries do not promise compiler-independent ABI compatibility. + +The private `etcd_client_factory.hpp` returns `std::unique_ptr` and is excluded from public mappings and installation. Explicit `DllEntryPlugin` / `DllExitPlugin` retain the existing platform exports and logger setup. Registration uses the original interface RTTI key and `module_update::disabled`; unregistration destroys the owned module. HTTP-client lookup, endpoint handling, timeout argument order, lease requests and callbacks are unchanged. The existing `keepalive` stub remains incomplete; this construction change adds no protocol or Registry capability. + +Etcd Client remains selected by default for application builds, with `ARK_BUILD_PLUGIN_ETCD_CLIENT=OFF` available to disable it. Its existing `nlohmann_json` package is required only when building the plugin (`etcd-client` vcpkg feature). With `ARK_BUILD_APPLICATIONS=ON`, `ARK_BUILD_PLUGIN_ETCD_CLIENT=ON`, `ARK_BUILD_PROTO=ON` and `ARK_BUILD_CORE_TESTS=ON`, run: + +```sh +cmake --build build --target check-etcd-client-plugin --parallel 1 +``` + +This target includes `check-etcd-client-construction`: actual factory/assembly doctest cases, plus a controlled-factory DSO using the production entry/exit source to check configuration handoff, interface metadata, disabled updates, exactly-once ownership and repeated stop. A separate process loads the actual production plugin, checks its entry/exit exports and unloads it. `check-core` includes both checks when the runtime plugin is selected; disabled-plugin core builds do not acquire the JSON dependency. Applications CI explicitly enables the plugin and runs these checks through its existing `check-core` job. + +These tests do not run an Etcd service or exercise production HTTP/Etcd interoperability. They establish construction and DSO boundaries, not a completed Registry or arbitrary hot-update safety. diff --git a/docs/reference/directory-structure_CN.md b/docs/reference/directory-structure_CN.md index 18665fdf1..809b09471 100644 --- a/docs/reference/directory-structure_CN.md +++ b/docs/reference/directory-structure_CN.md @@ -170,3 +170,19 @@ cmake --build build --target check-consul-construction --parallel 2 ``` `ARK_BUILD_PLUGIN_CONSUL=ON` 还要求 `ARK_BUILD_APPLICATIONS=ON`，默认保持关闭。插件直接链接生成协议目标 `ark::proto`。开启核心测试后，`check-consul-plugin` 还在独立进程中检查真实 Consul DSO 的导出符号；`check-core` 包含当前条件下可用的 Consul 检查。受控工厂不执行生产 HTTP 客户端或访问 Consul 服务。这些构造检查不代表真实 Consul 互通、协议验收或任意热更新安全。 + +## Etcd Client 类型化构造 + +**ABI／布局不兼容变更：** `etcd_client_module_impl` 现在继承 `module_base`，`etcd_client_plugin` 继承 `plugin_base`。Etcd Client 与消费者必须一起重建。全部五个公开头、`` 路径和公开类型继续通过 `ark::etcd_client` 提供，`etcd_client_module` 与 `etcd_client` 接口不变。C++ 插件边界不承诺跨编译器的 ABI 兼容性。 + +私有 `etcd_client_factory.hpp` 返回 `std::unique_ptr`，不映射、不安装。显式 `DllEntryPlugin`／`DllExitPlugin` 保留原有平台导出和日志初始化。注册沿用原接口 RTTI 键，显式使用 `module_update::disabled`；注销时销毁拥有的模块。HTTP 客户端查找、端点处理、超时参数顺序、租约请求与回调保持不变。原有 `keepalive` 空实现仍未完成，本次构造调整不增加协议或 Registry 能力。 + +Etcd Client 保留应用构建默认选中的策略，可通过 `ARK_BUILD_PLUGIN_ETCD_CLIENT=OFF` 关闭。仅构建该插件时需要现有 `nlohmann_json` 软件包（vcpkg 的 `etcd-client` feature）。开启 `ARK_BUILD_APPLICATIONS=ON`、`ARK_BUILD_PLUGIN_ETCD_CLIENT=ON`、`ARK_BUILD_PROTO=ON` 与 `ARK_BUILD_CORE_TESTS=ON` 后，运行： + +```sh +cmake --build build --target check-etcd-client-plugin --parallel 1 +``` + +该目标包含 `check-etcd-client-construction`：真实工厂／装配的 doctest 用例，以及使用生产入口／出口源码与受控工厂的 DSO 测试，覆盖配置传递、接口元数据、禁用 update、所有权恰好销毁一次和重复 stop。另一个独立进程加载真实生产插件，检查入口／出口导出后卸载。选中运行时插件时，`check-core` 包含这两项检查；关闭插件的核心构建不增加 JSON 依赖。应用 CI 显式启用该插件，通过现有 `check-core` 任务执行检查。 + +这些测试不启动 Etcd 服务，不执行生产 HTTP／Etcd 互通验收，只验证构造与 DSO 边界，不代表 Registry 已完成，也不证明任意热更新安全。 diff --git a/src/plugins/etcd_client/etcd_client_factory.hpp b/src/plugins/etcd_client/etcd_client_factory.hpp new file mode 100644 index 000000000..e521550f8 --- /dev/null +++ b/src/plugins/etcd_client/etcd_client_factory.hpp @@ -0,0 +1,13 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#pragma once + +#include + +#include "ark/etcd_client/etcd_client_module.hpp" + +// Plugin-private construction; deliberately absent from public header mappings. +namespace ark::detail { + +std::unique_ptr make_etcd_client_module(); + +} // namespace ark::detail diff --git a/src/plugins/etcd_client/etcd_client_module_impl.cpp b/src/plugins/etcd_client/etcd_client_module_impl.cpp index 7c5007fb1..4edfc4b98 100644 --- a/src/plugins/etcd_client/etcd_client_module_impl.cpp +++ b/src/plugins/etcd_client/etcd_client_module_impl.cpp @@ -1,8 +1,13 @@ #include "ark/etcd_client/etcd_client_impl.hpp" #include "ark/etcd_client/etcd_client_module_impl.hpp" +#include "etcd_client_factory.hpp" namespace ark { +std::unique_ptr detail::make_etcd_client_module() { + return std::make_unique(); +} + bool etcd_client_module_impl::init() { http_client_module_ = find_module(); return http_client_module_ != nullptr; diff --git a/src/plugins/etcd_client/etcd_client_module_impl.hpp b/src/plugins/etcd_client/etcd_client_module_impl.hpp index 88e659ece..8296b0aed 100644 --- a/src/plugins/etcd_client/etcd_client_module_impl.hpp +++ b/src/plugins/etcd_client/etcd_client_module_impl.hpp @@ -1,19 +1,17 @@ #pragma once #include "ark/etcd_client/etcd_client_module.hpp" +#include "ark/module_base.hpp" #include "ark/net/http_client_module.hpp" -#include "ark/plugin_manager.hpp" namespace ark { -class etcd_client_module_impl final : public etcd_client_module +class etcd_client_module_impl final : public module_base { - ARK_DECLARE_MODULE_FUNCTIONS - public: - virtual bool init() override; + bool init() override; - virtual std::shared_ptr create_etcd_client( + std::shared_ptr create_etcd_client( std::vector endpoints, int io_timeout = 5, int connect_timeout = 2) override; private: diff --git a/src/plugins/etcd_client/etcd_client_plugin.cpp b/src/plugins/etcd_client/etcd_client_plugin.cpp index a9f2edf23..ac6ccaef3 100644 --- a/src/plugins/etcd_client/etcd_client_plugin.cpp +++ b/src/plugins/etcd_client/etcd_client_plugin.cpp @@ -1,16 +1,31 @@ -#include "ark/etcd_client/etcd_client_module_impl.hpp" +#include + #include "ark/etcd_client/etcd_client_plugin.hpp" +#include "etcd_client_factory.hpp" namespace ark { -ARK_DECLARE_PLUGIN_DLL_FUNCTION(etcd_client_plugin) +ARK_EXPORT_FUNC void DllEntryPlugin(plugin_manager* manager, const std::string& name, logger* log) { + logger::init(log); + manager->register_plugin(name); +} + +ARK_EXPORT_FUNC void DllExitPlugin(plugin_manager* manager) { + manager->unregister_plugin(); +} void etcd_client_plugin::install() { - ARK_REGISTER_MODULE(etcd_client_module, etcd_client_module_impl); + if (!register_module(detail::make_etcd_client_module(), module_update::disabled)) + { + throw std::runtime_error("Etcd Client module registration rejected"); + } } void etcd_client_plugin::uninstall() { - ARK_UNREGISTER_MODULE(etcd_client_module, etcd_client_module_impl); + if (!unregister_module()) + { + throw std::runtime_error("Etcd Client module unregistration rejected"); + } } } // namespace ark diff --git a/src/plugins/etcd_client/etcd_client_plugin.hpp b/src/plugins/etcd_client/etcd_client_plugin.hpp index a199b52fb..684317dbb 100755 --- a/src/plugins/etcd_client/etcd_client_plugin.hpp +++ b/src/plugins/etcd_client/etcd_client_plugin.hpp @@ -1,8 +1,14 @@ -#include "ark/plugin.hpp" -#include "ark/plugin_manager.hpp" +#pragma once + +#include "ark/plugin_base.hpp" namespace ark { -ARK_DECLARE_PLUGIN(etcd_client_plugin) +class etcd_client_plugin final : public plugin_base +{ +public: + void install() override; + void uninstall() override; +}; } // namespace ark diff --git a/tests/ci/install-package.test.mjs b/tests/ci/install-package.test.mjs index 0b5daeec1..43eb8591d 100644 --- a/tests/ci/install-package.test.mjs +++ b/tests/ci/install-package.test.mjs @@ -72,10 +72,10 @@ include(cmake/ark_install.cmake) const text = readFileSync(file, 'utf8'); for (const forbidden of [root, producer, build, prefix]) assert.ok(!text.includes(forbidden), `${file}: ${forbidden}`); } - for (const name of ['mysql/mysql_module.hpp', 'kernel/kernel_plugin.hpp', 'physic/physic_module_impl.hpp', 'application_manifest.hpp']) { + for (const name of ['mysql/mysql_module.hpp', 'kernel/kernel_plugin.hpp', 'etcd_client/etcd_client_plugin.hpp', 'etcd_client/etcd_client_module_impl.hpp', 'physic/physic_module_impl.hpp', 'application_manifest.hpp']) { assert.ok(existsSync(join(relocated, 'include/ark', name)), name); } - for (const name of ['mysql/mysql_plugin.hpp', 'bus/bus_config_parser.hpp', 'http/http_plugin.hpp', 'net/http_client_module_impl.hpp', 'net/http_client_factory.hpp', 'kernel/kernel_factory.hpp', 'configuration_reader.hpp']) { + for (const name of ['mysql/mysql_plugin.hpp', 'bus/bus_config_parser.hpp', 'http/http_plugin.hpp', 'net/http_client_module_impl.hpp', 'net/http_client_factory.hpp', 'kernel/kernel_factory.hpp', 'etcd_client/etcd_client_factory.hpp', 'configuration_reader.hpp']) { assert.equal(existsSync(join(relocated, 'include/ark', name)), false, name); } rmSync(producer, { recursive: true }); diff --git a/tests/ci/package-dependencies.test.mjs b/tests/ci/package-dependencies.test.mjs index 9fee41e8e..840a16bc9 100644 --- a/tests/ci/package-dependencies.test.mjs +++ b/tests/ci/package-dependencies.test.mjs @@ -87,3 +87,20 @@ test('Applications CI enables Consul construction and production export checks w assert.match(checks, /add_dependencies\(check-core check-consul-plugin\)/); assert.match(checks, /add_dependencies\(check-core check-consul-construction\)/); }); + +test('Applications CI explicitly enables Etcd Client ownership and production DSO checks', () => { + const applications = read('.github/workflows/ci.yml').split(' applications:\n')[1].split('\n database:')[0]; + assert.match(applications, /-DARK_BUILD_PLUGIN_ETCD_CLIENT=ON/); + assert.match(applications, /run: cmake --build ci-build --target check-core/); + const core = read('tests/core/CMakeLists.txt'); + assert.match(core, /if\(TARGET ark_etcd_client_plugin\)[\s\S]*add_subdirectory\(\.\.\/etcd_client/); + const checks = read('tests/etcd_client/CMakeLists.txt'); + assert.match(checks, /add_dependencies\(check-core check-etcd-client-plugin check-etcd-client-construction\)/); + assert.match(checks, /etcd_client_impl\.cpp/); + const mappings = read('src/plugins/CMakeLists.txt').split('set(ark_etcd_client_public_headers')[1].split('\n)')[0]; + assert.doesNotMatch(mappings, /etcd_client_factory/); + const plugin = read('src/plugins/etcd_client/etcd_client_plugin.cpp'); + assert.match(plugin, /DllEntryPlugin/); + assert.match(plugin, /DllExitPlugin/); + assert.doesNotMatch(plugin, /ARK_DECLARE_PLUGIN|ARK_REGISTER_MODULE|ARK_UNREGISTER_MODULE/); +}); diff --git a/tests/consumer/etcd_client/main.cpp b/tests/consumer/etcd_client/main.cpp index ea35b4e3e..8b4071e70 100644 --- a/tests/consumer/etcd_client/main.cpp +++ b/tests/consumer/etcd_client/main.cpp @@ -8,8 +8,12 @@ #include static_assert(std::is_base_of_v); -static_assert(std::is_base_of_v); -static_assert(std::is_base_of_v); +static_assert(std::is_base_of_v, ark::etcd_client_module_impl>); +static_assert(std::is_base_of_v, ark::etcd_client_plugin>); + +#if __has_include() +#error "Etcd Client private factory must not be mapped or installed" +#endif int main() { return 0; diff --git a/tests/core/CMakeLists.txt b/tests/core/CMakeLists.txt index 768a291bb..ba3267c3f 100644 --- a/tests/core/CMakeLists.txt +++ b/tests/core/CMakeLists.txt @@ -226,6 +226,11 @@ if(ARK_BUILD_PROTO AND ARK_BUILD_CONFIG) add_subdirectory(../consul "${CMAKE_CURRENT_BINARY_DIR}/consul") endif() +# Keep JSON optional: construction checks follow the selected runtime plugin. +if(TARGET ark_etcd_client_plugin) + add_subdirectory(../etcd_client "${CMAKE_CURRENT_BINARY_DIR}/etcd_client") +endif() + if(ARK_BUILD_PROTO AND TARGET ark_kernel_plugin AND TARGET ark_utility_plugin) add_executable(ark_kernel_plugin_tests ../app/kernel_plugin.cpp) set_property(TARGET ark_kernel_plugin_tests PROPERTY LINK_LIBRARIES ark::kernel) diff --git a/tests/etcd_client/CMakeLists.txt b/tests/etcd_client/CMakeLists.txt new file mode 100644 index 000000000..66e391ace --- /dev/null +++ b/tests/etcd_client/CMakeLists.txt @@ -0,0 +1,35 @@ +# Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +find_package(nlohmann_json CONFIG REQUIRED) +set(etcd_client_source "${PROJECT_SOURCE_DIR}/src/plugins/etcd_client") + +# Compile real assembly/factory locally too: private C++ symbols are not DLL exports. +add_executable(ark_etcd_client_construction_tests ../core/main.cpp construction.cpp + "${etcd_client_source}/etcd_client_plugin.cpp" "${etcd_client_source}/etcd_client_module_impl.cpp" "${etcd_client_source}/etcd_client_impl.cpp") +set_property(TARGET ark_etcd_client_construction_tests PROPERTY LINK_LIBRARIES + ark::etcd_client nlohmann_json::nlohmann_json) +target_include_directories(ark_etcd_client_construction_tests PRIVATE "${etcd_client_source}") + +# A controlled module exercises the actual entry/exit functions without HTTP or Etcd Client services. +add_library(ark_etcd_client_entry_probe SHARED entry_factory.cpp "${etcd_client_source}/etcd_client_plugin.cpp") +set_property(TARGET ark_etcd_client_entry_probe PROPERTY LINK_LIBRARIES ark::etcd_client) +target_include_directories(ark_etcd_client_entry_probe PRIVATE "${etcd_client_source}") +set_target_properties(ark_etcd_client_entry_probe PROPERTIES PREFIX "") +add_executable(ark_etcd_client_entry_tests entry.cpp) +set_property(TARGET ark_etcd_client_entry_tests PROPERTY LINK_LIBRARIES ark::etcd_client) + +add_custom_target(check-etcd-client-construction + COMMAND $ + COMMAND $ "$/" "ark_etcd_client_entry_probe" + DEPENDS ark_etcd_client_construction_tests ark_etcd_client_entry_tests ark_etcd_client_entry_probe + USES_TERMINAL VERBATIM) + +if(TARGET ark_etcd_client_plugin) + # Keep real DSO export inspection separate from the host linking static AFProto. + add_executable(ark_etcd_client_exports_tests exports.cpp) + set_property(TARGET ark_etcd_client_exports_tests PROPERTY LINK_LIBRARIES ark::core) + add_custom_target(check-etcd-client-plugin + COMMAND $ "$/" "ark_etcd_client_plugin" + DEPENDS ark_etcd_client_exports_tests ark_etcd_client_plugin check-etcd-client-construction + USES_TERMINAL VERBATIM) + add_dependencies(check-core check-etcd-client-plugin check-etcd-client-construction) +endif() diff --git a/tests/etcd_client/construction.cpp b/tests/etcd_client/construction.cpp new file mode 100644 index 000000000..5d600fca3 --- /dev/null +++ b/tests/etcd_client/construction.cpp @@ -0,0 +1,56 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include +#include +#include + +#include "../network/doctest.h" + +#include +#include +#include "etcd_client_factory.hpp" + +#if __has_include() +#error "Etcd Client public mappings must exclude the private factory" +#endif + +static_assert(std::is_base_of_v, ark::etcd_client_module_impl>); +static_assert(std::is_base_of_v, ark::etcd_client_plugin>); +static_assert(std::is_same_v); +static_assert( + std::is_same_v>); + +TEST_CASE("Etcd Client real factory retains its public implementation and unpublished metadata") { + auto candidate = ark::detail::make_etcd_client_module(); + REQUIRE(candidate != nullptr); + const auto* instance = candidate.get(); + CHECK(typeid(*instance) == typeid(ark::etcd_client_module_impl)); + CHECK(candidate->get_plugin_manager() == nullptr); + CHECK(candidate->get_name().empty()); +} + +TEST_CASE("Etcd Client assembly retains the interface key, disabled updates and explicit ownership") { + ark::plugin_manager manager; + ark::etcd_client_plugin owner; + owner.set_plugin_manager(&manager); + CHECK(owner.get_plugin_version() == 0); + CHECK(owner.get_plugin_name() == typeid(ark::etcd_client_plugin).name()); + owner.set_plugin_conf("etcd-client-construction.toml"); + CHECK(ark::etcd_client_plugin::get_plugin_conf() == "etcd-client-construction.toml"); + owner.install(); + auto* instance = manager.find_module(); + REQUIRE(instance != nullptr); + CHECK(typeid(*instance) == typeid(ark::etcd_client_module_impl)); + CHECK(instance->get_plugin_manager() == &manager); + CHECK(instance->get_name() == typeid(ark::etcd_client_module).name()); + // Insertion succeeds only when assembly left the module out of the update set. + CHECK(manager.add_update_module(instance)); + manager.remove_update_module(instance->get_name()); + CHECK_THROWS_AS(owner.install(), std::runtime_error); + CHECK(manager.find_module() == instance); + owner.uninstall(); + // Reinstallation proves the public registration key and ownership were released. + CHECK_NOTHROW(owner.install()); + CHECK_NOTHROW(owner.uninstall()); + CHECK(manager.update(1)); + CHECK_THROWS_AS(owner.uninstall(), std::runtime_error); +} diff --git a/tests/etcd_client/entry.cpp b/tests/etcd_client/entry.cpp new file mode 100644 index 000000000..7420feeca --- /dev/null +++ b/tests/etcd_client/entry.cpp @@ -0,0 +1,45 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include +#include +#include + +#include +#include +#include + +namespace { + +void require(bool condition, const char* message) { + if (!condition) + { + std::cerr << message << std::endl; + std::_Exit(1); + } +} + +} // namespace + +int main(int argc, char* argv[]) { + require(argc == 3, "Expected Etcd Client entry-probe directory and library stem"); + ark::logger log; + ark::logger::init(&log); + log.set_log_func([](spdlog::level::level_enum, const std::string&) {}); + ark::dynamic_library library(argv[2]); + require(library.load(argv[1]), "Cannot load Etcd Client entry-probe library"); + auto events = reinterpret_cast(library.get_symbol("etcd_client_entry_events")); + require(events && events(0) == 0 && events(1) == 0, "Entry-probe state is not fresh"); + require(library.get_symbol("DllEntryPlugin") && library.get_symbol("DllExitPlugin"), + "Etcd Client explicit entry/exit exports are missing"); + ark::plugin_manager manager; + require(manager.start(0, argv[1], "conf/", {{argv[2], "entry-probe.toml"}}), "Etcd Client explicit entry failed"); + auto* instance = manager.find_module(); + require(instance && instance->get_plugin_manager() == &manager && + instance->get_name() == typeid(ark::etcd_client_module).name(), + "Etcd Client explicit entry lost interface registration metadata"); + require(events(0) == 1 && events(1) == 0, "Etcd Client entry did not transfer factory ownership"); + require(manager.update(1) && events(2) == 0, "Etcd Client entry changed disabled update selection"); + require(manager.stop(2) && events(1) == 1, "Etcd Client explicit exit did not destroy its module exactly once"); + require(manager.stop(3) && events(1) == 1, "Repeated stop invoked Etcd Client exit twice"); + require(library.unload(), "Cannot release Etcd Client observation handle"); + std::cout << "Etcd Client real assembly entry/exit, configuration, interface metadata and ownership passed\n"; +} diff --git a/tests/etcd_client/entry_factory.cpp b/tests/etcd_client/entry_factory.cpp new file mode 100644 index 000000000..ab69b24a0 --- /dev/null +++ b/tests/etcd_client/entry_factory.cpp @@ -0,0 +1,59 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include + +#include +#include +#include "etcd_client_factory.hpp" + +namespace { + +unsigned created{}; +unsigned destroyed{}; +unsigned updated{}; + +class entry_etcd_client final : public ark::module_base +{ +public: + entry_etcd_client() { + ++created; + } + + ~entry_etcd_client() override { + ++destroyed; + } + + bool init() override { + return ark::etcd_client_plugin::get_plugin_conf() == "conf/entry-probe.toml"; + } + + bool update() override { + ++updated; + return true; + } + + std::shared_ptr create_etcd_client(std::vector, int, int) override { + return {}; + } +}; + +} // namespace + +namespace ark::detail { + +std::unique_ptr make_etcd_client_module() { + return std::make_unique(); +} + +} // namespace ark::detail + +ARK_EXPORT_FUNC unsigned etcd_client_entry_events(unsigned event) { + switch (event) + { + case 0: + return created; + case 1: + return destroyed; + default: + return updated; + } +} diff --git a/tests/etcd_client/exports.cpp b/tests/etcd_client/exports.cpp new file mode 100644 index 000000000..f6ef4cb20 --- /dev/null +++ b/tests/etcd_client/exports.cpp @@ -0,0 +1,31 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include +#include +#include + +#include + +namespace { + +void require(bool condition, const char* message) { + if (!condition) + { + std::cerr << message << std::endl; + std::_Exit(1); + } +} + +} // namespace + +int main(int argc, char* argv[]) { + require(argc == 3, "Expected plugin directory and library stem"); + ark::logger log; + ark::logger::init(&log); + log.set_log_func([](spdlog::level::level_enum, const std::string&) {}); + ark::dynamic_library library(argv[2]); + require(library.load(argv[1]), "Cannot load actual Etcd Client shared library"); + require(library.get_symbol("DllEntryPlugin") != nullptr, "Missing explicit DllEntryPlugin export"); + require(library.get_symbol("DllExitPlugin") != nullptr, "Missing explicit DllExitPlugin export"); + require(library.unload(), "Cannot unload Etcd Client shared library"); + std::cout << argv[2] << ": actual DSO load, explicit entry/exit exports and unload passed\n"; +} From d38982c3cea351c07d51a8d634be0dd0aa217c73 Mon Sep 17 00:00:00 2001 From: nick Date: Mon, 26 Oct 2026 13:01:32 +0800 Subject: [PATCH 11/22] fix(tests): probe Net DSO listener and peer closure Check a successful live-listener negative control, retain the exercised TCP peer across manager shutdown, and require a remote EOF or reset before stopping that peer. Check listener closure using an exact refused new connection instead of global immediate port rebinding. Keep existing endpoint ownership, manager state, main-thread callbacks and pending HTTP cancellation assertions. The original macOS CI bind conflict holder was not reproduced locally; no production shutdown or reuse-address behavior changes. --- tests/app/net_plugin.cpp | 35 +++++++++++++++++++++++++++++++---- 1 file changed, 31 insertions(+), 4 deletions(-) diff --git a/tests/app/net_plugin.cpp b/tests/app/net_plugin.cpp index 6eb53f4a3..48a998c83 100644 --- a/tests/app/net_plugin.cpp +++ b/tests/app/net_plugin.cpp @@ -37,6 +37,13 @@ void wait_for(predicate ready, pump update) { require(ready(), "Timed out waiting for owned loopback peer"); } +arknet::error_code listener_probe(asio::io_context& context, const asio::ip::tcp::endpoint& endpoint) { + asio::ip::tcp::socket probe(context); + arknet::error_code error; + probe.connect(endpoint, error); + return error; +} + struct receiver { void on_message(const ark::net_msg* message, ark::conv_id_t session) { @@ -86,8 +93,17 @@ int main(int argc, char* argv[]) { receiver received; require(server->reg_msg_callback(45000, &received, &receiver::on_message), "Net message callback rejected"); std::weak_ptr endpoint = server->get_net(); + const asio::ip::tcp::endpoint listener_endpoint(asio::ip::make_address("127.0.0.1"), port); + const auto live_probe = listener_probe(context, listener_endpoint); + require(!live_probe, "Live Net listener negative control failed: " + live_probe.message()); + std::atomic peer_closed{}; + arknet::error_code peer_close_error; arknet::tcp_client client; client.set_auto_reconnect(false); + client.bind_disconnect([&] { + peer_close_error = arknet::get_last_error(); + peer_closed.store(true); + }); require(client.start("127.0.0.1", port), "Owned TCP peer failed to start"); ark::s_msg_header header; header.set_message_id(45000); @@ -143,16 +159,27 @@ int main(int argc, char* argv[]) { const auto cancelled = pending.get(); require(!cancelled.first && cancelled.second == arknet::error_code(asio::error::operation_aborted).message(), "HTTP DSO shutdown changed pending-request cancellation"); - client.stop(); peer.stop(); + require(client.is_started() && !peer_closed.load(), "Owned TCP peer must remain live across Net shutdown"); server.reset(); require(manager.stop(++tick), "Net DSO shutdown did not complete"); require(endpoint.expired(), "Net listener survived normal module unregistration"); require(manager.get_shutdown_state() == ark::plugin_manager::shutdown_state::stopped, "Net resources were retained instead of normally stopped"); + // Keep the peer open until the server closes its accepted socket. Stopping + // the client first would hide a retained server-side connection. + wait_for([&] { return peer_closed.load(); }, [] {}); + require(peer_close_error == asio::error::eof || peer_close_error == asio::error::connection_reset, + "Net shutdown did not remotely close the owned TCP peer: " + peer_close_error.message()); + client.stop(); require(manager.stop(++tick), "Repeated Net shutdown failed"); - asio::ip::tcp::acceptor rebound(context, {asio::ip::make_address("127.0.0.1"), port}); - require(rebound.is_open(), "Net shutdown did not release its listener"); - std::cout << "Net actual DSOs: interface RTTI lookup, manager-owned TCP/HTTP callbacks, listener release and " + // Binding the same port also tests unrelated kernel port-reuse state. A + // refused new connection checks listener closure directly, with no retry. + // The successful probe above is the negative control for a live listener. + const auto closed_probe = listener_probe(context, listener_endpoint); + require(closed_probe == asio::error::connection_refused, + "Net listener closure probe must be refused: " + closed_probe.message()); + std::cout << "Net actual DSOs: interface RTTI lookup, manager-owned TCP/HTTP callbacks, listener refusal, remote " + "peer closure and " "normal repeated shutdown passed\n"; } From 5c621eb117923068d1e220751cda19fdaf48bb80 Mon Sep 17 00:00:00 2001 From: nick Date: Mon, 26 Oct 2026 13:09:43 +0800 Subject: [PATCH 12/22] fix(tests): bound the Net listener shutdown probe Use a private io_context, async_connect and the existing five-second deadline. Cancel and close the probe socket on expiry, return typed timed_out rather than operation_aborted, and drain connect and timer callbacks before returning. Keep all live-listener, retained-peer and owner-state assertions and the existing test timeout. Production shutdown remains unchanged. --- tests/app/net_plugin.cpp | 28 ++++++++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-) diff --git a/tests/app/net_plugin.cpp b/tests/app/net_plugin.cpp index 48a998c83..e89307883 100644 --- a/tests/app/net_plugin.cpp +++ b/tests/app/net_plugin.cpp @@ -37,11 +37,31 @@ void wait_for(predicate ready, pump update) { require(ready(), "Timed out waiting for owned loopback peer"); } -arknet::error_code listener_probe(asio::io_context& context, const asio::ip::tcp::endpoint& endpoint) { +arknet::error_code listener_probe(asio::io_context&, const asio::ip::tcp::endpoint& endpoint) { + asio::io_context context; asio::ip::tcp::socket probe(context); - arknet::error_code error; - probe.connect(endpoint, error); - return error; + asio::steady_timer deadline(context, 5s); + arknet::error_code result = asio::error::operation_aborted; + bool completed = false; + bool timed_out = false; + deadline.async_wait([&](const arknet::error_code& error) { + if (!error && !completed) + { + timed_out = true; + arknet::error_code ignored; + probe.cancel(ignored); + probe.close(ignored); + } + }); + probe.async_connect(endpoint, [&](const arknet::error_code& error) { + completed = true; + result = timed_out ? arknet::error_code(asio::error::timed_out) : error; + deadline.cancel(); + }); + // Drain both the connect completion and the cancelled timer before their + // stack captures go out of scope. A deadline is not a refused connection. + context.run(); + return result; } struct receiver From ab1a01c24ed8af1a6403af05ce64cdafc8020a54 Mon Sep 17 00:00:00 2001 From: nick Date: Mon, 26 Oct 2026 13:53:29 +0800 Subject: [PATCH 13/22] refactor(osssdk)!: use typed plugin construction Retain public OSS SDK headers and types while replacing creation macros with module_base, an interface-returning private factory, plugin_base and explicit entry points. Replace the undefined deregistration call with typed owned unregistration while preserving the interface key and disabled updates. Add real-factory assembly and controlled DSO lifecycle regressions, production export checks and bilingual migration guidance. Keep the optional plugin off by default and retain its TODO capability boundary. BREAKING CHANGE: oss_sdk_module_impl and oss_sdk_plugin change public class layouts. Rebuild OSS SDK and all affected consumers together; original public header paths, types and the oss_sdk_module interface remain available. --- docs/guides/plugins.md | 2 + docs/guides/plugins_CN.md | 2 + docs/project-layout.md | 2 + docs/project-layout_CN.md | 2 + docs/reference/build-options.md | 4 +- docs/reference/build-options_CN.md | 4 +- docs/reference/directory-structure.md | 22 +++++++++ docs/reference/directory-structure_CN.md | 22 +++++++++ src/plugins/README.md | 4 +- src/plugins/README_CN.md | 4 +- src/plugins/osssdk/oss_sdk_factory.hpp | 13 +++++ src/plugins/osssdk/oss_sdk_module_impl.cpp | 5 +- src/plugins/osssdk/oss_sdk_module_impl.hpp | 5 +- src/plugins/osssdk/oss_sdk_plugin.cpp | 25 +++++++--- src/plugins/osssdk/oss_sdk_plugin.hpp | 10 ++-- tests/core/CMakeLists.txt | 2 + tests/osssdk/CMakeLists.txt | 36 ++++++++++++++ tests/osssdk/construction.cpp | 55 ++++++++++++++++++++++ tests/osssdk/entry.cpp | 45 ++++++++++++++++++ tests/osssdk/entry_factory.cpp | 55 ++++++++++++++++++++++ tests/osssdk/exports.cpp | 31 ++++++++++++ 21 files changed, 333 insertions(+), 17 deletions(-) create mode 100644 src/plugins/osssdk/oss_sdk_factory.hpp create mode 100644 tests/osssdk/CMakeLists.txt create mode 100644 tests/osssdk/construction.cpp create mode 100644 tests/osssdk/entry.cpp create mode 100644 tests/osssdk/entry_factory.cpp create mode 100644 tests/osssdk/exports.cpp diff --git a/docs/guides/plugins.md b/docs/guides/plugins.md index 7afc46cf7..e6b8d8bd2 100644 --- a/docs/guides/plugins.md +++ b/docs/guides/plugins.md @@ -215,3 +215,5 @@ Kernel also uses typed construction while retaining its public `*_impl` and `ker Consul follows the same construction pattern while retaining all three public headers and types. Its private factory is not installed; the module keeps its interface RTTI key and disabled updates. Rebuild Consul and consumers together because its public implementation/plugin layouts change. See [Consul construction migration](../reference/directory-structure.md#typed-consul-construction) for test entry points and dependency conditions. Etcd Client also retains all five public headers and types while using typed bases, a private interface-returning factory and explicit entry/exit functions. Its interface RTTI key and disabled updates are unchanged. Rebuild the plugin and consumers together because the public implementation/plugin layouts change. The [Etcd Client construction migration](../reference/directory-structure.md#typed-etcd-client-construction) describes optional JSON dependencies and tests without an Etcd service. + +OSS SDK uses the same typed bases and private interface-returning factory while retaining all three public headers/types, its interface RTTI key and disabled updates. Rebuild the plugin and consumers together for the public layout change. Typed unregistration replaces its undefined legacy deregistration call; the implementation remains a TODO scaffold. See [OSS SDK construction migration](../reference/directory-structure.md#typed-oss-sdk-construction) for build and test boundaries. diff --git a/docs/guides/plugins_CN.md b/docs/guides/plugins_CN.md index ef5d6defb..9f9a2126b 100644 --- a/docs/guides/plugins_CN.md +++ b/docs/guides/plugins_CN.md @@ -202,3 +202,5 @@ Kernel 同样使用类型化构造，并保留公开 `*_impl`、`kernel_plugin` Consul 使用相同构造方式，保留全部三个公开头与类型。私有工厂不安装；模块保留接口 RTTI 键和禁用 update 的选择。公开实现／插件布局发生变化，Consul 与消费者必须一起重建。测试入口与依赖条件见 [Consul 构造迁移](../reference/directory-structure_CN.md#consul-类型化构造)。 Etcd Client 使用类型化基类、返回接口的私有工厂与显式入口／出口，保留全部五个公开头与类型。接口 RTTI 键和禁用 update 的选择不变。公开实现／插件布局发生变化，插件与消费者必须一起重建。可选 JSON 依赖与无需 Etcd 服务的测试见 [Etcd Client 构造迁移](../reference/directory-structure_CN.md#etcd-client-类型化构造)。 + +OSS SDK 使用相同的类型化基类和返回接口的私有工厂，保留全部三个公开头／类型、接口 RTTI 键和禁用 update 的选择。公开布局发生变化，插件与消费者必须一起重建。类型化注销替代未定义的旧注销调用；实现仍为 TODO 骨架。构建与测试边界见 [OSS SDK 构造迁移](../reference/directory-structure_CN.md#oss-sdk-类型化构造)。 diff --git a/docs/project-layout.md b/docs/project-layout.md index 6ce9b31b3..b17dafa02 100644 --- a/docs/project-layout.md +++ b/docs/project-layout.md @@ -133,4 +133,6 @@ cmake -S tests/consumer/etcd_client -B etcd-client-headers -G Ninja \ cmake --build etcd-client-headers --target check-etcd-client-headers --parallel 2 ``` +OSS SDK retains its three public headers and types through typed construction; rebuild the plugin and consumers together because public class layouts change. See [OSS SDK construction migration](reference/directory-structure.md#typed-oss-sdk-construction) for optional build conditions and construction/export checks. The implementation remains a TODO scaffold. + Header mapping does not establish Consul, Kafka or OSS SDK runtime compatibility. Physic and `physics_scene` still require the unsupported legacy PhysX 3 integration and retain their explicit configuration failure; no passing PhysX build is claimed. diff --git a/docs/project-layout_CN.md b/docs/project-layout_CN.md index 82a8ee230..00faadbad 100644 --- a/docs/project-layout_CN.md +++ b/docs/project-layout_CN.md @@ -123,4 +123,6 @@ cmake -S tests/consumer/etcd_client -B etcd-client-headers -G Ninja \ cmake --build etcd-client-headers --target check-etcd-client-headers --parallel 2 ``` +OSS SDK 使用类型化构造，保留全部三个公开头和类型；公开类布局发生变化，插件与消费者必须一起重建。可选构建条件及构造／导出检查见 [OSS SDK 构造迁移](reference/directory-structure_CN.md#oss-sdk-类型化构造)。实现仍为 TODO 骨架。 + 头文件映射检查不代表 Consul、Kafka、OSS SDK 的运行时兼容性已经验证。Physic 与 `physics_scene` 仍依赖未受支持的旧版 PhysX 3 集成，保留明确的配置失败；此处没有宣称 PhysX 构建通过。 diff --git a/docs/reference/build-options.md b/docs/reference/build-options.md index 1825ad741..465a54877 100644 --- a/docs/reference/build-options.md +++ b/docs/reference/build-options.md @@ -39,10 +39,12 @@ cmake -S . -B build -DARK_BUILD_APPLICATIONS=ON \ cmake --build build --target ark_mysql_plugin ark_redis_plugin ``` -All official plugin public headers are copied from explicit lists at configuration time into each build tree's generated plugin header mapping. Each `ark::` API target carries that include root even with runtime plugins disabled. MySQL, Redis, Utility, Net, Bus and HTTP exclude private assembly and factories. Kernel retains all 38 existing headers, including `kernel_plugin.hpp`; Consul, Etcd Client, Kafka, OSS SDK and Physic retain all 22 existing public headers, including implementation and assembly headers. Registration keys, public type visibility and optional plugin policies are retained; [Kernel typed construction](kernel.md#typed-kernel-construction), [Consul typed construction](directory-structure.md#typed-consul-construction) and [Etcd Client typed construction](directory-structure.md#typed-etcd-client-construction) change public class layouts and require rebuilding consumers. Their private factory headers are excluded from mapping and installation; header mapping does not validate optional driver integrations. Each public header has one co-located source; its generated include mapping belongs to the selected build tree, rather than a second source layout. +All official plugin public headers are copied from explicit lists at configuration time into each build tree's generated plugin header mapping. Each `ark::` API target carries that include root even with runtime plugins disabled. MySQL, Redis, Utility, Net, Bus and HTTP exclude private assembly and factories. Kernel retains all 38 existing headers, including `kernel_plugin.hpp`; Consul, Etcd Client, Kafka, OSS SDK and Physic retain all 22 existing public headers, including implementation and assembly headers. Registration keys, public type visibility and optional plugin policies are retained; [Kernel typed construction](kernel.md#typed-kernel-construction), [Consul typed construction](directory-structure.md#typed-consul-construction), [Etcd Client typed construction](directory-structure.md#typed-etcd-client-construction) and [OSS SDK typed construction](directory-structure.md#typed-oss-sdk-construction) change public class layouts and require rebuilding consumers. Their private factory headers are excluded from mapping and installation; header mapping does not validate optional driver integrations. Each public header has one co-located source; its generated include mapping belongs to the selected build tree, rather than a second source layout. Applications CI explicitly enables Consul and Etcd Client on Linux, macOS and Windows. Its `check-core` run includes their construction, DLL lifecycle and production export checks; these checks require no Consul or Etcd service and do not establish service interoperability. Etcd Client keeps its existing default `ON` selection and can be disabled with `ARK_BUILD_PLUGIN_ETCD_CLIENT=OFF`. Building it requires the existing `nlohmann_json` package (`etcd-client` vcpkg feature); disabling it does not add that JSON dependency to core tests. +OSS SDK remains optional and off by default (`ARK_BUILD_PLUGIN_OSSSDK=OFF`). Core tests include its real factory/assembly and controlled DSO checks without Protobuf or a runtime plugin. With the runtime plugin selected, `check-oss-sdk-plugin` also loads the actual DSO and checks its exports; `check-core` includes the applicable checks. The implementation is still a TODO scaffold, with no object-storage client or service acceptance. See [OSS SDK construction migration](directory-structure.md#typed-oss-sdk-construction). + Link a consumer to `ark::mysql` and/or `ark::redis` for the public headers and usage requirements. These public interface targets do not load a runtime library. Deploy the selected libraries with the launcher and add their stems to the application's manifest: ```toml diff --git a/docs/reference/build-options_CN.md b/docs/reference/build-options_CN.md index d9d1cd082..2eec5a7a1 100644 --- a/docs/reference/build-options_CN.md +++ b/docs/reference/build-options_CN.md @@ -39,10 +39,12 @@ cmake -S . -B build -DARK_BUILD_APPLICATIONS=ON \ cmake --build build --target ark_mysql_plugin ark_redis_plugin ``` -全部官方插件的公开头通过显式清单在配置阶段复制到各构建树生成的插件公开头映射。各 `ark::` API target 传递该目录，插件关闭时仍可使用。MySQL、Redis、Utility、Net、Bus、HTTP 的私有装配头和工厂不进入映射；Kernel 保留包括 `kernel_plugin.hpp` 在内的原有 38 个头；Consul、Etcd Client、Kafka、OSS SDK、Physic 保留包括实现头与装配头在内的原有 22 个公开头。注册键、公开类型可见性和可选插件策略继续保留；[Kernel 类型化构造](kernel_CN.md#kernel-类型化构造)、[Consul 类型化构造](directory-structure_CN.md#consul-类型化构造)与 [Etcd Client 类型化构造](directory-structure_CN.md#etcd-client-类型化构造)改变公开类布局，需要重建消费者。这些私有工厂头均不映射、不安装；头文件映射不代表可选驱动集成已经验证。每个公共头仅有一份共置源码；生成的头文件映射属于选定构建树，不构成第二棵源码目录。 +全部官方插件的公开头通过显式清单在配置阶段复制到各构建树生成的插件公开头映射。各 `ark::` API target 传递该目录，插件关闭时仍可使用。MySQL、Redis、Utility、Net、Bus、HTTP 的私有装配头和工厂不进入映射；Kernel 保留包括 `kernel_plugin.hpp` 在内的原有 38 个头；Consul、Etcd Client、Kafka、OSS SDK、Physic 保留包括实现头与装配头在内的原有 22 个公开头。注册键、公开类型可见性和可选插件策略继续保留；[Kernel 类型化构造](kernel_CN.md#kernel-类型化构造)、[Consul 类型化构造](directory-structure_CN.md#consul-类型化构造)、[Etcd Client 类型化构造](directory-structure_CN.md#etcd-client-类型化构造)与 [OSS SDK 类型化构造](directory-structure_CN.md#oss-sdk-类型化构造)改变公开类布局，需要重建消费者。这些私有工厂头均不映射、不安装；头文件映射不代表可选驱动集成已经验证。每个公共头仅有一份共置源码；生成的头文件映射属于选定构建树，不构成第二棵源码目录。 应用 CI 在 Linux、macOS、Windows 显式启用 Consul 与 Etcd Client，`check-core` 包含两者的构造、动态库生命周期与生产库导出检查。这些检查不需要 Consul 或 Etcd 服务，也不代表服务互通验收。Etcd Client 保留现有默认 `ON` 选择，可通过 `ARK_BUILD_PLUGIN_ETCD_CLIENT=OFF` 关闭。构建它需要现有 `nlohmann_json` 软件包（vcpkg 的 `etcd-client` feature）；关闭它不会向核心测试增加该 JSON 依赖。 +OSS SDK 保持可选且默认关闭（`ARK_BUILD_PLUGIN_OSSSDK=OFF`）。核心测试包含真实工厂／装配与受控 DSO 检查，无需 Protobuf 或运行时插件。选中运行时插件后，`check-oss-sdk-plugin` 还加载真实 DSO 并检查导出；`check-core` 包含当前条件下可用的检查。实现仍为 TODO 骨架，没有对象存储客户端或服务验收。参见 [OSS SDK 构造迁移](directory-structure_CN.md#oss-sdk-类型化构造)。 + 业务插件链接 `ark::mysql` 和/或 `ark::redis`，以获取公开头文件和编译要求；公共接口目标不会加载运行时动态库。将所需动态库与启动器一起部署，并在应用清单中填写库名称： ```toml diff --git a/docs/reference/directory-structure.md b/docs/reference/directory-structure.md index 56ac10a5c..bf3360351 100644 --- a/docs/reference/directory-structure.md +++ b/docs/reference/directory-structure.md @@ -152,6 +152,8 @@ cmake -S tests/consumer/etcd_client -B etcd-client-headers -G Ninja \ cmake --build etcd-client-headers --target check-etcd-client-headers --parallel 2 ``` +OSS SDK retains its three public headers and types through typed construction; rebuild the plugin and consumers together because public class layouts change. See [OSS SDK construction migration](directory-structure.md#typed-oss-sdk-construction) for optional build conditions and construction/export checks. The implementation remains a TODO scaffold. + Header mapping does not establish Consul, Kafka or OSS SDK runtime compatibility. Physic and `physics_scene` still require the unsupported legacy PhysX 3 integration and retain their explicit configuration failure; no passing PhysX build is claimed. @@ -196,3 +198,23 @@ cmake --build build --target check-etcd-client-plugin --parallel 1 This target includes `check-etcd-client-construction`: actual factory/assembly doctest cases, plus a controlled-factory DSO using the production entry/exit source to check configuration handoff, interface metadata, disabled updates, exactly-once ownership and repeated stop. A separate process loads the actual production plugin, checks its entry/exit exports and unloads it. `check-core` includes both checks when the runtime plugin is selected; disabled-plugin core builds do not acquire the JSON dependency. Applications CI explicitly enables the plugin and runs these checks through its existing `check-core` job. These tests do not run an Etcd service or exercise production HTTP/Etcd interoperability. They establish construction and DSO boundaries, not a completed Registry or arbitrary hot-update safety. + +## Typed OSS SDK construction + +**Breaking ABI/layout change:** `oss_sdk_module_impl` now inherits `module_base` and `oss_sdk_plugin` inherits `plugin_base`. Rebuild OSS SDK and consumers together. All three public headers, their `` paths and public types remain available through `ark::osssdk`; the `oss_sdk_module` interface is unchanged. The C++ plugin boundary does not promise compiler-independent ABI compatibility. + +The private `oss_sdk_factory.hpp` returns `std::unique_ptr` and is excluded from public mappings and installation. Explicit `DllEntryPlugin` / `DllExitPlugin` retain platform exports and logger initialization. Installation retains the original `oss_sdk_module` interface RTTI key and disabled updates. Typed unregistration replaces the undefined `ARK_DEREGISTER_MODULE` invocation and destroys the owned module. Configuration storage and the inherited default update method are preserved. The implementation remains a TODO scaffold: no OSS client, request API or object-storage protocol is added. + +With `ARK_BUILD_CORE_TESTS=ON`, `check-oss-sdk-construction` compiles the real factory and assembly into doctest cases and uses the production entry/exit source with a counted controlled factory in a DSO. Checks cover retained public bases/types, factory metadata, interface registration, duplicate rejection, disabled updates, configuration handoff, exactly-once destruction and repeated stop. This target requires only the normal core/arknet dependencies; it needs no runtime plugin, Protobuf, JSON package or object-storage service: + +```sh +cmake --build build --target check-oss-sdk-construction --parallel 1 +``` + +The runtime plugin remains off by default. Enable `ARK_BUILD_APPLICATIONS=ON`, `ARK_BUILD_PLUGIN_OSSSDK=ON`, `ARK_BUILD_PROTO=ON` and `ARK_BUILD_CORE_TESTS=ON` to additionally build the actual `ark_oss_sdk_plugin` DSO and inspect its entry/exit exports in a separate process: + +```sh +cmake --build build --target check-oss-sdk-plugin --parallel 1 +``` + +Application builds retain their existing Protobuf, configuration and Boost.Asio requirements. `check-core` includes the applicable OSS SDK construction/export checks. No external OSS SDK dependency is introduced. These tests establish construction and DSO boundaries only; they do not establish OSS client functionality, service interoperability or arbitrary hot-update safety. diff --git a/docs/reference/directory-structure_CN.md b/docs/reference/directory-structure_CN.md index 25d79a69f..27b49cf93 100644 --- a/docs/reference/directory-structure_CN.md +++ b/docs/reference/directory-structure_CN.md @@ -142,6 +142,8 @@ cmake -S tests/consumer/etcd_client -B etcd-client-headers -G Ninja \ cmake --build etcd-client-headers --target check-etcd-client-headers --parallel 2 ``` +OSS SDK 使用类型化构造，保留全部三个公开头和类型；公开类布局发生变化，插件与消费者必须一起重建。可选构建条件及构造／导出检查见 [OSS SDK 构造迁移](directory-structure_CN.md#oss-sdk-类型化构造)。实现仍为 TODO 骨架。 + 头文件映射检查不代表 Consul、Kafka、OSS SDK 的运行时兼容性已经验证。Physic 与 `physics_scene` 仍依赖未受支持的旧版 PhysX 3 集成，保留明确的配置失败；此处没有宣称 PhysX 构建通过。 ## 安装目录 @@ -186,3 +188,23 @@ cmake --build build --target check-etcd-client-plugin --parallel 1 该目标包含 `check-etcd-client-construction`：真实工厂／装配的 doctest 用例，以及使用生产入口／出口源码与受控工厂的 DSO 测试，覆盖配置传递、接口元数据、禁用 update、所有权恰好销毁一次和重复 stop。另一个独立进程加载真实生产插件，检查入口／出口导出后卸载。选中运行时插件时，`check-core` 包含这两项检查；关闭插件的核心构建不增加 JSON 依赖。应用 CI 显式启用该插件，通过现有 `check-core` 任务执行检查。 这些测试不启动 Etcd 服务，不执行生产 HTTP／Etcd 互通验收，只验证构造与 DSO 边界，不代表 Registry 已完成，也不证明任意热更新安全。 + +## OSS SDK 类型化构造 + +**ABI／布局不兼容变更：** `oss_sdk_module_impl` 现在继承 `module_base`，`oss_sdk_plugin` 继承 `plugin_base`。OSS SDK 与消费者必须一起重建。全部三个公开头、`` 路径和公开类型继续通过 `ark::osssdk` 提供，`oss_sdk_module` 接口不变。C++ 插件边界不承诺跨编译器的 ABI 兼容性。 + +私有 `oss_sdk_factory.hpp` 返回 `std::unique_ptr`，不映射、不安装。显式 `DllEntryPlugin`／`DllExitPlugin` 保留平台导出和日志初始化。安装时沿用 `oss_sdk_module` 接口 RTTI 键，继续禁用 update。类型化注销替代未定义的 `ARK_DEREGISTER_MODULE` 调用，销毁拥有的模块。配置存储和继承的默认 update 方法保持不变。实现仍为 TODO 骨架，不增加 OSS 客户端、请求 API 或对象存储协议。 + +开启 `ARK_BUILD_CORE_TESTS=ON` 后，`check-oss-sdk-construction` 将真实工厂与装配编入 doctest 用例，并使用生产入口／出口源码与带计数的受控工厂构建 DSO。检查覆盖保留的公开基类／类型、工厂元数据、接口注册、重复注册拒绝、禁用 update、配置传递、恰好销毁一次和重复 stop。该目标仅需要通常的 core／arknet 依赖，无需运行时插件、Protobuf、JSON 软件包或对象存储服务： + +```sh +cmake --build build --target check-oss-sdk-construction --parallel 1 +``` + +运行时插件默认保持关闭。开启 `ARK_BUILD_APPLICATIONS=ON`、`ARK_BUILD_PLUGIN_OSSSDK=ON`、`ARK_BUILD_PROTO=ON` 与 `ARK_BUILD_CORE_TESTS=ON` 后，还可构建真实 `ark_oss_sdk_plugin` DSO，并在独立进程中检查入口／出口导出： + +```sh +cmake --build build --target check-oss-sdk-plugin --parallel 1 +``` + +应用构建仍需要现有 Protobuf、配置模块与 Boost.Asio。`check-core` 包含当前条件下可用的 OSS SDK 构造／导出检查，不增加外部 OSS SDK 依赖。这些测试只验证构造与 DSO 边界，不代表 OSS 客户端功能、服务互通或任意热更新安全已经通过验收。 diff --git a/src/plugins/README.md b/src/plugins/README.md index bb41192b6..24ba933db 100755 --- a/src/plugins/README.md +++ b/src/plugins/README.md @@ -28,4 +28,6 @@ Kernel co-locates its 38 existing headers with its cpp files in `src/plugins/ker Consul retains its three public headers and implementation/plugin types while using typed construction, a private owning factory and explicit entry points. Rebuild Consul and consumers together because the public class layouts change. The single interface registration key, disabled updates and request behavior are preserved. See [Consul construction migration](../../docs/reference/directory-structure.md#typed-consul-construction) for build conditions and service-free construction checks. -The other four legacy plugin families retain 19 byte-identical headers. Mapping checks do not validate Consul, Kafka or OSS SDK runtime integrations. Physic retains the unsupported PhysX 3 configuration failure; see [optional legacy dependencies](../../docs/dependencies.md#optional-legacy-integrations). +Kafka and Physic retain their original public headers. Mapping checks do not validate Consul, Kafka or OSS SDK runtime integrations. Physic retains the unsupported PhysX 3 configuration failure; see [optional legacy dependencies](../../docs/dependencies.md#optional-legacy-integrations). + +OSS SDK retains all three public headers and types while using typed construction with a private factory and explicit entry points. Rebuild the plugin and consumers together for the public layout change. It remains off by default and retains its TODO scaffold; construction/export checks add no object-storage client capability. See [OSS SDK construction migration](../../docs/reference/directory-structure.md#typed-oss-sdk-construction). diff --git a/src/plugins/README_CN.md b/src/plugins/README_CN.md index bed2c6eb9..97e8ded49 100644 --- a/src/plugins/README_CN.md +++ b/src/plugins/README_CN.md @@ -28,4 +28,6 @@ Kernel 的原有 38 个头与 cpp 共置于 `src/plugins/kernel/`，通过 `ark: Consul 保留原有三个公开头及实现／插件类型，同时使用类型化构造、私有拥有型工厂和显式入口。公开类布局发生变化，Consul 与消费者必须一起重建。唯一接口注册键、禁用 update 与请求行为保持不变。构建条件及无需服务的构造检查见 [Consul 构造迁移](../../docs/reference/directory-structure_CN.md#consul-类型化构造)。 -其他四类旧插件保留原有 19 个头的全部字节。映射检查不代表 Consul、Kafka、OSS SDK 的运行时集成已经验证。Physic 保留未受支持的 PhysX 3 配置失败；参见[可选旧集成依赖](../../docs/dependencies_CN.md)。 +Kafka 与 Physic 保留原有公开头。映射检查不代表 Consul、Kafka、OSS SDK 的运行时集成已经验证。Physic 保留未受支持的 PhysX 3 配置失败；参见[可选旧集成依赖](../../docs/dependencies_CN.md)。 + +OSS SDK 使用类型化构造、私有工厂与显式入口，保留全部三个公开头和类型。公开布局发生变化，插件与消费者必须一起重建。插件默认保持关闭，实现仍为 TODO 骨架；构造／导出检查不增加对象存储客户端能力。参见 [OSS SDK 构造迁移](../../docs/reference/directory-structure_CN.md#oss-sdk-类型化构造)。 diff --git a/src/plugins/osssdk/oss_sdk_factory.hpp b/src/plugins/osssdk/oss_sdk_factory.hpp new file mode 100644 index 000000000..c4c6fba0a --- /dev/null +++ b/src/plugins/osssdk/oss_sdk_factory.hpp @@ -0,0 +1,13 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#pragma once + +#include + +#include "ark/osssdk/oss_sdk_module.hpp" + +// Plugin-private construction; deliberately absent from public header mappings. +namespace ark::detail { + +std::unique_ptr make_oss_sdk_module(); + +} // namespace ark::detail diff --git a/src/plugins/osssdk/oss_sdk_module_impl.cpp b/src/plugins/osssdk/oss_sdk_module_impl.cpp index 061bfa3a8..c5619a994 100644 --- a/src/plugins/osssdk/oss_sdk_module_impl.cpp +++ b/src/plugins/osssdk/oss_sdk_module_impl.cpp @@ -19,9 +19,12 @@ */ #include "ark/osssdk/oss_sdk_module_impl.hpp" +#include "oss_sdk_factory.hpp" namespace ark { -//method realization +std::unique_ptr detail::make_oss_sdk_module() { + return std::make_unique(); +} } // namespace ark diff --git a/src/plugins/osssdk/oss_sdk_module_impl.hpp b/src/plugins/osssdk/oss_sdk_module_impl.hpp index fad7a70e5..734e3b0d3 100644 --- a/src/plugins/osssdk/oss_sdk_module_impl.hpp +++ b/src/plugins/osssdk/oss_sdk_module_impl.hpp @@ -20,14 +20,13 @@ #pragma once +#include "ark/module_base.hpp" #include "ark/osssdk/oss_sdk_module.hpp" -#include "ark/plugin_manager.hpp" namespace ark { -class oss_sdk_module_impl final : public oss_sdk_module +class oss_sdk_module_impl final : public module_base { - ARK_DECLARE_MODULE_FUNCTIONS public: // TODO diff --git a/src/plugins/osssdk/oss_sdk_plugin.cpp b/src/plugins/osssdk/oss_sdk_plugin.cpp index df8d665b8..d20d4321e 100644 --- a/src/plugins/osssdk/oss_sdk_plugin.cpp +++ b/src/plugins/osssdk/oss_sdk_plugin.cpp @@ -18,21 +18,34 @@ * */ -#include "ark/osssdk/oss_sdk_module_impl.hpp" +#include + #include "ark/osssdk/oss_sdk_plugin.hpp" +#include "oss_sdk_factory.hpp" namespace ark { -ARK_DECLARE_PLUGIN_DLL_FUNCTION(oss_sdk_plugin) +ARK_EXPORT_FUNC void DllEntryPlugin(plugin_manager* manager, const std::string& name, logger* log) { + logger::init(log); + manager->register_plugin(name); +} + +ARK_EXPORT_FUNC void DllExitPlugin(plugin_manager* manager) { + manager->unregister_plugin(); +} void oss_sdk_plugin::install() { - // install module - ARK_REGISTER_MODULE(oss_sdk_module, oss_sdk_module_impl); + if (!register_module(detail::make_oss_sdk_module(), module_update::disabled)) + { + throw std::runtime_error("OSS SDK module registration rejected"); + } } void oss_sdk_plugin::uninstall() { - // uninstall module - ARK_DEREGISTER_MODULE(oss_sdk_module, oss_sdk_module_impl); + if (!unregister_module()) + { + throw std::runtime_error("OSS SDK module unregistration rejected"); + } } } // namespace ark diff --git a/src/plugins/osssdk/oss_sdk_plugin.hpp b/src/plugins/osssdk/oss_sdk_plugin.hpp index 500bf3167..349ad0c5a 100644 --- a/src/plugins/osssdk/oss_sdk_plugin.hpp +++ b/src/plugins/osssdk/oss_sdk_plugin.hpp @@ -20,11 +20,15 @@ #pragma once -#include "ark/plugin.hpp" -#include "ark/plugin_manager.hpp" +#include "ark/plugin_base.hpp" namespace ark { -ARK_DECLARE_PLUGIN(oss_sdk_plugin) +class oss_sdk_plugin final : public plugin_base +{ +public: + void install() override; + void uninstall() override; +}; } // namespace ark \ No newline at end of file diff --git a/tests/core/CMakeLists.txt b/tests/core/CMakeLists.txt index 927446e32..3315fdc4d 100644 --- a/tests/core/CMakeLists.txt +++ b/tests/core/CMakeLists.txt @@ -224,6 +224,8 @@ if(ARK_BUILD_PROTO) add_dependencies(check-core check-kernel-initialization) endif() +add_subdirectory(../osssdk "${CMAKE_CURRENT_BINARY_DIR}/osssdk") + if(ARK_BUILD_PROTO AND ARK_BUILD_CONFIG) add_subdirectory(../consul "${CMAKE_CURRENT_BINARY_DIR}/consul") endif() diff --git a/tests/osssdk/CMakeLists.txt b/tests/osssdk/CMakeLists.txt new file mode 100644 index 000000000..cbf91e03f --- /dev/null +++ b/tests/osssdk/CMakeLists.txt @@ -0,0 +1,36 @@ +# Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +set(oss_sdk_source "${PROJECT_SOURCE_DIR}/src/plugins/osssdk") + +# Compile real assembly/factory locally too: private C++ symbols are not DLL exports. +add_executable(ark_oss_sdk_construction_tests ../core/main.cpp construction.cpp + "${oss_sdk_source}/oss_sdk_plugin.cpp" "${oss_sdk_source}/oss_sdk_module_impl.cpp") +set_property(TARGET ark_oss_sdk_construction_tests PROPERTY LINK_LIBRARIES + ark::osssdk) +target_include_directories(ark_oss_sdk_construction_tests PRIVATE "${oss_sdk_source}") + +# A controlled module exercises the actual entry/exit functions without object-storage services. +add_library(ark_oss_sdk_entry_probe SHARED entry_factory.cpp "${oss_sdk_source}/oss_sdk_plugin.cpp") +set_property(TARGET ark_oss_sdk_entry_probe PROPERTY LINK_LIBRARIES ark::osssdk) +target_include_directories(ark_oss_sdk_entry_probe PRIVATE "${oss_sdk_source}") +set_target_properties(ark_oss_sdk_entry_probe PROPERTIES PREFIX "") +add_executable(ark_oss_sdk_entry_tests entry.cpp) +set_property(TARGET ark_oss_sdk_entry_tests PROPERTY LINK_LIBRARIES ark::osssdk) + +add_custom_target(check-oss-sdk-construction + COMMAND $ + COMMAND $ "$/" "ark_oss_sdk_entry_probe" + DEPENDS ark_oss_sdk_construction_tests ark_oss_sdk_entry_tests ark_oss_sdk_entry_probe + USES_TERMINAL VERBATIM) + +if(TARGET ark_oss_sdk_plugin) + # Keep real DSO export inspection separate from the host linking static AFProto. + add_executable(ark_oss_sdk_exports_tests exports.cpp) + set_property(TARGET ark_oss_sdk_exports_tests PROPERTY LINK_LIBRARIES ark::core) + add_custom_target(check-oss-sdk-plugin + COMMAND $ "$/" "ark_oss_sdk_plugin" + DEPENDS ark_oss_sdk_exports_tests ark_oss_sdk_plugin check-oss-sdk-construction + USES_TERMINAL VERBATIM) + add_dependencies(check-core check-oss-sdk-plugin) +else() + add_dependencies(check-core check-oss-sdk-construction) +endif() diff --git a/tests/osssdk/construction.cpp b/tests/osssdk/construction.cpp new file mode 100644 index 000000000..66f268649 --- /dev/null +++ b/tests/osssdk/construction.cpp @@ -0,0 +1,55 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include +#include +#include + +#include "../network/doctest.h" + +#include +#include +#include "oss_sdk_factory.hpp" + +#if __has_include() +#error "OSS SDK public mappings must exclude the private factory" +#endif + +static_assert(std::is_base_of_v, ark::oss_sdk_module_impl>); +static_assert(std::is_base_of_v, ark::oss_sdk_plugin>); +static_assert(std::is_same_v); +static_assert(std::is_same_v>); + +TEST_CASE("OSS SDK real factory retains its public implementation and unpublished metadata") { + auto candidate = ark::detail::make_oss_sdk_module(); + REQUIRE(candidate != nullptr); + const auto* instance = candidate.get(); + CHECK(typeid(*instance) == typeid(ark::oss_sdk_module_impl)); + CHECK(candidate->get_plugin_manager() == nullptr); + CHECK(candidate->get_name().empty()); +} + +TEST_CASE("OSS SDK assembly retains the interface key, disabled updates and explicit ownership") { + ark::plugin_manager manager; + ark::oss_sdk_plugin owner; + owner.set_plugin_manager(&manager); + CHECK(owner.get_plugin_version() == 0); + CHECK(owner.get_plugin_name() == typeid(ark::oss_sdk_plugin).name()); + owner.set_plugin_conf("oss-sdk-construction.toml"); + CHECK(ark::oss_sdk_plugin::get_plugin_conf() == "oss-sdk-construction.toml"); + owner.install(); + auto* instance = manager.find_module(); + REQUIRE(instance != nullptr); + CHECK(typeid(*instance) == typeid(ark::oss_sdk_module_impl)); + CHECK(instance->get_plugin_manager() == &manager); + CHECK(instance->get_name() == typeid(ark::oss_sdk_module).name()); + // Insertion succeeds only when assembly left the module out of the update set. + CHECK(manager.add_update_module(instance)); + manager.remove_update_module(instance->get_name()); + CHECK_THROWS_AS(owner.install(), std::runtime_error); + CHECK(manager.find_module() == instance); + owner.uninstall(); + // Reinstallation proves the public registration key and ownership were released. + CHECK_NOTHROW(owner.install()); + CHECK_NOTHROW(owner.uninstall()); + CHECK(manager.update(1)); + CHECK_THROWS_AS(owner.uninstall(), std::runtime_error); +} diff --git a/tests/osssdk/entry.cpp b/tests/osssdk/entry.cpp new file mode 100644 index 000000000..d32165f9a --- /dev/null +++ b/tests/osssdk/entry.cpp @@ -0,0 +1,45 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include +#include +#include + +#include +#include +#include + +namespace { + +void require(bool condition, const char* message) { + if (!condition) + { + std::cerr << message << std::endl; + std::_Exit(1); + } +} + +} // namespace + +int main(int argc, char* argv[]) { + require(argc == 3, "Expected OSS SDK entry-probe directory and library stem"); + ark::logger log; + ark::logger::init(&log); + log.set_log_func([](spdlog::level::level_enum, const std::string&) {}); + ark::dynamic_library library(argv[2]); + require(library.load(argv[1]), "Cannot load OSS SDK entry-probe library"); + auto events = reinterpret_cast(library.get_symbol("oss_sdk_entry_events")); + require(events && events(0) == 0 && events(1) == 0, "Entry-probe state is not fresh"); + require(library.get_symbol("DllEntryPlugin") && library.get_symbol("DllExitPlugin"), + "OSS SDK explicit entry/exit exports are missing"); + ark::plugin_manager manager; + require(manager.start(0, argv[1], "conf/", {{argv[2], "entry-probe.toml"}}), "OSS SDK explicit entry failed"); + auto* instance = manager.find_module(); + require(instance && instance->get_plugin_manager() == &manager && + instance->get_name() == typeid(ark::oss_sdk_module).name(), + "OSS SDK explicit entry lost interface registration metadata"); + require(events(0) == 1 && events(1) == 0, "OSS SDK entry did not transfer factory ownership"); + require(manager.update(1) && events(2) == 0, "OSS SDK entry changed disabled update selection"); + require(manager.stop(2) && events(1) == 1, "OSS SDK explicit exit did not destroy its module exactly once"); + require(manager.stop(3) && events(1) == 1, "Repeated stop invoked OSS SDK exit twice"); + require(library.unload(), "Cannot release OSS SDK observation handle"); + std::cout << "OSS SDK real assembly entry/exit, configuration, interface metadata and ownership passed\n"; +} diff --git a/tests/osssdk/entry_factory.cpp b/tests/osssdk/entry_factory.cpp new file mode 100644 index 000000000..b51dad2e7 --- /dev/null +++ b/tests/osssdk/entry_factory.cpp @@ -0,0 +1,55 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include + +#include +#include +#include "oss_sdk_factory.hpp" + +namespace { + +unsigned created{}; +unsigned destroyed{}; +unsigned updated{}; + +class entry_oss_sdk final : public ark::module_base +{ +public: + entry_oss_sdk() { + ++created; + } + + ~entry_oss_sdk() override { + ++destroyed; + } + + bool init() override { + return ark::oss_sdk_plugin::get_plugin_conf() == "conf/entry-probe.toml"; + } + + bool update() override { + ++updated; + return true; + } +}; + +} // namespace + +namespace ark::detail { + +std::unique_ptr make_oss_sdk_module() { + return std::make_unique(); +} + +} // namespace ark::detail + +ARK_EXPORT_FUNC unsigned oss_sdk_entry_events(unsigned event) { + switch (event) + { + case 0: + return created; + case 1: + return destroyed; + default: + return updated; + } +} diff --git a/tests/osssdk/exports.cpp b/tests/osssdk/exports.cpp new file mode 100644 index 000000000..118770b0b --- /dev/null +++ b/tests/osssdk/exports.cpp @@ -0,0 +1,31 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include +#include +#include + +#include + +namespace { + +void require(bool condition, const char* message) { + if (!condition) + { + std::cerr << message << std::endl; + std::_Exit(1); + } +} + +} // namespace + +int main(int argc, char* argv[]) { + require(argc == 3, "Expected plugin directory and library stem"); + ark::logger log; + ark::logger::init(&log); + log.set_log_func([](spdlog::level::level_enum, const std::string&) {}); + ark::dynamic_library library(argv[2]); + require(library.load(argv[1]), "Cannot load actual OSS SDK shared library"); + require(library.get_symbol("DllEntryPlugin") != nullptr, "Missing explicit DllEntryPlugin export"); + require(library.get_symbol("DllExitPlugin") != nullptr, "Missing explicit DllExitPlugin export"); + require(library.unload(), "Cannot unload OSS SDK shared library"); + std::cout << argv[2] << ": actual DSO load, explicit entry/exit exports and unload passed\n"; +} From 663a4bd5561791d2f2538f00711c7c1ad11181d4 Mon Sep 17 00:00:00 2001 From: nick Date: Mon, 26 Oct 2026 13:55:41 +0800 Subject: [PATCH 14/22] refactor(kafka)!: replace plugin creation macros Use typed module/plugin bases, a private interface-returning factory and explicit DLL entries while preserving enabled updates and RdKafka callback inheritance. Add real factory/assembly, controlled lifecycle and production export checks. Cast callback log enums to integers for current fmt compatibility. BREAKING CHANGE: Kafka implementation and plugin public class layouts change. Rebuild Kafka and its C++ consumers together; retained public headers, interface keys and producer behavior remain available. --- docs/guides/plugins.md | 2 + docs/guides/plugins_CN.md | 2 + docs/reference/build-options.md | 4 +- docs/reference/build-options_CN.md | 4 +- docs/reference/directory-structure.md | 18 +++++++ docs/reference/directory-structure_CN.md | 18 +++++++ src/plugins/kafka/kafka_factory.hpp | 13 +++++ src/plugins/kafka/kafka_module_impl.cpp | 14 ++++-- src/plugins/kafka/kafka_module_impl.hpp | 7 ++- src/plugins/kafka/kafka_plugin.cpp | 23 +++++++-- src/plugins/kafka/kafka_plugin.hpp | 12 +++-- tests/ci/install-package.test.mjs | 4 +- tests/ci/package-dependencies.test.mjs | 15 ++++++ tests/core/CMakeLists.txt | 5 ++ tests/core/README.md | 10 ++++ tests/core/README_CN.md | 8 +++ tests/kafka/CMakeLists.txt | 35 +++++++++++++ tests/kafka/construction.cpp | 61 +++++++++++++++++++++++ tests/kafka/entry.cpp | 45 +++++++++++++++++ tests/kafka/entry_factory.cpp | 63 ++++++++++++++++++++++++ tests/kafka/exports.cpp | 31 ++++++++++++ 21 files changed, 376 insertions(+), 18 deletions(-) create mode 100644 src/plugins/kafka/kafka_factory.hpp create mode 100644 tests/kafka/CMakeLists.txt create mode 100644 tests/kafka/construction.cpp create mode 100644 tests/kafka/entry.cpp create mode 100644 tests/kafka/entry_factory.cpp create mode 100644 tests/kafka/exports.cpp diff --git a/docs/guides/plugins.md b/docs/guides/plugins.md index 7afc46cf7..9a5154bd3 100644 --- a/docs/guides/plugins.md +++ b/docs/guides/plugins.md @@ -215,3 +215,5 @@ Kernel also uses typed construction while retaining its public `*_impl` and `ker Consul follows the same construction pattern while retaining all three public headers and types. Its private factory is not installed; the module keeps its interface RTTI key and disabled updates. Rebuild Consul and consumers together because its public implementation/plugin layouts change. See [Consul construction migration](../reference/directory-structure.md#typed-consul-construction) for test entry points and dependency conditions. Etcd Client also retains all five public headers and types while using typed bases, a private interface-returning factory and explicit entry/exit functions. Its interface RTTI key and disabled updates are unchanged. Rebuild the plugin and consumers together because the public implementation/plugin layouts change. The [Etcd Client construction migration](../reference/directory-structure.md#typed-etcd-client-construction) describes optional JSON dependencies and tests without an Etcd service. + +Kafka retains all four public headers and both RdKafka callback bases while adopting typed construction and explicit entries. Its interface key and enabled frame updates remain unchanged, and the runtime plugin stays off by default. Rebuild Kafka and consumers together after the public layout change. See [Kafka construction migration](../reference/directory-structure.md#typed-kafka-construction) for driver requirements and tests without a broker connection. diff --git a/docs/guides/plugins_CN.md b/docs/guides/plugins_CN.md index ef5d6defb..3a8dc44f9 100644 --- a/docs/guides/plugins_CN.md +++ b/docs/guides/plugins_CN.md @@ -202,3 +202,5 @@ Kernel 同样使用类型化构造，并保留公开 `*_impl`、`kernel_plugin` Consul 使用相同构造方式，保留全部三个公开头与类型。私有工厂不安装；模块保留接口 RTTI 键和禁用 update 的选择。公开实现／插件布局发生变化，Consul 与消费者必须一起重建。测试入口与依赖条件见 [Consul 构造迁移](../reference/directory-structure_CN.md#consul-类型化构造)。 Etcd Client 使用类型化基类、返回接口的私有工厂与显式入口／出口，保留全部五个公开头与类型。接口 RTTI 键和禁用 update 的选择不变。公开实现／插件布局发生变化，插件与消费者必须一起重建。可选 JSON 依赖与无需 Etcd 服务的测试见 [Etcd Client 构造迁移](../reference/directory-structure_CN.md#etcd-client-类型化构造)。 + +Kafka 使用类型化构造与显式入口，保留原有四个公开头和两个 RdKafka 回调基类。接口键与启用逐帧 update 的选择不变，运行时插件继续默认关闭。公开布局发生变化，Kafka 与消费者必须一起重建。驱动要求及无需连接 broker 的测试见 [Kafka 构造迁移](../reference/directory-structure_CN.md#kafka-类型化构造)。 diff --git a/docs/reference/build-options.md b/docs/reference/build-options.md index 1825ad741..08bbbc6c5 100644 --- a/docs/reference/build-options.md +++ b/docs/reference/build-options.md @@ -39,7 +39,7 @@ cmake -S . -B build -DARK_BUILD_APPLICATIONS=ON \ cmake --build build --target ark_mysql_plugin ark_redis_plugin ``` -All official plugin public headers are copied from explicit lists at configuration time into each build tree's generated plugin header mapping. Each `ark::` API target carries that include root even with runtime plugins disabled. MySQL, Redis, Utility, Net, Bus and HTTP exclude private assembly and factories. Kernel retains all 38 existing headers, including `kernel_plugin.hpp`; Consul, Etcd Client, Kafka, OSS SDK and Physic retain all 22 existing public headers, including implementation and assembly headers. Registration keys, public type visibility and optional plugin policies are retained; [Kernel typed construction](kernel.md#typed-kernel-construction), [Consul typed construction](directory-structure.md#typed-consul-construction) and [Etcd Client typed construction](directory-structure.md#typed-etcd-client-construction) change public class layouts and require rebuilding consumers. Their private factory headers are excluded from mapping and installation; header mapping does not validate optional driver integrations. Each public header has one co-located source; its generated include mapping belongs to the selected build tree, rather than a second source layout. +All official plugin public headers are copied from explicit lists at configuration time into each build tree's generated plugin header mapping. Each `ark::` API target carries that include root even with runtime plugins disabled. MySQL, Redis, Utility, Net, Bus and HTTP exclude private assembly and factories. Kernel retains all 38 existing headers, including `kernel_plugin.hpp`; Consul, Etcd Client, Kafka, OSS SDK and Physic retain all 22 existing public headers, including implementation and assembly headers. Registration keys, public type visibility and optional plugin policies are retained; [Kernel typed construction](kernel.md#typed-kernel-construction), [Consul typed construction](directory-structure.md#typed-consul-construction), [Etcd Client typed construction](directory-structure.md#typed-etcd-client-construction) and [Kafka typed construction](directory-structure.md#typed-kafka-construction) change public class layouts and require rebuilding consumers. Their private factory headers are excluded from mapping and installation; header mapping does not validate optional driver integrations. Each public header has one co-located source; its generated include mapping belongs to the selected build tree, rather than a second source layout. Applications CI explicitly enables Consul and Etcd Client on Linux, macOS and Windows. Its `check-core` run includes their construction, DLL lifecycle and production export checks; these checks require no Consul or Etcd service and do not establish service interoperability. Etcd Client keeps its existing default `ON` selection and can be disabled with `ARK_BUILD_PLUGIN_ETCD_CLIENT=OFF`. Building it requires the existing `nlohmann_json` package (`etcd-client` vcpkg feature); disabling it does not add that JSON dependency to core tests. @@ -68,3 +68,5 @@ Core headers live under `src/core/`; `ark::core` supplies public header usage re `ARK_INSTALL` defaults to `ON` for the top-level project and `OFF` for embedded builds. It enables explicit public-header installation, relocatable CMake exports, `check-install` and TGZ/ZIP CPack archives. `ARK_USE_INSTALLED_ARKNET=ON` resolves a separately installed arknet package and takes precedence over the source/revision overrides. It defaults to `OFF`, preserving source dependency builds. See [installation and package consumption](../get-started/installation.md#install-and-consume-the-cmake-package). Packaging does not enable optional runtime plugins or validate their external drivers. Net's 18 public headers exclude `http_client_module_impl.hpp`; use `` and public interface lookup. This removal changes source/ABI compatibility; see [Net migration](network.md#api-and-private-construction). Private factory declarations are not installed. + +Kafka remains off by default (`ARK_BUILD_PLUGIN_KAFKA=OFF`). With the existing driver dependencies, applications, Protobuf and core tests enabled, `ARK_BUILD_PLUGIN_KAFKA=ON` adds `check-kafka-construction` and `check-kafka-plugin` to `check-core`. The first exercises actual factory/assembly plus controlled DSO lifecycle and enabled updates; the second also checks production DSO exports. No broker is contacted. Disabling Kafka avoids Kafka dependency discovery in core tests. See [Kafka construction migration](directory-structure.md#typed-kafka-construction) for rebuild requirements and verification boundaries. diff --git a/docs/reference/build-options_CN.md b/docs/reference/build-options_CN.md index d9d1cd082..1e287667c 100644 --- a/docs/reference/build-options_CN.md +++ b/docs/reference/build-options_CN.md @@ -39,7 +39,7 @@ cmake -S . -B build -DARK_BUILD_APPLICATIONS=ON \ cmake --build build --target ark_mysql_plugin ark_redis_plugin ``` -全部官方插件的公开头通过显式清单在配置阶段复制到各构建树生成的插件公开头映射。各 `ark::` API target 传递该目录，插件关闭时仍可使用。MySQL、Redis、Utility、Net、Bus、HTTP 的私有装配头和工厂不进入映射；Kernel 保留包括 `kernel_plugin.hpp` 在内的原有 38 个头；Consul、Etcd Client、Kafka、OSS SDK、Physic 保留包括实现头与装配头在内的原有 22 个公开头。注册键、公开类型可见性和可选插件策略继续保留；[Kernel 类型化构造](kernel_CN.md#kernel-类型化构造)、[Consul 类型化构造](directory-structure_CN.md#consul-类型化构造)与 [Etcd Client 类型化构造](directory-structure_CN.md#etcd-client-类型化构造)改变公开类布局，需要重建消费者。这些私有工厂头均不映射、不安装；头文件映射不代表可选驱动集成已经验证。每个公共头仅有一份共置源码；生成的头文件映射属于选定构建树，不构成第二棵源码目录。 +全部官方插件的公开头通过显式清单在配置阶段复制到各构建树生成的插件公开头映射。各 `ark::` API target 传递该目录，插件关闭时仍可使用。MySQL、Redis、Utility、Net、Bus、HTTP 的私有装配头和工厂不进入映射；Kernel 保留包括 `kernel_plugin.hpp` 在内的原有 38 个头；Consul、Etcd Client、Kafka、OSS SDK、Physic 保留包括实现头与装配头在内的原有 22 个公开头。注册键、公开类型可见性和可选插件策略继续保留；[Kernel 类型化构造](kernel_CN.md#kernel-类型化构造)、[Consul 类型化构造](directory-structure_CN.md#consul-类型化构造)、[Etcd Client 类型化构造](directory-structure_CN.md#etcd-client-类型化构造)与 [Kafka 类型化构造](directory-structure_CN.md#kafka-类型化构造)改变公开类布局，需要重建消费者。这些私有工厂头均不映射、不安装；头文件映射不代表可选驱动集成已经验证。每个公共头仅有一份共置源码；生成的头文件映射属于选定构建树，不构成第二棵源码目录。 应用 CI 在 Linux、macOS、Windows 显式启用 Consul 与 Etcd Client，`check-core` 包含两者的构造、动态库生命周期与生产库导出检查。这些检查不需要 Consul 或 Etcd 服务，也不代表服务互通验收。Etcd Client 保留现有默认 `ON` 选择，可通过 `ARK_BUILD_PLUGIN_ETCD_CLIENT=OFF` 关闭。构建它需要现有 `nlohmann_json` 软件包（vcpkg 的 `etcd-client` feature）；关闭它不会向核心测试增加该 JSON 依赖。 @@ -68,3 +68,5 @@ config = "" `ARK_INSTALL` 在顶层项目中默认 `ON`，嵌入构建时默认 `OFF`；启用显式公开头安装、可迁移 CMake 导出、`check-install` 与 TGZ／ZIP CPack 归档。`ARK_USE_INSTALLED_ARKNET=ON` 发现单独安装的 arknet 包，优先于源码与版本覆盖；默认 `OFF`，保留源码依赖构建。具体操作见[安装与包消费](../get-started/installation_CN.md#安装并消费-cmake-包)。打包不会启用可选运行时插件，也不代表外部驱动已经验收。 Net 的 18 个公开头不再包含 `http_client_module_impl.hpp`；使用 `` 并查找公开接口。此移除属于源码／ABI 兼容性变化，详见 [Net 迁移](network_CN.md#api-与私有创建)。私有工厂声明不安装。 + +Kafka 继续默认关闭（`ARK_BUILD_PLUGIN_KAFKA=OFF`）。准备原有驱动依赖并启用应用、Protobuf 和核心测试后，`ARK_BUILD_PLUGIN_KAFKA=ON` 会将 `check-kafka-construction`、`check-kafka-plugin` 加入 `check-core`。前者验证真实工厂／装配、受控动态库生命周期及启用 update；后者另检查生产动态库导出。测试不连接 broker。关闭 Kafka 时，核心测试不发现 Kafka 依赖。重建要求与验收边界见 [Kafka 构造迁移](directory-structure_CN.md#kafka-类型化构造)。 diff --git a/docs/reference/directory-structure.md b/docs/reference/directory-structure.md index 56ac10a5c..219e969ce 100644 --- a/docs/reference/directory-structure.md +++ b/docs/reference/directory-structure.md @@ -196,3 +196,21 @@ cmake --build build --target check-etcd-client-plugin --parallel 1 This target includes `check-etcd-client-construction`: actual factory/assembly doctest cases, plus a controlled-factory DSO using the production entry/exit source to check configuration handoff, interface metadata, disabled updates, exactly-once ownership and repeated stop. A separate process loads the actual production plugin, checks its entry/exit exports and unloads it. `check-core` includes both checks when the runtime plugin is selected; disabled-plugin core builds do not acquire the JSON dependency. Applications CI explicitly enables the plugin and runs these checks through its existing `check-core` job. These tests do not run an Etcd service or exercise production HTTP/Etcd interoperability. They establish construction and DSO boundaries, not a completed Registry or arbitrary hot-update safety. + +## Typed Kafka construction + +**Breaking ABI/layout change:** `kafka_module_impl` now inherits `module_base` and `kafka_plugin` inherits `plugin_base`. Rebuild Kafka and consumers together. All four public headers, their `` paths and public types remain available through `ark::kafka`. The module interface, producer/consumer wrappers and both `RdKafka::DeliveryReportCb` / `RdKafka::EventCb` bases are retained. C++ plugin boundaries do not promise compiler-independent ABI compatibility. + +The private `kafka_factory.hpp` returns `std::unique_ptr` and is excluded from public mappings and installation. Explicit `DllEntryPlugin` / `DllExitPlugin` retain platform exports and logger setup. Assembly retains the interface RTTI registration key and explicitly selects `module_update::enabled`, preserving periodic producer polling. Unregistration destroys the owned module. Producer storage and ownership, producer creation, delivery/event callbacks and message behavior are unchanged. + +Kafka remains optional and **off by default**. Its existing dependency discovery and linking are unchanged: enable `ARK_BUILD_APPLICATIONS=ON`, `ARK_BUILD_PROTO=ON`, `ARK_BUILD_CORE_TESTS=ON` and `ARK_BUILD_PLUGIN_KAFKA=ON`, with the platform's Kafka driver dependencies available. On Unix this includes librdkafka C/C++, OpenSSL, Zstandard, LZ4 and zlib. With that build configured, run: + +```sh +cmake --build build --target check-kafka-plugin --parallel 1 +``` + +`check-kafka-plugin` includes `check-kafka-construction`: actual factory/assembly doctest cases and a controlled-factory DSO using the production entry/exit source. They check typed bases, interface metadata, duplicate registration, enabled updates, configuration handoff, exactly-once ownership and repeated stop. A separate process loads the actual production DSO, checks its entry/exit exports and unloads it. `check-core` includes these checks only when the Kafka runtime target is selected; disabled-plugin core builds do not discover Kafka dependencies. + +Callback logs explicitly convert RdKafka error, severity and event-type enums to integers for current fmt compatibility, retaining their numeric log values. This does not change producer behavior, callback ordering or dependency discovery. + +These checks create no producer and require no broker connection. They establish construction and DSO boundaries, not Kafka delivery, flush behavior, service interoperability or arbitrary hot-update safety. The optional driver build still requires independent platform acceptance. diff --git a/docs/reference/directory-structure_CN.md b/docs/reference/directory-structure_CN.md index 25d79a69f..e5b6eeedf 100644 --- a/docs/reference/directory-structure_CN.md +++ b/docs/reference/directory-structure_CN.md @@ -186,3 +186,21 @@ cmake --build build --target check-etcd-client-plugin --parallel 1 该目标包含 `check-etcd-client-construction`：真实工厂／装配的 doctest 用例，以及使用生产入口／出口源码与受控工厂的 DSO 测试，覆盖配置传递、接口元数据、禁用 update、所有权恰好销毁一次和重复 stop。另一个独立进程加载真实生产插件，检查入口／出口导出后卸载。选中运行时插件时，`check-core` 包含这两项检查；关闭插件的核心构建不增加 JSON 依赖。应用 CI 显式启用该插件，通过现有 `check-core` 任务执行检查。 这些测试不启动 Etcd 服务，不执行生产 HTTP／Etcd 互通验收，只验证构造与 DSO 边界，不代表 Registry 已完成，也不证明任意热更新安全。 + +## Kafka 类型化构造 + +**ABI／布局不兼容变更：**`kafka_module_impl` 改为继承 `module_base`，`kafka_plugin` 改为继承 `plugin_base`。Kafka 与消费者必须一起重建。原有四个公开头、`` 路径和公开类型继续通过 `ark::kafka` 提供。模块接口、生产者／消费者封装及 `RdKafka::DeliveryReportCb`、`RdKafka::EventCb` 两个基类均保留。C++ 插件边界不承诺跨编译器 ABI 兼容。 + +私有 `kafka_factory.hpp` 返回 `std::unique_ptr`，不进入公开映射或安装。显式 `DllEntryPlugin`／`DllExitPlugin` 保留平台导出与日志初始化。装配保留接口 RTTI 注册键，并显式选择 `module_update::enabled`，继续定期轮询生产者。注销会销毁所属模块。生产者存储与所有权、生产者创建、投递／事件回调及消息行为均不变。 + +Kafka 继续作为可选插件，**默认关闭**。原有依赖发现与链接方式不变：启用 `ARK_BUILD_APPLICATIONS=ON`、`ARK_BUILD_PROTO=ON`、`ARK_BUILD_CORE_TESTS=ON`、`ARK_BUILD_PLUGIN_KAFKA=ON`，并准备对应平台的 Kafka 驱动依赖。Unix 平台包括 librdkafka C／C++、OpenSSL、Zstandard、LZ4 和 zlib。完成配置后运行： + +```sh +cmake --build build --target check-kafka-plugin --parallel 1 +``` + +`check-kafka-plugin` 包含 `check-kafka-construction`：真实工厂／装配 doctest，以及使用生产入口／出口源码、替换私有工厂的受控动态库。检查覆盖类型化基类、接口元数据、重复注册、启用 update、配置传递、仅销毁一次的所有权与重复停机。独立进程加载真实生产动态库，检查入口／出口导出后卸载。仅选择 Kafka 运行时目标时，`check-core` 才纳入这些检查；关闭插件的核心测试不发现 Kafka 依赖。 + +回调日志将 RdKafka 错误码、严重等级和事件类型枚举显式转换为整数，以兼容当前 fmt，并保留原有数值信息。生产者行为、回调顺序与依赖发现方式均不变。 + +这些检查不创建生产者，也不连接 broker。它们验证构造与动态库边界，不代表 Kafka 消息投递、flush 行为、服务互通或任意热更新安全性。可选驱动构建仍需单独进行各平台验收。 diff --git a/src/plugins/kafka/kafka_factory.hpp b/src/plugins/kafka/kafka_factory.hpp new file mode 100644 index 000000000..c9f6118ed --- /dev/null +++ b/src/plugins/kafka/kafka_factory.hpp @@ -0,0 +1,13 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#pragma once + +#include + +#include "ark/kafka/kafka_module.hpp" + +// Plugin-private construction; deliberately absent from public header mappings. +namespace ark::detail { + +std::unique_ptr make_kafka_module(); + +} // namespace ark::detail diff --git a/src/plugins/kafka/kafka_module_impl.cpp b/src/plugins/kafka/kafka_module_impl.cpp index 9035e5b28..ee2e13227 100644 --- a/src/plugins/kafka/kafka_module_impl.cpp +++ b/src/plugins/kafka/kafka_module_impl.cpp @@ -1,9 +1,14 @@ #include "ark/kafka/kafka_module_impl.hpp" #include "ark/kafka/kafka_plugin.hpp" #include "ark/logger.hpp" +#include "kafka_factory.hpp" namespace ark { +std::unique_ptr detail::make_kafka_module() { + return std::make_unique(); +} + bool kafka_module_impl::init() { return true; } @@ -63,20 +68,23 @@ void kafka_module_impl::event_cb(RdKafka::Event& event) { switch (event.type()) { case RdKafka::Event::EVENT_ERROR: - ARK_LOG_WARN("Kafka error! err=[{}]:{} msg={}", event.err(), RdKafka::err2str(event.err()), event.str()); + ARK_LOG_WARN("Kafka error! err=[{}]:{} msg={}", static_cast(event.err()), RdKafka::err2str(event.err()), + event.str()); break; case RdKafka::Event::EVENT_STATS: ARK_LOG_INFO("Kafka stats: {}", event.str()); break; case RdKafka::Event::EVENT_LOG: - ARK_LOG_INFO("Kafka event log! severity={} facility={} msg={}", event.severity(), event.fac(), event.str()); + ARK_LOG_INFO("Kafka event log! severity={} facility={} msg={}", static_cast(event.severity()), event.fac(), + event.str()); break; case RdKafka::Event::EVENT_THROTTLE: ARK_LOG_INFO("Kafka throttled time={}ms by {} ! brokerID={}", event.throttle_time(), event.broker_name(), event.broker_id()); break; default: - ARK_LOG_INFO("Kafka event. type={} err=[{}] msg={}", event.type(), RdKafka::err2str(event.err()), event.str()); + ARK_LOG_INFO("Kafka event. type={} err=[{}] msg={}", static_cast(event.type()), + RdKafka::err2str(event.err()), event.str()); break; } } diff --git a/src/plugins/kafka/kafka_module_impl.hpp b/src/plugins/kafka/kafka_module_impl.hpp index a843df2ba..83cbb5fc0 100644 --- a/src/plugins/kafka/kafka_module_impl.hpp +++ b/src/plugins/kafka/kafka_module_impl.hpp @@ -3,16 +3,15 @@ #include "ark/container/array_map.hpp" #include "ark/kafka/kafka.hpp" #include "ark/kafka/kafka_module.hpp" -#include "ark/plugin_manager.hpp" +#include "ark/module_base.hpp" namespace ark { class kafka_module_impl - : public kafka_module + : public module_base , public RdKafka::DeliveryReportCb , public RdKafka::EventCb { - ARK_DECLARE_MODULE_FUNCTIONS public: bool init() override; bool update() override; @@ -28,4 +27,4 @@ class kafka_module_impl array_map producers_; }; -} // namespace ark \ No newline at end of file +} // namespace ark diff --git a/src/plugins/kafka/kafka_plugin.cpp b/src/plugins/kafka/kafka_plugin.cpp index 3a3900e74..37a6ea18a 100644 --- a/src/plugins/kafka/kafka_plugin.cpp +++ b/src/plugins/kafka/kafka_plugin.cpp @@ -1,16 +1,31 @@ -#include "ark/kafka/kafka_module_impl.hpp" +#include + #include "ark/kafka/kafka_plugin.hpp" +#include "kafka_factory.hpp" namespace ark { -ARK_DECLARE_PLUGIN_DLL_FUNCTION(kafka_plugin) +ARK_EXPORT_FUNC void DllEntryPlugin(plugin_manager* manager, const std::string& name, logger* log) { + logger::init(log); + manager->register_plugin(name); +} + +ARK_EXPORT_FUNC void DllExitPlugin(plugin_manager* manager) { + manager->unregister_plugin(); +} void kafka_plugin::install() { - ARK_REGISTER_MODULE(kafka_module, kafka_module_impl); + if (!register_module(detail::make_kafka_module(), module_update::enabled)) + { + throw std::runtime_error("Kafka module registration rejected"); + } } void kafka_plugin::uninstall() { - ARK_UNREGISTER_MODULE(kafka_module, kafka_module_impl); + if (!unregister_module()) + { + throw std::runtime_error("Kafka module unregistration rejected"); + } } } // namespace ark diff --git a/src/plugins/kafka/kafka_plugin.hpp b/src/plugins/kafka/kafka_plugin.hpp index cee80bcd2..db562c284 100644 --- a/src/plugins/kafka/kafka_plugin.hpp +++ b/src/plugins/kafka/kafka_plugin.hpp @@ -1,8 +1,14 @@ -#include "ark/plugin.hpp" -#include "ark/plugin_manager.hpp" +#pragma once + +#include "ark/plugin_base.hpp" namespace ark { -ARK_DECLARE_PLUGIN(kafka_plugin) +class kafka_plugin final : public plugin_base +{ +public: + void install() override; + void uninstall() override; +}; } // namespace ark diff --git a/tests/ci/install-package.test.mjs b/tests/ci/install-package.test.mjs index 43eb8591d..e91f348b5 100644 --- a/tests/ci/install-package.test.mjs +++ b/tests/ci/install-package.test.mjs @@ -72,10 +72,10 @@ include(cmake/ark_install.cmake) const text = readFileSync(file, 'utf8'); for (const forbidden of [root, producer, build, prefix]) assert.ok(!text.includes(forbidden), `${file}: ${forbidden}`); } - for (const name of ['mysql/mysql_module.hpp', 'kernel/kernel_plugin.hpp', 'etcd_client/etcd_client_plugin.hpp', 'etcd_client/etcd_client_module_impl.hpp', 'physic/physic_module_impl.hpp', 'application_manifest.hpp']) { + for (const name of ['mysql/mysql_module.hpp', 'kernel/kernel_plugin.hpp', 'etcd_client/etcd_client_plugin.hpp', 'etcd_client/etcd_client_module_impl.hpp', 'kafka/kafka.hpp', 'kafka/kafka_module.hpp', 'kafka/kafka_module_impl.hpp', 'kafka/kafka_plugin.hpp', 'physic/physic_module_impl.hpp', 'application_manifest.hpp']) { assert.ok(existsSync(join(relocated, 'include/ark', name)), name); } - for (const name of ['mysql/mysql_plugin.hpp', 'bus/bus_config_parser.hpp', 'http/http_plugin.hpp', 'net/http_client_module_impl.hpp', 'net/http_client_factory.hpp', 'kernel/kernel_factory.hpp', 'etcd_client/etcd_client_factory.hpp', 'configuration_reader.hpp']) { + for (const name of ['mysql/mysql_plugin.hpp', 'bus/bus_config_parser.hpp', 'http/http_plugin.hpp', 'net/http_client_module_impl.hpp', 'net/http_client_factory.hpp', 'kernel/kernel_factory.hpp', 'etcd_client/etcd_client_factory.hpp', 'kafka/kafka_factory.hpp', 'configuration_reader.hpp']) { assert.equal(existsSync(join(relocated, 'include/ark', name)), false, name); } rmSync(producer, { recursive: true }); diff --git a/tests/ci/package-dependencies.test.mjs b/tests/ci/package-dependencies.test.mjs index 840a16bc9..58061aa9f 100644 --- a/tests/ci/package-dependencies.test.mjs +++ b/tests/ci/package-dependencies.test.mjs @@ -104,3 +104,18 @@ test('Applications CI explicitly enables Etcd Client ownership and production DS assert.match(plugin, /DllExitPlugin/); assert.doesNotMatch(plugin, /ARK_DECLARE_PLUGIN|ARK_REGISTER_MODULE|ARK_UNREGISTER_MODULE/); }); + +test('Optional Kafka construction preserves update selection without public factories', () => { + const core = read('tests/core/CMakeLists.txt'); + assert.match(core, /if\(TARGET ark_kafka_plugin\)[\s\S]*add_subdirectory\(\.\.\/kafka/); + const checks = read('tests/kafka/CMakeLists.txt'); + assert.match(checks, /add_dependencies\(check-core check-kafka-plugin check-kafka-construction\)/); + assert.match(checks, /kafka_module_impl\.cpp/); + const mappings = read('src/plugins/CMakeLists.txt').split('set(ark_kafka_public_headers')[1].split('\n)')[0]; + assert.doesNotMatch(mappings, /kafka_factory/); + const plugin = read('src/plugins/kafka/kafka_plugin.cpp'); + assert.match(plugin, /module_update::enabled/); + assert.match(plugin, /DllEntryPlugin/); + assert.match(plugin, /DllExitPlugin/); + assert.doesNotMatch(plugin, /ARK_DECLARE_PLUGIN|ARK_REGISTER_MODULE|ARK_UNREGISTER_MODULE/); +}); diff --git a/tests/core/CMakeLists.txt b/tests/core/CMakeLists.txt index 927446e32..440b326a9 100644 --- a/tests/core/CMakeLists.txt +++ b/tests/core/CMakeLists.txt @@ -233,6 +233,11 @@ if(TARGET ark_etcd_client_plugin) add_subdirectory(../etcd_client "${CMAKE_CURRENT_BINARY_DIR}/etcd_client") endif() +# Keep Kafka driver discovery optional: checks follow its selected runtime plugin. +if(TARGET ark_kafka_plugin) + add_subdirectory(../kafka "${CMAKE_CURRENT_BINARY_DIR}/kafka") +endif() + if(ARK_BUILD_PROTO AND TARGET ark_kernel_plugin AND TARGET ark_utility_plugin) add_executable(ark_kernel_plugin_tests ../app/kernel_plugin.cpp) set_property(TARGET ark_kernel_plugin_tests PROPERTY LINK_LIBRARIES ark::kernel) diff --git a/tests/core/README.md b/tests/core/README.md index bfe66a033..748362bb4 100644 --- a/tests/core/README.md +++ b/tests/core/README.md @@ -159,3 +159,13 @@ With Protobuf and the Kernel/Utility runtime targets enabled, `check-kernel-plug The XXTEA cases cover exact-size short-key allocations, 16-byte and embedded-zero keys, binary round trips and output appending. See the [key contract](../../docs/reference/xxtea.md). + +With the optional Kafka runtime enabled (`ARK_BUILD_APPLICATIONS=ON`, `ARK_BUILD_PROTO=ON`, +`ARK_BUILD_CORE_TESTS=ON`, `ARK_BUILD_PLUGIN_KAFKA=ON`) and its existing driver dependencies, +`check-core` includes `check-kafka-plugin` and `check-kafka-construction`. The focused checks +compile the real factory/assembly, verify interface RTTI metadata and enabled updates, run +production entry/exit code with a controlled factory, and load the actual production DSO to +inspect exports. They create no producer or broker connection. Kafka stays off by default; +disabled-plugin core builds do not discover its dependencies. See the +[construction migration](../../docs/reference/directory-structure.md#typed-kafka-construction) +for the public-layout rebuild requirement and service-validation limits. diff --git a/tests/core/README_CN.md b/tests/core/README_CN.md index 56005e362..eb8d5ddb2 100644 --- a/tests/core/README_CN.md +++ b/tests/core/README_CN.md @@ -136,3 +136,11 @@ Net 注册的 HTTP 客户端现在由私有接口工厂创建。`check-net-plugi 开启 Protobuf 且存在 Kernel／Utility 运行时 target 时，`check-kernel-plugin` 通过独立公开接口 runner 加载真实 DSO，检查原有四个接口键与管理器指针、元数据／静态记录加载、地图／实体注册和回调、管理器更新、停机销毁与重复 stop。它与现有直接初始化／实体默认值测试一起纳入 `check-core`；不在卸载后检查注册表，也不证明任意热更新安全。Kernel 公开头消费者检查全部保留类型的基类和 update 声明。 XXTEA 用例覆盖精确长度分配的短密钥、16 字节与内嵌零字节密钥、二进制往返及输出追加，参见[密钥约定](../../docs/reference/xxtea_CN.md)。 + +启用可选 Kafka 运行时（`ARK_BUILD_APPLICATIONS=ON`、`ARK_BUILD_PROTO=ON`、 +`ARK_BUILD_CORE_TESTS=ON`、`ARK_BUILD_PLUGIN_KAFKA=ON`）并准备原有驱动依赖后， +`check-core` 包含 `check-kafka-plugin` 与 `check-kafka-construction`。专项检查编译真实 +工厂／装配，验证接口 RTTI 元数据与启用 update，通过受控工厂执行生产入口／出口， +并加载真实生产动态库检查导出。测试不创建生产者、不连接 broker。Kafka 继续默认关闭； +关闭插件的核心测试不发现其依赖。公开布局重建要求与服务验收边界见 +[构造迁移](../../docs/reference/directory-structure_CN.md#kafka-类型化构造)。 diff --git a/tests/kafka/CMakeLists.txt b/tests/kafka/CMakeLists.txt new file mode 100644 index 000000000..85bcebba8 --- /dev/null +++ b/tests/kafka/CMakeLists.txt @@ -0,0 +1,35 @@ +# Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +# Match production's Unix-only discovery; Windows keeps its existing external setup. +if(NOT WIN32) + find_package(librdkafka REQUIRED) +endif() +set(kafka_source "${PROJECT_SOURCE_DIR}/src/plugins/kafka") + +# Compile real assembly/factory too: private C++ symbols are not DLL exports. +add_executable(ark_kafka_construction_tests ../core/main.cpp construction.cpp + "${kafka_source}/kafka_plugin.cpp" "${kafka_source}/kafka_module_impl.cpp") +set_property(TARGET ark_kafka_construction_tests PROPERTY LINK_LIBRARIES + ark::kafka "${LIBRDKAFKA_CPP_LIBRARIES}" "${LIBRDKAFKA_LIBRARIES}") +target_include_directories(ark_kafka_construction_tests PRIVATE "${kafka_source}" "${LIBRDKAFKA_INCLUDE_DIR}") + +# The controlled module exercises production entry/exit without a producer or broker. +add_library(ark_kafka_entry_probe SHARED entry_factory.cpp "${kafka_source}/kafka_plugin.cpp") +set_property(TARGET ark_kafka_entry_probe PROPERTY LINK_LIBRARIES ark::kafka) +target_include_directories(ark_kafka_entry_probe PRIVATE "${kafka_source}") +set_target_properties(ark_kafka_entry_probe PROPERTIES PREFIX "") +add_executable(ark_kafka_entry_tests entry.cpp) +set_property(TARGET ark_kafka_entry_tests PROPERTY LINK_LIBRARIES ark::kafka) + +add_custom_target(check-kafka-construction + COMMAND $ + COMMAND $ "$/" "ark_kafka_entry_probe" + DEPENDS ark_kafka_construction_tests ark_kafka_entry_tests ark_kafka_entry_probe + USES_TERMINAL VERBATIM) + +add_executable(ark_kafka_exports_tests exports.cpp) +set_property(TARGET ark_kafka_exports_tests PROPERTY LINK_LIBRARIES ark::core) +add_custom_target(check-kafka-plugin + COMMAND $ "$/" "ark_kafka_plugin" + DEPENDS ark_kafka_exports_tests ark_kafka_plugin check-kafka-construction + USES_TERMINAL VERBATIM) +add_dependencies(check-core check-kafka-plugin check-kafka-construction) diff --git a/tests/kafka/construction.cpp b/tests/kafka/construction.cpp new file mode 100644 index 000000000..02be84f74 --- /dev/null +++ b/tests/kafka/construction.cpp @@ -0,0 +1,61 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include +#include +#include + +#include "../network/doctest.h" + +#include +#include +#include "kafka_factory.hpp" + +#if __has_include() +#error "Kafka public mappings must exclude the private factory" +#endif + +static_assert(std::is_base_of_v, ark::kafka_module_impl>); +static_assert(std::is_base_of_v, ark::kafka_plugin>); +static_assert(std::is_base_of_v); +static_assert(std::is_base_of_v); +static_assert(!std::is_same_v); +static_assert(std::is_same_v>); + +TEST_CASE("Kafka real factory retains its public implementation and unpublished metadata") { + auto candidate = ark::detail::make_kafka_module(); + REQUIRE(candidate != nullptr); + const auto* instance = candidate.get(); + CHECK(typeid(*instance) == typeid(ark::kafka_module_impl)); + CHECK(candidate->get_plugin_manager() == nullptr); + CHECK(candidate->get_name().empty()); +} + +TEST_CASE("Kafka assembly retains the interface key, enabled updates and explicit ownership") { + ark::plugin_manager manager; + ark::kafka_plugin owner; + owner.set_plugin_manager(&manager); + CHECK(owner.get_plugin_version() == 0); + CHECK(owner.get_plugin_name() == typeid(ark::kafka_plugin).name()); + owner.set_plugin_conf("kafka-construction.toml"); + CHECK(ark::kafka_plugin::get_plugin_conf() == "kafka-construction.toml"); + owner.install(); + auto* instance = manager.find_module(); + REQUIRE(instance != nullptr); + CHECK(typeid(*instance) == typeid(ark::kafka_module_impl)); + CHECK(instance->get_plugin_manager() == &manager); + CHECK(instance->get_name() == typeid(ark::kafka_module).name()); + // Duplicate update insertion fails only when assembly already selected updates. + CHECK_FALSE(manager.add_update_module(instance)); + CHECK(instance->init()); + CHECK(manager.update(1)); + // Empty parameters avoid creating a producer or connecting to any broker. + CHECK_FALSE(instance->init_producer("", "construction")); + CHECK_FALSE(instance->init_producer("unused", "")); + CHECK_THROWS_AS(owner.install(), std::runtime_error); + CHECK(manager.find_module() == instance); + owner.uninstall(); + // Reinstallation proves the public registration key and ownership were released. + CHECK_NOTHROW(owner.install()); + CHECK_NOTHROW(owner.uninstall()); + CHECK(manager.update(1)); + CHECK_THROWS_AS(owner.uninstall(), std::runtime_error); +} diff --git a/tests/kafka/entry.cpp b/tests/kafka/entry.cpp new file mode 100644 index 000000000..a72464d22 --- /dev/null +++ b/tests/kafka/entry.cpp @@ -0,0 +1,45 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include +#include +#include + +#include +#include +#include + +namespace { + +void require(bool condition, const char* message) { + if (!condition) + { + std::cerr << message << std::endl; + std::_Exit(1); + } +} + +} // namespace + +int main(int argc, char* argv[]) { + require(argc == 3, "Expected Kafka entry-probe directory and library stem"); + ark::logger log; + ark::logger::init(&log); + log.set_log_func([](spdlog::level::level_enum, const std::string&) {}); + ark::dynamic_library library(argv[2]); + require(library.load(argv[1]), "Cannot load Kafka entry-probe library"); + auto events = reinterpret_cast(library.get_symbol("kafka_entry_events")); + require(events && events(0) == 0 && events(1) == 0, "Entry-probe state is not fresh"); + require(library.get_symbol("DllEntryPlugin") && library.get_symbol("DllExitPlugin"), + "Kafka explicit entry/exit exports are missing"); + ark::plugin_manager manager; + require(manager.start(0, argv[1], "conf/", {{argv[2], "entry-probe.toml"}}), "Kafka explicit entry failed"); + auto* instance = manager.find_module(); + require(instance && instance->get_plugin_manager() == &manager && + instance->get_name() == typeid(ark::kafka_module).name(), + "Kafka explicit entry lost interface registration metadata"); + require(events(0) == 1 && events(1) == 0, "Kafka entry did not transfer factory ownership"); + require(manager.update(1) && events(2) == 1, "Kafka entry lost enabled update selection"); + require(manager.stop(2) && events(1) == 1, "Kafka explicit exit did not destroy its module exactly once"); + require(manager.stop(3) && events(1) == 1, "Repeated stop invoked Kafka exit twice"); + require(library.unload(), "Cannot release Kafka observation handle"); + std::cout << "Kafka real assembly entry/exit, configuration, interface metadata and ownership passed\n"; +} diff --git a/tests/kafka/entry_factory.cpp b/tests/kafka/entry_factory.cpp new file mode 100644 index 000000000..3c6c3a35f --- /dev/null +++ b/tests/kafka/entry_factory.cpp @@ -0,0 +1,63 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include + +#include +#include +#include "kafka_factory.hpp" + +namespace { + +unsigned created{}; +unsigned destroyed{}; +unsigned updated{}; + +class entry_kafka final : public ark::module_base +{ +public: + entry_kafka() { + ++created; + } + + ~entry_kafka() override { + ++destroyed; + } + + bool init() override { + return ark::kafka_plugin::get_plugin_conf() == "conf/entry-probe.toml"; + } + + bool update() override { + ++updated; + return true; + } + + bool init_producer(const std::string&, const std::string&) override { + return false; + } + + int produce(const std::string&, const std::string&, void*) override { + return 0; + } +}; + +} // namespace + +namespace ark::detail { + +std::unique_ptr make_kafka_module() { + return std::make_unique(); +} + +} // namespace ark::detail + +ARK_EXPORT_FUNC unsigned kafka_entry_events(unsigned event) { + switch (event) + { + case 0: + return created; + case 1: + return destroyed; + default: + return updated; + } +} diff --git a/tests/kafka/exports.cpp b/tests/kafka/exports.cpp new file mode 100644 index 000000000..238f8f035 --- /dev/null +++ b/tests/kafka/exports.cpp @@ -0,0 +1,31 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include +#include +#include + +#include + +namespace { + +void require(bool condition, const char* message) { + if (!condition) + { + std::cerr << message << std::endl; + std::_Exit(1); + } +} + +} // namespace + +int main(int argc, char* argv[]) { + require(argc == 3, "Expected plugin directory and library stem"); + ark::logger log; + ark::logger::init(&log); + log.set_log_func([](spdlog::level::level_enum, const std::string&) {}); + ark::dynamic_library library(argv[2]); + require(library.load(argv[1]), "Cannot load actual Kafka shared library"); + require(library.get_symbol("DllEntryPlugin") != nullptr, "Missing explicit DllEntryPlugin export"); + require(library.get_symbol("DllExitPlugin") != nullptr, "Missing explicit DllExitPlugin export"); + require(library.unload(), "Cannot unload Kafka shared library"); + std::cout << argv[2] << ": actual DSO load, explicit entry/exit exports and unload passed\n"; +} From fb85954715ea33cfd651c4811bb0569f0d986360 Mon Sep 17 00:00:00 2001 From: nick Date: Mon, 26 Oct 2026 15:43:40 +0800 Subject: [PATCH 15/22] docs(roadmap): align supported behavior and remaining acceptance --- docs/concepts/application-architecture.md | 4 +- docs/concepts/application-architecture_CN.md | 4 +- docs/project/roadmap-history.md | 71 +++++++++++++++ docs/project/roadmap-history_CN.md | 88 +++++++++++++++++++ docs/project/roadmap.md | 73 ++++------------ docs/project/roadmap_CN.md | 90 ++++---------------- docs/site/src/navigation.ts | 5 ++ 7 files changed, 202 insertions(+), 133 deletions(-) create mode 100644 docs/project/roadmap-history.md create mode 100644 docs/project/roadmap-history_CN.md diff --git a/docs/concepts/application-architecture.md b/docs/concepts/application-architecture.md index bde15849d..e1d190cf8 100644 --- a/docs/concepts/application-architecture.md +++ b/docs/concepts/application-architecture.md @@ -53,7 +53,9 @@ Adding a public header or CMake API target only makes an interface available to ## Ordering and ownership -The manager installs plugins before running the module initialization stages. It does not topologically sort module dependencies or make manifest list order a lifecycle-order contract. Missing-module lookups can return null, and module callback return values are not currently aggregated into a transactional startup result. Verify required plugins/resources and application logs; a running process alone does not prove every service initialized successfully. +The manager installs plugins before running `init()`, `post_init()`, `check_config()` and `pre_update()`. The first `false` return stops startup; a propagated startup exception also reaches the launcher. The launcher reports the failed phase and available module/plugin details, then terminates with exit code `1` before entering the update loop. It does not roll back partially started plugins or run normal shutdown hooks on this failure path. See [startup failure handling](../guides/applications.md#check-startup-and-stop-the-process). + +The manager does not topologically sort module dependencies or make manifest list order a lifecycle-order contract. Missing-module lookups can return null. Verify required plugins/resources, initialization logs and application-specific readiness before routing traffic; a running process alone does not prove every service initialized successfully. Pointers returned by `find_module()` belong to the loaded plugin. Stop asynchronous producers before teardown and do not retain those pointers after unload. Keeping connections and state through an in-flight business-plugin replacement is not an automatic consequence of this composition model. diff --git a/docs/concepts/application-architecture_CN.md b/docs/concepts/application-architecture_CN.md index 1c6799916..c36480be4 100644 --- a/docs/concepts/application-architecture_CN.md +++ b/docs/concepts/application-architecture_CN.md @@ -53,7 +53,9 @@ config = "" ## 顺序与所有权 -管理器先安装插件，再执行模块初始化阶段。它不按依赖关系排序模块，清单中的插件顺序也不是生命周期顺序保证。查找缺失模块可能返回空指针，模块回调返回值目前没有汇总为可回滚的启动结果。需要核对必需插件、资源和应用日志；进程仍在运行，不代表每个服务都已初始化成功。 +管理器先安装插件，再执行 `init()`、`post_init()`、`check_config()` 和 `pre_update()`。首个 `false` 返回值会中止启动；向外传播的启动异常也会到达启动器。启动器报告失败阶段及可用的模块／插件信息，在进入更新循环前以退出码 `1` 终止。这条失败路径不回滚部分启动的插件，也不执行正常停机钩子。见[启动失败处理](../guides/applications_CN.md#确认启动并停止进程)。 + +管理器不按依赖关系排序模块，清单中的插件顺序也不是生命周期顺序保证。查找缺失模块可能返回空指针。接收流量前，需要核对必需插件、资源、初始化日志与应用就绪状态；进程仍在运行，不代表每个服务都已初始化成功。 `find_module()` 返回的指针属于已加载插件。释放前先停止异步生产者，插件卸载后不再保留这些指针。在业务函数执行期间替换插件，同时保留连接和状态，需要另外的协调机制，不是这种组合方式自动提供的能力。 diff --git a/docs/project/roadmap-history.md b/docs/project/roadmap-history.md new file mode 100644 index 000000000..cb012016b --- /dev/null +++ b/docs/project/roadmap-history.md @@ -0,0 +1,71 @@ +# Historical roadmap + +This snapshot preserves the previous roadmap, including its original wording and checkbox states. It is a historical record, not a list of supported APIs, accepted capabilities or current priorities. English and Chinese snapshots retain their original differences. Use the [current roadmap](roadmap.md) for supported behavior and remaining acceptance. + +--- + +## Current modernization + +- [x] Complete the `include/ark/`, `src/` and `src/plugins//` layout and snake_case API migration, including Protobuf source naming. +- [x] Release native single-node MySQL and Redis clients in arknet 0.2.0. +- [ ] Replace ark database plugins' legacy drivers, with Asio coroutine entry points, native arknet owning results and module-thread completion. +- [ ] Integrate the remaining independent official plugins with optional builds and tests. +- [ ] Review Nimbus documentation locally while preserving existing articles and entry points. + +Hot-update discussions and implementation are deferred. The original planning list below retains its historical checkbox state; it is not a current API or capability list. Use the current items above, source code and acceptance records to determine implementation status. + +## High priority + +- [ ] Put all schemas in one Excel workbook and keep other configuration tables separate, while remaining compatible with the existing format. +- [x] Unify code style. +- [ ] Connect to logserver before process startup and fail startup if the connection cannot be established, so logging is available from the beginning. +- [x] Upgrade spdlog. +- [ ] Add a dirty data layer that batches changed data after each client request, instead of sending every change separately in small packets with repeated protocol headers. +- [ ] Complete documentation for concepts, design principles, server architecture and usage examples. +- [x] Align all basic data structures with C++ containers. +- [ ] Add service registry support, considering etcd or consul. +- [ ] Use timers for periodic logs and heartbeats instead of update and HeartBeat. +- [ ] Update configuration packaging: edit in Excel and generate `csv` files for the project to read. Use contributor-specific suffixes such as Item.A.xlsx and Item.B.xlsx to reduce collaboration conflicts. +- [ ] ~~Control plugin loading order through configuration fields or configuration order.~~ +- [x] Use the AFXml wrapper instead of accessing rapidxml directly. +- [x] Simplify interface class design. +- [ ] Generate repetitive code (_plugin and module classes_). + +## Low priority + +- [ ] Update protocol and client handling for Map and Layer (**the concepts have changed; client integration is pending**). +- [ ] Use a different approach to sorting and swapping entries in inventory tables. +- [ ] Reduce data synchronization traffic: remove repeated strings such as `NodeName` and `TableName`, send index-to-name mappings at login, then use indices for updates and names for persistence. +- [ ] Extend DataTable to support more complex structures, such as items containing gems with experience and enchantments, or weapons with random attributes. The existing fixed two-dimensional structure is difficult to extend. +- [ ] Add network library and performance benchmarks under benchmark/unittest. +- [ ] Add kcp. +- [ ] Use SWIG for the scripting module. +- [ ] Evaluate ECS: represent all objects as entities and use GUIDs to identify their group and position for fast lookup. Research references include Overwatch, ET and [ECS-unity](https://github.com/sschmid/Entitas-CSharp). +- [ ] Add coroutines, referring to libgo. +- [ ] Introduce tcmalloc. +- [ ] Add a monitoring web page using vue-admin. +- [ ] Add a Deployment web tool. +- [ ] Add message queue processing. +- [ ] Add operations tools (tcm & deploy). +- [ ] Use multiple threads for selected CPU-intensive work. + +## Partially finished + +- [x] Add spdlog sinks that organize files by directory and rotate by size or hour (*partially changed; further changes are needed for the tlog format, /data/binlog/1001/20180627/game1_hour.log*). +- [x] Add consistent_hash_map (**added, not yet enabled**). + +## Finished + +- [x] Implement the new architecture. +- [x] Configure connection relationships instead of setting them manually in code (bus_relation, server_settting). Detect IP addresses and generate ports dynamically instead of specifying them, and express network addresses as URLs such as tcp://0.0.0.0:9001 and wss://0.0.0.0:9001. +- [x] Place Debug and Release binaries together under bin, with separate Debug and Release settings so they can share one configuration set. +- [x] Change guid to an internal pointer type to reduce memory use for other non-guid types. ***Changed to int64_t; the memory concern no longer applies.*** +- [x] Add a network plugin to separate network modules from application logic (NetClient and NetServer are still too closely coupled to application logic). +- [x] Add endpoint to the net plugin. +- [x] Merge the kernel and config plugins because kernel depends heavily on config. + +## Bugs + +- [ ] ClearRow shifts subsequent row indices. Either preserve the indices and clear only the row contents, or have the client update subsequent indices when it receives DeleteRow. +- [x] Fix ConfigPlugin data loading. +- [x] Fix the return values of AFMapEx First and Next. diff --git a/docs/project/roadmap-history_CN.md b/docs/project/roadmap-history_CN.md new file mode 100644 index 000000000..6a5eb24f3 --- /dev/null +++ b/docs/project/roadmap-history_CN.md @@ -0,0 +1,88 @@ +# 历史路线图 + +本页保留原路线图的文字与勾选状态，仅供追溯，不作为支持的 API、已验收能力或当前优先级清单。中英文快照保留原有差异。支持范围与剩余验收见[当前路线图](roadmap_CN.md)。 + +--- + +## 当前改造 + +- [x] 完成 `include/ark/`、`src/` 与 `src/plugins/<名称>/` 的目录整理和 snake_case 接口迁移，包含 Protobuf 源码命名。 +- [x] 随 arknet 0.2.0 发布原生 MySQL、Redis 单节点客户端。 +- [ ] 替换 ark 数据库插件的旧驱动，提供 Asio 协程入口，直接使用 arknet 拥有型结果，并在模块线程恢复。 +- [ ] 整合其余独立官方插件，按需构建并补齐测试。 +- [ ] 在本地审阅 Nimbus 文档，保留原有文章和入口。 + +业务热更新的讨论与实现暂缓。下方保留原有规划的历史勾选状态，不作为当前 API 或能力清单；当前实现以以上条目、源码和验收记录为准。 + +## High priority + +- [x] excel格式修改为一张excel里放所有的schema,其他配置表分开来写,兼容现在的格式 +- [x] 调整代码风格统一 +- [x] spdlog升级 +- [x] 增加dirty data layer, 当一个客户端请求处理完成后统一下发变化数据，不能有变有发，会导致小包太碎，并且浪费协议的head(改为非实时数据下一帧统一发送，实时数据有变有发) +- [ ] 文档逐步补齐，概念介绍、设计理念、服务器架构、使用例子 +- [x] 所有的基础数据结构向C++容器看齐 +- [x] 服务中心注册(考虑etcd/consul来做), 已经使用consul来处理了 +- [x] 定时log和心跳改为timer方式，不要再是用update和HeartBeat了 +- [x] 修改配置打包工具(用Excel编辑，生成`xml`来给工程读取。为解决多人协作冲突, excel可以用名字后缀区分，类如Item.A.xlsx Item.B.xlsx) +- [x] 修改xml不再直接用rapidxml，用封装后的AFXml +- [x] 接口类重新设计纯粹化 +- [x] 增加生成器来生成重复代码(_插件类和模块类_) +- [x] 使用`std::chrono`改造`AFDateTime` +- [x] session id 使用GUID类型 +- [x] 增加murmur hash函数用来做consistent hash map +- [x] 增加xoshiro/xoroshiro随机算法 +- [x] 增加xxtea, crc16, crc32, crc64加密算法 +- [ ] 增加codec类，用来剥离框架对特定协议的依赖，方便外部替换协议类型 +- [ ] 增加插件动态热更新 +- [ ] 增加工具获取插件动态链接库版本号 +- [x] Linux下强杀进程处理(kill -9 xxx) +- [ ] 配置meta和entity meta的区分 +- [ ] 网络修改为asio系 +- [ ] web serever/client修改为asio/beast系 +- [ ] Redis 插件接入 arknet 自主实现的 Redis 客户端 + +## Low priority + +- [x] Map, Layer,对应修改协议和Client的处理(**概念已经修改, 还未对接客户端**) +- [x] 关于背包类的table，整理或者交换，需要改成别的方式 +- [x] 优化数据同步的流量,减小属性同步的体积(去掉`NodeName`和`TableName`类似的string同步, 上线先同步index和name的映射，后续全部用index下发，入库用name) +- [x] 拓展DataTable的组成，例如道具包含了宝石(带经验，附魔等)、武器(带随机属性)等东西，DataTable现有的二维结构是特定的，不方便做扩展 +- [ ] 增加网络库和性能的基准测试用例，放到目录benchmark/unittest +- [ ] 增加kcp +- [ ] 脚本模块使用SWIG +- [x] 考虑使用ECS，全部对象Entity化，~~通过GUID来确定属于哪个分组的哪个位置，能快速定位和查找(通过守望先锋、ET、[ECS-unity](https://github.com/sschmid/Entitas-CSharp)，具体还要再调研)~~ +- [ ] 增加coroutine，参考libgo +- [ ] 引入tcmalloc +- [ ] 增加监控web页面(ant-design-for-vue) +- [ ] 增加Deployment web工具 +- [ ] ~~增加消息队列处理~~ +- [ ] 增加运维工具(tcm & deploy) +- [ ] 部分计算密集的地方使用多线程 +- [ ] 增加协议wrapper层，这样框架层包装后，可以甩出来给业务用，无需知道业务层用的协议类型 +- [ ] ~~升级至C++17~~ +- [ ] ~~`std::string_veiw`/`discard`/...~~ +- [ ] ~~插件加载顺序(通过配置中的字段, 或者通过配置的顺序)~~ +- [ ] ~~修改进程启动优先链接logserver，如果logserver连不上，则直接不能启动，这样就可以从一开始就能打log了~~ + +## Part finished + +- [x] spdlog增加按照文件夹/大小/小时滚动的sink(*已经修改了一部分，还需要进一步修改成tlog的格式，/data/binlog/1001/20180627/game1_hour.log*) +- [x] 增加consistent_hash_map + +## Finished + +- [x] 新架构修改 +- [x] 增加连接关系的配置，不再通过程序中手动设置关系(bus_relation, server_settting, IP和port不用再指定了, 通过获取IP和动态生成port), 网络地址修改为tcp://0.0.0.0:9001 wss://0.0.0.0:9001这样的URL +- [x] 修改bin文件夹debug和release在一起，debug和release分开配置(方便配置是一套) +- [x] guid改为内部指针类型(降低其他非guid类型的内存占用)***修改为int64_t了,不需要再考虑内存占用了*** +- [x] 增加网络插件,这样可以将网络模块和业务逻辑模块拆分开(现在NetClient和NetServer还是跟业务关联太紧密了) +- [x] net plugin增加endpoint +- [x] kernel和config插件还是得合并为一个，因为kernel非常依赖config插件 + +## Bugs + +- [x] ClearRow的时候后面的行号会变化，所以要考虑要么行号不变，只是置空，要么就得客户端来根据DeleteRow消息同步修改后续的row变化 +- [x] 现在的ConfigPlugin数据加载有问题 +- [x] AFMapEx的First Next返回值有问题 +- [ ] 单个进程开多个监听服务，busid会被覆盖和连接管理有问题 diff --git a/docs/project/roadmap.md b/docs/project/roadmap.md index 36c0b28ed..0b1f720ea 100644 --- a/docs/project/roadmap.md +++ b/docs/project/roadmap.md @@ -1,67 +1,26 @@ # Roadmap -## Current modernization +This page distinguishes supported source behavior from remaining design and deployment acceptance. C++20 is the build baseline; it does not mean every modernization or optional integration is complete. The [historical roadmap](roadmap-history.md) preserves the previous planning list and its checkbox states. -- [x] Complete the `include/ark/`, `src/` and `src/plugins//` layout and snake_case API migration, including Protobuf source naming. -- [x] Release native single-node MySQL and Redis clients in arknet 0.2.0. -- [ ] Replace ark database plugins' legacy drivers, with Asio coroutine entry points, native arknet owning results and module-thread completion. -- [ ] Integrate the remaining independent official plugins with optional builds and tests. -- [ ] Review Nimbus documentation locally while preserving existing articles and entry points. +## Supported behavior -Hot-update discussions and implementation are deferred. The original planning list below retains its historical checkbox state; it is not a current API or capability list. Use the current items above, source code and acceptance records to determine implementation status. +- Core code lives in `src/core/`, the launcher in `src/application/`, shared manifest/configuration reading in `src/config/`, and official plugin headers and implementation together in `src/plugins//`. Explicit CMake public-header mappings preserve `` include paths. See [project layout and public interfaces](../reference/directory-structure.md) for targets, rebuild requirements and optional plugin boundaries. +- Public [MySQL](../reference/mysql.md) and [Redis](../reference/redis.md) modules use arknet native clients through Asio coroutine entry points. Inputs and results own their storage; manager-owned task groups resume on the module owner thread. MySQL includes prepared statements and transactions; Redis includes commands and pipelines. Legacy database drivers are no longer the public plugin interface. +- Startup callbacks propagate the first failure. The launcher reports the failed phase and exits with code `1` before the update loop. Shutdown stops managed task admission, drains accepted tasks and waits for module readiness. A configured cooperative wait timeout retains the process/resources and disables unloading. These behaviors do not provide partial-startup rollback or a hard process-stop deadline. See [application startup and shutdown](../guides/applications.md) and [managed tasks](../guides/tasks.md). -## High priority +The four public database service/failure modes passed against isolated Redis and MySQL servers on macOS. The [database service acceptance record](../guides/database-validation.md) identifies the tested revisions, environment and exact scope; those results do not cover every deployment or current source revision. -- [ ] Put all schemas in one Excel workbook and keep other configuration tables separate, while remaining compatible with the existing format. -- [x] Unify code style. -- [ ] Connect to logserver before process startup and fail startup if the connection cannot be established, so logging is available from the beginning. -- [x] Upgrade spdlog. -- [ ] Add a dirty data layer that batches changed data after each client request, instead of sending every change separately in small packets with repeated protocol headers. -- [ ] Complete documentation for concepts, design principles, server architecture and usage examples. -- [x] Align all basic data structures with C++ containers. -- [ ] Add service registry support, considering etcd or consul. -- [ ] Use timers for periodic logs and heartbeats instead of update and HeartBeat. -- [ ] Update configuration packaging: edit in Excel and generate `csv` files for the project to read. Use contributor-specific suffixes such as Item.A.xlsx and Item.B.xlsx to reduce collaboration conflicts. -- [ ] ~~Control plugin loading order through configuration fields or configuration order.~~ -- [x] Use the AFXml wrapper instead of accessing rapidxml directly. -- [x] Simplify interface class design. -- [ ] Generate repetitive code (_plugin and module classes_). +## Remaining acceptance and design -## Low priority +| Area | Remaining work and acceptance boundary | +| --- | --- | +| Database deployment | Run separate real-service acceptance for request cancellation, server termination and dynamic-library unloading. Add target-environment and cross-machine capacity evidence before making deployment or capacity claims. Local protocol endpoints and construction checks do not replace these runs. | +| Startup and shutdown | Partial-startup rollback is not provided. A common dependency-aware shutdown protocol remains design work; managed task drain alone does not prove that arbitrary callbacks, native workers or background cleanup have ended. Define ownership and admission phases before extending unload guarantees. | +| Optional plugins | Kafka construction/export checks do not test broker delivery, flush behavior or service interoperability; its driver build needs independent platform acceptance. OSS SDK remains a scaffold without client/request functionality. Physic retains the explicit unsupported legacy PhysX 3 configuration failure. See [optional plugin boundaries](../reference/directory-structure.md). | +| C++20 modernization | Review each remaining module and optional integration against its own API, build and runtime requirements. The language baseline, header layout and focused construction checks do not establish completion of the whole modernization. | -- [ ] Update protocol and client handling for Map and Layer (**the concepts have changed; client integration is pending**). -- [ ] Use a different approach to sorting and swapping entries in inventory tables. -- [ ] Reduce data synchronization traffic: remove repeated strings such as `NodeName` and `TableName`, send index-to-name mappings at login, then use indices for updates and names for persistence. -- [ ] Extend DataTable to support more complex structures, such as items containing gems with experience and enchantments, or weapons with random attributes. The existing fixed two-dimensional structure is difficult to extend. -- [ ] Add network library and performance benchmarks under benchmark/unittest. -- [ ] Add kcp. -- [ ] Use SWIG for the scripting module. -- [ ] Evaluate ECS: represent all objects as entities and use GUIDs to identify their group and position for fast lookup. Research references include Overwatch, ET and [ECS-unity](https://github.com/sschmid/Entitas-CSharp). -- [ ] Add coroutines, referring to libgo. -- [ ] Introduce tcmalloc. -- [ ] Add a monitoring web page using vue-admin. -- [ ] Add a Deployment web tool. -- [ ] Add message queue processing. -- [ ] Add operations tools (tcm & deploy). -- [ ] Use multiple threads for selected CPU-intensive work. +## Deferred directions -## Partially finished +Registry design and business-plugin hot update remain paused. In-flight work, state ownership, dependency order and compatibility need an agreed design before implementation or broader unload claims. Business-configuration export changes are also deferred; the shared application-manifest reader does not replace business-resource formats or their export tools. -- [x] Add spdlog sinks that organize files by directory and rotate by size or hour (*partially changed; further changes are needed for the tlog format, /data/binlog/1001/20180627/game1_hour.log*). -- [x] Add consistent_hash_map (**added, not yet enabled**). - -## Finished - -- [x] Implement the new architecture. -- [x] Configure connection relationships instead of setting them manually in code (bus_relation, server_settting). Detect IP addresses and generate ports dynamically instead of specifying them, and express network addresses as URLs such as tcp://0.0.0.0:9001 and wss://0.0.0.0:9001. -- [x] Place Debug and Release binaries together under bin, with separate Debug and Release settings so they can share one configuration set. -- [x] Change guid to an internal pointer type to reduce memory use for other non-guid types. ***Changed to int64_t; the memory concern no longer applies.*** -- [x] Add a network plugin to separate network modules from application logic (NetClient and NetServer are still too closely coupled to application logic). -- [x] Add endpoint to the net plugin. -- [x] Merge the kernel and config plugins because kernel depends heavily on config. - -## Bugs - -- [ ] ClearRow shifts subsequent row indices. Either preserve the indices and clear only the row contents, or have the client update subsequent indices when it receives DeleteRow. -- [x] Fix ConfigPlugin data loading. -- [x] Fix the return values of AFMapEx First and Next. +Historical ideas in the archive are not automatically scheduled work. Future scope should name the affected API or deployment, its prerequisites and concrete acceptance evidence before it is promoted into this roadmap. diff --git a/docs/project/roadmap_CN.md b/docs/project/roadmap_CN.md index a089a6b9f..55895a2d5 100644 --- a/docs/project/roadmap_CN.md +++ b/docs/project/roadmap_CN.md @@ -1,84 +1,26 @@ # 路线图 -## 当前改造 +本页区分源码支持的行为与尚缺的设计、部署验收。C++20 是构建基线，不代表全部现代化改造或可选集成都已完成。[历史路线图](roadmap-history_CN.md)保留原规划清单与勾选状态。 -- [x] 完成 `include/ark/`、`src/` 与 `src/plugins/<名称>/` 的目录整理和 snake_case 接口迁移，包含 Protobuf 源码命名。 -- [x] 随 arknet 0.2.0 发布原生 MySQL、Redis 单节点客户端。 -- [ ] 替换 ark 数据库插件的旧驱动，提供 Asio 协程入口，直接使用 arknet 拥有型结果，并在模块线程恢复。 -- [ ] 整合其余独立官方插件，按需构建并补齐测试。 -- [ ] 在本地审阅 Nimbus 文档，保留原有文章和入口。 +## 支持的行为 -业务热更新的讨论与实现暂缓。下方保留原有规划的历史勾选状态，不作为当前 API 或能力清单；当前实现以以上条目、源码和验收记录为准。 +- 核心代码位于 `src/core/`，启动器位于 `src/application/`，共享清单与配置读取位于 `src/config/`，官方插件的头文件与实现共置于 `src/plugins//`。CMake 显式公开头文件映射保留 `` 包含路径。目标、重建要求和可选插件边界见[项目布局与公开接口](../reference/directory-structure_CN.md)。 +- 公开 [MySQL](../reference/mysql_CN.md) 与 [Redis](../reference/redis_CN.md) 模块通过 Asio 协程入口使用 arknet 原生客户端。输入与结果拥有自身存储；管理器拥有的任务组在模块所属线程恢复。MySQL 提供预处理语句与事务，Redis 提供命令与流水线。旧数据库驱动已不再作为公开插件接口。 +- 启动回调的首个失败会向上传递。启动器报告失败阶段，在进入更新循环前以退出码 `1` 终止。停止时关闭托管任务接收，排空已接收任务并等待模块就绪。配置的协作式等待超时会保留进程和资源，并禁止卸载。这些行为不提供部分启动回滚，也不构成强制进程退出期限。见[应用启动与停止](../guides/applications_CN.md)和[托管任务](../guides/tasks_CN.md)。 -## High priority +公开数据库接口的四种正常／故障模式已在 macOS 上通过隔离 Redis 与 MySQL 真实服务验收。[数据库服务验收记录](../guides/database-validation_CN.md)列明测试版本、环境与范围；这些结果不覆盖所有部署环境或当前源码版本。 -- [x] excel格式修改为一张excel里放所有的schema,其他配置表分开来写,兼容现在的格式 -- [x] 调整代码风格统一 -- [x] spdlog升级 -- [x] 增加dirty data layer, 当一个客户端请求处理完成后统一下发变化数据，不能有变有发，会导致小包太碎，并且浪费协议的head(改为非实时数据下一帧统一发送，实时数据有变有发) -- [ ] 文档逐步补齐，概念介绍、设计理念、服务器架构、使用例子 -- [x] 所有的基础数据结构向C++容器看齐 -- [x] 服务中心注册(考虑etcd/consul来做), 已经使用consul来处理了 -- [x] 定时log和心跳改为timer方式，不要再是用update和HeartBeat了 -- [x] 修改配置打包工具(用Excel编辑，生成`xml`来给工程读取。为解决多人协作冲突, excel可以用名字后缀区分，类如Item.A.xlsx Item.B.xlsx) -- [x] 修改xml不再直接用rapidxml，用封装后的AFXml -- [x] 接口类重新设计纯粹化 -- [x] 增加生成器来生成重复代码(_插件类和模块类_) -- [x] 使用`std::chrono`改造`AFDateTime` -- [x] session id 使用GUID类型 -- [x] 增加murmur hash函数用来做consistent hash map -- [x] 增加xoshiro/xoroshiro随机算法 -- [x] 增加xxtea, crc16, crc32, crc64加密算法 -- [ ] 增加codec类，用来剥离框架对特定协议的依赖，方便外部替换协议类型 -- [ ] 增加插件动态热更新 -- [ ] 增加工具获取插件动态链接库版本号 -- [x] Linux下强杀进程处理(kill -9 xxx) -- [ ] 配置meta和entity meta的区分 -- [ ] 网络修改为asio系 -- [ ] web serever/client修改为asio/beast系 -- [ ] Redis 插件接入 arknet 自主实现的 Redis 客户端 +## 剩余验收与设计 -## Low priority +| 领域 | 剩余工作与验收边界 | +| --- | --- | +| 数据库部署 | 为请求取消、服务端终止和动态库卸载分别执行真实服务验收。发布部署或容量结论前，补充目标环境与跨机器容量证据。本机协议端点与构造检查不能替代这些验收。 | +| 启动与停机 | 框架不提供部分启动回滚。通用的依赖感知停机协议仍需设计；托管任务排空不代表任意回调、原生工作线程或后台清理均已结束。扩展卸载保证前，先明确所有权与接收阶段。 | +| 可选插件 | Kafka 构造／导出检查不验证 broker 投递、flush 行为或服务互通；驱动构建仍需独立平台验收。OSS SDK 保持骨架状态，不提供客户端／请求功能。Physic 保留旧版 PhysX 3 不受支持的显式配置失败。见[可选插件边界](../reference/directory-structure_CN.md)。 | +| C++20 改造 | 按各模块与可选集成的 API、构建和运行要求继续审查。语言基线、头文件布局和局部构造检查不足以证明全部改造完成。 | -- [x] Map, Layer,对应修改协议和Client的处理(**概念已经修改, 还未对接客户端**) -- [x] 关于背包类的table，整理或者交换，需要改成别的方式 -- [x] 优化数据同步的流量,减小属性同步的体积(去掉`NodeName`和`TableName`类似的string同步, 上线先同步index和name的映射，后续全部用index下发，入库用name) -- [x] 拓展DataTable的组成，例如道具包含了宝石(带经验，附魔等)、武器(带随机属性)等东西，DataTable现有的二维结构是特定的，不方便做扩展 -- [ ] 增加网络库和性能的基准测试用例，放到目录benchmark/unittest -- [ ] 增加kcp -- [ ] 脚本模块使用SWIG -- [x] 考虑使用ECS，全部对象Entity化，~~通过GUID来确定属于哪个分组的哪个位置，能快速定位和查找(通过守望先锋、ET、[ECS-unity](https://github.com/sschmid/Entitas-CSharp)，具体还要再调研)~~ -- [ ] 增加coroutine，参考libgo -- [ ] 引入tcmalloc -- [ ] 增加监控web页面(ant-design-for-vue) -- [ ] 增加Deployment web工具 -- [ ] ~~增加消息队列处理~~ -- [ ] 增加运维工具(tcm & deploy) -- [ ] 部分计算密集的地方使用多线程 -- [ ] 增加协议wrapper层，这样框架层包装后，可以甩出来给业务用，无需知道业务层用的协议类型 -- [ ] ~~升级至C++17~~ -- [ ] ~~`std::string_veiw`/`discard`/...~~ -- [ ] ~~插件加载顺序(通过配置中的字段, 或者通过配置的顺序)~~ -- [ ] ~~修改进程启动优先链接logserver，如果logserver连不上，则直接不能启动，这样就可以从一开始就能打log了~~ +## 暂缓方向 -## Part finished +Registry 设计与业务插件热更新仍暂缓。扩展实现或卸载保证前，需要先确定在途任务、状态所有权、依赖顺序与兼容性设计。业务配置导出调整也暂缓；共享应用清单读取器不替换业务资源格式及其导出工具。 -- [x] spdlog增加按照文件夹/大小/小时滚动的sink(*已经修改了一部分，还需要进一步修改成tlog的格式，/data/binlog/1001/20180627/game1_hour.log*) -- [x] 增加consistent_hash_map - -## Finished - -- [x] 新架构修改 -- [x] 增加连接关系的配置，不再通过程序中手动设置关系(bus_relation, server_settting, IP和port不用再指定了, 通过获取IP和动态生成port), 网络地址修改为tcp://0.0.0.0:9001 wss://0.0.0.0:9001这样的URL -- [x] 修改bin文件夹debug和release在一起，debug和release分开配置(方便配置是一套) -- [x] guid改为内部指针类型(降低其他非guid类型的内存占用)***修改为int64_t了,不需要再考虑内存占用了*** -- [x] 增加网络插件,这样可以将网络模块和业务逻辑模块拆分开(现在NetClient和NetServer还是跟业务关联太紧密了) -- [x] net plugin增加endpoint -- [x] kernel和config插件还是得合并为一个，因为kernel非常依赖config插件 - -## Bugs - -- [x] ClearRow的时候后面的行号会变化，所以要考虑要么行号不变，只是置空，要么就得客户端来根据DeleteRow消息同步修改后续的row变化 -- [x] 现在的ConfigPlugin数据加载有问题 -- [x] AFMapEx的First Next返回值有问题 -- [ ] 单个进程开多个监听服务，busid会被覆盖和连接管理有问题 +归档中的历史设想不自动成为排期任务。将新范围纳入路线图前，应明确受影响的 API 或部署、前置条件与具体验收证据。 diff --git a/docs/site/src/navigation.ts b/docs/site/src/navigation.ts index 2949f2b9b..0c2f1aa64 100644 --- a/docs/site/src/navigation.ts +++ b/docs/site/src/navigation.ts @@ -256,6 +256,11 @@ const tree: Entry[] = [ "project/roadmap", "Roadmap", "路线图" + ], + [ + "project/roadmap-history", + "Historical roadmap", + "历史路线图" ] ] }, From b66d345864d4633ea284fe1385069647301c9639 Mon Sep 17 00:00:00 2001 From: nick Date: Mon, 26 Oct 2026 16:09:34 +0800 Subject: [PATCH 16/22] fix(buffer)!: transfer receive storage with move-only ownership Own the receive array with a private unique_ptr deleter, prohibit shallow copies and transfer allocation/cursor state through noexcept moves. Leave moved-from buffers empty and reusable, preserve self-move, capacity growth and throwing allocation behavior, and route growth through matching configured array hooks. Reject out-of-range consumption and avoid null-pointer operations for empty writes and storage. Add permanent ownership, framing-state, deterministic allocation-failure and isolated custom-hook regressions with bilingual guidance. BREAKING CHANGE: ark::buffer no longer supports copy construction or copy assignment and its private storage representation changes. Use explicit moves for ownership transfer and rebuild affected Core/Net consumers and plugins together. Define compatible ARK_NEW_ARRAY/ARK_DELETE_ARRAY overrides consistently; growth now uses the same hook pair as construction and destruction. --- docs/reference/directory-structure.md | 2 + docs/reference/directory-structure_CN.md | 2 + docs/reference/network.md | 21 +++ docs/reference/network_CN.md | 21 +++ src/core/buffer.hpp | 109 +++++++++----- tests/core/CMakeLists.txt | 15 ++ tests/core/buffer_failure.cpp | 68 +++++++++ tests/core/buffer_hooks.cpp | 151 +++++++++++++++++++ tests/core/buffers.cpp | 177 +++++++++++++++++++++++ 9 files changed, 526 insertions(+), 40 deletions(-) create mode 100644 tests/core/buffer_failure.cpp create mode 100644 tests/core/buffer_hooks.cpp create mode 100644 tests/core/buffers.cpp diff --git a/docs/reference/directory-structure.md b/docs/reference/directory-structure.md index 8df5530d6..212f435e0 100644 --- a/docs/reference/directory-structure.md +++ b/docs/reference/directory-structure.md @@ -34,6 +34,8 @@ The MySQL/Redis interfaces have one source each: `src/plugins/mysql/mysql_module Core has one source copy under `src/core/`. `ark::core` maps its explicit public headers into the selected build tree, preserving `` and nested include paths. Edit source headers rather than generated copies. The `app`/`app_d` executable and manifest format are unchanged. +`` retains its public path with move-only ownership; rebuild affected Core/Net consumers for the source/layout change. See [receive-buffer ownership](network.md#receive-buffer-ownership) for migration, allocation-hook compatibility and focused tests. + ## CMake targets | Target | Usage | diff --git a/docs/reference/directory-structure_CN.md b/docs/reference/directory-structure_CN.md index baf4885d8..132eca7e7 100644 --- a/docs/reference/directory-structure_CN.md +++ b/docs/reference/directory-structure_CN.md @@ -32,6 +32,8 @@ MySQL、Redis 的唯一公开接口源码分别是 `src/plugins/mysql/mysql_modu 核心头文件只有 `src/core/` 中的一份源码。`ark::core` 将显式公开清单映射到选定构建目录，保留 `` 等包含路径。修改源码头文件，不编辑生成副本。启动程序仍为 `app`／`app_d`，清单格式保持不变。 +`` 保留原有公开路径，所有权改为仅支持移动；源码／布局发生变化，需要重建受影响的 Core／Net 消费者。迁移、分配钩子兼容性与定向测试见[收包缓冲的所有权](network_CN.md#收包缓冲的所有权)。 + ## CMake 目标 | 目标 | 用途 | diff --git a/docs/reference/network.md b/docs/reference/network.md index 56f5729af..6f4324a7a 100644 --- a/docs/reference/network.md +++ b/docs/reference/network.md @@ -95,6 +95,27 @@ TCP, WebSocket and RUDP endpoints use a 4 KiB initial application receive buffer This is the framing buffer for application messages, configured separately from kernel socket send/receive buffers. It is not a 4 KiB maximum message size. The [local receive-buffer report](../performance/receive-buffers.md) retains the allocation, fragmented-input and large-batch throughput tradeoff. It does not establish network RTT or connection capacity. +## Receive-buffer ownership + +**Breaking source/layout change:** `` exposes a move-only `ark::buffer`. Copy construction and copy assignment are deleted to prevent two objects from deleting the same array. Rebuild the Core/Net consumers and their plugins together. The tracked Net sessions already prohibit copying and own their buffer in place; copying a `shared_ptr` session handle does not copy that buffer. This repository audit does not establish whether external consumers relied on the former implicit copy operations. + +Move construction and move assignment are `noexcept` and transfer the allocation, unread bytes, capacity and cursor state. Assignment releases the destination's previous allocation; self-move leaves it unchanged. A moved-from buffer has zero capacity, zero unread bytes and a null data pointer; it can be destroyed, assigned or written to again. Existing public method signatures remain available. Standalone construction still defaults to 512 KiB, while Net sessions retain their separately configured 4 KiB default. + +```cpp +ark::buffer pending(4096); +pending.write("abc", 3); +ark::buffer processing(std::move(pending)); // transfers ownership +pending.write("next", 4); // allocates fresh storage +``` + +Include `` for `std::move`. `get_data()` returns a borrowed pointer into the current allocation; keep its owner alive and reacquire the pointer after compaction, growth or ownership replacement. This change introduces no clone API. Applications that need independent data can explicitly write the readable bytes into another buffer while the source remains valid. + +The buffer keeps `ARK_NEW_ARRAY` / `ARK_DELETE_ARRAY` as its narrow allocation compatibility boundary, now using the same pair for construction, growth and release. This corrects the previous growth path that bypassed custom hooks with raw `new[]` / `delete[]`. Define a compatible, nonthrowing release hook consistently across translation units before including the header. Without overrides, allocation still uses throwing `new[]`: `std::bad_alloc` propagates and failed growth retains the old bytes, capacity and cursors. If a custom allocator returns null, construction retains an empty zero-capacity buffer and `write()` returns `false` without changing the existing state. + +A zero-length write succeeds without dereferencing its pointer; a null source with nonzero length returns `false`. Removing more than the unread length is a no-op, including an overflowing count. Unrepresentable growth throws `std::length_error` before allocation. Normal reads, writes, compaction and capacity growth retain their existing behavior. + +With `ARK_BUILD_CORE_TESTS=ON`, `check-buffer` runs the lifetime/move, compaction/growth, boundary, deterministic default-allocation failure and separate custom-hook regressions; it is also part of `check-core`. The allocation-failure test fails one small array request instead of requesting excessive memory. `check-network` retains the existing Net framing/fragmented-input coverage. These checks do not establish service or capacity acceptance. + ## Member callbacks and borrowed arguments Endpoint constructors and client/server service registration bind the supplied diff --git a/docs/reference/network_CN.md b/docs/reference/network_CN.md index 097069151..696049bfe 100644 --- a/docs/reference/network_CN.md +++ b/docs/reference/network_CN.md @@ -80,6 +80,27 @@ TCP、WebSocket 与 RUDP 端点使用 4 KiB 初始应用层收包缓冲，随后 这是应用消息的拆包缓冲，与内核 socket 收发缓冲分别配置，也不是 4 KiB 的最大消息限制。[本地收包缓冲报告](../performance/receive-buffers_CN.md)保留了内存申请、分片输入与大批量吞吐量之间的取舍，不把这些结果视为网络 RTT 或连接容量。 +## 收包缓冲的所有权 + +**源码／布局不兼容变更：** `` 提供的 `ark::buffer` 仅支持移动。复制构造和复制赋值被显式删除，避免两个对象重复释放同一数组。Core／Net 消费者及其插件必须一起重建。仓库中的 Net Session 原本就禁止复制，并在对象内部持有缓冲；复制 `shared_ptr` Session 句柄不会复制该缓冲。这次仓库调用检查不能证明外部消费者从未使用原有隐式复制操作。 + +移动构造和移动赋值均为 `noexcept`，转移数组、未读数据、容量和游标状态。移动赋值释放目标原有数组；自身移动保持原状态。移出对象的容量和未读长度均为零，数据指针为空，可继续销毁、赋值或写入。原有公开方法签名继续保留。独立构造默认容量仍为 512 KiB，Net Session 继续使用独立配置的 4 KiB 默认容量。 + +```cpp +ark::buffer pending(4096); +pending.write("abc", 3); +ark::buffer processing(std::move(pending)); // 转移所有权 +pending.write("next", 4); // 重新申请存储 +``` + +使用 `std::move` 时包含 ``。`get_data()` 返回当前数组中的借用指针；所属对象必须保持有效，在压缩、扩容或替换所有权后重新获取指针。本次不增加 clone API。需要独立数据的应用可在源数据有效期间，将可读字节显式写入另一个缓冲。 + +缓冲通过 `ARK_NEW_ARRAY`／`ARK_DELETE_ARRAY` 保留最小分配兼容边界，构造、扩容和释放现在使用同一对钩子。这修正了原扩容路径通过裸 `new[]`／`delete[]` 绕过自定义钩子的问题。覆盖钩子时，须在包含头文件前定义匹配且释放时不抛异常的实现，并确保各编译单元定义一致。默认分配仍使用会抛异常的 `new[]`：`std::bad_alloc` 继续向外传播，扩容失败保留原有数据、容量和游标。自定义分配器返回空指针时，构造得到容量为零的空缓冲；`write()` 返回 `false`，不改变已有状态。 + +零长度写入成功且不解引用传入指针；非零长度的空源指针返回 `false`。移除数量超过未读长度时不改变状态，包括会使加法溢出的数量。容量无法表示时，在分配前抛出 `std::length_error`。正常读写、压缩和容量增长行为保持不变。 + +开启 `ARK_BUILD_CORE_TESTS=ON` 后，`check-buffer` 运行生命周期／移动、压缩／扩容、边界、默认分配确定性失败及独立自定义钩子回归，并纳入 `check-core`。分配失败测试只使一次小数组申请失败，不申请巨量内存。`check-network` 保留既有 Net 拆包与分片输入覆盖。这些检查不代表服务或容量验收。 + ## 成员回调与借用参数 端点构造函数与客户端／服务端服务的注册接口通过 C++20 `std::bind_front`， diff --git a/src/core/buffer.hpp b/src/core/buffer.hpp index 7af5763a9..1483967dc 100644 --- a/src/core/buffer.hpp +++ b/src/core/buffer.hpp @@ -21,8 +21,11 @@ #pragma once #include +#include #include +#include #include +#include #include "ark/macros.hpp" @@ -30,17 +33,49 @@ namespace ark { class buffer final { + struct data_deleter + { + void operator()(char* value) const noexcept { + ARK_DELETE_ARRAY(char, value); + } + }; + + using owned_data = std::unique_ptr; + + static char* allocate_data(size_t size) { + return ARK_NEW_ARRAY(char, size); + } + public: explicit buffer(size_t buffer_size = 1024 * 512) { - data_ = ARK_NEW_ARRAY(char, buffer_size); + data_.reset(allocate_data(buffer_size)); if (data_ != nullptr) { data_size_ = buffer_size; } } - ~buffer() { - ARK_DELETE_ARRAY(char, data_); + ~buffer() = default; + + buffer(const buffer&) = delete; + buffer& operator=(const buffer&) = delete; + + buffer(buffer&& other) noexcept + : data_(std::move(other.data_)) + , data_size_(std::exchange(other.data_size_, 0)) + , write_pos_(std::exchange(other.write_pos_, 0)) + , read_pos_(std::exchange(other.read_pos_, 0)) { + } + + buffer& operator=(buffer&& other) noexcept { + if (this != &other) + { + data_ = std::move(other.data_); + data_size_ = std::exchange(other.data_size_, 0); + write_pos_ = std::exchange(other.write_pos_, 0); + read_pos_ = std::exchange(other.read_pos_, 0); + } + return *this; } size_t get_capacity() const noexcept { @@ -48,39 +83,29 @@ class buffer final } bool write(const char* data, size_t len) { - bool write_ret = true; - - if (get_write_valid_count() >= len) + if (len == 0) { - memcpy(get_write_ptr(), data, len); - add_write_pos(len); + return true; } - else + if (data == nullptr) { - size_t left_len = data_size_ - get_length(); - + return false; + } + if (get_write_valid_count() < len) + { + const size_t left_len = data_size_ - get_length(); if (left_len >= len) { adjust_to_head(); - write(data, len); } - else + else if (!grow(len - left_len)) { - size_t need_len = len - left_len; - - if (need_len > 0) - { - grow(need_len); - write(data, len); - } - else - { - write_ret = false; - } + return false; } } - - return write_ret; + std::memcpy(get_write_ptr(), data, len); + add_write_pos(len); + return true; } size_t get_length() { @@ -90,7 +115,7 @@ class buffer final char* get_data() { if (read_pos_ < data_size_) { - return data_ + read_pos_; + return data_.get() + read_pos_; } else { @@ -99,11 +124,9 @@ class buffer final } void remove_data(size_t value) { - size_t temp = read_pos_ + value; - - if (temp <= data_size_) + if (value <= get_length()) { - read_pos_ = temp; + read_pos_ += value; } } @@ -120,7 +143,7 @@ class buffer final if (len > 0) { - memmove(data_, data_ + read_pos_, len); + std::memmove(data_.get(), data_.get() + read_pos_, len); } read_pos_ = 0; @@ -156,7 +179,7 @@ class buffer final char* get_write_ptr() { if (write_pos_ < data_size_) { - return data_ + write_pos_; + return data_.get() + write_pos_; } else { @@ -164,7 +187,7 @@ class buffer final } } - void grow(size_t len) { + bool grow(size_t len) { if (len > std::numeric_limits::max() - data_size_) { throw std::length_error("Receive buffer size exceeds the addressable range"); @@ -172,18 +195,24 @@ class buffer final const size_t required = data_size_ + len; const size_t doubled = data_size_ <= std::numeric_limits::max() / 2 ? data_size_ * 2 : required; const size_t n = std::max(required, doubled); - char* d = new char[n]; - + owned_data candidate(allocate_data(n)); + if (!candidate) + { + return false; + } const size_t unread = get_length(); - memcpy(d, data_ + read_pos_, unread); + if (unread > 0) + { + std::memcpy(candidate.get(), data_.get() + read_pos_, unread); + } + data_ = std::move(candidate); read_pos_ = 0; write_pos_ = unread; data_size_ = n; - delete[] data_; - data_ = d; + return true; } - char* data_{nullptr}; + owned_data data_; size_t data_size_{0}; size_t write_pos_{0}; diff --git a/tests/core/CMakeLists.txt b/tests/core/CMakeLists.txt index b162013ad..5766fb32b 100644 --- a/tests/core/CMakeLists.txt +++ b/tests/core/CMakeLists.txt @@ -32,6 +32,21 @@ add_custom_target(check-logging USES_TERMINAL) add_dependencies(check-core check-logging) +# Separate binaries keep custom allocation hooks and new[] replacement isolated. +add_executable(ark_buffer_tests main.cpp buffers.cpp) +set_property(TARGET ark_buffer_tests PROPERTY LINK_LIBRARIES ark::core) +add_executable(ark_buffer_failure_tests main.cpp buffer_failure.cpp) +set_property(TARGET ark_buffer_failure_tests PROPERTY LINK_LIBRARIES ark::core) +add_executable(ark_buffer_hook_tests main.cpp buffer_hooks.cpp) +set_property(TARGET ark_buffer_hook_tests PROPERTY LINK_LIBRARIES ark::core) +add_custom_target(check-buffer + COMMAND $ + COMMAND $ + COMMAND $ + DEPENDS ark_buffer_tests ark_buffer_failure_tests ark_buffer_hook_tests + USES_TERMINAL VERBATIM) +add_dependencies(check-core check-buffer) + find_package(Threads REQUIRED) add_library(ark_task_header_checks OBJECT headers/task.cpp) # This SDK consumer uses the public core target without aggregate plugin APIs. diff --git a/tests/core/buffer_failure.cpp b/tests/core/buffer_failure.cpp new file mode 100644 index 000000000..84b63c804 --- /dev/null +++ b/tests/core/buffer_failure.cpp @@ -0,0 +1,68 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include +#include +#include + +#include "../network/doctest.h" + +#include + +namespace { + +bool fail_next_array{}; + +} // namespace + +// Fail one ordinary array allocation deterministically, without exhausting memory. +void* operator new[](std::size_t size) { + if (fail_next_array) + { + fail_next_array = false; + throw std::bad_alloc{}; + } + if (auto* result = std::malloc(size ? size : 1)) + { + return result; + } + throw std::bad_alloc{}; +} + +void operator delete[](void* value) noexcept { + std::free(value); +} + +void operator delete[](void* value, std::size_t) noexcept { + std::free(value); +} + +TEST_CASE("Buffer default construction allocation propagates bad_alloc") { + fail_next_array = true; + bool caught{}; + try + { ark::buffer candidate(8); } + catch (const std::bad_alloc&) + { caught = true; } + CHECK_FALSE(fail_next_array); + CHECK(caught); +} + +TEST_CASE("Buffer default growth allocation propagates bad_alloc and retains unread state") { + ark::buffer value(4); + REQUIRE(value.write("abc", 3)); + value.remove_data(1); + const auto* unread = value.get_data(); + fail_next_array = true; + bool caught{}; + try + { value.write("defg", 4); } + catch (const std::bad_alloc&) + { caught = true; } + CHECK_FALSE(fail_next_array); + CHECK(caught); + CHECK(value.get_capacity() == 4); + CHECK(value.get_data() == unread); + CHECK(value.get_length() == 2); + CHECK(std::string(value.get_data(), value.get_length()) == "bc"); + REQUIRE(value.write("defg", 4)); + CHECK(std::string(value.get_data(), value.get_length()) == "bcdefg"); +} diff --git a/tests/core/buffer_hooks.cpp b/tests/core/buffer_hooks.cpp new file mode 100644 index 000000000..54a7dd83c --- /dev/null +++ b/tests/core/buffer_hooks.cpp @@ -0,0 +1,151 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include +#include +#include +#include +#include + +#include "../network/doctest.h" + +namespace { + +std::array live{}; +unsigned allocations{}; +unsigned releases{}; +bool return_null{}; +bool throw_allocation{}; + +char* allocate_bytes(size_t size) { + if (throw_allocation) + { + throw_allocation = false; + throw std::bad_alloc{}; + } + if (return_null) + { + return_null = false; + return nullptr; + } + auto* result = static_cast(std::malloc(size ? size : 1)); + if (!result) + { + throw std::bad_alloc{}; + } + for (auto& entry : live) + { + if (!entry) + { + entry = result; + ++allocations; + return result; + } + } + std::abort(); +} + +void release_bytes(char*& value) noexcept { + for (auto& entry : live) + { + if (entry == value) + { + entry = nullptr; + std::free(value); + value = nullptr; + ++releases; + return; + } + } + std::abort(); +} + +} // namespace + +// A separate executable prevents differently expanded inline definitions from mixing. +#define ARK_NEW_ARRAY(element_type, size) allocate_bytes(size) +#define ARK_DELETE_ARRAY(element_type, pointer) release_bytes(pointer) +#include + +TEST_CASE("Buffer uses matching custom hooks for growth replacement moves and destruction") { + const auto initial_allocations = allocations; + const auto initial_releases = releases; + { + ark::buffer source(4); + REQUIRE(source.write("abc", 3)); + source.remove_data(2); + REQUIRE(source.write("defg", 4)); + CHECK(allocations == initial_allocations + 2); + CHECK(releases == initial_releases + 1); + CHECK(std::string(source.get_data(), source.get_length()) == "cdefg"); + ark::buffer destination(2); + destination = std::move(source); + CHECK(releases == initial_releases + 2); + auto* same = &destination; + destination = std::move(*same); + CHECK(releases == initial_releases + 2); + ark::buffer moved(std::move(destination)); + CHECK(allocations == initial_allocations + 3); + CHECK(releases == initial_releases + 2); + REQUIRE(source.write("r", 1)); + CHECK(std::string(moved.get_data(), moved.get_length()) == "cdefg"); + } + CHECK(allocations - initial_allocations == 4); + CHECK(releases - initial_releases == 4); + for (const auto* entry : live) + { + CHECK(entry == nullptr); + } +} + +TEST_CASE("Buffer null custom allocation preserves the empty constructor and failed write states") { + const auto initial_releases = releases; + return_null = true; + { + ark::buffer empty(4); + CHECK(empty.get_capacity() == 0); + CHECK(empty.get_length() == 0); + CHECK(empty.get_data() == nullptr); + CHECK(empty.write(nullptr, 0)); + return_null = true; + CHECK_FALSE(empty.write("x", 1)); + CHECK(empty.get_capacity() == 0); + CHECK(empty.get_length() == 0); + REQUIRE(empty.write("x", 1)); + CHECK(empty.get_capacity() == 1); + } + CHECK(releases == initial_releases + 1); + { + ark::buffer value(4); + REQUIRE(value.write("abc", 3)); + value.remove_data(1); + const auto* unread = value.get_data(); + return_null = true; + CHECK_FALSE(value.write("defg", 4)); + CHECK(value.get_capacity() == 4); + CHECK(value.get_data() == unread); + CHECK(value.get_length() == 2); + CHECK(std::string(value.get_data(), value.get_length()) == "bc"); + REQUIRE(value.write("defg", 4)); + CHECK(std::string(value.get_data(), value.get_length()) == "bcdefg"); + } +} + +TEST_CASE("Buffer throwing custom growth allocation preserves state and matching release ownership") { + const auto initial_allocations = allocations; + const auto initial_releases = releases; + { + ark::buffer value(4); + REQUIRE(value.write("abc", 3)); + value.remove_data(1); + const auto* unread = value.get_data(); + throw_allocation = true; + CHECK_THROWS_AS(value.write("defg", 4), std::bad_alloc); + CHECK_FALSE(throw_allocation); + CHECK(value.get_capacity() == 4); + CHECK(value.get_data() == unread); + CHECK(value.get_length() == 2); + CHECK(std::string(value.get_data(), value.get_length()) == "bc"); + CHECK(allocations == initial_allocations + 1); + CHECK(releases == initial_releases); + } + CHECK(releases == initial_releases + 1); +} diff --git a/tests/core/buffers.cpp b/tests/core/buffers.cpp new file mode 100644 index 000000000..94027f089 --- /dev/null +++ b/tests/core/buffers.cpp @@ -0,0 +1,177 @@ +// Copyright (c) 2026 OpenArkStudio. Licensed under Apache-2.0. See LICENSE. +#include +#include +#include +#include +#include +#include + +#include "../network/doctest.h" + +#include + +static_assert(!std::is_copy_constructible_v); +static_assert(!std::is_copy_assignable_v); +static_assert(std::is_nothrow_move_constructible_v); +static_assert(std::is_nothrow_move_assignable_v); + +namespace { + +std::string contents(ark::buffer& value) { + return value.get_length() ? std::string(value.get_data(), value.get_length()) : std::string{}; +} + +} // namespace + +TEST_CASE("Buffer retains default explicit and zero capacities with empty writes") { + ark::buffer defaults; + CHECK(defaults.get_capacity() == 512 * 1024); + CHECK(defaults.get_length() == 0); + ark::buffer configured(7); + CHECK(configured.get_capacity() == 7); + CHECK(configured.get_length() == 0); + CHECK(configured.write(nullptr, 0)); + CHECK(configured.get_length() == 0); + CHECK_FALSE(configured.write(nullptr, 1)); + CHECK(configured.get_length() == 0); + ark::buffer zero(0); + CHECK(zero.get_capacity() == 0); + CHECK(zero.get_data() == nullptr); + CHECK(zero.write(nullptr, 0)); + CHECK(zero.get_capacity() == 0); + REQUIRE(zero.write("abc", 3)); + CHECK(zero.get_capacity() == 3); + CHECK(contents(zero) == "abc"); +} + +TEST_CASE("Buffer compacts unread binary bytes without growing") { + ark::buffer value(8); + const char bytes[] = {'a', 'b', '\0', 'd', 'e', 'f'}; + REQUIRE(value.write(bytes, sizeof(bytes))); + const auto* allocation = value.get_data(); + value.remove_data(2); + REQUIRE(value.get_length() == 4); + REQUIRE(value.write("XY", 2)); + CHECK(value.get_capacity() == 8); + value.remove_data(2); + REQUIRE(value.write("1234", 4)); + CHECK(value.get_data() == allocation); + CHECK(value.get_capacity() == 8); + CHECK(contents(value) == "efXY1234"); + value.remove_data(8); + CHECK(value.get_length() == 0); + CHECK(value.write("z", 1)); + CHECK(value.get_capacity() == 8); + CHECK(contents(value) == "z"); +} + +TEST_CASE("Buffer growth retains unread bytes and existing capacity selection") { + ark::buffer value(4); + REQUIRE(value.write("abc", 3)); + value.remove_data(2); + REQUIRE(value.write("defg", 4)); + CHECK(contents(value) == "cdefg"); + CHECK(value.get_capacity() == 8); + value.remove_data(5); + REQUIRE(value.write("123456789", 9)); + CHECK(contents(value) == "123456789"); + CHECK(value.get_capacity() == 16); + value.remove_data(8); + REQUIRE(value.write("abcdef", 6)); + CHECK(contents(value) == "9abcdef"); + CHECK(value.get_capacity() == 16); +} + +TEST_CASE("Buffer move construction transfers unread state and leaves a reusable source") { + ark::buffer source(8); + REQUIRE(source.write("abcdef", 6)); + source.remove_data(2); + const auto* unread = source.get_data(); + ark::buffer destination(std::move(source)); + CHECK(destination.get_data() == unread); + CHECK(destination.get_capacity() == 8); + CHECK(contents(destination) == "cdef"); + CHECK(source.get_capacity() == 0); + CHECK(source.get_length() == 0); + CHECK(source.get_data() == nullptr); + CHECK(source.write(nullptr, 0)); + source.remove_data(std::numeric_limits::max()); + CHECK(source.get_length() == 0); + REQUIRE(source.write("reuse", 5)); + CHECK(contents(source) == "reuse"); + CHECK(contents(destination) == "cdef"); +} + +TEST_CASE("Buffer move assignment replaces ownership and self move preserves contents") { + ark::buffer source(12); + REQUIRE(source.write("source", 6)); + source.remove_data(1); + const auto* unread = source.get_data(); + ark::buffer destination(4); + REQUIRE(destination.write("old", 3)); + destination = std::move(source); + CHECK(destination.get_data() == unread); + CHECK(destination.get_capacity() == 12); + CHECK(contents(destination) == "ource"); + CHECK(source.get_capacity() == 0); + CHECK(source.get_length() == 0); + CHECK(source.get_data() == nullptr); + auto* same = &destination; + destination = std::move(*same); + CHECK(destination.get_data() == unread); + CHECK(contents(destination) == "ource"); + REQUIRE(source.write("new", 3)); + CHECK(contents(source) == "new"); + ark::buffer empty(std::move(source)); + source = std::move(destination); + CHECK(contents(source) == "ource"); + destination = std::move(source); + CHECK(contents(destination) == "ource"); + CHECK(source.get_length() == 0); + CHECK(contents(empty) == "new"); +} + +TEST_CASE("Buffer assignment from an empty moved source releases the previous allocation") { + ark::buffer survivor(2); + { + ark::buffer source(8); + REQUIRE(source.write("kept", 4)); + survivor = std::move(source); + } + CHECK(contents(survivor) == "kept"); + ark::buffer source(std::move(survivor)); + source = std::move(survivor); + CHECK(source.get_capacity() == 0); + CHECK(source.get_length() == 0); + CHECK(source.get_data() == nullptr); + REQUIRE(source.write("again", 5)); + CHECK(contents(source) == "again"); +} + +TEST_CASE("Buffer owning storage survives container relocation") { + std::vector values; + values.emplace_back(4); + REQUIRE(values.front().write("first", 5)); + const auto* stored = values.front().get_data(); + values.reserve(values.capacity() + 1); + CHECK(values.front().get_data() == stored); + CHECK(contents(values.front()) == "first"); +} + +TEST_CASE("Buffer rejects invalid consumption and overflowing growth without mutating state") { + ark::buffer value(8); + REQUIRE(value.write("abc", 3)); + value.remove_data(4); + CHECK(contents(value) == "abc"); + value.remove_data(1); + const auto* unread = value.get_data(); + value.remove_data(std::numeric_limits::max()); + CHECK(value.get_data() == unread); + CHECK(contents(value) == "bc"); + CHECK_THROWS_AS(value.write("x", std::numeric_limits::max()), std::length_error); + CHECK(value.get_data() == unread); + CHECK(contents(value) == "bc"); + CHECK(value.get_capacity() == 8); + REQUIRE(value.write("de", 2)); + CHECK(contents(value) == "bcde"); +} From 6697839ae189cf308063b6f3e44572c6c9ae4f56 Mon Sep 17 00:00:00 2001 From: nick Date: Mon, 26 Oct 2026 16:27:33 +0800 Subject: [PATCH 17/22] perf(buffer): isolate write capacity preparation Move compaction and growth preparation into a private helper behind the C++20 unlikely branch for insufficient tail space. Keep allocation and exception handling off the steady receive-write path without changing any ownership, bounds, hook or failure contract. AppleClang Release disassembly moves grow out of the framing loop and reduces the local benchmark's parse_frames body from 335 to 270 instructions. The bounded timing comparison remains separate evidence and is retained locally. --- src/core/buffer.hpp | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/src/core/buffer.hpp b/src/core/buffer.hpp index 1483967dc..561aedc4a 100644 --- a/src/core/buffer.hpp +++ b/src/core/buffer.hpp @@ -91,14 +91,9 @@ class buffer final { return false; } - if (get_write_valid_count() < len) + if (get_write_valid_count() < len) [[unlikely]] { - const size_t left_len = data_size_ - get_length(); - if (left_len >= len) - { - adjust_to_head(); - } - else if (!grow(len - left_len)) + if (!prepare_write(len)) { return false; } @@ -131,6 +126,16 @@ class buffer final } private: + bool prepare_write(size_t len) { + const size_t left_len = data_size_ - get_length(); + if (left_len >= len) + { + adjust_to_head(); + return true; + } + return grow(len - left_len); + } + void adjust_to_head() { size_t len = 0; From 9923f33b230e052cacceb09bff21f80c95c5027f Mon Sep 17 00:00:00 2001 From: nick Date: Mon, 26 Oct 2026 16:47:45 +0800 Subject: [PATCH 18/22] fix(lifecycle): retain libraries after native unload failure --- docs/guides/applications.md | 2 +- docs/guides/applications_CN.md | 2 +- docs/guides/tasks.md | 6 +- docs/guides/tasks_CN.md | 6 +- src/core/dynamic_library.hpp | 31 ++++- src/core/plugin_manager.hpp | 167 ++++++++++++----------- tests/core/CMakeLists.txt | 10 ++ tests/core/README.md | 8 ++ tests/core/README_CN.md | 8 ++ tests/core/dynamic_library.cpp | 241 +++++++++++++++++++++++++++++++++ 10 files changed, 397 insertions(+), 84 deletions(-) create mode 100644 tests/core/dynamic_library.cpp diff --git a/docs/guides/applications.md b/docs/guides/applications.md index 0e7c7dc7f..785fd81ba 100644 --- a/docs/guides/applications.md +++ b/docs/guides/applications.md @@ -89,7 +89,7 @@ On macOS/Linux, Ctrl+C or `SIGTERM` requests shutdown through the application's After task completion and the active callback stack returns, the manager waits for every module to report `ready_to_shut()`, then synchronously invokes shutdown hooks and unloading. A module must use its owner-frame readiness check to stop/drain native work and confirm native thread exit before allowing unload. When all hooks succeed, Timers logs `PreShut` and then `Shut`, and the launcher exits. A hook may block; the handler-count budget is not a shutdown-duration bound. -If a shutdown hook returns `false` or throws, the launcher reports its phase, module and available exception details once, keeps the process/resources alive, and does not continue unloading. `plugin_manager::update()` returning `false` or throwing during drain also locks the launcher into retention; owner updates continue where possible, but later completion cannot resume unloading. +If a shutdown hook returns `false` or throws, the launcher reports its phase, module and available exception details once, keeps the process/resources alive, and does not continue unloading. A failed native library close similarly reports `unload: `, retains that library handle and wrapper, and stops before closing later libraries. Repeated `stop()` stays failed without repeating plugin exits or close calls. Earlier shutdown work and successful library closes are not rolled back. A successful `FreeLibrary` or `dlclose` call does not prove that the OS library image was unmapped or that arbitrary hot update is safe. `plugin_manager::update()` returning `false` or throwing during drain also locks the launcher into retention; owner updates continue where possible, but later completion cannot resume unloading. `--shutdown_timeout_ms=N` configures a non-negative int64 wait in milliseconds; the default `0` waits indefinitely. A positive limit uses a monotonic clock and reports elapsed time, pending managed-task count and the waiting phase once. On expiry the process and resources stay alive, owner updates continue, and unloading remains disabled even if work completes later. This is a cooperative owner-loop wait limit: it cannot interrupt a blocking user `update()`, readiness check, shutdown hook or unload call. It is not a hard process-stop deadline, cancellation policy or successful shutdown. This managed-task drain does not prove that arbitrary callbacks, native pool workers or background cleanup have finished; see [module task integration](tasks.md#shared-business-executor-for-modules). Timers's logs alone are not a readiness or shutdown guarantee for every plugin combination. diff --git a/docs/guides/applications_CN.md b/docs/guides/applications_CN.md index 8d18350d9..abf3977fd 100644 --- a/docs/guides/applications_CN.md +++ b/docs/guides/applications_CN.md @@ -89,7 +89,7 @@ macOS/Linux 可使用 Ctrl+C 或 `SIGTERM`，由应用主循环处理停止请 任务完成且当前回调返回后，管理器等待所有模块的 `ready_to_shut()` 返回就绪，再同步执行关闭钩子与卸载。模块应在 owner frame 的就绪检查中停止并排空原生工作，确认原生线程退出后才允许卸载。所有钩子成功时，Timers 依次输出 `PreShut` 和 `Shut`，随后启动器退出。钩子可能阻塞；handler 数量预算不代表停机耗时上限。 -关闭钩子返回 `false` 或抛出异常时，启动器输出一次阶段、模块及可用的异常信息，保留进程和资源，不再继续卸载。排空期间的 `plugin_manager::update()` 返回 `false` 或抛出异常，也会锁定资源保留状态；启动器尽可能继续 owner 更新，但后续完成不能恢复卸载。 +关闭钩子返回 `false` 或抛出异常时，启动器输出一次阶段、模块及可用的异常信息，保留进程和资源，不再继续卸载。原生动态库关闭失败时同样报告 `unload: `，保留该库句柄与包装对象，并停止关闭后续库。重复 `stop()` 继续返回失败，不重复执行插件出口或关闭调用。先前完成的停机工作与成功关闭的库不会回滚。`FreeLibrary` 或 `dlclose` 成功不证明操作系统已移除库映像，也不证明任意热更新安全。排空期间的 `plugin_manager::update()` 返回 `false` 或抛出异常，也会锁定资源保留状态；启动器尽可能继续 owner 更新，但后续完成不能恢复卸载。 `--shutdown_timeout_ms=N` 设置等待时限，单位为毫秒，接受非负 int64 整数；默认值 `0` 表示无限等待。正数使用单调时钟计时，超时后输出一次实际等待时间、待完成的业务任务数和等待阶段。进程与资源继续保留，owner 更新继续运行；即使工作随后完成，也不再卸载。这是 owner loop 的协作式等待时限，不能中断阻塞中的用户 `update()`、就绪检查、关闭钩子或卸载调用，不构成强制进程退出期限、自动取消策略或成功停机。业务任务排空不证明外部回调、原生连接池线程或后台清理已经完成，详见[模块任务集成](tasks_CN.md#模块共用业务执行器)。仅凭 Timers 的日志，不能推断所有插件组合都具备相同的就绪条件或停止保证。 diff --git a/docs/guides/tasks.md b/docs/guides/tasks.md index 8e3f6c0d6..f8e566ba0 100644 --- a/docs/guides/tasks.md +++ b/docs/guides/tasks.md @@ -72,11 +72,13 @@ Private database bridges use `manager.get_task_runtime()` to share that exact ex | --- | --- | | `running` | Update normally and accept tasks | | `draining` | Continue module updates and the central poll; retain business state and services | -| `failed` | A shutdown hook returned failure or threw; do not continue unloading, and retain process/resources | -| `stopped` | Shutdown hooks completed; repeated `stop()` returns `true` | +| `failed` | A shutdown hook failed/threw or a native library close failed; stop further unloading and retain remaining resources | +| `stopped` | Shutdown hooks and native close calls succeeded; repeated `stop()` returns `true` | Use `get_shutdown_state()`, `get_shutdown_failure()` and `get_shutdown_exception()` to distinguish waiting from failure. The launcher keeps pumping while tasks remain, and reports a hook failure once while retaining the process. There is no implicit shutdown timeout, cancellation, detach or frame destruction as a drain substitute. The pending-task check does not wait for completion. The poll budget limits handler count, not execution time, and does not bound user-hook or total shutdown latency. +`dynamic_library::unload()` returns `false` on a native close failure and keeps its handle. Success clears the handle; unloading an empty or already closed wrapper returns `true` without another native call. Manager failure records `unload: ` and latches the existing failed state, retaining the failed wrapper and remaining libraries. It does not retry plugin exits or closes on a later `stop()`, roll back previous teardown, or establish OS image removal. + Use the manager and its groups on their constructing thread. Do not mutate module registries or recursively update from update/shutdown hooks or executor callbacks. While any managed task is pending, no module can unregister: another module’s task may still borrow that provider. Destroying a manager with tasks, on an active callback stack or from another thread terminates; destruction is not a drain entry point.