Skip to content

Commit 4188620

Browse files
mcpplibs.mwrap.mimalloc 3.6.1: mimalloc-m (mimalloc as C++23 modules) (#664)
Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com>
1 parent f2b71fe commit 4188620

9 files changed

Lines changed: 229 additions & 0 deletions

File tree

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# Add mcpplibs.mwrap.mimalloc (mimalloc-m 3.6.1)
2+
3+
## Form
4+
5+
- Third-party upstream [microsoft/mimalloc](https://github.com/microsoft/mimalloc), release v3.6.1, MIT. The library is
6+
`compat.mimalloc` 3.6.1 of this index (`pkgs/c/compat.mimalloc.lua`), the archive openkal-musl 0.26.0 embeds.
7+
- Form A external mwrap module repository [mcpplibs/mimalloc-m](https://github.com/mcpplibs/mimalloc-m), package
8+
`mcpplibs.mwrap.mimalloc`, version 3.6.1 (first release of the identity), tag `mwrap-3.6.1` on the merged main commit
9+
(`0582ff7`); release asset `mimalloc-m-3.6.1.tar.gz` (`62ae5121…9abe`), CN mirror `gitcode.com/mcpp-res/mwrap-mimalloc`
10+
with identical bytes.
11+
- Implementation ownership: the library is compat.mimalloc's. The wrapper vendors no upstream code; it retains upstream's
12+
README.md and LICENSE under `upstream/mimalloc/` with checksums in `upstream/documents.json`, and deploys them with its
13+
own LICENSE/NOTICE (`runtime.deploy`). The adaptation a C library needs to embed mimalloc is in that C library
14+
(openkal-musl `port/src/mimalloc`), not here (mcpp design 2026-11-02 part 3 §7.1).
15+
16+
## Modules
17+
18+
- `mcpplibs.mwrap.mimalloc`: the functions, heaps, arenas, sub-processes, options and STL allocators of `<mimalloc.h>`
19+
by `export using`, in the global namespace where upstream declares them; it re-exports the three below.
20+
- `mcpplibs.mwrap.mimalloc.stats` (`<mimalloc-stats.h>`) and `mcpplibs.mwrap.mimalloc.profile` (`<mimalloc-profile.h>`);
21+
upstream's `static inline` helpers (`mi_malloc_csize`, `mi_zalloc_csize`, `mi_free_csize`, `mi_stats_init`,
22+
`mi_stats_header_init`) adapted under their names in `mcpplibs::mwrap::mimalloc`.
23+
- `mcpplibs.mwrap.mimalloc.macros`: the value macros (`MI_MALLOC_VERSION`, `MI_STAT_VERSION`, `MI_SMALL_SIZE_MAX` …)
24+
as constants. The macros that take a type (`mi_malloc_tp` and its eleven companions) and `mi_stats_t_decl` stay
25+
macros, in `<mwrap/mimalloc/macros.hpp>`.
26+
27+
## Verification
28+
29+
- Wrapper CI (Linux x86_64 GCC 16.1 and LLVM 23.1.3, macOS LLVM, Windows LLVM): the module tests, the export check
30+
(`tools/check_exports.py`) and upstream's `test-api` and `test-stress` over the same library; the default template in
31+
a workflow of its own.
32+
- `tests/mwrap/mimalloc-heap`: the real release archive through the candidate index; a block that grows and keeps its
33+
bytes, alignments to 64 KiB, a private heap visited and destroyed, a container over `mi_stl_allocator`, an option,
34+
blocks freed by another thread, the statistics; and the side header's typed macros beside the import.
35+
- `python3 tests/mwrap/check_installed_template.py --package mcpplibs.mwrap.mimalloc@3.6.1 --deploy`: the unmodified
36+
default `allocate` template from the real archive, then a relocated bundle whose `licenses/mimalloc/` holds the
37+
wrapper's LICENSE/NOTICE and mimalloc's README and LICENSE byte for byte.
38+
39+
## Motivation
40+
41+
mcpp design 2026-11-02 part 3 (§7, §7.1): openkal-musl 0.26 embeds mimalloc as the allocator of the ok image, and a
42+
program that wants mimalloc's own interface (heaps, options, statistics) imports it through this module, over the same
43+
`compat.mimalloc`.

‎.github/workflows/validate.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1666,6 +1666,11 @@ jobs:
16661666
shell: bash
16671667
run: python3 tests/mwrap/check_installed_template.py --mcpp "$MCPP" --package mcpplibs.mwrap.box2d@3.2.0 --deploy
16681668

1669+
- name: Installed mimalloc wrapper archive, actual default allocation template and its relocated notices
1670+
if: contains(format(' {0} ', env.MEMBERS), ' mimalloc-heap ') && matrix.toolchain == 'default'
1671+
shell: bash
1672+
run: python3 tests/mwrap/check_installed_template.py --mcpp "$MCPP" --package mcpplibs.mwrap.mimalloc@3.6.1 --deploy
1673+
16691674
- name: Installed libwebp wrapper archive, actual default image template and its relocated notices
16701675
if: contains(format(' {0} ', env.MEMBERS), ' libwebp-image ') && matrix.toolchain == 'default'
16711676
shell: bash

‎CHANGELOG.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,12 @@
1515

1616
### Added
1717

18+
- **mcpplibs.mwrap.mimalloc 3.6.1(新包)。** mimalloc-m:以 C++23 模块(`import mcpplibs.mwrap.mimalloc;`)使用 compat.mimalloc
19+
3.6.1。`<mimalloc.h>` 的函数、堆、arena、子进程、选项与 STL 分配器以上游的全局名导出;`.stats`、`.profile` 两个子模块对应
20+
`<mimalloc-stats.h>`、`<mimalloc-profile.h>`;上游的 `static inline` 辅助函数以同名适配在 `mcpplibs::mwrap::mimalloc`;值宏在
21+
`.macros` 中为常量,带类型参数的宏与 `mi_stats_t_decl` 留在 `<mwrap/mimalloc/macros.hpp>`。消费者测试
22+
`tests/mwrap/mimalloc-heap` 与已安装模板检查(`allocate` 模板与重定位后的许可文件)。GLOBAL 为 GitHub 发布资产,CN 为
23+
GitCode `mcpp-res/mwrap-mimalloc` 镜像,sha256 两处一致。
1824
- **openkal-ld/run/tools/bind 0.3.0。** openkal 0.21 的一轮:okld 把不可写、文件与内存大小相同、与文件偏移模 64 KiB 同余且独占所在
1925
粒度的段从映像文件映射(`okld_host::map`,即 `kal_exec_map_part`),其余段照旧复制;每个重定位须写入可写段,否则拒绝映像;
2026
okrun 在程序运行期间保持映像文件打开。测试成员 `openkal-tools`、`openkal-bind` 随之移动。GLOBAL 为 GitHub 标签归档,CN 为

‎mcpp.toml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ members = [
3030
"tests/mwrap/opus-ogg",
3131
"tests/mwrap/libvpx-webm",
3232
"tests/mwrap/box2d-physics",
33+
"tests/mwrap/mimalloc-heap",
3334
"tests/mwrap/miniaudio-engine",
3435
"tests/mwrap/freetype-render",
3536
"tests/mwrap/harfbuzz-shape",

‎pkgs/mwrap/mimalloc.lua‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
-- Immutable upstream-aligned mwrap source release (mimalloc-m mwrap-3.6.1);
2+
-- GLOBAL and CN bytes are identical.
3+
--
4+
-- The wrapper is Apache-2.0; the library it links, compat.mimalloc (mimalloc
5+
-- 3.6.1), is MIT, linked statically: the package deploys mimalloc's README and
6+
-- LICENSE beside programs (runtime.deploy).
7+
package = {
8+
spec = "1",
9+
namespace = "mcpplibs.mwrap",
10+
name = "mimalloc",
11+
description = "mimalloc-m: mimalloc 3.6.1 (Microsoft's general-purpose allocator) as C++23 modules: its heaps, options, statistics, profiler and STL allocators",
12+
licenses = { "Apache-2.0" },
13+
repo = "https://github.com/mcpplibs/mimalloc-m",
14+
type = "package",
15+
extensions = {
16+
autoupdate = {
17+
tag = "mwrap-${version}",
18+
url = "https://github.com/mcpplibs/mimalloc-m/releases/download/${tag}/mimalloc-m-${version}.tar.gz",
19+
},
20+
},
21+
xpm = {
22+
linux = {
23+
["3.6.1"] = {
24+
url = {
25+
GLOBAL = "https://github.com/mcpplibs/mimalloc-m/releases/download/mwrap-3.6.1/mimalloc-m-3.6.1.tar.gz",
26+
CN = "https://gitcode.com/mcpp-res/mwrap-mimalloc/releases/download/3.6.1/mimalloc-m-3.6.1.tar.gz",
27+
},
28+
sha256 = "62ae512171fcbff73f557b1b1af505850d6e87a482a63cd956a409a1fe569abe",
29+
},
30+
},
31+
macosx = {
32+
["3.6.1"] = {
33+
url = {
34+
GLOBAL = "https://github.com/mcpplibs/mimalloc-m/releases/download/mwrap-3.6.1/mimalloc-m-3.6.1.tar.gz",
35+
CN = "https://gitcode.com/mcpp-res/mwrap-mimalloc/releases/download/3.6.1/mimalloc-m-3.6.1.tar.gz",
36+
},
37+
sha256 = "62ae512171fcbff73f557b1b1af505850d6e87a482a63cd956a409a1fe569abe",
38+
},
39+
},
40+
windows = {
41+
["3.6.1"] = {
42+
url = {
43+
GLOBAL = "https://github.com/mcpplibs/mimalloc-m/releases/download/mwrap-3.6.1/mimalloc-m-3.6.1.tar.gz",
44+
CN = "https://gitcode.com/mcpp-res/mwrap-mimalloc/releases/download/3.6.1/mimalloc-m-3.6.1.tar.gz",
45+
},
46+
sha256 = "62ae512171fcbff73f557b1b1af505850d6e87a482a63cd956a409a1fe569abe",
47+
},
48+
},
49+
},
50+
}

‎tests/mwrap/check_installed_template.py‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -247,6 +247,26 @@ def main():
247247
notice = (roots[0] / "NOTICE").read_bytes()
248248
assert b"mcpp-community" in notice and b"Erin Catto" in notice
249249
print(f"Verified {len(declared)} byte-identical wrapper/Box2D license files", flush=True)
250+
elif identity == "mcpplibs.mwrap.mimalloc":
251+
# mimalloc is MIT and linked statically: its LICENSE and README
252+
# travel with the program beside the wrapper's LICENSE/NOTICE.
253+
sources = [path for store in (home / "xpkgs", home / "registry/data/xpkgs")
254+
for path in store.glob(f"mcpplibs.mwrap-x-mimalloc/{version}/*/src/mimalloc.cppm")]
255+
assert len(sources) == 1, sources
256+
source = sources[0].parent.parent
257+
declared = tomllib.loads((source / "mcpp.toml").read_text(encoding="utf-8"))["runtime"]["deploy"]
258+
roots = list(relocated.rglob("licenses/mimalloc"))
259+
assert len(roots) == 1, f"Missing mimalloc runtime license tree: {roots}"
260+
for entry in declared:
261+
notice = source / entry["from"]
262+
copied = roots[0].parent.parent / entry["to"] / notice.name
263+
assert copied.is_file() and copied.read_bytes() == notice.read_bytes(), copied
264+
assert b"MIT License" in (roots[0] / "upstream/mimalloc/LICENSE").read_bytes()
265+
assert b"Microsoft Corporation" in (roots[0] / "upstream/mimalloc/LICENSE").read_bytes()
266+
assert b"Apache License" in (roots[0] / "LICENSE").read_bytes()
267+
notice = (roots[0] / "NOTICE").read_bytes()
268+
assert b"mcpp-community" in notice and b"Daan Leijen" in notice
269+
print(f"Verified {len(declared)} byte-identical wrapper/mimalloc license files", flush=True)
250270
elif identity == "mcpplibs.mwrap.libwebp":
251271
# libwebp (BSD-3-Clause with Google's patent grant) is linked
252272
# statically: the wrapper's LICENSE/NOTICE and libwebp's README,
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
[package]
2+
name = "mimalloc-heap"
3+
version = "3.6.1"
4+
description = "Consume the immutable mimalloc wrapper: private heaps visited and destroyed, aligned blocks, an STL container over mimalloc, options and statistics, threads that free each other's blocks, and the side header beside the import"
5+
6+
[indices]
7+
# Use the complete candidate built-in index prepared by tests/run_members.sh.
8+
# An explicit empty table overrides the legacy workspace redirect.
9+
10+
[dependencies]
11+
"mcpplibs.mwrap.mimalloc" = "3.6.1"
Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
// The installed wrapper, as a consumer reaches it: each check names what the
2+
// allocator promises.
3+
import std;
4+
import mcpplibs.mwrap.mimalloc;
5+
6+
namespace {
7+
int failures = 0;
8+
void check(bool ok, std::string_view what)
9+
{
10+
std::println("{} {}", ok ? "ok " : "FAIL", what);
11+
if (!ok) ++failures;
12+
}
13+
}
14+
15+
int main()
16+
{
17+
namespace m = mcpplibs::mwrap::mimalloc::macros;
18+
check(mi_version() == m::MI_MALLOC_VERSION && mi_version() == 30601, "the library is mimalloc 3.6.1");
19+
20+
void* p = mi_malloc(1000);
21+
std::memset(p, 0x5a, 1000);
22+
p = mi_realloc(p, 1 << 20);
23+
check(p && static_cast<unsigned char*>(p)[999] == 0x5a && mi_usable_size(p) >= (1u << 20),
24+
"a block grows and keeps its bytes");
25+
mi_free(p);
26+
27+
for (std::size_t align : {16u, 256u, 4096u, 65536u}) {
28+
void* a = mi_malloc_aligned(align + 3, align);
29+
check(a && reinterpret_cast<std::uintptr_t>(a) % align == 0, std::format("a block aligned to {}", align));
30+
mi_free(a);
31+
}
32+
33+
mi_heap_t* heap = mi_heap_new();
34+
std::vector<void*> blocks;
35+
for (int i = 0; i < 2000; ++i) blocks.push_back(mi_heap_malloc(heap, 8 + i % 700));
36+
std::size_t seen = 0;
37+
mi_heap_visit_blocks(heap, true, [](const mi_heap_t*, const mi_heap_area_t*, void* block, std::size_t, void* arg) {
38+
if (block) ++*static_cast<std::size_t*>(arg);
39+
return true;
40+
}, &seen);
41+
check(seen == blocks.size() && mi_heap_contains(heap, blocks.back()),
42+
"a private heap holds and visits every block it handed out");
43+
mi_heap_destroy(heap);
44+
45+
std::vector<std::string, mi_stl_allocator<std::string>> names;
46+
for (int i = 0; i < 5000; ++i) names.push_back(std::format("module{}.cppm", i));
47+
check(names.size() == 5000 && names[4999] == "module4999.cppm", "a container's elements live in mimalloc");
48+
49+
const long before = mi_option_get(mi_option_purge_delay);
50+
mi_option_set(mi_option_purge_delay, before + 5);
51+
check(mi_option_get(mi_option_purge_delay) == before + 5, "an option is set and read back");
52+
mi_option_set(mi_option_purge_delay, before);
53+
54+
// Blocks freed by a thread other than the one that allocated them, and the
55+
// memory taken again afterwards.
56+
std::vector<void*> handed(20000);
57+
std::thread([&] { for (auto& b : handed) b = mi_malloc(48); }).join();
58+
const bool all = std::ranges::all_of(handed, [](void* b) { return b != nullptr; });
59+
std::thread([&] { for (auto* b : handed) mi_free(b); }).join();
60+
mi_collect(true);
61+
void* again = mi_malloc(48);
62+
check(all && again != nullptr, "blocks allocated in one thread are freed in another");
63+
mi_free(again);
64+
65+
mi_stats_t stats;
66+
mcpplibs::mwrap::mimalloc::mi_stats_init(&stats);
67+
check(mi_stats_get(&stats) && stats.size == sizeof stats, "the process's statistics are read");
68+
69+
std::println("-- failures: {} --", failures);
70+
return failures == 0 ? 0 : 1;
71+
}
Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
// The side header from the installed package, beside the import: the typed
2+
// macros with the module's entities, one allocator under both spellings.
3+
#include <mwrap/mimalloc/macros.hpp>
4+
5+
import std;
6+
import mcpplibs.mwrap.mimalloc;
7+
8+
struct pair { long a, b; };
9+
10+
int main() {
11+
pair* p = mi_malloc_tp(pair);
12+
if (!p) return 1;
13+
p->a = 20; p->b = 22;
14+
const long sum = p->a + p->b;
15+
mi_free_tp(pair, p);
16+
pair* many = mi_mallocn_tp(pair, 64);
17+
if (!many || mi_usable_size(many) < 64 * sizeof(pair)) return 2;
18+
mi_free(many);
19+
mi_stats_t_decl(stats);
20+
if (static_cast<int>(stats.version) != mcpplibs::mwrap::mimalloc::macros::MI_STAT_VERSION) return 3;
21+
return sum == 42 ? 0 : 4;
22+
}

0 commit comments

Comments
 (0)