Skip to content

Make Kuna nicer to use as a library - #1602

Open
evanrichter wants to merge 4 commits into
Noelo-Lab:mainfrom
evanrichter:pr/librarification
Open

evanrichter wants to merge 4 commits into
Noelo-Lab:mainfrom
evanrichter:pr/librarification

Conversation

@evanrichter

Copy link
Copy Markdown

Hi Zion! I'm using Kuna as a library in a side project and it's been great except for just a few rough edges. Mainly removing the "cli-isms" like allowing options from environment vars and reading spec files from disk. Also being able to load headerless images that will support firmware or raw shellcode loading directly.

Also, with this PR it's possible to embed the spec files/dirs into the kuna cli directly so that separate files aren't needed in the distribution. (although with how infrequently they change, I'm not sure if the binary size tradeoff would be worth it).

One thing in the new code that might be a future footgun: LoadOptions uses thread locals to manage binary load options. In a single threaded async runtime, it's technically possible for callers to yield one kuna analysis with one set of LoadOptions to another async task that was set up with a different LoadOptions. I think in practice this usage would be rare, but certainly the errors would be strange and possibly look like unexpected decompilation failures. We went with the implementation tradeoff of being able to more closely match the current code that took options from env. LMK if you want a second pass at that.

An automated PR description follows:


[AUTOMATED] Load binaries as a library: in-memory specs, explicit LoadOptions, one Bootstrap builder

A program that embeds Kuna can now load many binaries in one process, on several threads, each with its own settings, from specs it holds in memory.

On main, each load scans a specs/ directory and reparses every .ldefs. Each load also reads its settings from process-wide KUNA_* environment variables, and an object load writes KUNA_GOVERSION, so two threads cannot load with different settings.

use kuna_console::engine::{language_database, Bootstrap};
use kuna_decomp::kuna_loadoptions::LoadOptions;
use kuna_decomp::kuna_memoryspecs::MemorySpecs;

// One language's .ldefs, .sla, .pspec and .cspec, compiled into the caller.
static X86_64: &[(&str, &[u8])] = &[/* ("x86.ldefs", include_bytes!(..)), .. */];

let db = language_database(&[MemorySpecs::new(X86_64)])?; // scans once
let options = LoadOptions { load_strings: false, ..Default::default() };
let load = Bootstrap::new(&db).options(&options); // reads no environment
std::thread::scope(|s| {
    for path in ["a.so", "b.so"] {
        s.spawn(move || inspect(load.object(path)));
    }
});

On main this does not compile. These tests show it working:

make specs
cd decompiler
cargo test -p kuna-console --test integration -- verify_memoryspecs:: verify_language_database:: verify_bootstrap::
cargo test -p kuna-console --test verify_loadoptions

What changed

  • MemorySpecs holds a compiled spec tree as (path, bytes) pairs. Each bootstrap gains a *_with_specs form that takes directories, memory trees or both. One language's files, flat, form a complete tree.
  • LoadOptions holds every load setting: the load-time catalog options, the fat slice, the .pdb, .fid and FLIRT paths, the Go release and the KUNA_LOAD_* switches. A load with explicit LoadOptions neither reads nor writes the environment. The Architecture keeps its LoadOptions and re-enters them for analysis and decompilation.
  • language_database(roots) scans the specs once, and the *_with_db bootstraps reuse its result.
  • Bootstrap::new(&languages).target(..).isa(..).options(..) replaces the growing set of bootstrap_* variants. It has one method per input: object, image, file, raw, te, te_bytes, mz_bytes, xml_image, xml_root. The builder is Copy, so threads can share it.
  • Existing callers see no change. Each bootstrap_* function keeps its signature and calls the builder. A load without LoadOptions reads the environment as before.

Tests

  • Specs in memory decompile the fauxware fixture as the specs/ directory does, and never fall back to the filesystem.
  • Loads on separate threads keep their own LoadOptions and leave the environment untouched.
  • One database serves two threads, and a bootstrap over it does not rescan the specs.
  • For every input and setting, the builder builds the same program as the old entry point.

No existing expectation moved: make test stays at 675/675 and make test-stages at its baseline.

evanrichter and others added 4 commits November 15, 2026 21:01
A library caller can now pass its compiled specs as bytes instead of
shipping a specs/ directory. MemorySpecs holds a spec tree as
(path, contents) pairs, over a static table or over owned data such as
Vec<u8> or Arc<[u8]>. SpecRoot names a directory or such a tree.

Each public bootstrap gains a *_with_specs form that takes any list of
spec roots, and the &[String] functions delegate to it.
LanguageDatabase owns the trees it was given, so nothing is registered
process-wide.

A tree need not follow Ghidra's layout: one language's .ldefs, .sla,
.pspec and .cspec, flat, form a complete root. Tests check that
memory roots decompile the fauxware fixture as the specs directory
does, never fall back to the filesystem, and mix with directory roots.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Library callers can now load several images at once in one process
with different settings. LoadOptions holds every load-time option
(ppc32picimports, ifuncfpret, ppcapu, kernelcache, goabi,
symbolnamechars, libctypes, calltargettype, macho-arm64e, sourcelang,
objcivars, objcsdk, dyldimage, dyldcrossrefs), the fat slice, the .pdb,
.fid and FLIRT paths, the Go release, and the load-pass and mapping
switches that KUNA_LOAD_OBJC, KUNA_LOAD_STRINGS and KUNA_LOAD_MMAP set.
bootstrap_from_object_with_options and bootstrap_from_image_with_options
put it in force on the loading thread, and each Architecture keeps its
options and re-enters them around analysis, decompilation, name lookup
and console commands. An explicit-options load neither reads nor writes
the environment; environment_fallbacks() counts reads that fell back to
it on a thread.

The entry points that take no options load from the environment,
publish KUNA_GOVERSION, and build programs that read the environment
live. set_kuna_option and reset_wire_defaults write the environment for
a later load.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A library that loads many binaries can now scan and parse the .ldefs
files once. language_database(spec_roots) scans the roots, and
bootstrap_from_object_with_db and bootstrap_from_image_with_db take the
database in place of the spec roots, alongside LoadOptions. A test
loads two images on two threads from one database and shows that a
bootstrap over it does not rescan the roots.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Kuna has 26 bootstrap_* entry points, one per combination
of input format, language source (spec roots or a scanned
LanguageDatabase), ARM ISA (environment, explicit) and load options
(environment, explicit). Each new setting doubled the set.

Bootstrap::new(&languages).target(..).arm_isa(..).options(&opts) now
holds those settings, and one method per input loads with them:
object, image, file, raw, te, te_bytes, mz_bytes, xml_image,
xml_root. LanguageSource, a sealed trait, accepts spec roots as a
slice, array or Vec, or a LanguageDatabase. The builder is Copy, so one
configured builder can load many inputs, on several threads.

Without options, a load puts the environment in force for that load on
the calling thread, and an unset ISA comes from KUNA_ARM_ISA. With
options it reads neither. Explicit options now also reach raw, TE, MZ
and XML loads.

Each bootstrap_* function becomes a one-line wrapper over the builder.
Tests check that explicit options reach every input without the
environment, that a builder without options reads the environment and
one with options does not, and that one builder loads the same program
on several threads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant