Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions .github/workflows/plugins.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,14 @@ jobs:
- run: zsh scripts/test-plugin-macos.sh
- run: |
python3 tools/plugins/fixture.py
build/plugins/plugin-surface-probe build/plugins/miucs-plugin-worker plugins/staff-note build/plugin-fixture.json
MIUCS_FORCE_CPU=1 build/plugins/plugin-surface-probe build/plugins/miucs-plugin-worker plugins/staff-note build/plugin-fixture.json
build/plugins/plugin-surface-probe build/plugins/miucs-plugin-worker plugins/pixel-keyboard build/plugin-fixture.json
MIUCS_FORCE_CPU=1 build/plugins/plugin-surface-probe build/plugins/miucs-plugin-worker plugins/pixel-keyboard build/plugin-fixture.json
build/plugins/plugin-surface-probe build/plugins/miucs-plugin-worker plugins/niulai-keyboard build/plugin-fixture.json
MIUCS_FORCE_CPU=1 build/plugins/plugin-surface-probe build/plugins/miucs-plugin-worker plugins/niulai-keyboard build/plugin-fixture.json
build/plugins/plugin-surface-probe build/plugins/miucs-plugin-worker plugins/muffin-puppies build/plugin-fixture.json
MIUCS_FORCE_CPU=1 build/plugins/plugin-surface-probe build/plugins/miucs-plugin-worker plugins/muffin-puppies build/plugin-fixture.json
build/plugins/plugin-surface-probe build/plugins/miucs-plugin-worker plugins/heiji-kazuha build/plugin-fixture.json
MIUCS_FORCE_CPU=1 build/plugins/plugin-surface-probe build/plugins/miucs-plugin-worker plugins/heiji-kazuha build/plugin-fixture.json
build/plugins/plugin-benchmark plugins build/plugin-fixture.json > build/plugin-benchmark.json
- uses: actions/upload-artifact@v4
if: always()
Expand Down Expand Up @@ -68,6 +70,10 @@ jobs:
if ($LASTEXITCODE -ne 0) { throw 'Muffin lifecycle validation failed' }
python tests/plugins/muffin_replay.py
if ($LASTEXITCODE -ne 0) { throw 'Muffin input replay failed' }
python tools/plugins/validate.py plugins/heiji-kazuha
if ($LASTEXITCODE -ne 0) { throw 'Heiji Kazuha lifecycle validation failed' }
python tests/plugins/heiji_kazuha_replay.py
if ($LASTEXITCODE -ne 0) { throw 'Heiji Kazuha input replay failed' }
python tools/plugins/preview.py
if ($LASTEXITCODE -ne 0) { throw 'CPU preview failed' }
build/plugins/Release/plugin-surface-probe.exe build/plugins/Release/miucs-plugin-worker.exe plugins build/plugin-fixture.json
Expand Down
6 changes: 3 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

## 平台支持

| 平台 | 音频采集 / 恢复 | 分析后端 | 铅笔飞鸟绘制 | 五线谱音符绘制 | 应用外壳 |
| 平台 | 音频采集 / 恢复 | 分析后端 | 四个插件绘制 | 插件适配 | 应用外壳 |
| --- | --- | --- | --- | --- | --- |
| macOS 14+ / Apple Silicon | 已实现,含自动恢复 | Swift 分析 + Accelerate FFT | Lua/Skia,共用五种笔色 | Lua/Skia,共用五种笔色 | 已实现 |
| Windows 10/11 x64 | WASAPI 已实现;实机待验 | C# FFT/混音旋律/节拍;合成测试 | Lua/Skia;实机待验 | Lua/Skia;实机待验 | 托盘/透明窗口已实现;实机待验 |
Expand Down Expand Up @@ -77,11 +77,11 @@ python3 scripts/benchmark-pop-melody.py --prepare --adc-dataset build/adc2004 --
sh scripts/build-plugins.sh
sh scripts/test-plugins.sh # C++ 运行库、固定数值回放、预览、传输与生命周期
zsh scripts/test-plugin-macos.sh # macOS 插件选择、独立 worker、热重载和托盘
python3 tools/plugins/validate.py plugins/staff-note
python3 tools/plugins/validate.py plugins/pixel-keyboard
python3 tools/plugins/preview.py
```

数值回放使用 `tests/plugins/reference.json.gz` 中的固定参考,覆盖三个原有插件各 600 帧。参考来自移除前的模拟器,输入哈希防止测试数据无意漂移;维护说明见 [测试基线](tests/plugins/README.md)。性能测试使用 `build/plugins/plugin-benchmark`,持续内存检查使用 `tests/plugins/soak.py`。
数值回放使用 `tests/plugins/reference.json.gz` 中的固定参考,覆盖奶牛猫的 600 帧头部与尾巴动画;麦芬、牛来和平次与和叶由各自规则回放验证。参考来自移除前的模拟器,输入哈希防止测试数据无意漂移;维护说明见 [测试基线](tests/plugins/README.md)。性能测试使用 `build/plugins/plugin-benchmark`,持续内存检查使用 `tests/plugins/soak.py`。

## Windows 接入

Expand Down
78 changes: 25 additions & 53 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,88 +1,60 @@
# Miucs · 桌面音乐可视化
# Miucs · 奶牛猫桌面伙伴

**让音乐自己飞。**
**打字时伸爪,听音乐时摇尾。**

把音乐和打字变成桌面上的动画:随旋律飞行的小鸟、随节奏摆动的音符,以及跟着按键敲击的像素猫。画面透明、默认鼠标穿透,不抢键盘焦点。
奶牛猫是 Miucs 的默认桌面伙伴:黑白猫爪跟随按键,猫头随打字轻晃,尾巴响应音乐节拍。画面透明、默认鼠标穿透,不抢键盘焦点。

`macOS 14+ · Apple Silicon · Swift 6` · `Windows 10/11 · 实机待验`

![像素猫键盘:双爪跟随按键,猫头随打字摇动,尾巴随音乐摆动](docs/assets/miucs-pixel-keyboard.gif)
![奶牛猫:猫爪跟随打字,猫头与尾巴响应输入](docs/assets/plugins/pixel-keyboard.gif)

> 合成输入驱动原生绘制器生成的演示,未录制真实输入或桌面;GIF 无声。
> 预览使用合成音乐与按键输入,经正式 Lua/Skia 绘制器生成;GIF 无声,不是桌面录屏。

## 三种可视化
## 四个桌面伙伴

[查看工作台生成的全部插件 GIF 预览](docs/plugin-previews.md),包含「牛来」和「麦芬与线条小狗」。

| 模式 | 效果 | 详细说明 |
| 插件 | 效果 | 详细说明 |
| --- | --- | --- |
| 像素音乐键盘 | 猫爪敲击真实物理键位,猫头随打字速度摇动,尾巴响应音乐节拍 | [像素键盘](docs/pixel-keyboard.md) |
| 五线谱音符 | 主旋律驱动音符与轨迹,节拍展开轻弧,按键散出星星 | [五线谱](docs/staff-note.md) |
| 铅笔飞鸟 | 音高决定飞行高度,音量与起音带动振翅,管道自动避让 | [飞鸟](docs/sound-flight.md) |
| **奶牛猫 · 默认主打** | 黑白猫爪敲键、猫头轻晃、尾巴随节拍摆动 | [奶牛猫](docs/pixel-keyboard.md) |
| 麦芬 | 两只线条小狗随打字跳跃,麦芬随节拍吹泡泡 | [麦芬](plugins/muffin-puppies/README.md) |
| 牛来 | 牛蹄响应按键,上方角色随音乐和打字运动 | [牛来](plugins/niulai-keyboard/README.md) |
| 平次与和叶 | 平次随打字挥木剑,和叶随节拍转身 | [平次与和叶](plugins/heiji-kazuha/README.md) |

音频和键盘可以同时驱动画面,也能独立使用。每种模式分别记住位置和大小,切换时不重启输入采集。仅使用 Lua/Skia 插件,也可以添加自己编写的 Lua 可视化。
[查看四个插件的动态预览](docs/plugin-previews.md)。首次启动默认选择奶牛猫;之后记住你的选择。每个插件分别保存位置和大小,切换不重启输入采集。

## 构建与启动

### macOS

需要 Apple Silicon、macOS 14+、支持 Swift 6 的 Xcode 或 Command Line Tools、Python 3.10+ 和 CMake 3.24+;首次创建本地开发签名需要 OpenSSL 3。在仓库根目录执行:
需要 Apple Silicon、macOS 14+、Swift 6、Python 3.10+、CMake 3.24+;首次本地开发签名需要 OpenSSL 3。

```sh
zsh build.sh
open dist/Miucs.app
```

首次运行按提示授权系统音频捕获;需要打字效果时,授予「输入监控」权限。菜单中的「键盘监听」可查看状态。休眠唤醒或音频中断后会自动重连,也可手动选择「重新连接音频」。
首次运行按提示授权系统音频捕获;要让猫爪响应打字,授予「输入监控」权限。音频和键盘可以同时使用,也能独立运行。系统休眠或音频中断后自动恢复,也可在菜单选择「重新连接音频」。更新后先退出旧版,再打开新构建。

### Windows

已有 .NET 8/WPF、WASAPI 实现,实机采集、绘制及多屏验收尚未完成。构建步骤见 [Windows README](apps/windows/README.md),支持边界见 [验收表](docs/windows-acceptance.md)。
提供 .NET 8/WPF、WASAPI 实现。构建方法见 [Windows README](apps/windows/README.md),实机验证范围见 [验收表](docs/windows-acceptance.md)。

## 常用操作

从菜单栏或托盘打开菜单:

| 操作 | 用途 |
| --- | --- |
| 可视化 | 切换当前画面 |
| 显示 / 隐藏拾音器 | 控制悬浮画面的显示 |
| 画面大小 | 调整缩放比例 |
| 线条颜色 | 选择五种笔色;像素键盘保留固定配色 |
| 画面不透明度 | 10%~100%,即时生效并自动保存 |
| 调整位置与大小… | 拖动画面移动、拖动右下角缩放,完成后恢复鼠标穿透 |

也可以将鼠标停留在画面上 3 秒,点击出现的小锁来解锁或锁定。菜单栏猫图标属于应用本身,不随可视化插件切换。

## 自己制作插件
- **可视化**:切换四个桌面伙伴,奶牛猫排在首位。
- **显示 / 隐藏画面**:控制浮窗显示。
- **画面大小、不透明度**:等比缩放,调整 10%~100% 不透明度。
- **调整位置与大小**:拖动画面或右下角缩放,完成后恢复鼠标穿透;也可悬停 3 秒点击小锁。
- **托盘猫摇头**:应用自己的猫图标,独立于当前插件。

插件使用 **Lua 5.4 + Skia**,一个文件夹对应一个可视化。复制模板后,编写场景与动画,放入用户插件目录即可加载,无需重新编译应用。
## 插件与隐私

所有构建均包含插件运行库和内置插件。首次构建会下载并校验 Lua、JSON 库和 Skia 依赖:
应用内置四个 Lua 5.4/Skia 插件,也可安装自己编写的插件。首次构建自动下载并校验运行库依赖。开发和调试见 [插件开发指南](docs/plugin-development.md) 与 [插件工作台](docs/plugin-lab.md)。

```sh
zsh build.sh
open dist/Miucs.app
```

更新后请先退出已运行的 Miucs,再打开新版本。应用只支持插件可视化;运行库缺失时会显示错误。

**[插件开发指南 →](docs/plugin-development.md)**:包含完整示例、双端安装步骤、输入数据、热重载、验证预览和常见问题。

## 本地处理

系统音频在本机分析,不依赖网络服务或外部模型;应用不保存真实音频、屏幕画面或输入文字。键盘效果使用物理键位,不受输入法影响。

主旋律是对混音中突出音高的近似估计,复杂伴奏可能影响结果;音符用于可视化,不生成完整乐谱。算法与限制见[混音主旋律说明](docs/predominant-mixture.md)。
音频与物理按键只在本机分析,不依赖网络模型;应用不保存真实音频、屏幕或输入文字。

## 开发文档

- [贡献指南与测试命令](CONTRIBUTING.md)
- [插件开发指南](docs/plugin-development.md) · [插件 API 契约](contracts/plugins/README.md)
- [架构与依赖边界](docs/architecture.md)
- [贡献指南与测试](CONTRIBUTING.md) · [架构与依赖边界](docs/architecture.md)
- [插件契约](contracts/plugins/README.md) · [可视化契约](contracts/visualization/README.md)
- [音频数据](contracts/audio-features/README.md) · [键盘数据](contracts/keyboard-input/README.md)
- [外观设置](docs/appearance.md) · [验证记录](docs/validation/README.md)

插件版另带 **「牛来」**,详见[牛来说明](plugins/niulai-keyboard/README.md)。

插件版另带 **「麦芬与线条小狗」**,详见[麦芬皮肤说明](plugins/muffin-puppies/README.md)。皮肤采用断续手绘键盘,两只小狗随按键播放模型绘制的跳跃帧,麦芬随音乐节拍吹泡泡。
2 changes: 1 addition & 1 deletion apps/macos/PluginVisualizations.swift
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import Foundation
/// The catalog is the only source of visualization sessions.
@MainActor
enum PluginVisualizations {
static let defaultID = "pencil-flight"
static let defaultID = "pixel-keyboard"
static func registry() -> VisualizationRegistry {
let registry = VisualizationRegistry()
PluginHost.shared.register(in: registry)
Expand Down
6 changes: 0 additions & 6 deletions apps/macos/VisualizationAppearanceStore.swift
Original file line number Diff line number Diff line change
Expand Up @@ -14,17 +14,11 @@ final class VisualizationAppearanceStore: ObservableObject {
private let preferences: UserDefaults
var appearance: VisualizationAppearance { selection.appearance }

// Decode only the saved ink needed to migrate existing installations.
private struct PreviousPalette: Decodable { let bird: VisualizationColor }

init(preferences: UserDefaults = .standard) {
self.preferences = preferences
transparency = min(90, max(0, preferences.object(forKey: Self.transparencyKey) as? Int ?? 0))
if let raw = preferences.string(forKey: Self.preferencesKey) {
selection = VisualizationInk(rawValue: raw) ?? .graphite
} else if let data = preferences.data(forKey: "MiucsPencilPalette.v1"),
let legacy = try? JSONDecoder().decode(PreviousPalette.self, from: data) {
selection = VisualizationInk.nearest(to: legacy.bird)
} else { selection = .graphite }
}
// Called only by a normal launch, so offline diagnostics remain read-only.
Expand Down
12 changes: 5 additions & 7 deletions apps/macos/VisualizerPanel.swift
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ import AppKit
final class VisualizerPanel: NSPanel, NSWindowDelegate {
static let selectionPreferencesKey = "MiucsSelectedVisualization.v1"
static func framePreferencesKey(for id: String) -> String {
id == "pencil-flight" ? "MiucsPencilFlightFrame" : "MiucsVisualizationFrame.v1.\(id)"
"MiucsVisualizationFrame.v1.\(id)"
}
private static func autosaveName(for id: String) -> String {
id == "pencil-flight" ? "MiucsPencilFlight" : "MiucsVisualization.\(id)"
"MiucsVisualization.\(id)"
}
let visualizationHost: VisualizationHostView
private let preferences: UserDefaults
Expand All @@ -33,7 +33,7 @@ final class VisualizerPanel: NSPanel, NSWindowDelegate {
preferences: UserDefaults = .standard, persistsFrame: Bool = true,
keyboardSource: (any KeyboardFeatureSource)? = nil, monitorsHover: Bool = true) {
let requestedID = selectedID ?? preferences.string(forKey: Self.selectionPreferencesKey)
let initial = requestedID.flatMap { registry.descriptor(id: $0) } ?? registry.descriptors.first!
let initial = requestedID.flatMap { registry.descriptor(id: $0) } ?? registry.descriptor(id: PluginVisualizations.defaultID) ?? registry.descriptors.first!
descriptor = initial
visualizationHost = VisualizationHostView(source: source, registry: registry, selectedID: initial.id,
keyboardSource: keyboardSource)
Expand Down Expand Up @@ -94,7 +94,7 @@ final class VisualizerPanel: NSPanel, NSWindowDelegate {
}

private func updateTitle() {
title = descriptor.id == "pencil-flight" ? "Miucs · Pencil Flight" : "Miucs · \(descriptor.name)"
title = "Miucs · \(descriptor.name)"
}

private func restoreGeometry(fallbackTopLeft: NSPoint?) {
Expand All @@ -112,8 +112,7 @@ final class VisualizerPanel: NSPanel, NSWindowDelegate {
setFrame(NSRect(x: topLeft.x, y: topLeft.y - layout.height,
width: layout.width, height: layout.height), display: false)
}
let legacyWidth = descriptor.id == "pencil-flight" ? preferences.object(forKey: "MiucsPencilFlightWidth") as? Double : nil
let width: Double = saved.map { Double($0.width) } ?? (restored ? Double(frame.width) : legacyWidth ?? layout.width)
let width: Double = saved.map { Double($0.width) } ?? (restored ? Double(frame.width) : layout.width)
setDrawingWidth(width, persist: false)
visualizationHost.updateViewport()
if persistsFrame { setFrameAutosaveName(Self.autosaveName(for: descriptor.id)) }
Expand Down Expand Up @@ -151,7 +150,6 @@ final class VisualizerPanel: NSPanel, NSWindowDelegate {
frame.width.isFinite, frame.height.isFinite, frame.width > 0, frame.height > 0 else { return }
preferences.set(["x": frame.minX, "y": frame.minY,
"width": frame.width, "height": frame.height], forKey: Self.framePreferencesKey(for: descriptor.id))
if descriptor.id == "pencil-flight" { preferences.set(frame.width, forKey: "MiucsPencilFlightWidth") }
preferences.set(descriptor.id, forKey: Self.selectionPreferencesKey)
if persistsFrame { saveFrame(usingName: Self.autosaveName(for: descriptor.id)) }
}
Expand Down
4 changes: 2 additions & 2 deletions apps/windows/Core/Preferences.cs
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ public static WindowFrame Fit(WindowFrame requested, WindowFrame workArea, doubl

public sealed class Preferences
{
public string Visualization { get; set; } = "pencil-flight";
public string Visualization { get; set; } = "pixel-keyboard";
public string Color { get; set; } = "graphite";
private int transparency;
public int Transparency { get => transparency; set => transparency = Math.Clamp(value, 0, 90); }
Expand All @@ -42,7 +42,7 @@ public static Preferences Load(string path)
var result = JsonSerializer.Deserialize<Preferences>(File.ReadAllText(path));
if (result == null) return new();
result.Frames ??= new();
if (string.IsNullOrWhiteSpace(result.Visualization) || result.Visualization.Length > 128) result.Visualization = "pencil-flight";
if (string.IsNullOrWhiteSpace(result.Visualization) || result.Visualization.Length > 128) result.Visualization = "pixel-keyboard";
result.Color = Palette.ContainsKey(result.Color ?? "") ? result.Color! : "graphite";
foreach (var key in new List<string>(result.Frames.Keys))
if (result.Frames[key] == null || !result.Frames[key].IsValid) result.Frames.Remove(key);
Expand Down
2 changes: 1 addition & 1 deletion apps/windows/DesktopHost.cs
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ private void Dispatch(Action action)
private void SwitchVisualization(string id)
{
if (!pluginHost.Views.ContainsKey(id))
id=pluginHost.Views.Keys.FirstOrDefault() ?? "";
id=pluginHost.Views.ContainsKey("pixel-keyboard") ? "pixel-keyboard" : pluginHost.Views.Keys.FirstOrDefault() ?? "";
if (view != null && view.Id == id) return;
WindowFrame? previous = view == null ? null : CurrentFrame();
hoverLock?.Reset();
Expand Down
2 changes: 1 addition & 1 deletion apps/windows/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

解压 `Miucs-windows-x64.zip`,保留整个目录,运行 `Miucs.exe`。自包含版本无需单独安装 .NET,也不需要管理员权限。退出后可直接删除目录。无安装器、无签名、无开机自启、无注册表持久化;未签名的新程序可能显示 Windows 信任提示,应先核对来源及 SHA-256,不要关闭系统安全设置。

启动后在通知区域寻找 Miucs 图标(可能位于折叠区)。默认显示飞鸟,捕获**当前默认多媒体播放设备**的系统混音,不使用麦克风。播放音乐后自动驱动画面。WASAPI 空闲设备可能没有 PCM;此时不伪造静音帧,也不因为没有音乐不断重连。
启动后在通知区域寻找 Miucs 图标(可能位于折叠区)。默认显示奶牛猫,捕获**当前默认多媒体播放设备**的系统混音,不使用麦克风。播放音乐后自动驱动画面。WASAPI 空闲设备可能没有 PCM;此时不伪造静音帧,也不因为没有音乐不断重连。

- **可视化**:仅加载插件目录中的画面;运行库缺失时显示启动错误。切换不重启两路输入。
- **托盘猫摇头**:所有模式统一显示同一只像素猫;托盘动画独立于插件,开启键盘输入后随打字速度摇头,停下后回正。隐藏画面仍可响应,切换模式不换猫、不重启动作。可从菜单关闭托盘动画;锁屏/休眠或 Windows 关闭界面动画时保持静止。不会因此自动开启键盘监听。
Expand Down
Loading
Loading