From 89a1fa9b92f529eec314690b97eebeeab8a68a92 Mon Sep 17 00:00:00 2001 From: Ziyi Zhang Date: Tue, 11 Aug 2026 18:50:04 -0400 Subject: [PATCH 1/7] feat: add the input, audio and platform skills --- README.md | 4 + .../android-add-adaptive-performance/SKILL.md | 92 ++ .../resources/ADAPTIVE_PERFORMANCE_PLAN.md | 83 ++ .../AdaptivePerformanceSignalManager.cs | 269 +++++ .../resources/AdaptiveQualityAdapter.cs | 297 ++++++ skills/asset-transformer-toolkit/SKILL.md | 28 + .../api-docs/rule-api.md | 154 +++ .../api-docs/ruleblock-api.md | 37 + .../api-docs/ruleset-api.md | 138 +++ .../references/create-importer.md | 121 +++ .../references/lods.md | 61 ++ .../references/rulesets-and-actions.md | 227 +++++ skills/setup-audiorandomcontainer/SKILL.md | 35 + .../references/api.md | 132 +++ skills/setup-game-inputs/SKILL.md | 32 + .../references/input-system.md | 924 ++++++++++++++++++ 16 files changed, 2634 insertions(+) create mode 100644 skills/android-add-adaptive-performance/SKILL.md create mode 100644 skills/android-add-adaptive-performance/resources/ADAPTIVE_PERFORMANCE_PLAN.md create mode 100644 skills/android-add-adaptive-performance/resources/AdaptivePerformanceSignalManager.cs create mode 100644 skills/android-add-adaptive-performance/resources/AdaptiveQualityAdapter.cs create mode 100644 skills/asset-transformer-toolkit/SKILL.md create mode 100644 skills/asset-transformer-toolkit/api-docs/rule-api.md create mode 100644 skills/asset-transformer-toolkit/api-docs/ruleblock-api.md create mode 100644 skills/asset-transformer-toolkit/api-docs/ruleset-api.md create mode 100644 skills/asset-transformer-toolkit/references/create-importer.md create mode 100644 skills/asset-transformer-toolkit/references/lods.md create mode 100644 skills/asset-transformer-toolkit/references/rulesets-and-actions.md create mode 100644 skills/setup-audiorandomcontainer/SKILL.md create mode 100644 skills/setup-audiorandomcontainer/references/api.md create mode 100644 skills/setup-game-inputs/SKILL.md create mode 100644 skills/setup-game-inputs/references/input-system.md diff --git a/README.md b/README.md index e60ed11..90cbce7 100644 --- a/README.md +++ b/README.md @@ -22,6 +22,10 @@ npx skills add Unity-Technologies/skills | `ui-uitk` | UI Toolkit (Unity 6.0+) — author UXML/USS, flex layout, custom elements, Painter2D, runtime binding | | `ui-ugui` | uGUI — Canvas hierarchies, RectTransform anchoring, Layout Groups, prefab UI | | `ui-imgui` | IMGUI editor tooling — EditorWindows, custom Inspectors, PropertyDrawers | +| `setup-game-inputs` | Input System — action maps, bindings, control schemes, rebinding | +| `setup-audiorandomcontainer` | AudioRandomContainer assets for randomized playback | +| `android-add-adaptive-performance` | Android thermal and power signals mapped to dynamic quality tiers | +| `asset-transformer-toolkit` | 3D model and point-cloud import, RuleSets and Actions, LOD generation | ## Usage diff --git a/skills/android-add-adaptive-performance/SKILL.md b/skills/android-add-adaptive-performance/SKILL.md new file mode 100644 index 0000000..bdbd6ad --- /dev/null +++ b/skills/android-add-adaptive-performance/SKILL.md @@ -0,0 +1,92 @@ +--- +name: android-add-adaptive-performance +description: Implements Unity Adaptive Performance for Android (Unity >= 6.0 / 6000.0.0) by + handling hardware thermal/power signals and mapping them to quality tiers with dynamic + graphics and simulation quality adjustments. Use when the user asks about adaptive + performance, thermal throttling, dynamic quality scaling, FPS drops on Android, or + optimizing Android game performance. +required_editor_version: ">=6000.0.0" +required_packages: + com.unity.adaptiveperformance: ">=4.0.0" +--- + +## Quick Start + +Adds a full Adaptive Performance integration to a Unity Android project. Creates two MonoBehaviours (`AdaptivePerformanceSignalManager` and `AdaptiveQualityAdapter`), optionally configures URP quality settings, and bootstraps the system into the user's chosen scene. + +## Critical Rules + +- Do not make any subjective calls — ask the user when in doubt +- Follow steps in strict order; never jump ahead +- STOP at every `WAIT` checkpoint and await the user's response before continuing +- Do not install any packages +- Do not add the bootstrap GameObject until all scripts are written and in place +- Use `AdaptivePerformanceIntegration` as the namespace for all generated code +- Do not create placeholder files when creating folders + +## Workflow + +### Step 1: Gather Required Information + +Do not output any code until all answers are collected. + +1. Attempt to detect the graphics pipeline (URP / HDRP / Built-in). If there is any doubt or you cannot determine it, ask the user directly. +2. Ask the user how many quality tiers they want. Default is 3 (Best / Medium / Low). Determine appropriate tier names yourself based on their answer. +3. Ask whether quality should snap back to Best when conditions normalize, or use a sticky downgrade (stay at the lowest tier reached for the session, preventing oscillation). + +**WAIT for the user to answer all three questions before proceeding.** + +### Step 2: Create URP Quality Settings (skip entirely if not URP) + +- Create `mobile_adaptive` and `mobile_max` quality settings, configured to affect Android only. +- For each, create a matching URP Render Pipeline Asset under `Assets/AdaptivePerformanceManager/Settings/`: + - `urp_mobile_adaptive` — Enable Adaptive Performance: **ON** + - `urp_mobile_max` — Enable Adaptive Performance: **OFF** + +Tell the user what was created, then continue. + +### Step 3: Generate the Signal Handler Script + +Copy `AdaptivePerformanceSignalManager` from `resources/AdaptivePerformanceSignalManager.cs` into `Assets/AdaptivePerformanceManager/`. Adjust tier count and names to match the user's answers from Step 1. + +Tell the user the script has been added, then continue. + +### Step 4: Generate the Quality Adapter Script + +Copy `AdaptiveQualityAdapter` from `resources/AdaptiveQualityAdapter.cs` into `Assets/AdaptivePerformanceManager/`. Apply all scalers — never skip any. Add a `// HIGH VISUAL IMPACT` comment above any scaler line that may visually disrupt gameplay (leave the code intact). + +For FPS targets, use display Hz divisors: +- 60 Hz displays: 60, 30 FPS +- 90 Hz displays: 90, 45, 30 FPS +- 120 Hz displays: 120, 60, 40, 30 FPS + +See `resources/ADAPTIVE_PERFORMANCE_PLAN.md` for the full scaler table and URP-specific scalers. + +Tell the user both scripts are ready, then continue. + +### Step 5: Add Bootstrap GameObject + +Ask the user which scene to place the `AdaptivePerformanceSignalManager` GameObject in. + +**WAIT for the user to respond before continuing.** + +1. Add the GameObject to the specified scene. +2. Attach `AdaptivePerformanceSignalManager` and `AdaptiveQualityAdapter` components to it and configure all references. +3. Save the scene. + +### Step 6: Produce a Final Checklist + +List everything that was done, then list what the user must do manually: + +- Go to **Project Settings → Adaptive Performance** and enable "Enable Adaptive Performance" if not already checked. +- In the Providers section, check **Android Provider**. +- If new Quality Settings were added, review and adjust each one's settings and verify the URP Render Pipeline Assets are configured correctly. +- `AdaptivePerformanceSignalManager.cs` is the primary script to customize tier logic. +- Optional: `AdaptiveLayerCulling` via `Camera.main.layerCullDistances` is available but requires detailed per-project setup by the user. +- Optional: Post-processing `VolumeProfiles` in scene Volumes are not controlled by this integration and can be tuned manually for additional adaptive gains. + +## Detailed References + +- **Full implementation plan and scaler tables:** [resources/ADAPTIVE_PERFORMANCE_PLAN.md](resources/ADAPTIVE_PERFORMANCE_PLAN.md) +- **Signal handler template:** [resources/AdaptivePerformanceSignalManager.cs](resources/AdaptivePerformanceSignalManager.cs) +- **Quality adapter template:** [resources/AdaptiveQualityAdapter.cs](resources/AdaptiveQualityAdapter.cs) diff --git a/skills/android-add-adaptive-performance/resources/ADAPTIVE_PERFORMANCE_PLAN.md b/skills/android-add-adaptive-performance/resources/ADAPTIVE_PERFORMANCE_PLAN.md new file mode 100644 index 0000000..aae7c80 --- /dev/null +++ b/skills/android-add-adaptive-performance/resources/ADAPTIVE_PERFORMANCE_PLAN.md @@ -0,0 +1,83 @@ +# Adaptive Performance Signal Handler Skill Plan +Target: Unity >=6.0 (6000.0.0), Android only +Constraints: +- Do not make any subjective calls +- Follow the full implementation plan strictly +- All steps should be done by you, unless it is explicitly mentioned that the user should do it +- Do not install any packages +- No flicker detection +- Stop at each step that says to ask the user a question and ask the question. You should not get all the way to the end and ask a lot of questions at once. This overwhelms the user. +- Do not add the bootstrap GameObject before adding and writing all necessary code +- Use AdaptivePerformanceIntegration as the namespace for any code +- Do not create any placeholder files when creating folders + +## Step 1: Gather minimum required information +Do not output code until all answers are collected. + +1. Attempt to detect graphics pipeline (URP/HDRP/Built-in) but if there is any doubt or inablity to do so, ask the user directly. +2. Tier structure: + - Default: 3 tiers (0..2) meaning Best/Medium/Low + - Ask the user if they want to change the number of tiers and determine good names to map them to on your own. +3. Hardware condition reversal behavior: + - Ask the user if they want to return to “Best” when conditions normalize or keep the game running at the lowest detected quality (sticky downgrade). + +## Step 2 (Skip if not using URP) Create quality settings and URP Render Pipeline assets +- Create one "mobile_adaptive" and one "mobile_max" quality setting. +- Each one should be set to affect Android only +- For each of the mobile quality settings, create and set a matching URP Render Pipeline Asset in Assets/AdaptivePerformanceManager/Settings/ named "urp_mobile_adaptive" and "urp_mobile_max" respectively. +- Turn on "Use Enable Adaptive Performance" for the URP Render Pipeline asset "urp_mobile_adaptive". +- Turn off "Use Enable Adaptive Performance" for the URP Render Pipeline asset "urp_mobile_max". + +## Step 3: Generate the “signal handling” script only +Add AdaptivePerformanceSignalManager from resources/AdaptivePerformanceSignalManager.cs to Assets/AdaptivePerformanceManager/ and adjust based on the decided tier structure + +## Step 4: Write an integration the user can customize +Here are the frame rates that each display type supports so you will want to try and determine the Hz of the display to know which FPS numbers you will be able to drop to. +60 Hz displays: 60 FPS, 30 FPS +90 Hz displays: 90 FPS, 45 FPS, 30 FPS +120 Hz displays: 120 FPS, 60 FPS, 40 FPS, 30 FPS + +Include all available scalers. Important: Do not skip any scaler. +Warn the user with a code comment in AdaptiveQualityAdapter if any particular scaler may negatively affect gameplay or high visual impact. + +For all pipelines: +General performance scalers (Do not skip ANY of these under any circumstances): +General scalers Min Scale Max Scale Max Level Visual Impact Target Setting scaled +AdaptiveLOD 0.4 1 3 High GPU QualitySettings.lodBias +AdaptiveResolution 0.5 1 9 Low GPU/FillRate AdaptivePerformanceRenderSettings.RenderScaleMultiplier and AdaptivePerformanceRenderSettings.ScalableBuffers +AdaptiveFramerate 15 60 45 High CPU/GPU/FillRate Application.targetFrameRate +AdaptiveViewDistance 50 1,000 40 High GPU Camera.main.farClipPlane +AdaptivePhysics 0.5 1 5 Low CPU Time.fixedDeltaTime + +Additional scalers for URP pipelines only: +Universal Render Pipeline (URP) scalers (Do not skip ANY of these under any circumstances) +URP scalers Min Scale Max Scale Max Level Visual Impact Target Setting scaled +AdaptiveBatching 0 1 1 Medium CPU AdaptivePerformanceRenderSettings.SkipDynamicBatching +AdaptiveLUT 0 1 1 Medium CPU/GPU AdaptivePerformanceRenderSettings.LutBias +AdaptiveMSAA 0 1 2 Medium GPU/FillRate AdaptivePerformanceRenderSettings.AntiAliasingQualityBias +AdaptiveShadowCascade 0 1 2 Medium CPU/GPU AdaptivePerformanceRenderSettings.MainLightShadowCascadesCountBias +AdaptiveShadowDistance 0.15 1 3 Low GPU AdaptivePerformanceRenderSettings.MaxShadowDistanceMultiplier +AdaptiveShadowQuality 0 1 3 High CPU/GPU AdaptivePerformanceRenderSettings.ShadowQualityBias +AdaptiveShadowmapResolution 0.15 1 3 Low GPU AdaptivePerformanceRenderSettings.MainLightShadowmapResolutionMultiplier +AdaptiveSorting 0 1 1 Medium CPU AdaptivePerformanceRenderSettings.SkipFrontToBackSorting +AdaptiveTransparency 0 1 1 High GPU AdaptivePerformanceRenderSettings.SkipTransparentObjects +AdaptiveDecals 0.01 1 20 Medium GPU AdaptivePerformanceRenderSettings.DecalsDrawDistance + +Add the example adapter script from resources/AdaptiveQualityAdapter.cs and place it in Assets/AdaptivePerformanceManager/. Add code comments above any lines for scalers that cause high visual impact (but leave the code intact). + +## Step 5: Add bootstrap GameObject +- Determine what scene the "AdaptivePerformanceSignalManager" bootstrap GameObject will be added to +- Add the "AdaptivePerformanceSignalManager" bootstrap GameObject to the scene +- Add the AdaptivePerformanceSignalManager and AdaptiveQualityAdapter components to the AdaptivePerformanceSignalManager GameObject and configure them +- Save the scene + +## Step 6: Produce a final checklist +- Include a checklist of what was done to the project and what steps the user should do on their own. +- Ask the user to go to Project Settings -> Adaptive Performance and check the "Enable Adaptive Performance" checkbox if it is not checked. +- Ask the user to check the "Android Provider" box in the Providers section in the Adaptive Performance settings. +- If new Quality settings were added, encourage the user to adjust the settings for each one and to check and adjust the URP Render Pipeline assets associated with each. + - The URP Render Pipeline asset for mobile_adaptive should have Enable Adaptive Performance turned on so ask the user to check. + - The URP Render Pipeline asset for mobile_max should have Enable Adaptive Performance turned off on so ask the user to check. +- Give a brief explanation of how "AdaptivePerformanceSignalManager.cs" is the main script they would edit to expand or change what was done. +- Let the user know they can also implement AdaptiveLayerCulling via Camera.main.layerCullDistances but it takes detailed. setup from the user +- Another thing that isn't really controlled in any of the settings we've talked about above is the postprocessing stack, the VolumeProfiles defined in Volumes in scene. Tell them this is something they can implement on their own for even better adaptive performance. \ No newline at end of file diff --git a/skills/android-add-adaptive-performance/resources/AdaptivePerformanceSignalManager.cs b/skills/android-add-adaptive-performance/resources/AdaptivePerformanceSignalManager.cs new file mode 100644 index 0000000..ab5074f --- /dev/null +++ b/skills/android-add-adaptive-performance/resources/AdaptivePerformanceSignalManager.cs @@ -0,0 +1,269 @@ +using System; +using UnityEngine; +using UnityEngine.AdaptivePerformance; + +namespace AdaptivePerformanceIntegration +{ + /// + /// Game-agnostic Adaptive Performance signal handler. + /// Reads thermal/performance state via the official API and reports the effective state index. + /// + /// This MonoBehaviour polls the Adaptive Performance subsystem at a configurable interval, + /// evaluates the current thermal warning level and temperature, and maps them to a simplified + /// integer state index (0–3). Downstream systems (e.g., quality adapters) subscribe to + /// to react to thermal changes without depending + /// on the Adaptive Performance API directly. + /// + /// Effective States: + /// 0: Normal (Cool) – No thermal concerns. + /// 1: Elevated (Pre-warning) – Temperature has risen past the preemptive threshold + /// but no official warning has been issued yet. + /// 2: Throttling Imminent – The subsystem reports . + /// 3: Throttling – The subsystem reports ; + /// the device is actively reducing clock speeds. + /// + public class AdaptivePerformanceSignalManager : MonoBehaviour + { + /// + /// Serializable policy block that controls how and when thermal state is evaluated. + /// Exposed in the Inspector so designers can tune thresholds per-project. + /// + [Serializable] + public class StatePolicy + { + /// + /// Normalized temperature level (0–1) above which the manager reports state 1 + /// ("Elevated") even when no official thermal warning has been raised. This lets + /// the game begin reducing load before the hardware signals a problem. + /// + [Tooltip("Temperature level (0-1) above which we report 'Elevated' (1) even if there is no official thermal warning.")] + [Range(0f, 1f)] + public float PreemptiveTemperatureThreshold = 0.5f; + + /// + /// How often (in seconds) the manager re-evaluates thermal state in Update(). + /// Lower values give faster reactions but cost more CPU. + /// + [Header("Polling interval (seconds)")] + [Min(0.1f)] + public float PollIntervalSeconds = 1.0f; + + /// + /// Master switch. When false, polling, evaluation, and event firing are all skipped. + /// + [Header("Enable/disable adaptation")] + public bool Enabled = true; + } + + /// + /// Inspector-exposed policy controlling thresholds, polling rate, and the enable flag. + /// + [Header("Policy")] + public StatePolicy Policy = new StatePolicy(); + + /// + /// When true, state transitions and subsystem acquisition are logged to the console + /// via . + /// + [Header("Debug")] + public bool VerboseLogging = true; + + /// + /// Fired whenever the effective hardware state index (0–3) changes. + /// Subscribers receive the new state index. + /// + public event Action HardwareStateChangedEvent; + + /// + /// Optional debug/telemetry event that carries the raw warning-level name and + /// normalized temperature (0–1) each time the state transitions. + /// + public event Action AdaptiveStateChangedEvent; + + /// + /// The most recently computed effective hardware state index (0–3). + /// Initialized to -1 to guarantee the first evaluation always triggers events. + /// + public int CurrentStateIndex { get; private set; } = -1; + + /// Cached reference to the Adaptive Performance subsystem instance. + private IAdaptivePerformance _ap; + + /// Unscaled timestamp of the next scheduled poll. + private float _nextPollTime; + + /// + /// Guards against double-subscribing to the thermal event if + /// is called more than once. + /// + private bool _subscribedToThermalEvent; + + /// + /// Marks this GameObject as persistent across scene loads so thermal monitoring + /// is never interrupted by scene transitions. + /// + private void Awake() + { + DontDestroyOnLoad(gameObject); + } + + /// + /// Attempts to acquire the Adaptive Performance subsystem on the first frame + /// and schedules the initial poll. + /// + private void Start() + { + if (!Policy.Enabled) return; + + TryAcquireInstance(); + _nextPollTime = Time.unscaledTime + Policy.PollIntervalSeconds; + } + + /// + /// Cleans up the thermal event subscription when this component is destroyed. + /// + private void OnDestroy() + { + UnsubscribeThermalEvent(); + } + + /// + /// Per-frame update. If the subsystem has not been acquired yet, retries at the + /// configured poll interval. Once acquired, evaluates thermal state on each poll tick. + /// Uses so polling is unaffected by time scale. + /// + private void Update() + { + if (!Policy.Enabled) return; + + // Subsystem not yet available – retry acquisition on the next poll tick. + if (_ap == null || !_ap.Active) + { + if (Time.unscaledTime >= _nextPollTime) + { + _nextPollTime = Time.unscaledTime + Policy.PollIntervalSeconds; + TryAcquireInstance(); + } + return; + } + + // Wait until the next scheduled poll. + if (Time.unscaledTime < _nextPollTime) return; + _nextPollTime = Time.unscaledTime + Policy.PollIntervalSeconds; + + EvaluateAndReport(); + } + + /// + /// Fetches the singleton instance from + /// . If successful, subscribes to thermal events + /// and performs an immediate evaluation so there is no gap until the first poll. + /// + private void TryAcquireInstance() + { + _ap = Holder.Instance; + if (_ap == null || !_ap.Active) return; + + Log($"Adaptive Performance acquired. Active={_ap.Active}"); + SubscribeThermalEvent(); + EvaluateAndReport(); + } + + /// + /// Subscribes to the subsystem's push-based + /// so the manager can react immediately when the device raises or clears a warning, + /// rather than waiting for the next poll tick. + /// + private void SubscribeThermalEvent() + { + if (_subscribedToThermalEvent || _ap?.ThermalStatus == null) return; + _ap.ThermalStatus.ThermalEvent += OnThermalEvent; + _subscribedToThermalEvent = true; + } + + /// + /// Removes the thermal event subscription, preventing callbacks after this + /// component has been destroyed or disabled. + /// + private void UnsubscribeThermalEvent() + { + if (!_subscribedToThermalEvent || _ap?.ThermalStatus == null) return; + _ap.ThermalStatus.ThermalEvent -= OnThermalEvent; + _subscribedToThermalEvent = false; + } + + /// + /// Callback invoked by the Adaptive Performance subsystem when thermal conditions + /// change. Delegates to . + /// + /// Latest thermal metrics snapshot from the subsystem. + private void OnThermalEvent(ThermalMetrics metrics) + { + if (!Policy.Enabled) return; + EvaluateAndReport(metrics); + } + + /// + /// Convenience overload that pulls the latest + /// from the cached subsystem reference and forwards to the evaluation logic. + /// + private void EvaluateAndReport() + { + if (_ap?.ThermalStatus == null) return; + EvaluateAndReport(_ap.ThermalStatus.ThermalMetrics); + } + + /// + /// Core evaluation logic. Maps the subsystem's and + /// normalized temperature to an integer state index (0–3). + /// + /// State mapping: + /// → 3 + /// → 2 + /// with temp ≥ threshold → 1 + /// Otherwise → 0 + /// + /// If the computed index differs from , both + /// and + /// are fired. + /// + /// Thermal metrics snapshot to evaluate. + private void EvaluateAndReport(ThermalMetrics metrics) + { + int stateIndex = 0; + + switch (metrics.WarningLevel) + { + case WarningLevel.Throttling: + stateIndex = 3; + break; + case WarningLevel.ThrottlingImminent: + stateIndex = 2; + break; + case WarningLevel.NoWarning: + default: + if (metrics.TemperatureLevel >= Policy.PreemptiveTemperatureThreshold) + stateIndex = 1; + break; + } + + if (stateIndex != CurrentStateIndex) + { + CurrentStateIndex = stateIndex; + Log($"Hardware State changed: {stateIndex} (warning={metrics.WarningLevel}, temp={metrics.TemperatureLevel:F2})"); + HardwareStateChangedEvent?.Invoke(CurrentStateIndex); + AdaptiveStateChangedEvent?.Invoke(metrics.WarningLevel.ToString(), metrics.TemperatureLevel); + } + } + + /// + /// Writes a timestamped debug message to the Unity console when + /// is enabled. + /// + /// Message to log, automatically prefixed with "[AdaptivePerformance]". + private void Log(string msg) + { + if (VerboseLogging) Debug.Log($"[AdaptivePerformance] {msg}"); + } + } +} diff --git a/skills/android-add-adaptive-performance/resources/AdaptiveQualityAdapter.cs b/skills/android-add-adaptive-performance/resources/AdaptiveQualityAdapter.cs new file mode 100644 index 0000000..0b84a9c --- /dev/null +++ b/skills/android-add-adaptive-performance/resources/AdaptiveQualityAdapter.cs @@ -0,0 +1,297 @@ +using UnityEngine; +using UnityEngine.Rendering; +using UnityEngine.AdaptivePerformance; + +namespace AdaptivePerformanceIntegration +{ + /// + /// Dynamically adjusts visual quality settings in response to hardware thermal/performance + /// signals relayed by . + /// + /// The adapter maps discrete hardware states (Normal, Elevated, Throttling Imminent, Throttling) + /// to designer-defined quality tiers, then applies both general Unity quality knobs and + /// (optionally) URP-specific scalers for each tier. + /// + /// An optional "sticky downgrade" policy ensures that once the device has been stressed, + /// quality never climbs back up during the session—useful for preventing oscillation on + /// devices that hover near a thermal boundary. + /// + public class AdaptiveQualityAdapter : MonoBehaviour + { + /// + /// Defines the full set of rendering parameters for a single quality tier. + /// + [System.Serializable] + public struct QualityTierSettings + { + public string Name; + + // --- General Scalers --- + + [Header("General Scalers")] + + /// Multiplier layered on top of the base render scale. + public float RenderScaleMultiplier; + + /// Divisor applied to the device refresh rate to determine target FPS (e.g. 1, 2, 4). + public int FrameRateDivisor; + + /// LOD bias multiplier—lower values force lower-detail meshes earlier. + public float LODBias; + + /// Physics step interval; larger values reduce physics CPU cost at the expense of accuracy. + public float FixedDeltaTime; + + /// Main camera far clip plane; reducing it culls distant geometry. + public float FarClipPlane; + + // --- URP Adaptive Performance Scalers (Note: Only applies if using URP) --- + + [Header("URP Scalers (Inactive in Built-in)")] + + /// Multiplier for the maximum shadow draw distance inside URP. + public float MaxShadowDistanceMultiplier; + + /// Multiplier for the main directional light's shadow map resolution. + public float MainLightShadowmapResolutionMultiplier; + + /// Bias added to the shadow cascade count (negative values reduce cascades). + public int ShadowCascadesBias; + + /// Bias added to the anti-aliasing quality level (negative values lower AA quality). + public int AAQualityBias; + + /// Bias added to the shadow quality level (negative values lower shadow fidelity). + public int ShadowQualityBias; + + /// When true, dynamic batching is skipped to save CPU overhead. + public bool SkipDynamicBatching; + + /// When true, transparent objects are not rendered—a significant GPU savings. + public bool SkipTransparentObjects; + + /// When true, front-to-back sorting is skipped to reduce CPU sort time. + public bool SkipFrontToBackSorting; + + /// Draw distance multiplier for decal projectors. + public float DecalsDrawDistance; + + /// Bias for the color grading LUT resolution (lower = faster, less accurate color). + public float LutBias; + } + + [Header("Components")] + /// + /// Reference to the signal manager that translates raw Adaptive Performance data + /// into actionable hardware-state events this adapter subscribes to. + /// + public AdaptivePerformanceSignalManager SignalManager; + + [Header("Settings")] + /// + /// When enabled, the adapter only ever moves to a *lower* quality tier during the + /// session—it will never return to a higher one even if thermal conditions improve. + /// This prevents visual "popping" on devices that repeatedly cross a thermal boundary. + /// + [Tooltip("If true, once the game quality drops, it never returns to a higher tier.")] + public bool StickyDowngrade = true; + + /// + /// Ordered array of quality tiers from highest fidelity (index 0) to lowest. + /// + public QualityTierSettings[] Tiers = new QualityTierSettings[] + { + new QualityTierSettings + { + Name = "Best", + RenderScaleMultiplier = 1.0f, + FrameRateDivisor = 1, + LODBias = 2.0f, + FixedDeltaTime = 0.0166f, + FarClipPlane = 1000f, + MaxShadowDistanceMultiplier = 1.0f, + MainLightShadowmapResolutionMultiplier = 1.0f, + ShadowCascadesBias = 0, + AAQualityBias = 0, + ShadowQualityBias = 0, + SkipDynamicBatching = false, + SkipTransparentObjects = false, + SkipFrontToBackSorting = false, + DecalsDrawDistance = 1.0f, + LutBias = 1.0f + }, + new QualityTierSettings + { + Name = "Medium", + RenderScaleMultiplier = 0.85f, + FrameRateDivisor = 2, + LODBias = 1.5f, + FixedDeltaTime = 0.02f, + FarClipPlane = 800f, + MaxShadowDistanceMultiplier = 0.75f, + MainLightShadowmapResolutionMultiplier = 0.75f, + ShadowCascadesBias = -1, + AAQualityBias = -1, + ShadowQualityBias = -1, + SkipDynamicBatching = false, + SkipTransparentObjects = false, + SkipFrontToBackSorting = false, + DecalsDrawDistance = 0.75f, + LutBias = 0.75f + }, + new QualityTierSettings + { + Name = "Low", + RenderScaleMultiplier = 0.7f, + FrameRateDivisor = 4, + LODBias = 1.0f, + FixedDeltaTime = 0.0333f, + FarClipPlane = 500f, + MaxShadowDistanceMultiplier = 0.5f, + MainLightShadowmapResolutionMultiplier = 0.5f, + ShadowCascadesBias = -2, + AAQualityBias = -2, + ShadowQualityBias = -2, + SkipDynamicBatching = true, + SkipTransparentObjects = true, + SkipFrontToBackSorting = true, + DecalsDrawDistance = 0.5f, + LutBias = 0.5f + } + }; + + [Header("Hardware State Mapping")] + public HardwareStateMapping[] StateMappings = new HardwareStateMapping[] + { + new HardwareStateMapping { StateLabel = "Normal", TargetTierName = "Best" }, + new HardwareStateMapping { StateLabel = "Elevated", TargetTierName = "Medium" }, + new HardwareStateMapping { StateLabel = "Throttling Imminent", TargetTierName = "Low" }, + new HardwareStateMapping { StateLabel = "Throttling", TargetTierName = "Low" } + }; + + [System.Serializable] + public struct HardwareStateMapping + { + public string StateLabel; + public string TargetTierName; + } + + [Header("Runtime Info (Read Only)")] + public int MaxTierIndexReached = 0; + public int CurrentTierIndex = 0; + public int AppliedTierIndex = 0; + + private bool _urpEnabled = false; + + private void OnEnable() + { + if (SignalManager != null) + SignalManager.HardwareStateChangedEvent += OnHardwareStateChanged; + } + + private void OnDisable() + { + if (SignalManager != null) + SignalManager.HardwareStateChangedEvent -= OnHardwareStateChanged; + } + + private void Start() + { + // Detect URP without direct reference to its types to avoid compilation errors in Built-in projects + _urpEnabled = GraphicsSettings.currentRenderPipeline != null && + GraphicsSettings.currentRenderPipeline.GetType().Name.Contains("Universal"); + + Debug.Log("[AdaptiveQualityAdapter] Initialized and monitoring hardware signals."); + } + + private void OnHardwareStateChanged(int stateIndex) + { + int targetTierIndex = ResolveStateToTierIndex(stateIndex); + CurrentTierIndex = targetTierIndex; + + if (StickyDowngrade) + { + if (targetTierIndex > MaxTierIndexReached) + { + MaxTierIndexReached = targetTierIndex; + ApplyQuality(targetTierIndex); + AppliedTierIndex = targetTierIndex; + } + else + { + Debug.Log($"[AdaptiveQualityAdapter] Sticky policy: Ignoring return to tier index {targetTierIndex}. Current Max Tier index is {MaxTierIndexReached}."); + } + } + else + { + ApplyQuality(targetTierIndex); + AppliedTierIndex = targetTierIndex; + } + } + + private int ResolveStateToTierIndex(int stateIndex) + { + string targetName = (StateMappings != null && stateIndex < StateMappings.Length) + ? StateMappings[stateIndex].TargetTierName + : string.Empty; + + for (int i = 0; i < Tiers.Length; i++) + { + if (Tiers[i].Name == targetName) return i; + } + + return Mathf.Max(0, Tiers.Length - 1); + } + + private void ApplyQuality(int tierIndex) + { + if (tierIndex < 0 || tierIndex >= Tiers.Length) return; + + QualityTierSettings settings = Tiers[tierIndex]; + Debug.Log($"[AdaptiveQualityAdapter] Applied Quality Tier {tierIndex} ({settings.Name})"); + + // --- General Scalers --- + Application.targetFrameRate = CalculateTargetFrameRate(settings.FrameRateDivisor); + QualitySettings.lodBias = settings.LODBias; + Time.fixedDeltaTime = settings.FixedDeltaTime; + + if (Camera.main != null) + Camera.main.farClipPlane = settings.FarClipPlane; + + // AdaptivePerformanceRenderSettings applies generally if the package is present + AdaptivePerformanceRenderSettings.RenderScaleMultiplier = settings.RenderScaleMultiplier; + + if (!_urpEnabled) return; + + // --- URP Specific Scalers (Only if URP is active) --- + AdaptivePerformanceRenderSettings.MaxShadowDistanceMultiplier = settings.MaxShadowDistanceMultiplier; + AdaptivePerformanceRenderSettings.MainLightShadowmapResolutionMultiplier = settings.MainLightShadowmapResolutionMultiplier; + AdaptivePerformanceRenderSettings.MainLightShadowCascadesCountBias = settings.ShadowCascadesBias; + AdaptivePerformanceRenderSettings.AntiAliasingQualityBias = settings.AAQualityBias; + AdaptivePerformanceRenderSettings.ShadowQualityBias = settings.ShadowQualityBias; + AdaptivePerformanceRenderSettings.SkipDynamicBatching = settings.SkipDynamicBatching; + AdaptivePerformanceRenderSettings.SkipTransparentObjects = settings.SkipTransparentObjects; + AdaptivePerformanceRenderSettings.SkipFrontToBackSorting = settings.SkipFrontToBackSorting; + AdaptivePerformanceRenderSettings.DecalsDrawDistance = settings.DecalsDrawDistance; + AdaptivePerformanceRenderSettings.LutBias = settings.LutBias; + } + + private int CalculateTargetFrameRate(int divisor) + { + // Unity 6+ uses RefreshRateRatio for precise display frequencies + double refreshRate = Screen.currentResolution.refreshRateRatio.value; + + // On some platforms or when not yet available, fallback to a sensible default + if (refreshRate <= 0) refreshRate = 60.0; + + // Enforce "progressive halving" (1, 2, 4, 8...) by snapping the divisor to the next power of two. + // This ensures target FPS aligns with display VSync intervals for smooth pacing. + int effectiveDivisor = Mathf.NextPowerOfTwo(Mathf.Max(1, divisor)); + + int target = Mathf.RoundToInt((float)(refreshRate / effectiveDivisor)); + + // Requirement: Frame rate can't go below 30. + return Mathf.Max(30, target); + } + } +} diff --git a/skills/asset-transformer-toolkit/SKILL.md b/skills/asset-transformer-toolkit/SKILL.md new file mode 100644 index 0000000..a16f5ae --- /dev/null +++ b/skills/asset-transformer-toolkit/SKILL.md @@ -0,0 +1,28 @@ +--- +name: asset-transformer-toolkit +description: Imports 3D models and point clouds into Unity using Asset Transformer Toolkit (formerly Pixyz Plugin), and creates, modifies, and executes RuleSets and Actions for optimization and transformation. Use this skill for any task involving RuleSets (.asset files) or ImporterScriptableObjects, including explaining, inspecting, creating, or modifying rules and actions. It also handles LOD generation. This is the primary method to import 3D models and pointclouds. +required_packages: + com.unity.industry.toolkit: ">=4.0.0" +--- +### API Reference +Tool functions are provided by `Unity.Pixyz.Plugin4Unity.Editor.AI.ATTAssistantUtilities`. They are documented inline in each reference file below alongside the classes they operate on. + +### Technical Notes +Pixyz Plugin is the former name of Asset Transformer Toolkit. Prefer using the term 'Asset Transformer Toolkit' when addressing the user, unless the user is using the term 'Pixyz' +Most classes and code are in the Unity.Pixyz.Plugin4Unity.Editor assembly. +Asset Transformer Toolkit is NOT the same thing as Asset Transformer Studio/Pixyz Studio. NEVER rely on information about Asset Transformer Studio. + +### Importers +Asset Transformer Toolkit can import 3D file from outside the project. +When modifying Importers, NEVER assume it should be immediately followed by a reimport. The import process can be very long, so it must only be launched when the user requests it. +Modify fields in Importers using the ScriptableObject API. NEVER use reflection to access or modify Importer fields — reflection is not available. +To create a Pixyz/Asset Transformer Toolkit Importer for importing a file, or to reimport a model using an existing ImporterScriptableObject, read [references/create-importer](references/create-importer.md) + +### RuleSets and Actions +Read [references/rulesets-and-actions](references/rulesets-and-actions.md) +Read [api-docs/ruleset-api](api-docs/ruleset-api.md) when the RuleSet API is needed. +Read [api-docs/rule-api](api-docs/rule-api.md) when the Rule API is needed. +Read [api-docs/ruleblock-api](api-docs/ruleblock-api.md) when the RuleBlock API is needed, including `ActionBase.Id` for constructing `RuleBlock` instances. + +### Levels of Detail +Read [references/lods](references/lods.md) diff --git a/skills/asset-transformer-toolkit/api-docs/rule-api.md b/skills/asset-transformer-toolkit/api-docs/rule-api.md new file mode 100644 index 0000000..8c0fa7d --- /dev/null +++ b/skills/asset-transformer-toolkit/api-docs/rule-api.md @@ -0,0 +1,154 @@ +## Contents +- [Rule constructors](#rule-constructors) — `Rule()` +- [Rule methods](#rule-methods) — `GetBlock`, `GetBlockIndex`, `RemoveBlockAt`, `RemoveBlock`, `AppendBlock`, `InsertBlock`, `IsLastBlock` +- [Rule properties](#rule-properties) — `Name`, `IsEnabled`, `BlocksCount`, `Blocks` + +--- + +## Rule constructors + +The Rule API reference contains the following constructors. + +### `Rule()` + +This constructor creates an empty Rule with no blocks. + +```csharp +public Rule() +``` + +## Rule methods + +The Rule API reference contains the following methods. + +### `GetBlock` + +This method retrieves the `RuleBlock` at the specified index. + +```csharp +public RuleBlock GetBlock(int i) +``` + +`GetBlock` accepts the following parameter. + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `i` | int | required | Zero-based index of the block to retrieve. | + +This method returns the `RuleBlock` at the given index. + +### `GetBlockIndex` + +This method returns the index of the given `RuleBlock` within the Rule. + +```csharp +public int GetBlockIndex(RuleBlock block) +``` + +`GetBlockIndex` accepts the following parameter. + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `block` | RuleBlock | required | The `RuleBlock` whose index to find. | + +This method returns an `int` index, or `-1` if the block is not found. + +### `RemoveBlockAt` + +This method removes the `RuleBlock` at the specified index. + +```csharp +public void RemoveBlockAt(int index) +``` + +`RemoveBlockAt` accepts the following parameter. + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `index` | int | required | Zero-based index of the block to remove. | + +### `RemoveBlock` + +This method removes a specific `RuleBlock` instance from the Rule. + +```csharp +public void RemoveBlock(RuleBlock block) +``` + +`RemoveBlock` accepts the following parameter. + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `block` | RuleBlock | required | The `RuleBlock` instance to remove. | + +### `AppendBlock` + +This method adds a `RuleBlock` to the end of the Rule and sets its back-reference to this Rule. + +```csharp +public void AppendBlock(RuleBlock block) +``` + +`AppendBlock` accepts the following parameter. + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `block` | RuleBlock | required | The `RuleBlock` instance to append. | + +### `InsertBlock` + +This method inserts a `RuleBlock` at the specified index, shifting subsequent blocks down. If `index` is beyond the last position, the block is appended instead. + +```csharp +public void InsertBlock(RuleBlock block, int index) +``` + +`InsertBlock` accepts the following parameters. + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `block` | RuleBlock | required | The `RuleBlock` instance to insert. | +| `index` | int | required | Zero-based position at which to insert the block. | + +## Rule properties + +The Rule API reference contains the following properties. + +### `Name` + +```csharp +public string Name { get; set; } +``` + +The display name of the Rule. Setting this property notifies the UI. + +> **Note:** C# property names are case-sensitive. Use `Name` (capital N) — `name` does not exist and will not compile. + +```csharp +rule.Name = "My Rule"; +string ruleName = rule.Name; +``` + +### `IsEnabled` + +```csharp +public bool IsEnabled { get; set; } +``` + +Controls whether the Rule is active within its RuleSet. When `false`, the Rule is skipped during execution. Setting this property notifies the UI. + +### `BlocksCount` + +```csharp +public int BlocksCount { get; } +``` + +The number of `RuleBlock` instances currently in the Rule. + +### `Blocks` + +```csharp +public IEnumerable Blocks { get; } +``` + +An enumerable over all `RuleBlock` instances in the Rule, in execution order. diff --git a/skills/asset-transformer-toolkit/api-docs/ruleblock-api.md b/skills/asset-transformer-toolkit/api-docs/ruleblock-api.md new file mode 100644 index 0000000..97cf80a --- /dev/null +++ b/skills/asset-transformer-toolkit/api-docs/ruleblock-api.md @@ -0,0 +1,37 @@ +## ActionBase properties + +### `Id` + +```csharp +public abstract int Id { get; } +``` + +A unique integer identifier for an action type. Pass this to the `RuleBlock(int actionId)` constructor. + +## RuleBlock constructors + +The RuleBlock API reference contains the following constructors. + +### `RuleBlock(int actionId)` + +This constructor creates a RuleBlock that will execute the action identified by `actionId`. The action instance is created lazily on first access. + +```csharp +public RuleBlock(int actionId) +``` + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `actionId` | int | required | The ID of the action this RuleBlock will trigger. Use `ActionBase.Id` to obtain this value from an action instance. | + +## RuleBlock properties + +The RuleBlock API reference contains the following properties. + +### `IsEnabled` + +```csharp +public bool IsEnabled { get; set; } +``` + +Controls whether this block is active within its Rule. When `false`, the block is skipped during execution. diff --git a/skills/asset-transformer-toolkit/api-docs/ruleset-api.md b/skills/asset-transformer-toolkit/api-docs/ruleset-api.md new file mode 100644 index 0000000..fbc40ee --- /dev/null +++ b/skills/asset-transformer-toolkit/api-docs/ruleset-api.md @@ -0,0 +1,138 @@ +## Contents +- [RuleSet methods](#ruleset-methods) — `GetRule`, `GetRuleIndex`, `RemoveRuleAt`, `RemoveRule`, `InsertRule`, `AppendRule`, `IsValid` +- [RuleSet properties](#ruleset-properties) — `RulesCount` +- [RuleSet utility functions](#ruleset-utility-functions) — `RunRuleSet` + +--- + +## RuleSet methods + +The RuleSet API reference contains the following methods. + +### `GetRule` + +This method retrieves the `Rule` at the specified index. + +```csharp +public Rule GetRule(int i) +``` + +`GetRule` accepts the following parameter. + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `i` | int | required | Zero-based index of the rule to retrieve. | + +This method returns the `Rule` at the given index. + +### `GetRuleIndex` + +This method returns the index of the given `Rule` within the RuleSet. Returns `-1` and logs an error if the RuleSet is currently running. + +```csharp +public int GetRuleIndex(Rule rule) +``` + +`GetRuleIndex` accepts the following parameter. + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `rule` | Rule | required | The `Rule` whose index to find. | + +This method returns an `int` index, or `-1` if the RuleSet is running or the rule is not found. + +### `RemoveRuleAt` + +This method removes the rule at the specified index. Has no effect and logs an error if the RuleSet is currently running. + +```csharp +public void RemoveRuleAt(int index) +``` + +`RemoveRuleAt` accepts the following parameter. + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `index` | int | required | Zero-based index of the rule to remove. | + +### `RemoveRule` + +This method removes a specific `Rule` instance from the RuleSet. Has no effect and logs an error if the RuleSet is currently running. + +```csharp +public void RemoveRule(Rule rule) +``` + +`RemoveRule` accepts the following parameter. + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `rule` | Rule | required | The `Rule` instance to remove. | + +### `InsertRule` + +This method inserts a `Rule` at the specified index, shifting subsequent rules down. Has no effect and logs an error if the RuleSet is currently running. + +```csharp +public void InsertRule(int index, Rule rule, bool notify) +``` + +`InsertRule` accepts the following parameters. + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `index` | int | required | Zero-based position at which to insert the rule. | +| `rule` | Rule | required | The `Rule` instance to insert. | +| `notify` | bool | required | When `true`, notifies the UI that the RuleSet has changed. | + +### `AppendRule` + +This method adds a `Rule` to the end of the RuleSet. Has no effect and logs an error if the RuleSet is currently running. + +```csharp +public void AppendRule(Rule rule) +``` + +`AppendRule` accepts the following parameter. + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `rule` | Rule | required | The `Rule` instance to append. | + +### `IsValid` + +This method validates that every enabled action in the RuleSet has valid input. It skips disabled rules. + +```csharp +public bool IsValid() +``` + +This method accepts no parameters. It returns `true` if all enabled actions pass validation, or `false` if any action reports an error. + +## RuleSet properties + +The RuleSet API reference contains the following properties. + +### `RulesCount` + +```csharp +public int RulesCount { get; } +``` + +The number of `Rule` instances currently in the RuleSet. + +## RuleSet utility functions + +The following functions are from `Unity.Pixyz.Plugin4Unity.Editor.AI.ATTAssistantUtilities`. + +### `RunRuleSet` + +This function executes all rules in a RuleSet asset against the current scene selection, or the entire scene if nothing is selected. + +```csharp +public static void RunRuleSet(string rulesetPath) +``` + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `rulesetPath` | string | required | Project-relative path to the RuleSet asset to run. | diff --git a/skills/asset-transformer-toolkit/references/create-importer.md b/skills/asset-transformer-toolkit/references/create-importer.md new file mode 100644 index 0000000..f0342d7 --- /dev/null +++ b/skills/asset-transformer-toolkit/references/create-importer.md @@ -0,0 +1,121 @@ +## Contents +- [Workflow steps](#workflow) — Pre-Flight, Create Importer, Configure Importer, Import +- [Importer utility functions](#importer-utility-functions) — `EnsureImporterSaveFolder`, `GetImporterTypes`, `GetImporterProperties`, `CreateImporter`, `CanImportFile`, `ImportFile` + +--- + +If a new Importer does not need to be created, skip to Step 3 + +### Step 1: Pre-Flight +- Check whether an importer asset already exists for this file. If it does, skip to Step 3. +- Verify the file to be imported exists. +- Verify the file type is supported by Pixyz/Asset Transformer Toolkit using `ATTAssistantUtilities.CanImportFile()`. +- Check what kind of file (eg. point cloud, CAD) is it. Find the Importer type that would best match it. + +### Step 2: Create Importer +- Create the appropriate Importer asset with the path to the file to be imported. The path must be relative to the Application.dataPath. +- If there are issues, fix and revalidate. Do not exceed 3 iterations. +Do not continue if this step cannot be completed successfully. + +### Step 3: Configure Importer +- If requested, change the importer's settings. +- If requested, assign the requested RuleSet to the Importer. +- If requested, make changes to the LOD generation. +Continue only if asked to also import the file. The import process can be very long, so NEVER assume you must import the file unless it was requested. + +### Step 4: Import +- Start the asynchronous import using `ATTAssistantUtilities.ImportFile()`. +- Report whether the import process started successfully. Remind the user this is a background process. + +## Importer utility functions + +The following functions are from `Unity.Pixyz.Plugin4Unity.Editor.AI.ATTAssistantUtilities`. + +### `EnsureImporterSaveFolder` + +Returns the project-relative asset save folder configured in Asset Transformer Toolkit Project Settings, creating it if it does not already exist. Use this as the destination path when creating a new `ImporterScriptableObject` asset. + +```csharp +public static string EnsureImporterSaveFolder() +``` + +Returns a `string` such as `"Assets/3DModels"`. + +### `GetImporterTypes` + +Returns the names of all `ImporterScriptableObject` types available in the project, including user-defined importers. Always call this before creating or referencing an importer type — never assume a type exists. + +```csharp +public static string[] GetImporterTypes() +``` + +Returns a `string[]` of unqualified type names (e.g. `"CADImporterScriptableObject"`). + +### `GetImporterProperties` + +Returns the public serialized fields of an `ImporterScriptableObject` type by name, including fields from intermediate base classes. Use this to discover what settings are available on any importer type — concrete importer classes may be internal or user-defined. + +```csharp +public static ImporterPropertyInfo[] GetImporterProperties(string typeName) +``` + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `typeName` | string | required | Unqualified type name as returned by `GetImporterTypes()`. | + +Returns an `ImporterPropertyInfo[]`. Each entry has the following fields: + +| Field | Type | Description | +|-------|------|-------------| +| `Name` | string | Human-readable property name. | +| `Type` | string | Unqualified type name of the field. | +| `SerializedPropertyPath` | string | The exact string to pass to `SerializedObject.FindProperty`. For auto-properties declared with `[field: SerializeField]`, this differs from `Name` (e.g. `k__BackingField`). Always use this field — never construct the path from `Name` yourself. | + +### `CreateImporter` + +Creates a new `ImporterScriptableObject` asset for the given file. Call `GetImporterTypes()` first to confirm the type name — never assume or invent one. Use `EnsureImporterSaveFolder()` to get the correct save path. + +```csharp +public static string CreateImporter(string filePath, string typeName, string savePath) +``` + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `filePath` | string | required | Absolute system path to the 3D file the importer will reference. | +| `typeName` | string | required | Unqualified importer type name as returned by `GetImporterTypes()`. | +| `savePath` | string | required | Project-relative folder path where the importer asset will be saved. | + +Returns a `string` with the project-relative path to the created asset (e.g. `"Assets/3DModels/MyModel.asset"`). Pass this path to `ImportFile` to trigger import. + +### `CanImportFile` + +Checks whether the Asset Transformer Toolkit supports a file format. Call this before `ImportFile` to avoid runtime errors. Throws if no file exists at `filePath`. + +```csharp +public static bool CanImportFile(string filePath) +``` + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `filePath` | string | required | Absolute system path to the 3D file to check. | + +Returns `true` if the file format is supported, `false` otherwise. + +### `ImportFile` + +Triggers import or re-import using an existing `ImporterScriptableObject` asset. Requires an `ImporterScriptableObject` to already exist at `importerPath`. + +```csharp +public static void ImportFile(string importerPath) +``` + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `importerPath` | string | required | Project-relative path to the `ImporterScriptableObject` asset. | + +### Technical Notes +- Because import is an asynchronous process, the imported files may not yet exist in the project by the time the prompt finishes execution. +- The imported model will always be saved to the same directory that the ImporterScriptableObject lives in. +- Importers are designed to be extended. Check which Importer types exist in the project using `ATTAssistantUtilities.GetImporterTypes()` instead of making assumptions like a point cloud file always maps to the PointCloudImporterScriptableObject. +- Only one import should be processing at a time. +- When creating a new ImporterScriptableObject, save it under the folder returned by `ATTAssistantUtilities.EnsureImporterSaveFolder()`. This reflects the user-configured save folder from Project Settings and defaults to `"Assets/3DModels"`. diff --git a/skills/asset-transformer-toolkit/references/lods.md b/skills/asset-transformer-toolkit/references/lods.md new file mode 100644 index 0000000..9e4a768 --- /dev/null +++ b/skills/asset-transformer-toolkit/references/lods.md @@ -0,0 +1,61 @@ +Choose a path depending on the task. + +### Path A: CADImporterScriptableObject +Choose this path if working with an object that is or inherits from CADImporterScriptableObject. +- Identify the types of LODs available in the project. Choose the one(s) most appropriate for your case. +- Find the LODGenerator property in the Importer. +- Execute the requested task. +- Verify the number of rules present in the Generator does not exceed 7. + +### Path B: Scene Object +Choose this path when asked to work with a scene asset directly. +- Add a UnityEngine.PixyzPlugin4Unity.Components.LODGeneratorComponent script to the GameObject if one doesn't exist anywhere in the hierarchy. +- Execute the given task. + +### Path C: PointCloudImporter +Choose this path when working with a PointCloudImporterScriptableObject object. +- The PointCloudImporter only permits enabling/disabling LOD generation and setting the number of LODs to be generated. Other actions like choosing the type of LODRule to apply is unsupported. +- Models imported by the PointCloudImporter will not have a LODGeneratorComponent script attached to them. +- If the task is permitted, execute it. + +### Path D: Other Importer +Choose this path if working with a different importer type. +- This is a user defined importer and not part of the base Asset Transformer Toolkit package. Precise instructions cannot be provided. +- Attempt to perform the requested task, but do not exceed three iterations. +- If the task cannot be performed successfully, inform the user working with this object is not currently supported by Assistant. + +## LOD utility functions + +The following functions are from `Unity.Pixyz.Plugin4Unity.Editor.AI.ATTAssistantUtilities`. + +### `GetLODRules` + +Returns all available `LODRule` implementations and their parameters. Use this before constructing or modifying a LOD configuration to discover valid types. + +```csharp +public static LODRuleDescription[] GetLODRules() +``` + +Returns a `LODRuleDescription[]`, each containing the type name, assembly-qualified name, and parameter list. + +### `LODRuleDescription` + +| Field | Type | Description | +|-------|------|-------------| +| `Name` | string | Unqualified class name of the LODRule type. | +| `QualifiedName` | string | Assembly-qualified name, suitable for `Type.GetType`. | +| `Parameters` | `LODParameterInfo[]` | Configurable properties on this LODRule type. | + +### `LODParameterInfo` + +| Field | Type | Description | +|-------|------|-------------| +| `Name` | string | Property name. | +| `Type` | string | Property type name. | + +### Technical Notes +- The user will need to refresh the Inspector to see changes applied by Assistant. +- In an Importer's settings for LOD generation, the number of LODs set to generate does not include LOD0. There will always be one more LOD than the setting says. For example, setting the PointCloudImporter's NumberofLODs setting to 1 will result in the model having two LODs: LOD0 and LOD1. +- While Unity supports up to 8 LODs including LOD0, the PointCloudImporter is a special case that only supports 7. +- With the exception of models imported by the PointCloudImporter, models that were imported with LODs will have a LODGeneratorComponent script. +- LODs have nothing to do with RuleSets and Actions. RuleSets and Actions will NOT help with any LOD-related task. diff --git a/skills/asset-transformer-toolkit/references/rulesets-and-actions.md b/skills/asset-transformer-toolkit/references/rulesets-and-actions.md new file mode 100644 index 0000000..834f21a --- /dev/null +++ b/skills/asset-transformer-toolkit/references/rulesets-and-actions.md @@ -0,0 +1,227 @@ +## Contents +- [Actions](#actions) +- [RuleSets](#rulesets) + - Description + - Modifying RuleSets + - Creating RuleSets + - Setting Action Parameters + - Running RuleSets + - Validation Checklist +- [Action utility functions](#action-utility-functions) — `GetActionsList`, `GetActionDefinitions`, `SetActionParameter` +- [Action utility output types](#action-utility-output-types) — `ActionInfo`, `ActionDefinition`, `ActionParameterInfo`, `EnumInfo` + +--- + +## Actions + +Actions are classes derived from UnityEditor.PixyzPlugin4Unity.Actions.ActionBase that execute a task on a list of input objects. +It is expected that users will create additional Action classes to extend the Actions available in the base package. + +## RuleSets + +### Description +RuleSets are a SerializedObject containing a list of Rule instances, which contain a list of Action instances. RuleSets are used to ensure a set of Actions are executed in a specific order. +RuleSets are derived from ScriptableObject and must end with the '.asset' extension. +RuleSets support conversion to json. This will include information about the Actions they contain. + +### Modifying RuleSets + +#### Step 1: Load RuleSet +- Use AssetDatabase.LoadAssetAtPath to load the asset from memory. + +#### Step 2: Construct the script +- Read the API reference for the RuleSet class. Also read the API reference for the Rule and RuleBlock classes if required. +- Construct a script using the APIs from those files, to run as C# in a live Editor. Here is an + example that adds a Decimate Action to a RuleSet: + +```csharp +using UnityEngine; +using UnityEditor; +using UnityEditor.PixyzPlugin4Unity.RuleEngine; + +string path = "Assets/Rulesets/OptimizationRuleSet.asset"; +RuleSet ruleSet = AssetDatabase.LoadAssetAtPath(path); + +if (ruleSet == null) + throw new System.Exception($"RuleSet not found at {path}"); + +Undo.RecordObject(ruleSet, "Add Decimate action"); + +if (ruleSet.RulesCount == 0) + throw new System.Exception($"No rules found in RuleSet {path}"); + +Rule rule = ruleSet.GetRule(0); + +// Decimate Action ID is 277054868 +RuleBlock decimateBlock = new RuleBlock(277054868); + +rule.AppendBlock(decimateBlock); + +EditorUtility.SetDirty(ruleSet); +AssetDatabase.SaveAssets(); + +return $"Added Decimate action to {path} (Rule index 0)"; +``` + +A proper script for modifying a RuleSet has the following traits: +- Does not use the ScriptableObject API (this bypasses necessary event triggers). + +#### Step 3: Validation +- Run the script as C# in a live Editor. +- Validate the RuleSet against the validation checklist. + + +### Creating RuleSets + +#### Step 1: Create RuleSet +- Create the UnityEditor.PixyzPlugin4Unity.RuleEngine.RuleSet asset. +Continue to Step 2 if actions need to be added to the RuleSet. If not, add the GetContextGameObjects action and jump to Step 3. + +#### Step 2: Add Actions + +**Preflight** +- Ensure the RuleSet exists. +- Choose the combination of Actions that will best perform the requested procedure. NEVER create new Actions without explicit permission. Instead, use the Actions returned by `GetActionsList`. +- Divide Actions into Rules based on the GameObject they need to act upon. Each Rule initially executes on every GameObject unless the input is narrowed with a Filter action. Example: If only lights need to be disabled and only meshes with >10000 vertices need to be decimated, two rules will be needed as this is two different groups of GameObjects. + +**Adding Rules** +- Check whether the existing Rule(s) in the RuleSet is just the GetContextGameObjects action. If it is, append the group of actions to it rather than creating a new Rule. +- If a new Rule needs to be created, add it to the RuleSet. +- Add Actions to the Rules. +- Set action parameters if required — see Setting Action Parameters below. + +**Technical Notes** +- All Actions derive from the ActionBase class. +- Actions are located in the UnityEditor.PixyzPlugin4Unity.Actions namespace. + +#### Step 3: Validation +- Validate the RuleSet logic using the validation checklist. + + +### Setting Action Parameters + +#### Step 1: Gather data +- Gather any missing information needed to call `ATTAssistantUtilities.SetActionParameter`. If you need to retrieve a GlobalObjectId, first read the GlobalObjectId class to choose the correct function to call. +- Call `ATTAssistantUtilities.SetActionParameter` to set the parameter. +- If the result is false and not an exception, retry a maximum of three times. +- Follow a path based on the result. + +#### Path A: AITypeSecurityException +Follow these steps if `ATTAssistantUtilities.SetActionParameter` threw an AITypeSecurityException. +- Inform the user the parameter cannot be set programmatically for security reasons. +- Advise the user to manually set the parameter and what value to set it to. + +#### Path B: Exception +Follow these steps if `ATTAssistantUtilities.SetActionParameter` threw any other exception. +- Warn the user the property was unable to be set. +- Advise the user to manually set the parameter and what value to set it to. + +#### Path C: Success +Perform these steps if `ATTAssistantUtilities.SetActionParameter` returned true. +- If the property was set to a scene GameObject, NEVER validate it because it is not persistent. INSTEAD warn the user the property value is temporary and will be lost. +- Report the success to the user. + +#### Path D: Failure +Perform this step if `ATTAssistantUtilities.SetActionParameter` always returns false. +- Report the failure to the user and advise them to set the parameter manually. Tell them what the property should be set to. + +All paths are exclusive. + +**Technical Notes** +- For the Decimate action specifically, if mesh quality is going to be set to a preset, the Criterion parameter must also be set to Quality. +- Prefer using presets when possible rather than individually setting each value. +- If a preset is used, avoid changing values the preset changed unless requested otherwise. + +**Safety & Constraints** +1. **One-Strike Rule**: If `ATTAssistantUtilities.SetActionParameter` throws an AITypeSecurityException, you MUST TERMINATE the task immediately. Do NOT use raw C# execution, reflection, or any other method to bypass this. Follow the steps in Path A as your final actions. + + +### Running RuleSets +Use `ATTAssistantUtilities.RunRuleSet()` instead of the RuleSet's public API to run a RuleSet. +Only one RuleSet must be running at a time. +When running a RuleSet, remind the user it is a background task/asynchronous. + + +### Validation Checklist +- The first Action in each Rule is GetContextGameObjects or RunRules. +- If the RunRules Action is in a Rule, it is the only Action. +- Each Rule has at least one Action. + + +## Action utility functions + +The following functions are from `Unity.Pixyz.Plugin4Unity.Editor.AI.ATTAssistantUtilities`. + +### `GetActionsList` + +Returns all Rule Engine actions available in the project, including user-defined actions. Use this when you do not already know an action's ID. Pass the returned IDs to `GetActionDefinitions` to inspect parameters. + +```csharp +public static ActionInfo[] GetActionsList() +``` + +Returns an `ActionInfo[]` containing the name, tooltip, and ID of every available action. + +### `GetActionDefinitions` + +Returns parameter definitions for one or more actions by unqualified class name (e.g. `"Decimate"`, not `"UnityEditor.PixyzPlugin4Unity.Actions.Decimate"`). Use this before `SetActionParameter` to obtain correct parameter names and types. Throws if an action class name is not found. + +```csharp +public static ActionDefinition[] GetActionDefinitions(string[] actionClassNames) +``` + +Returns an `ActionDefinition[]`, each containing the action ID and its full parameter list. + +### `SetActionParameter` + +Sets a `UserParameter` field value on an action within a RuleSet. Use `GetActionDefinitions` first to obtain the correct parameter name. Returns `false` if the field was not found. + +```csharp +public static bool SetActionParameter(string ruleSetPath, int ruleIndex, int ruleblockIndex, string parameterName, string value) +``` + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `ruleSetPath` | string | required | Project-relative path to the RuleSet asset. | +| `ruleIndex` | int | required | Zero-based index of the rule containing the action. | +| `ruleblockIndex` | int | required | Zero-based index of the action block within the rule. | +| `parameterName` | string | required | The `ParameterPath` value from `GetActionDefinitions`. Must not be fully qualified. | +| `value` | string | required | String representation of the value to set. For Unity assets or scene objects, provide a `GlobalObjectId` string. For `LayerMask`, use layer names separated by `\|`. | + +Returns `true` if the parameter was set and the RuleSet saved, `false` if the field was not found. + +## Action utility output types + +### `ActionInfo` + +| Field | Type | Description | +|-------|------|-------------| +| `Name` | string | Fully qualified class name of the action. | +| `Description` | string | Tooltip text describing what the action does. | +| `ID` | int | Unique integer ID. Pass to `GetActionDefinitions` or use as `RuleBlock` action ID. | + +### `ActionDefinition` + +| Field | Type | Description | +|-------|------|-------------| +| `ID` | int | Unique integer ID of the action. | +| `Parameters` | `ActionParameterInfo[]` | All configurable `UserParameter` fields on the action. | + +### `ActionParameterInfo` + +| Field | Type | Description | +|-------|------|-------------| +| `Name` | string | Immediate field name. | +| `ParameterPath` | string | Full dot-separated path to pass as `parameterName` to `SetActionParameter` (e.g. `"advancedParametersQuality.surfacicTolerance"`). | +| `Type` | string | Fully qualified type name of the field. | +| `Description` | string | Tooltip describing the parameter. | +| `IsConditional` | bool | `true` if this parameter is only visible under certain conditions. | +| `PossibleEnumValues` | `EnumInfo[]` | Valid values if the parameter is an enum type. | +| `NestedParameters` | `ActionParameterInfo[]` | Child parameters for struct fields. | + +### `EnumInfo` + +| Field | Type | Description | +|-------|------|-------------| +| `Label` | string | Name of the enum value. | +| `Value` | Int64 | Underlying integer value of the enum member. | diff --git a/skills/setup-audiorandomcontainer/SKILL.md b/skills/setup-audiorandomcontainer/SKILL.md new file mode 100644 index 0000000..d593dd6 --- /dev/null +++ b/skills/setup-audiorandomcontainer/SKILL.md @@ -0,0 +1,35 @@ +--- +name: setup-audiorandomcontainer +description: Creates AudioRandomContainer assets with randomized audio playback. Use for dynamic sounds from multiple clips with variation of volume, pitch and playback timing. +required_editor_version: ">=6000.3.13" +--- +# Set up AudioRandomContainer + +## When to use this skill +Use when the user wants to: +- Create dynamic audio that randomizes between multiple clips +- Set up audio with randomized volume, pitch, or timing +- Configure automatic or triggered audio playback with variation + +## DO's +- Use the internal API from `UnityEngine.Audio` and `UnityEditor.Audio` namespaces +- Use standard C# construction: `new AudioRandomContainer()` +- Use `AudioRandomContainerUtilities.AddElements()` or `container.AddElements()` to add clips +- Save the asset immediately after creation using `AssetDatabase.SaveAssetIfDirty()` +- Default filename: "New Audio Random Container.asset" (auto-enumerate if exists) + +## DON'Ts +- ❌ Never use reflection +- ❌ Never use `ScriptableObject.CreateInstance` +- ❌ Never use `SerializedObject`/`SerializedProperty` +- ❌ Do not save to disk after setting properties (leave for user) +- ❌ Do not auto-play `AudioSource` after assigning container unless explicitly requested + +## Procedure +1. Create the container: `var container = new AudioRandomContainer()` +2. Generate unique path and save to disk with `AssetDatabase.CreateAsset()` and `AssetDatabase.SaveAssetIfDirty()` +3. **Only if requested**: Add audio clips using `container.AddElements(clipArray)` +4. **Only if requested**: Update properties using the API (see references) + +## References +See [references/api.md](references/api.md) \ No newline at end of file diff --git a/skills/setup-audiorandomcontainer/references/api.md b/skills/setup-audiorandomcontainer/references/api.md new file mode 100644 index 0000000..f92477b --- /dev/null +++ b/skills/setup-audiorandomcontainer/references/api.md @@ -0,0 +1,132 @@ +# AudioRandomContainer API Reference + +## Table of Contents +- [Architecture](#architecture) + - [Key Types](#key-types) + - [Assignment to AudioSource](#assignment-to-audiosource) + - [Scope Restrictions](#scope-restrictions) +- [Asset Creation](#asset-creation) + - [Construction](#construction) + - [Saving](#saving) +- [Getting and Setting Properties](#getting-and-setting-properties) + - [Access Methods](#access-methods) + - [Adding Elements (Clips)](#adding-elements-clips) + - [Saving After Property Changes](#saving-after-property-changes) + - [Conditional Property Requirements](#conditional-property-requirements) +- [Code Example](#code-example) + +## Architecture + +### Key Types +- **`AudioRandomContainer`**: Implements `IAudioGenerator` for dynamic audio playback +- **`AudioContainerElement`**: Holds an `AudioClip` reference plus additional properties (not yet accessible via API) + +### Assignment to AudioSource +- ✅ **DO**: Use `AudioSource.generator` to assign the container +- ❌ **DON'T**: Use `AudioSource.resource` (will be deprecated) + +### Scope Restrictions +- `AudioRandomContainer` is **internal only** - reach it from C# running inside the Editor +- **Never reference** this type in generated project scripts (causes compilation errors) +- **Always import** `UnityEngine.Audio` namespace when working with this type + +## Asset Creation + +### Construction +- ✅ **DO**: Use standard C# construction: `new AudioRandomContainer()` +- ❌ **DON'T**: Use reflection or `ScriptableObject.CreateInstance` +- Proper initialization is handled in the native backing layer + +### Saving +- **Required**: Save to disk immediately after creation using: + ```csharp + AssetDatabase.CreateAsset(container, path); + AssetDatabase.SaveAssetIfDirty(container); + ``` + +## Getting and Setting Properties + +### Access Methods +- ✅ **DO**: Access all properties through the API +- ❌ **DON'T**: Use reflection or `SerializedObject`/`SerializedProperty` + +### Adding Elements (Clips) +- ✅ **DO**: Use `container.AddElements(clipArray)` or `AudioRandomContainerUtilities.AddElements()` +- ❌ **DON'T**: Set the `elements` property directly + +### Saving After Property Changes +- ❌ **DON'T**: Save the asset to disk after setting properties +- Leave saving for the user to do manually + +### Conditional Property Requirements +These properties only take effect when their corresponding "enabled" flag is true: + +| Property | Requires This Flag Set to `true` | +|----------|----------------------------------| +| `volumeRandomizationRange` | `volumeRandomizationEnabled` | +| `pitchRandomizationRange` | `pitchRandomizationEnabled` | +| `automaticTriggerTimeRandomizationRange` | `automaticTriggerTimeRandomizationEnabled` | +| `loopCountRandomizationRange` | `loopCountRandomizationEnabled` | + +## Code Example + +```csharp +// ============================================================================ +// STEP 1: Create the asset +// ============================================================================ +var container = new AudioRandomContainer(); + +// ============================================================================ +// STEP 2: Save to disk (REQUIRED immediately after creation) +// ============================================================================ +var path = "Assets/New Audio Random Container.asset"; +path = AssetDatabase.GenerateUniqueAssetPath(path); +AssetDatabase.CreateAsset(container, path); +AssetDatabase.SaveAssetIfDirty(container); + +// ============================================================================ +// STEP 3: Add audio clips +// ============================================================================ + +// Option A: Add empty elements (no clips assigned yet) +container.AddElements(2); + +// Option B: Add elements with clips assigned +const string clip1Path = "Assets/clip1.wav"; +const string clip2Path = "Assets/clip2.wav"; +var clip1 = AssetDatabase.LoadAssetAtPath(clip1Path); +var clip2 = AssetDatabase.LoadAssetAtPath(clip2Path); +container.AddElements(new[] { clip1, clip2 }); + +// ============================================================================ +// STEP 4: Set properties (ONLY if requested by user) +// ============================================================================ + +// Volume settings +container.volume = -20; // Range: [-80, 0], unit: decibels +container.volumeRandomizationEnabled = true; // Required for range to work +container.volumeRandomizationRange = new Vector2(-10, 10); // Range: [-80, 80], unit: decibels + +// Pitch settings +container.pitch = 500; // Range: [-1200, 1200], unit: cents +container.pitchRandomizationEnabled = true; // Required for range to work +container.pitchRandomizationRange = new Vector2(-100, 100); // Range: [-1200, 1200], unit: cents + +// Playback settings +container.playbackMode = AudioRandomContainerPlaybackMode.Random; +container.avoidRepeatingLast = 0; // Only for Random mode; must be < number of clips + +// Trigger settings +container.triggerMode = AudioRandomContainerTriggerMode.Automatic; +container.automaticTriggerMode = AudioRandomContainerAutomaticTriggerMode.Pulse; +container.automaticTriggerTime = 2f; // Unit: seconds +container.automaticTriggerTimeRandomizationEnabled = true; // Required for range to work +container.automaticTriggerTimeRandomizationRange = new Vector2(1f, 3f); // Range: [-60, 60] + +// Loop settings +container.loopMode = AudioRandomContainerLoopMode.Clips; +container.loopCount = 5; // Disabled if loop mode is Infinite +container.loopCountRandomizationEnabled = true; // Required for range to work +container.loopCountRandomizationRange = new Vector2(3, 5); // Range: [-10, 10] + +``` diff --git a/skills/setup-game-inputs/SKILL.md b/skills/setup-game-inputs/SKILL.md new file mode 100644 index 0000000..1ffb1da --- /dev/null +++ b/skills/setup-game-inputs/SKILL.md @@ -0,0 +1,32 @@ +--- +name: setup-game-inputs +description: Handles game input setup and configuration — player controls, action maps, bindings, control schemes, and Input Actions assets. Use when setting up keyboard, mouse, or gamepad input, configuring PlayerInput or UI input, generating C# input wrappers, implementing rebinding, or working with the Input System package or Legacy Input Manager. +--- + +## Step 1: Determine Active Input Handler + +The system prompt context should provide the current active input system. If not check the project setting. + +- **Input System Package (New)** — new Input System only; `ENABLE_INPUT_SYSTEM` is defined. +- **Input Manager (Old)** — legacy Input Manager only; `ENABLE_LEGACY_INPUT_MANAGER` is defined. +- **Both** — both defines are set. + +Changing this setting requires an Editor restart. Read the matching path below before deep implementation. + +## Path A: Input System (new) + +**When:** Active Input Handler is **Input System Package (New)**. +Read [input-system.md](references/input-system.md) + +## Path B: Legacy Input Manager (old) + +**When:** Active Input Handler is **Input Manager (Old)** only. +Use **Project Settings > Input Manager** axes and `UnityEngine.Input` (`GetAxis`, `GetButton`, etc.). + + +## Path C: Both + +**When:** Active Input Handler is **Both**. + +Treat **Input System (new)** as the default. Read reference [input-system.md](references/input-system.md) + diff --git a/skills/setup-game-inputs/references/input-system.md b/skills/setup-game-inputs/references/input-system.md new file mode 100644 index 0000000..13abb6e --- /dev/null +++ b/skills/setup-game-inputs/references/input-system.md @@ -0,0 +1,924 @@ +## Table of Contents +- [Performance Notes](#performance-notes) +- [0. Package Installation + Project Setting Check (Must Do First)](#0-package-installation-project-setting-check-must-do-first) +- [1. Pre-Flight Check (Crucial)](#1-pre-flight-check-crucial) +- [2. Gather Missing Information](#2-gather-missing-information) +- [3. Planning & Execution Steps](#3-planning-execution-steps) +- [4. Validation Checklist (Must Confirm)](#4-validation-checklist-must-confirm) +- [5. Final Confirmation Message (What reporting back)](#5-final-confirmation-message-what-reporting-back) +- [Important API notes](#important-api-notes) +- [Core Concepts Reference](#core-concepts-reference) +- [Responding to Actions Reference](#responding-to-actions-reference) +- [PlayerInput Component Reference](#playerinput-component-reference) +- [PlayerInputManager Reference (Multiplayer)](#playerinputmanager-reference-multiplayer) +- [UI Support Reference](#ui-support-reference) +- [Interactions Reference](#interactions-reference) +- [Composite Bindings Reference](#composite-bindings-reference) +- [Interactive Rebinding Reference](#interactive-rebinding-reference) +- [Processors Reference](#processors-reference) +- [Direct Device Access Reference (Prototyping Only)](#direct-device-access-reference-prototyping-only) +- [Migration from Legacy Input Manager](#migration-from-legacy-input-manager) +- [Common Mistakes to Avoid](#common-mistakes-to-avoid) + + +## Performance Notes +- Do this thoroughly. +- Quality is more important than speed. + +## 0. Package Installation + Project Setting Check (Must Do First) + +1. Package Installation Check +First of all **verify that the com.unity.inputsystem package is installed** +**Install if Missing:** add the package to the project manifest — `Packages/manifest.json`, +under `dependencies`: + +```json +"com.unity.inputsystem": "" +``` + +Don't invent the version string. Read the current one from the Unity registry — +`https://packages.unity.com/com.unity.inputsystem` lists every published version — or copy the version an +adjacent Unity package in this manifest already uses. A version that doesn't exist makes +Unity fail resolution **silently**, so a wrong guess looks like nothing happened. + +Unity resolves the new dependency the next time the Editor regains focus. This needs no +Editor connection, which is why it's the default route here. + +If you do have a live Editor to run C# in, the equivalent is: + +```csharp +using UnityEditor.PackageManager; + +var request = Client.Add("com.unity.inputsystem"); +UnityEngine.Debug.Log("Requested com.unity.inputsystem. Progress shows in the Package Manager window."); +``` +**Proceed:** Only continue to the next steps once InputSystem is confirmed to be installed. + +2. Active Input Handling Check +After verifying the package is installed, check the project's Active Input Handling setting: +- **Input System Package (New)** — only the new Input System is active. `ENABLE_INPUT_SYSTEM` is defined. +- **Input Manager (Old)** — only the legacy Input Manager is active. `ENABLE_LEGACY_INPUT_MANAGER` is defined. +- **Both** — both systems are active. Both defines are set. Use Input System (new) as default. + +Changing the Active Input Handling setting requires an Editor restart. + +## 1. Pre-Flight Check (Crucial) + +### Check Project-Wide Actions (CRITICAL — DO NOT GREP PROJECT FILES) + +**DO NOT** search or grep `ProjectSettings/ProjectSettings.asset`, `ProjectSettings/EditorBuildSettings.asset`, or any other project settings files to find the project-wide actions asset. The reference is stored internally via `EditorBuildSettings` config objects and is **not human-readable** in project files. Attempting to grep these files will fail and waste time. + +**The ONLY correct way** to check and manage project-wide actions is the C# API below, run in a live Editor: + +**To check if project-wide actions are assigned and inspect their contents:** + +Two routes. The file route needs no Editor: `.inputactions` assets are JSON on disk, so +glob for `*.inputactions` and read one directly to see its action maps, actions and +bindings. What a file cannot tell you is which asset is *assigned* project-wide — that +lives in project settings. + +With a live Editor to run C# in: + +```csharp +using UnityEngine; +using UnityEngine.InputSystem; + +var actions = InputSystem.actions; +if (actions == null) +{ + Debug.Log("No project-wide Input Actions asset is currently assigned."); + Debug.Log("To create one: Edit > Project Settings > Input System Package > Create a new project-wide Action Asset"); + + // Also check if any .inputactions assets exist in the project that could be assigned + var guids = UnityEditor.AssetDatabase.FindAssets("t:InputActionAsset"); + if (guids.Length > 0) + { + Debug.Log($"Found {guids.Length} InputActionAsset(s) in the project that could be assigned:"); + foreach (var guid in guids) + { + var assetPath = UnityEditor.AssetDatabase.GUIDToAssetPath(guid); + Debug.Log($" - {assetPath}"); + } + } + return "no project-wide actions assigned"; +} + +var path = UnityEditor.AssetDatabase.GetAssetPath(actions); +var report = new System.Text.StringBuilder(); +report.AppendLine($"Project-wide actions asset: {actions.name} ({path})"); +report.AppendLine($"Action Maps ({actions.actionMaps.Count}):"); +foreach (var map in actions.actionMaps) +{ + report.AppendLine($" - {map.name} ({map.actions.Count} actions)"); + foreach (var action in map.actions) + { + report.AppendLine($" {action.name} (Type: {action.type}, ExpectedControlType: {action.expectedControlType}, Bindings: {action.bindings.Count})"); + } +} +report.AppendLine($"Control Schemes ({actions.controlSchemes.Count}):"); +foreach (var scheme in actions.controlSchemes) +{ + report.AppendLine($" - {scheme.name}"); +} +return report.ToString(); +``` + +Return the report rather than only logging it: logs land in the Editor console, while the +returned value is what comes back to whoever ran the snippet. + +**To assign an existing .inputactions asset as project-wide:** +```csharp +var asset = UnityEditor.AssetDatabase.LoadAssetAtPath("Assets/MyActions.inputactions"); +if (asset != null) +{ + InputSystem.actions = asset; + UnityEngine.Debug.Log($"Assigned '{asset.name}' as project-wide actions."); +} +``` +Note: `InputSystem.actions` can only be assigned in Edit mode (not Play mode) and the asset must be a persistent file on disk inside the Assets folder. + +**To find all .inputactions assets in the project:** +```csharp +var guids = UnityEditor.AssetDatabase.FindAssets("t:InputActionAsset"); +foreach (var guid in guids) +{ + var assetPath = UnityEditor.AssetDatabase.GUIDToAssetPath(guid); + UnityEngine.Debug.Log($"Found: {assetPath}"); +} +``` + +### UI Input Module sanity (if the project uses UI) +If the user uses Unity UI (uGUI): +- Find (or create) an EventSystem. +- Ensure it has `InputSystemUIInputModule`. +- If `StandaloneInputModule` is present, remove it to avoid conflicts. + +For UI Toolkit (Unity 2023.2+): The UI actions defined in the default project-wide actions directly map to UI Toolkit input. `InputSystemUIInputModule` component is not needed. + +For UI Toolkit (pre-2023.2): `InputSystemUIInputModule` component must be used. + +## 2. Gather Missing Information +Before invoking tools, ensure the following details exist. If not, ask the user: + +### Core questions +* **Asset name/path:** Where input actions should be? (Default: Assets/Input/InputActions.inputactions) +* **Action maps:** e.g. Player, UI, Vehicle, Debug +* **Target GameObject(s):** which object gets PlayerInput / input scripts? (player prefab, character root, etc.) +* **Platforms/devices:** Keyboard&Mouse, Gamepad, Touch, XR? +* **Gameplay actions needed:** e.g. Move, Look, Jump, Sprint, Crouch, Interact, Fire, Aim, Pause, Navigate UI + +### Per-action details (important) +* **For each action, gather:** + * **Action Type:** Value / Button / PassThrough + * **Expected Control Type:** Vector2, Axis, Button, Delta, etc. + * **Bindings:** default keys/buttons, plus optional composites (2D Vector WASD, arrow keys) + * **Interactions/Processors:** Hold/Tap, Press behavior, Deadzone, Normalize, Invert Y, Sensitivity + +### Action Type Selection Guide + +| Action Type | Use When | Behavior | +|-------------|----------|----------| +| **Value** (default) | Continuous inputs: movement sticks, triggers, mouse delta | Tracks the most actuated control. Performs initial state check on enable. Conflict resolution picks highest magnitude. | +| **Button** | Discrete press actions: jump, fire, interact | Like Value but only binds to `ButtonControl`. No initial state check (avoids re-triggering held buttons on enable). | +| **PassThrough** | Multi-device monitoring, UI pointer actions, raw input | No conflict resolution. Every bound control change fires a callback. No single "driving" control. | + + +## 3. Planning & Execution Steps + +0. Identify existing input patterns and architecture in the project and follow them + +1. Create/Update the Input Actions asset +- Reuse existing Input Actions asset (or create new one) +- Create Input control schemes for required devices (or reuse existing). Don't skip this step, it is important for local multiplayer to have control schemes. +- Create required Action Maps (or reuse existing). +- Create Actions with correct types/control types. +- Add Bindings (including composites like WASD for Move). + +2. Generate a C# wrapper (Optional) +If the user wants strongly-typed code or it is the pattern in the project: +- Enable Generate C# Class on the .inputactions asset +- Set wrapper class name (e.g. GameInput) +- Ensure it regenerates when the asset changes + +3. Hook into gameplay +Base it on existing input patterns in the project. + +* **Option 1 - PlayerInput-based setup** +- Add PlayerInput to the player root (or ensure it exists). +- Assign the Input Actions asset to PlayerInput.actions. +- Set: + - Default Map (e.g. Player) + - Notification Behavior based on the project patterns or what the user specified (Prefer "Invoke CSharp Events" if no preference nor existing pattern exist): + * **Send Messages** when the PlayerInputs sends messages (Default) + - Implement or make sure there are functions in a script to take the messages + - Make sure the PlayerInput.NotificationBehaviour uses SendMessages + - Method signature: `public void OnActionName()` or `public void OnActionName(InputValue value)` + - The component must be on the same GameObject as PlayerInput + - `InputValue` is only valid during the callback; do not store it + * **Broadcast Messages** same as Send Messages but also sends to child GameObjects + - Method signature: same as Send Messages + - Component can be on the same or any child GameObject + * **Invoke CSharp Events** when a C# script subscribes for the PlayerInput + - Make a script to subscribe for the `onActionTriggered` event on the PlayerInput + - Make sure the PlayerInput.NotificationBehaviour uses InvokeCSharpEvents + - Method signature: receives `InputAction.CallbackContext` + * **Invoke Unity Events** when the PlayerInputs has set up unity events to call functions + - Implement or make sure there are functions in a script to take unity event + - Set up the unity events on the PlayerInput to call the functions + - Make sure the PlayerInput.NotificationBehaviour uses InvokeUnityEvents + - Method signature: `public void OnActionName(InputAction.CallbackContext context)` +- If using control schemes, set Default Control Scheme (optional). + +**CRITICAL PlayerInput rule:** When writing input code that works with PlayerInput, do NOT use `InputSystem.actions`. Use `playerInput.actions` instead. PlayerInput creates private copies of actions for device filtering in multiplayer. Using `InputSystem.actions` bypasses automatic device assignment. + +**Project-wide actions + PlayerInput caveat:** With project-wide actions, all action maps may be enabled by default. Disable `InputSystem.actions` and enable only the map PlayerInput should use: +```csharp +void Start() +{ + playerInput = GetComponent(); + InputSystem.actions.Disable(); + playerInput.currentActionMap?.Enable(); +} +``` + +* **Option 2 - InputAction asset reference** (Default) + - Create a script that owns an InputActionAsset / generated wrapper instance. + - Enable/disable maps in OnEnable/OnDisable. + - Subscribe to performed/canceled events. + +Example with generated C# wrapper: +```csharp +public class MyPlayerScript : MonoBehaviour, IGameplayActions +{ + MyPlayerControls controls; + + public void OnEnable() + { + if (controls == null) + { + controls = new MyPlayerControls(); + controls.gameplay.SetCallbacks(this); + } + controls.gameplay.Enable(); + } + + public void OnDisable() + { + controls.gameplay.Disable(); + } + + public void OnMove(InputAction.CallbackContext context) + { + var value = context.ReadValue(); + } + + public void OnJump(InputAction.CallbackContext context) { } +} +``` + +* **Option 3 - Project-Wide Actions** (Simplest) + - First check if a project-wide asset is assigned using the script from Section 1 "Check Project-Wide Actions". Do NOT grep ProjectSettings files. + - If no project-wide asset is assigned, either create one via the asset creation API (see API note 1) and assign it with `InputSystem.actions = asset;`, or instruct the user to go to Edit > Project Settings > Input System Package > Create a new project-wide Action Asset. + - Project-wide actions are enabled by default and ready to use. + - Hook actions into gameplay using `InputSystem.actions.FindAction("Move")`. Cache references in `Start()`, do NOT call `FindAction` every frame. + +Example: +```csharp +using UnityEngine; +using UnityEngine.InputSystem; + +public class PlayerController : MonoBehaviour +{ + InputAction moveAction; + InputAction jumpAction; + + void Start() + { + moveAction = InputSystem.actions.FindAction("Move"); + jumpAction = InputSystem.actions.FindAction("Jump"); + } + + void Update() + { + Vector2 moveValue = moveAction.ReadValue(); + + if (jumpAction.WasPressedThisFrame()) + { + // Jump logic + } + } +} +``` + +4. UI support (if requested) +- Ensure EventSystem + InputSystemUIInputModule exists. +- Ensure there's a UI action map (or use Unity's default UI actions pattern). +- Confirm the UI module references correct actions (depending on project setup). + +Required UI actions (names and types must match for UI Toolkit compatibility via Project-Wide Input Actions: `InputSystem.actions`): + +| Action | Action Type | Control Type | Description | +|--------|-------------|--------------|-------------| +| Navigate | PassThrough | Vector2 | D-pad / arrow key navigation | +| Submit | Button | Button | Confirm selection | +| Cancel | Button | Button | Exit interaction | +| Point | PassThrough | Vector2 | Cursor position | +| Click | PassThrough | Button | Primary click | +| RightClick | PassThrough | Button | Secondary click | +| MiddleClick | PassThrough | Button | Middle click | +| ScrollWheel | PassThrough | Vector2 | Scroll input | +| Tracked Device Position | PassThrough | Vector3 | XR position | +| Tracked Device Orientation | PassThrough | Quaternion | XR rotation | + +**IMPORTANT:** Pointer-type UI actions (Point, Click, RightClick, MiddleClick, ScrollWheel) MUST be set to PassThrough type so multiple devices can feed input without filtering. + +## 4. Validation Checklist (Must Confirm) +- Input System package installed +- Active Input Handling set correctly +- No UI module conflicts (StandaloneInputModule removed if necessary, InputSystemUIInputModule not required by UI Toolkit) +- The input action asset is not corrupted after the changes +- Modifications to input action assets do not result in missing input action references +- The input actions references assigned to the scripts where it is needed +- The input action asset has input control schemes +- Actions have correct types (Value for continuous, Button for discrete, PassThrough for multi-device) +- Composite bindings are correctly configured (2D Vector for WASD, 1D Axis for left/right, etc.) + +## 5. Final Confirmation Message (What reporting back) +Summarize what was created/changed: +- Input Actions asset path + action maps/actions +- Control schemes + bindings +- PlayerInput setup (target object, default map, notification behavior) +- UI EventSystem module state +- Any restart requirement (Active Input Handling change) + + +## Important API notes + +0. Never edit inputaction asset Json directly, always use the InputActionAsset API, run in a live Editor, to edit the asset. + +1. CreateAsset() should not be used to create a file of type 'inputactions'. +To create and save the '.inputaction' files use the next code example: +```csharp +InputActionAsset asset = ScriptableObject.CreateInstance(); + +string json = asset.ToJson(); +File.WriteAllText(path, json); +``` + +2. Do NOT use 'Input.' class to handle inputs, it might result with exceptions at runtime. Use `InputSystem.actions.FindAction()` or action references instead. + +3. To add or remove input control scheme use `asset.AddControlScheme(InputControlScheme)` + +4. To add an action to a map use the API example that follows `public static InputAction AddAction(this InputActionMap map, string name, InputActionType type = InputActionType.Value, string binding = null, string interactions = null, string processors = null, string groups = null, string expectedControlLayout = null)` + +5. To add a composite binding use `AddCompositeBinding`: +```csharp +moveAction.AddCompositeBinding("2DVector") + .With("Up", "/w") + .With("Down", "/s") + .With("Left", "/a") + .With("Right", "/d"); +``` + +6. To add a simple binding use `AddBinding`: +```csharp +fireAction.AddBinding("/leftButton"); +fireAction.AddBinding("/rightTrigger"); +``` + +7. Enable/Disable actions and maps: +```csharp +// Enable a single action +myAction.Enable(); + +// Enable an entire action map +gameplayMap.Enable(); + +// Disable +myAction.Disable(); +gameplayMap.Disable(); +``` +DO not change bindings while an action is enabled. Disable first, modify, then re-enable. + +8. To find an action in an asset or project-wide actions: +```csharp +// By action name (searches all maps) +var action = asset.FindAction("Jump"); + +// By map/action path (disambiguates if name collisions exist) +var action = asset.FindAction("Player/Jump"); +``` + +9. CRITICAL: Project-Wide Actions are NOT in ProjectSettings files. +**NEVER** search, grep, or read `ProjectSettings/ProjectSettings.asset`, `ProjectSettings/EditorBuildSettings.asset`, or any other settings files to find input actions. The project-wide actions reference is stored internally via `EditorBuildSettings` config objects (binary format, not greppable). Always use `InputSystem.actions` to read the current project-wide actions, and `InputSystem.actions = asset` to assign them. See Section 1 "Check Project-Wide Actions" for the complete script. + +10. CRITICAL: Correct Unity Input System API Names + +| WRONG (Hallucinated) | CORRECT | +|---------------------|---------| +| `InputSystem.GetDevice()` | `Keyboard.current` | +| `InputSystem.GetDevice()` | `Mouse.current` | +| `InputSystem.GetDevice()` | `Gamepad.current` | +| `Input.GetAxis("Horizontal")` | `InputSystem.actions.FindAction("Move").ReadValue().x` | +| `Input.GetButtonDown("Jump")` | `InputSystem.actions.FindAction("Jump").WasPressedThisFrame()` | +| `Input.GetButton("Jump")` | `InputSystem.actions.FindAction("Jump").IsPressed()` | +| `Input.GetButtonUp("Jump")` | `InputSystem.actions.FindAction("Jump").WasReleasedThisFrame()` | +| `Input.mousePosition` | `Mouse.current.position.ReadValue()` | +| `Input.GetMouseButtonDown(0)` | `Mouse.current.leftButton.wasPressedThisFrame` | +| `Input.GetKey(KeyCode.Space)` | `Keyboard.current.spaceKey.isPressed` | +| `Input.GetKeyDown(KeyCode.Space)` | `Keyboard.current.spaceKey.wasPressedThisFrame` | +| `InputActionMap.FromJson` creating an asset | `InputActionAsset.FromJson` for full assets | + +## Core Concepts Reference + +### Actions +Actions are named, game-meaningful inputs ("Jump", "Move") decoupled from hardware. They allow separating the purpose of an input from the device controls that perform it. + +Each action has: +- A **name** (unique within its action map) +- A unique **ID** (persists across renames) +- An **Action Type** (Value, Button, or PassThrough) +- An **Expected Control Type** (Vector2, Button, Axis, etc.) + +Actions are a runtime-only feature. Do NOT use them in Editor window code. + +### Action Maps +Action maps group actions for a context (e.g., "Player", "UI", "Vehicle"). Enable/disable entire maps as a unit to switch input contexts. + +### Input Action Assets +`.inputactions` files stored in JSON format containing action maps, actions, bindings, and control schemes. The recommended workflow is one asset assigned as project-wide actions. + +### Project-Wide Actions +One asset designated globally via **Edit > Project Settings > Input System Package**. Accessible as `InputSystem.actions`. Preloaded at startup. Actions are enabled by default. + +To create and assign default project-wide actions, go to **Edit > Project Settings > Input System Package** and click "Create a new project-wide Action Asset". This creates `InputSystem_Actions.inputactions` with default Player and UI action maps. + +### Control Schemes +Groups of bindings and devices (e.g., "Keyboard&Mouse", "Gamepad"). Used for: +- Enabling/disabling sets of bindings +- PlayerInput automatic device pairing +- UI device switching feedback + +### Bindings +Links an action to device control(s) via control paths. Types: +- **Normal binding**: direct path like `/leftStick` +- **Composite binding**: synthesizes a value from multiple part bindings (e.g., WASD → Vector2) + +Key binding properties: + +| Property | Description | +|----------|-------------| +| `path` | Control path identifying the control(s). Example: `"/leftStick"` | +| `overridePath` | Non-destructive override of `path`. Used for runtime rebinding. | +| `effectivePath` | Returns `overridePath` if set, otherwise `path`. | +| `action` | Name or ID of the action this binding triggers. | +| `groups` | Semicolon-separated binding groups (used for control schemes). Example: `"Keyboard&Mouse;Gamepad"` | +| `interactions` | Semicolon-separated interactions. Example: `"hold(duration=0.75)"` | +| `processors` | Semicolon-separated processors. Example: `"invertVector2(invertX=false)"` | +| `isComposite` | Whether this binding is a composite root. | +| `isPartOfComposite` | Whether this binding is a part of a composite. | + +Control path syntax: +- `/buttonSouth` — matches on any gamepad +- `/buttonSouth` — matches only PlayStation controllers +- `/button*` — wildcard matching +- `*/{Submit}` — matches any control with "Submit" usage on any device + +## Responding to Actions Reference + +### Polling (Recommended for Gameplay) +Read values in `Update()`. Cache action references in `Start()`. + +| Method | Description | +|--------|-------------| +| `ReadValue()` | Current value of the action. Type must match the bound control's value type. | +| `IsPressed()` | True if actuation is above press point and hasn't fallen to release threshold. | +| `WasPressedThisFrame()` | True if actuation crossed press point this frame. | +| `WasReleasedThisFrame()` | True if actuation fell from above press point to at/below release threshold this frame. | +| `WasPerformedThisFrame()` | True if the action's phase became Performed this frame (interaction-driven). | +| `WasCompletedThisFrame()` | True if the action's phase changed away from Performed this frame. | + +### Callbacks (Event-Driven) +Subscribe to action phase callbacks for sporadic or multi-listener setups. + +```csharp +action.started += ctx => { /* Interaction started */ }; +action.performed += ctx => { /* Interaction completed */ }; +action.canceled += ctx => { /* Interaction interrupted/released */ }; +``` + +`InputAction.CallbackContext` is only valid during the callback. Do not store it. + +**Action Phases:** + +| Phase | Description | +|-------|-------------| +| `Disabled` | Action is disabled and can't receive input. | +| `Waiting` | Action is enabled and waiting for input. | +| `Started` | Input has started an interaction with the action. | +| `Performed` | An interaction with the action has been completed. | +| `Canceled` | An interaction with the action has been interrupted. | + +### Default Interaction Behavior by Action Type + +| Callback | Value | Button | PassThrough | +|----------|-------|--------|-------------| +| `started` | Control changed away from default value | Button started being pressed | Not used | +| `performed` | Control changed value | Button crossed press threshold | Control changed value | +| `canceled` | Controls no longer actuated | Button released | Action disabled | + +### Other Callback Options +- `InputActionMap.actionTriggered` — single callback for all actions in a map (receives started, performed, canceled) +- `InputSystem.onActionChange` — global callback for all action-related changes + +## PlayerInput Component Reference + +The PlayerInput component provides: +- Configuring how Actions map to methods or callbacks +- Handling local multiplayer: device filtering, screen splitting + +### Configuration Properties + +| Property | Description | +|----------|-------------| +| **Actions** | The Input Actions asset (project-wide or standalone asset) | +| **Default Scheme** | Control scheme to enable by default | +| **Default Map** | Action map to enable by default. If None, no actions are enabled. | +| **Camera** | Player camera (only needed for split-screen) | +| **Behavior** | Notification method: Send Messages, Broadcast Messages, Invoke Unity Events, Invoke C# Events | + +### Notification Behaviors + +| Behavior | How it Works | Method Signature | +|----------|-------------|-----------------| +| **Send Messages** | `GameObject.SendMessage` on the PlayerInput's GameObject | `void OnActionName()` or `void OnActionName(InputValue value)` | +| **Broadcast Messages** | `GameObject.BroadcastMessage` down the hierarchy | Same as Send Messages | +| **Invoke Unity Events** | Separate UnityEvent per action, configurable in Inspector | `void OnActionName(InputAction.CallbackContext context)` | +| **Invoke C# Events** | Plain C# events: `onActionTriggered`, `onDeviceLost`, `onDeviceRegained` | `void Handler(InputAction.CallbackContext context)` | + +### Action Map Switching +```csharp +// Switch by name +playerInput.SwitchCurrentActionMap("UI"); + +// Check current +var currentMap = playerInput.currentActionMap; + +// Deactivate/Activate all input +playerInput.DeactivateInput(); +playerInput.ActivateInput(); // Re-enables default action map +``` + +### Device Lost/Regained +PlayerInput sends `DeviceLostMessage` and `DeviceRegainedMessage` notifications when devices disconnect/reconnect. + +### UI Integration +Assign an `InputSystemUIInputModule` reference to PlayerInput's `UI Input Module` field. Both must use the same Input Actions asset. PlayerInput will configure the UI module to use the same action/device configuration. + +For multiplayer UI, use `MultiplayerEventSystem` instead of `EventSystem`. Each player gets their own `MultiplayerEventSystem` + `InputSystemUIInputModule` + `PlayerInput`. + +## PlayerInputManager Reference (Multiplayer) + +Used alongside PlayerInput for local multiplayer. + +| Property | Description | +|----------|-------------| +| **Player Prefab** | Must have a PlayerInput component | +| **Join Behavior** | Join When Button Is Pressed / Join When Join Action Is Triggered / Manual | +| **Max Players** | Maximum player count (-1 = unlimited) | +| **Split Screen** | Enable/configure split-screen rendering | + +Each PlayerInput instance gets a private copy of actions with device filtering. Players are automatically paired to unique devices. + +## UI Support Reference + +### InputSystemUIInputModule +Required for Unity UI (uGUI). Replaces `StandaloneInputModule`. + +| Property | Description | +|----------|-------------| +| Move Repeat Delay | Initial delay before repeat navigation events | +| Move Repeat Rate | Interval between repeat navigation events | +| Actions Asset | Input Action Asset driving the UI | +| Deselect on Background Click | Clear selection when clicking empty space (default: true) | +| Pointer Behavior | How multiple pointers are handled | + +### Pointer Behaviors + +| Mode | Description | +|------|-------------| +| **Single Mouse or Pen But Multi Touch And Track** | Default. Mouse/pen unified; touch and tracked devices are separate. | +| **Single Unified Pointer** | All input unified into one pointer. | +| **All Pointers As Is** | Every device is its own pointer. | + +### UI Toolkit Compatibility + +| UI Solution | Compatible | UI Input Module Required | +|-------------|------------|-------------------------| +| UI Toolkit (2023.2+) | Yes | Not required | +| UI Toolkit (pre-2023.2) | Yes | Required | +| Unity UI (uGUI) | Yes | Required | +| IMGUI | No (use "Both" Active Input Handling for IMGUI + Input System coexistence) | + +## Interactions Reference + +Interactions are input patterns that drive action phase transitions. Applied to bindings or actions. + +### Built-in Interactions + +| Interaction | Description | Key Parameters | +|-------------|-------------|----------------| +| **Default** | Applied when no interaction is specified. Behavior varies by action type. | — | +| **Press** | Explicit button-press pattern. | `pressPoint`, `behavior` (PressOnly/ReleaseOnly/PressAndRelease) | +| **Hold** | Requires holding a control for a duration. | `duration` (default: `InputSettings.defaultHoldTime`), `pressPoint` | +| **Tap** | Press and release within a duration. | `duration` (default: `InputSettings.defaultTapTime`), `pressPoint` | +| **SlowTap** | Hold for minimum duration, then release to trigger. | `duration` (default: `InputSettings.defaultSlowTapTime`), `pressPoint` | +| **MultiTap** | Multiple taps in succession (e.g., double-click). | `tapCount` (default: 2), `tapTime`, `tapDelay`, `pressPoint` | + +### Interaction Phase Behavior + +**Hold:** +- `started` → control crosses press point +- `performed` → held above press point for >= duration +- `canceled` → released before duration elapsed + +**Tap:** +- `started` → control crosses press point +- `performed` → released before duration elapsed +- `canceled` → held too long (>= duration) + +**Adding Interactions:** +```csharp +// In code +action.AddBinding("/buttonSouth") + .WithInteractions("hold(duration=0.4)"); + +// On action directly +var action = new InputAction(interactions: "hold(duration=0.4)"); +``` + +Multiple interactions on a binding are processed in order. The first to trigger "consumes" the input. + +### Timeout Completion +```csharp +// Get progress of hold/tap interaction (0 to 1) +float progress = action.GetTimeoutCompletionPercentage(); +``` + +## Composite Bindings Reference + +Composites combine multiple controls into a single value. + +### Built-in Composites + +| Composite | Output Type | Parts | Usage | +|-----------|-------------|-------|-------| +| **1D Axis** | `float` | Positive, Negative | Left/Right, triggers | +| **2D Vector** (Dpad) | `Vector2` | Up, Down, Left, Right | WASD movement, D-pad | +| **3D Vector** | `Vector3` | Up, Down, Left, Right, Forward, Backward | 3D movement | +| **One Modifier** | Any | Modifier, Binding | SHIFT+Key shortcuts | +| **Two Modifiers** | Any | Modifier1, Modifier2, Binding | CTRL+SHIFT+Key | + +### Code Examples + +```csharp +// 1D Axis +myAction.AddCompositeBinding("1DAxis") + .With("Positive", "/d") + .With("Negative", "/a"); + +// 2D Vector (WASD) +myAction.AddCompositeBinding("2DVector") + .With("Up", "/w") + .With("Down", "/s") + .With("Left", "/a") + .With("Right", "/d"); + +// 2D Vector with mode +myAction.AddCompositeBinding("2DVector(mode=2)") // mode=2 is Analog + .With("Up", "/leftStick/up") + .With("Down", "/leftStick/down") + .With("Left", "/leftStick/left") + .With("Right", "/leftStick/right"); + +// One Modifier (SHIFT+1) +myAction.AddCompositeBinding("OneModifier") + .With("Binding", "/1") + .With("Modifier", "/ctrl"); + +// Two Modifiers (CTRL+SHIFT+1) +myAction.AddCompositeBinding("TwoModifiers") + .With("Button", "/1") + .With("Modifier1", "/leftCtrl") + .With("Modifier2", "/leftShift"); +``` + +### 2D Vector Mode Parameter + +| Mode | Value | Description | +|------|-------|-------------| +| DigitalNormalized | 0 | Default. Inputs treated as on/off, vector normalized (diamond-shaped range). | +| Digital | 1 | On/off but not normalized. Diagonals have magnitude > 1. | +| Analog | 2 | Full floating-point values. Down and Left inverted. | + +Each composite part can have multiple bindings (e.g., both WASD and arrow keys for the same 2D Vector). + +## Interactive Rebinding Reference + +Allow users to customize bindings at runtime. + +### Performing a Rebind +```csharp +void RemapButtonClicked(InputAction actionToRebind) +{ + var rebindOperation = actionToRebind + .PerformInteractiveRebinding() + .Start(); +} +``` +IMPORTANT: Dispose `RebindingOperation` instances via `Dispose()` to prevent memory leaks. + +### Configuration Options +- `WithExpectedControlType()` — filter by control type +- `WithControlsExcluding()` — exclude specific controls +- `WithCancelingThrough()` — set a cancel control +- `WithTargetBinding()` / `WithBindingGroup()` — target specific bindings + +### Save and Load Rebinds +```csharp +// Save +var rebinds = playerInput.actions.SaveBindingOverridesAsJson(); +PlayerPrefs.SetString("rebinds", rebinds); + +// Load (removes existing overrides by default) +var rebinds = PlayerPrefs.GetString("rebinds"); +playerInput.actions.LoadBindingOverridesFromJson(rebinds); +``` + +### Restore Defaults +```csharp +// Remove overrides from a single action +playerInput.actions["fire"].RemoveAllBindingOverrides(); + +// Remove all overrides from all actions +playerInput.actions.RemoveAllBindingOverrides(); +``` + +### Display Binding Strings +```csharp +// Get display string for an action +string displayStr = action.GetBindingDisplayString(); + +// Get display string for a specific binding index +string displayStr = action.GetBindingDisplayString(1); + +// Get with device/control info (for icon replacement) +string displayStr = action.GetBindingDisplayString(0, out string deviceLayout, out string controlPath); +``` + +### Apply Binding Overrides (Non-Interactive) +```csharp +// Override a binding path +playerInput.actions["fire"].ApplyBindingOverride("/leftTrigger"); + +// Override by binding index +var jumpAction = playerInput.actions["Jump"]; +var bindingIndex = jumpAction.GetBindingIndexForControl(Keyboard.current.spaceKey); +jumpAction.ApplyBindingOverride(bindingIndex, "/enter"); +``` + +Override properties (`overridePath`, `overrideProcessors`, `overrideInteractions`) are NOT saved with the asset JSON. Use `SaveBindingOverridesAsJson` / `LoadBindingOverridesFromJson` separately. + +## Processors Reference + +Processors transform input values. Applied to bindings or actions. Stack with processors on controls. + +Common processors: +- `invertVector2(invertX=true,invertY=true)` — invert axes +- `scaleVector2(x=1,y=1)` — scale axes +- `stickDeadzone(min=0.125,max=0.925)` — apply deadzone to stick input +- `axisDeadzone(min=0.125,max=0.925)` — apply deadzone to single axis +- `normalize(min=0,max=1,zero=0)` — normalize to range +- `clamp(min=0,max=1)` — clamp value +- `invert` — invert a single float value +- `scale(factor=1)` — scale a single float value + +```csharp +// In code +action.AddBinding("/leftStick") + .WithProcessors("stickDeadzone(min=0.2,max=0.9)"); + +// On action +var action = new InputAction(processors: "invertVector2(invertX=false)"); +``` + +### Parameter Overrides for Sensitivity +```csharp +// Adjust mouse sensitivity separately from gamepad +var look = new InputAction("look", type: InputActionType.Value); +look.AddBinding("/delta", groups: "KeyboardMouse", processors: "scaleVector2"); +look.AddBinding("/rightStick", groups: "Gamepad", processors: "scaleVector2"); + +look.ApplyParameterOverride("scaleVector2:x", 0.5f, InputBinding.MaskByGroup("KeyboardMouse")); +look.ApplyParameterOverride("scaleVector2:y", 0.5f, InputBinding.MaskByGroup("KeyboardMouse")); + +look.ApplyParameterOverride("scaleVector2:x", 2f, InputBinding.MaskByGroup("Gamepad")); +look.ApplyParameterOverride("scaleVector2:y", 2f, InputBinding.MaskByGroup("Gamepad")); +``` + +## Direct Device Access Reference (Prototyping Only) + +For quick prototyping or fixed-device scenarios. Less flexible than actions. + +```csharp +// Keyboard +if (Keyboard.current.spaceKey.wasPressedThisFrame) { } +if (Keyboard.current.wKey.isPressed) { } + +// Mouse +Vector2 mousePos = Mouse.current.position.ReadValue(); +if (Mouse.current.leftButton.wasPressedThisFrame) { } +Vector2 mouseDelta = Mouse.current.delta.ReadValue(); + +// Gamepad +var gamepad = Gamepad.current; +if (gamepad == null) return; // No gamepad connected +Vector2 move = gamepad.leftStick.ReadValue(); +if (gamepad.buttonSouth.wasPressedThisFrame) { } +if (gamepad.rightTrigger.wasPressedThisFrame) { } +``` + +Always null-check `*.current` as devices may not be connected. + +## Migration from Legacy Input Manager + +When migrating from old `Input` class to Input System: + +| Legacy (Old) | Input System (New) — Actions Approach | Input System (New) — Direct Approach | +|--------------|--------------------------------------|--------------------------------------| +| `Input.GetAxis("Horizontal")` | `moveAction.ReadValue().x` | `Keyboard.current.dKey.ReadValue() - Keyboard.current.aKey.ReadValue()` | +| `Input.GetButton("Fire1")` | `fireAction.IsPressed()` | `Mouse.current.leftButton.isPressed` | +| `Input.GetButtonDown("Jump")` | `jumpAction.WasPressedThisFrame()` | `Keyboard.current.spaceKey.wasPressedThisFrame` | +| `Input.GetButtonUp("Jump")` | `jumpAction.WasReleasedThisFrame()` | `Keyboard.current.spaceKey.wasReleasedThisFrame` | +| `Input.mousePosition` | `pointerAction.ReadValue()` | `Mouse.current.position.ReadValue()` | +| `Input.GetMouseButtonDown(0)` | `clickAction.WasPressedThisFrame()` | `Mouse.current.leftButton.wasPressedThisFrame` | +| `Input.GetKey(KeyCode.Space)` | `action.IsPressed()` | `Keyboard.current.spaceKey.isPressed` | +| `Input.touches` / `Input.touchCount` | Use `EnhancedTouchSupport` | `EnhancedTouch.Touch.activeTouches` | + +Preprocessor defines for conditional compilation: +- `#if ENABLE_INPUT_SYSTEM` — new Input System is active +- `#if ENABLE_LEGACY_INPUT_MANAGER` — old Input Manager is active +- Both can be true when Active Input Handling is set to "Both" + +Old API surface "Unity Input" resides in class `UnityEngine.Input`. Input System package API surface resides in root namespace `UnityEngine.InputSystem`. +The following exceptions exist and may be used regardless of Active Input Handling setting: +- `UnityEngine.Input.location` +- `UnityEngine.Input.stylusTouchSupported` +- `UnityEngine.Input.mousePresent` +- `UnityEngine.Input.multiTouchEnabled` + +## Common Mistakes to Avoid + +### 1. Using `Input.` Class with Input System +**Problem:** `Input.GetAxis`, `Input.GetButtonDown` etc. throw exceptions when only Input System (new) is active. +**Solution:** Use `InputSystem.actions.FindAction()` or direct device access (`Keyboard.current`, etc.). + +### 2. Calling FindAction Every Frame +**Problem:** `InputSystem.actions.FindAction("Move")` in `Update()` is wasteful. +**Solution:** Cache the `InputAction` reference in `Start()` or `Awake()`. + +### 3. Using InputSystem.actions with PlayerInput +**Problem:** `InputSystem.actions` is the singleton copy. PlayerInput creates private copies for device filtering. +**Solution:** Use `playerInput.actions` when working with PlayerInput. + +### 4. Not Disabling Actions Before Modifying Bindings +**Problem:** Changing bindings while actions are enabled causes temporary disable/re-enable of all actions. +**Solution:** Disable the action or map, make changes, then re-enable. + +### 5. Storing InputAction.CallbackContext +**Problem:** Context struct is only valid during the callback. +**Solution:** Read values during the callback; don't store the context for later use. + +### 6. Wrong Action Type for UI +**Problem:** Using Value type for UI pointer actions causes only one device to drive input. +**Solution:** UI pointer actions (Point, Click, ScrollWheel, etc.) must be PassThrough. + +### 7. Missing Control Schemes +**Problem:** Local multiplayer doesn't pair devices correctly. +**Solution:** Always create control schemes with required devices. PlayerInput uses these for automatic device pairing. + +### 8. Forgetting to Dispose RebindingOperation +**Problem:** `PerformInteractiveRebinding()` allocates unmanaged memory. +**Solution:** Always call `Dispose()` on the `RebindingOperation` when done. + +### 9. Not Saving Binding Overrides Separately +**Problem:** `overridePath` is not saved with `InputActionAsset.ToJson()`. +**Solution:** Use `SaveBindingOverridesAsJson()` / `LoadBindingOverridesFromJson()` and persist via PlayerPrefs or file. + +### 10. Editing .inputactions JSON Directly +**Problem:** Manual JSON edits can corrupt the asset, break binding IDs, or lose data. +**Solution:** Always modify assets programmatically through the InputActionAsset API, run in a live Editor. + +### 11. Searching ProjectSettings Files for Input Actions +**Problem:** Grepping or reading `ProjectSettings/ProjectSettings.asset` or other settings files to find the project-wide actions asset. The reference is stored via `EditorBuildSettings` config objects in binary format and is not searchable in text files. This always fails and wastes multiple tool calls. +**Solution:** Always use `InputSystem.actions` to check the current project-wide actions. See Section 1 "Check Project-Wide Actions" for the complete script. From 8f6d112e12e17ca16afdb391bdb9aa056bc26ea6 Mon Sep 17 00:00:00 2001 From: Ziyi Zhang Date: Wed, 12 Aug 2026 09:11:56 -0400 Subject: [PATCH 2/7] fix: leave setup-audiorandomcontainer out of this batch --- README.md | 1 - skills/setup-audiorandomcontainer/SKILL.md | 35 ----- .../references/api.md | 132 ------------------ 3 files changed, 168 deletions(-) delete mode 100644 skills/setup-audiorandomcontainer/SKILL.md delete mode 100644 skills/setup-audiorandomcontainer/references/api.md diff --git a/README.md b/README.md index 90cbce7..5d1ed7e 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,6 @@ npx skills add Unity-Technologies/skills | `ui-ugui` | uGUI — Canvas hierarchies, RectTransform anchoring, Layout Groups, prefab UI | | `ui-imgui` | IMGUI editor tooling — EditorWindows, custom Inspectors, PropertyDrawers | | `setup-game-inputs` | Input System — action maps, bindings, control schemes, rebinding | -| `setup-audiorandomcontainer` | AudioRandomContainer assets for randomized playback | | `android-add-adaptive-performance` | Android thermal and power signals mapped to dynamic quality tiers | | `asset-transformer-toolkit` | 3D model and point-cloud import, RuleSets and Actions, LOD generation | diff --git a/skills/setup-audiorandomcontainer/SKILL.md b/skills/setup-audiorandomcontainer/SKILL.md deleted file mode 100644 index d593dd6..0000000 --- a/skills/setup-audiorandomcontainer/SKILL.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -name: setup-audiorandomcontainer -description: Creates AudioRandomContainer assets with randomized audio playback. Use for dynamic sounds from multiple clips with variation of volume, pitch and playback timing. -required_editor_version: ">=6000.3.13" ---- -# Set up AudioRandomContainer - -## When to use this skill -Use when the user wants to: -- Create dynamic audio that randomizes between multiple clips -- Set up audio with randomized volume, pitch, or timing -- Configure automatic or triggered audio playback with variation - -## DO's -- Use the internal API from `UnityEngine.Audio` and `UnityEditor.Audio` namespaces -- Use standard C# construction: `new AudioRandomContainer()` -- Use `AudioRandomContainerUtilities.AddElements()` or `container.AddElements()` to add clips -- Save the asset immediately after creation using `AssetDatabase.SaveAssetIfDirty()` -- Default filename: "New Audio Random Container.asset" (auto-enumerate if exists) - -## DON'Ts -- ❌ Never use reflection -- ❌ Never use `ScriptableObject.CreateInstance` -- ❌ Never use `SerializedObject`/`SerializedProperty` -- ❌ Do not save to disk after setting properties (leave for user) -- ❌ Do not auto-play `AudioSource` after assigning container unless explicitly requested - -## Procedure -1. Create the container: `var container = new AudioRandomContainer()` -2. Generate unique path and save to disk with `AssetDatabase.CreateAsset()` and `AssetDatabase.SaveAssetIfDirty()` -3. **Only if requested**: Add audio clips using `container.AddElements(clipArray)` -4. **Only if requested**: Update properties using the API (see references) - -## References -See [references/api.md](references/api.md) \ No newline at end of file diff --git a/skills/setup-audiorandomcontainer/references/api.md b/skills/setup-audiorandomcontainer/references/api.md deleted file mode 100644 index f92477b..0000000 --- a/skills/setup-audiorandomcontainer/references/api.md +++ /dev/null @@ -1,132 +0,0 @@ -# AudioRandomContainer API Reference - -## Table of Contents -- [Architecture](#architecture) - - [Key Types](#key-types) - - [Assignment to AudioSource](#assignment-to-audiosource) - - [Scope Restrictions](#scope-restrictions) -- [Asset Creation](#asset-creation) - - [Construction](#construction) - - [Saving](#saving) -- [Getting and Setting Properties](#getting-and-setting-properties) - - [Access Methods](#access-methods) - - [Adding Elements (Clips)](#adding-elements-clips) - - [Saving After Property Changes](#saving-after-property-changes) - - [Conditional Property Requirements](#conditional-property-requirements) -- [Code Example](#code-example) - -## Architecture - -### Key Types -- **`AudioRandomContainer`**: Implements `IAudioGenerator` for dynamic audio playback -- **`AudioContainerElement`**: Holds an `AudioClip` reference plus additional properties (not yet accessible via API) - -### Assignment to AudioSource -- ✅ **DO**: Use `AudioSource.generator` to assign the container -- ❌ **DON'T**: Use `AudioSource.resource` (will be deprecated) - -### Scope Restrictions -- `AudioRandomContainer` is **internal only** - reach it from C# running inside the Editor -- **Never reference** this type in generated project scripts (causes compilation errors) -- **Always import** `UnityEngine.Audio` namespace when working with this type - -## Asset Creation - -### Construction -- ✅ **DO**: Use standard C# construction: `new AudioRandomContainer()` -- ❌ **DON'T**: Use reflection or `ScriptableObject.CreateInstance` -- Proper initialization is handled in the native backing layer - -### Saving -- **Required**: Save to disk immediately after creation using: - ```csharp - AssetDatabase.CreateAsset(container, path); - AssetDatabase.SaveAssetIfDirty(container); - ``` - -## Getting and Setting Properties - -### Access Methods -- ✅ **DO**: Access all properties through the API -- ❌ **DON'T**: Use reflection or `SerializedObject`/`SerializedProperty` - -### Adding Elements (Clips) -- ✅ **DO**: Use `container.AddElements(clipArray)` or `AudioRandomContainerUtilities.AddElements()` -- ❌ **DON'T**: Set the `elements` property directly - -### Saving After Property Changes -- ❌ **DON'T**: Save the asset to disk after setting properties -- Leave saving for the user to do manually - -### Conditional Property Requirements -These properties only take effect when their corresponding "enabled" flag is true: - -| Property | Requires This Flag Set to `true` | -|----------|----------------------------------| -| `volumeRandomizationRange` | `volumeRandomizationEnabled` | -| `pitchRandomizationRange` | `pitchRandomizationEnabled` | -| `automaticTriggerTimeRandomizationRange` | `automaticTriggerTimeRandomizationEnabled` | -| `loopCountRandomizationRange` | `loopCountRandomizationEnabled` | - -## Code Example - -```csharp -// ============================================================================ -// STEP 1: Create the asset -// ============================================================================ -var container = new AudioRandomContainer(); - -// ============================================================================ -// STEP 2: Save to disk (REQUIRED immediately after creation) -// ============================================================================ -var path = "Assets/New Audio Random Container.asset"; -path = AssetDatabase.GenerateUniqueAssetPath(path); -AssetDatabase.CreateAsset(container, path); -AssetDatabase.SaveAssetIfDirty(container); - -// ============================================================================ -// STEP 3: Add audio clips -// ============================================================================ - -// Option A: Add empty elements (no clips assigned yet) -container.AddElements(2); - -// Option B: Add elements with clips assigned -const string clip1Path = "Assets/clip1.wav"; -const string clip2Path = "Assets/clip2.wav"; -var clip1 = AssetDatabase.LoadAssetAtPath(clip1Path); -var clip2 = AssetDatabase.LoadAssetAtPath(clip2Path); -container.AddElements(new[] { clip1, clip2 }); - -// ============================================================================ -// STEP 4: Set properties (ONLY if requested by user) -// ============================================================================ - -// Volume settings -container.volume = -20; // Range: [-80, 0], unit: decibels -container.volumeRandomizationEnabled = true; // Required for range to work -container.volumeRandomizationRange = new Vector2(-10, 10); // Range: [-80, 80], unit: decibels - -// Pitch settings -container.pitch = 500; // Range: [-1200, 1200], unit: cents -container.pitchRandomizationEnabled = true; // Required for range to work -container.pitchRandomizationRange = new Vector2(-100, 100); // Range: [-1200, 1200], unit: cents - -// Playback settings -container.playbackMode = AudioRandomContainerPlaybackMode.Random; -container.avoidRepeatingLast = 0; // Only for Random mode; must be < number of clips - -// Trigger settings -container.triggerMode = AudioRandomContainerTriggerMode.Automatic; -container.automaticTriggerMode = AudioRandomContainerAutomaticTriggerMode.Pulse; -container.automaticTriggerTime = 2f; // Unit: seconds -container.automaticTriggerTimeRandomizationEnabled = true; // Required for range to work -container.automaticTriggerTimeRandomizationRange = new Vector2(1f, 3f); // Range: [-60, 60] - -// Loop settings -container.loopMode = AudioRandomContainerLoopMode.Clips; -container.loopCount = 5; // Disabled if loop mode is Infinite -container.loopCountRandomizationEnabled = true; // Required for range to work -container.loopCountRandomizationRange = new Vector2(3, 5); // Range: [-10, 10] - -``` From 5323bf5fe51301a271881852b5d2f057284cfbf7 Mon Sep 17 00:00:00 2001 From: Ziyi Zhang Date: Wed, 12 Aug 2026 10:08:57 -0400 Subject: [PATCH 3/7] feat: add the optimize-audio skill --- README.md | 1 + skills/optimize-audio/SKILL.md | 199 ++++++++++++++++++ .../resources/audio-import-api.md | 146 +++++++++++++ .../resources/platform-settings.md | 48 +++++ 4 files changed, 394 insertions(+) create mode 100644 skills/optimize-audio/SKILL.md create mode 100644 skills/optimize-audio/resources/audio-import-api.md create mode 100644 skills/optimize-audio/resources/platform-settings.md diff --git a/README.md b/README.md index 5d1ed7e..8b60a5c 100644 --- a/README.md +++ b/README.md @@ -22,6 +22,7 @@ npx skills add Unity-Technologies/skills | `ui-uitk` | UI Toolkit (Unity 6.0+) — author UXML/USS, flex layout, custom elements, Painter2D, runtime binding | | `ui-ugui` | uGUI — Canvas hierarchies, RectTransform anchoring, Layout Groups, prefab UI | | `ui-imgui` | IMGUI editor tooling — EditorWindows, custom Inspectors, PropertyDrawers | +| `optimize-audio` | Reduce audio memory and DSP CPU cost through import settings, load types, and mixer topology | | `setup-game-inputs` | Input System — action maps, bindings, control schemes, rebinding | | `android-add-adaptive-performance` | Android thermal and power signals mapped to dynamic quality tiers | | `asset-transformer-toolkit` | 3D model and point-cloud import, RuleSets and Actions, LOD generation | diff --git a/skills/optimize-audio/SKILL.md b/skills/optimize-audio/SKILL.md new file mode 100644 index 0000000..8c6a5a0 --- /dev/null +++ b/skills/optimize-audio/SKILL.md @@ -0,0 +1,199 @@ +--- +name: optimize-audio +description: Optimizes Unity 6 audio memory, CPU cost, and playback quality through correct import settings and mixer configuration. Use when the user wants to reduce audio memory usage, choose the right Load Type for short clips versus music versus ambient beds, configure platform-appropriate sample rates and codecs, force 3D audio to mono, or reduce AudioMixer CPU cost from deep group trees or effects running on silent paths. +--- +## Critical Rules + +- Do not make changes before reporting findings to the user +- Follow steps in strict order; never jump ahead +- STOP at every `WAIT` checkpoint and await the user's response before continuing +- Quality is more important than speed: measure before and after every change +- Always verify results in a device build; Editor audio stats are indicative only + +## 0. Set up the execution path + +Every C# step below runs inside a live Editor through the Unity CLI. **The `unity-cli` skill owns +getting you there** — installing the CLI, confirming a connected Editor, adding the project's +`com.unity.pipeline` package, telling a genuinely absent Editor apart from one stuck in Safe Mode, +and discovering the Editor's command catalog. Follow it first; don't re-derive any of it here. + +Two things it can't know for you: + +- **You need `eval` in particular**, not just a reachable Editor. Confirm it appears in the + catalog. Its presence depends on the Pipeline package version, not on the CLI, so a healthy + install can still lack it — if it's missing, say so and stop. +- **Do not hand-edit `.meta` files to change import settings.** Importer values only take effect + through `SaveAndReimport()` in a live Editor, so an unreachable Editor is a stop, not a cue to + edit metadata directly. + +Run C# with `unity command eval --code ''`. Discover the parameter shape from +`unity command --format json` rather than assuming one. `unity command` defaults to a 30 second +timeout. + +### Passing C# to `eval` + +`eval` compiles a **statement block, not a file**. Two consequences, both of which cause a compile +error rather than a warning: + +- **No `using` directives.** The compiler reads `using UnityEngine;` as a resource-disposal + statement and rejects it (`CS0210`). +- **Types must be fully qualified.** A bare `AssetDatabase` or `AudioImporter` does not resolve + (`CS0246` / `CS0103`), and a bare `Object` is ambiguous with `object` (`CS0104`). + +The recipes in [resources/audio-import-api.md](resources/audio-import-api.md) are written +fully qualified so they can be passed to `eval` as-is. + +## 1. Pre-Flight: Detect Audio System + +Before doing anything else, establish the audio environment: + +1. **Detect platform and sample rate:** Use `eval` to read `EditorUserBuildSettings.activeBuildTarget` and `AudioSettings.outputSampleRate`. The output sample rate affects whether overriding clip sample rates will actually save memory. +2. **Detect AudioMixer presence:** Use the mixer-asset query recipe in [resources/audio-import-api.md](resources/audio-import-api.md) to see if a mixer graph exists. If none exists, note that routing and effect costs are not a concern. +3. **Detect AudioListener:** Use the scene-component query recipe in [resources/audio-import-api.md](resources/audio-import-api.md) for `UnityEngine.AudioListener` to confirm exactly one listener is present. Multiple listeners produce incorrect spatialization; zero listeners produce silence. +4. **Proceed** only after platform and listener state are confirmed. + +## 2. Assess Current State + +Before recommending any change, gather observable data: + +1. **Find all AudioSources:** Use the scene-component query recipe in [resources/audio-import-api.md](resources/audio-import-api.md) for `UnityEngine.AudioSource`. For each result, use **one** `eval` call to batch-read properties — see the batch read recipe in [resources/audio-import-api.md](resources/audio-import-api.md). +2. **Inspect mixer topology:** If a mixer was found in Pre-Flight, use `eval` to read the AudioMixer's exposed parameters and group count. A group count above ~8 or effects on the Master group are immediate flags. +3. **Check DSP buffer size:** Use the DSP buffer recipe in [resources/audio-import-api.md](resources/audio-import-api.md) to read buffer size. See DSP Buffer Size Guidelines in [resources/platform-settings.md](resources/platform-settings.md) for recommended values. +4. **Report findings before making changes:** Summarize ALL detected sources, the listener count, and mixer depth to the user. Flag any immediate risks (e.g., stereo clip with `spatialBlend = 1`, Decompress On Load on a clip > 1 MB, reverb on the Master group). + +**WAIT for the user to review the assessment before proceeding.** + +## 3. Understand Request + +Route to the correct section based on what the user needs: + +| User Says | Path | +|-----------|------| +| "audio memory too high" / "memory profiler shows audio" | Section 4 — Import settings audit | +| "load times slow" / "decompression stall" | Section 4 — Load Type review | +| "DSP spike" / "mixer CPU" / "audio CPU high" | Section 4B — Mixer audit | +| "3D sound wrong" / "only left channel plays" / "stereo in 3D" | Section 4A — Force To Mono + spatial settings | +| "quality artifacts" / "voice sounds bad" / "Vorbis crackling" | Section 4C — Compression quality tuning | +| "mobile audio battery" / "mobile memory" | Section 4D — Mobile sample rate override | +| "set import settings on all clips" / "batch audio settings" | Section 4 — Bulk import audit | +| "streaming" / "background loading" / "Addressables audio" | Section 4E — Streaming and async load | + +If the symptom is ambiguous, ask: "Is the problem audio memory usage, DSP CPU spikes, or audio playback quality?" + +## 4. Primary Diagnostic Workflow + +Use the findings from Section 2 to determine which sub-section applies. More than one may apply simultaneously. + +### 4A. Force To Mono and Spatial Settings + +For any AudioSource where `spatialBlend > 0` (3D positioned sound): + +1. **Check clip channel count:** Use `eval` to read `audioSource.clip.channels`. If `channels == 2` and `spatialBlend == 1`, only the left channel plays — this is a bug, not a feature. +2. **Recommend Force To Mono:** Use the read importer recipe in [resources/audio-import-api.md](resources/audio-import-api.md) to inspect current settings, then apply Force To Mono using the force-to-mono recipe. +3. **Apply and reimport:** Report before/after channel counts to the user. +4. **Verify spatial blend:** Use `eval` to confirm `audioSource.spatialBlend` is `1.0` (full 3D) and `audioSource.rolloffMode` is set to an appropriate curve. + +### 4B. AudioMixer Audit + +1. **Measure group depth:** Use `eval` to walk the mixer's group tree and count levels. More than 3 levels (Master → SFX / Music / Voice → sub-bus) adds routing overhead every frame, even when children are silent. +2. **Check effects on silent groups:** Use `eval` to query each group's effects list. Effects such as `AudioReverbFilter` run their DSP at full cost even when no AudioSource routes to that group. +3. **Flag SFX Reverb on parent groups:** This is the most expensive built-in effect. If found on the Master or a high-level group, flag it explicitly. +4. **Present recommendations to the user:** + - Remove or bypass effects on groups that have no active sources. + - Use **snapshots** to switch mix states (combat / explore / pause) rather than toggling effects at runtime. + - Flatten unnecessary sub-buses; redirect sources to a shallower ancestor. + + **WAIT for the user to approve the mixer changes before applying.** + +5. **Verify DSP buffer size:** If `bufferLength` from Pre-Flight is very small (< 256), recommend increasing it — see DSP Buffer Size Guidelines in [resources/platform-settings.md](resources/platform-settings.md). + +### 4C. Compression Quality Tuning + +1. **Read current compression format:** Use the read importer recipe in [resources/audio-import-api.md](resources/audio-import-api.md) to read `compressionFormat` and `quality` for the clips reported by the user. +2. **Apply the platform matrix:** See the Compression Format Matrix in [resources/platform-settings.md](resources/platform-settings.md) for per-platform recommendations. +3. **Warn about lossy sources:** Use the lossy source check recipe in [resources/audio-import-api.md](resources/audio-import-api.md). If the original file is MP3, warn the user that lossy source quality is lost permanently after Unity re-encodes. Recommend WAV or AIFF sources. + +### 4D. Mobile Sample Rate Override + +1. **Identify SFX clips on mobile target:** Use the scene-component query recipe for `UnityEngine.AudioSource` and filter for non-music, non-dialogue clips. +2. **Read current sample rate setting:** Use the read importer recipe in [resources/audio-import-api.md](resources/audio-import-api.md) to read `sampleRateSetting` and `sampleRateOverride` for each clip. +3. **Apply mobile override:** Use the sample rate override recipe in [resources/audio-import-api.md](resources/audio-import-api.md). See Sample Rate Recommendations in [resources/platform-settings.md](resources/platform-settings.md) for per-use-case rates. +4. **Report savings:** Halving the sample rate halves the PCM memory cost. Report the estimated saving for each clip changed. + +### 4E. Load Type and Streaming + +1. **Audit Load Type per clip:** Use `eval` to read `clip.loadType` for each clip found in Section 2. +2. **Apply the decision rule:** See Load Type Decision Table in [resources/platform-settings.md](resources/platform-settings.md). +3. **Flag mismatches:** See Load Type Mismatch Flags in [resources/platform-settings.md](resources/platform-settings.md). Report both types of mismatches to the user. +4. **Apply `Load In Background`** for any Streaming clip — use the Load In Background recipe in [resources/audio-import-api.md](resources/audio-import-api.md). + +## 5. Validation + +After any import setting or mixer change: + +1. **Re-read clip stats:** Use `eval` to re-read `clip.loadType`, `clip.channels`, `AudioSettings.outputSampleRate`, and the importer's `compressionFormat` to confirm the change applied after reimport. +2. **Confirm AudioSource routing:** Use the scene-component query recipe for `UnityEngine.AudioSource` and verify `audioSource.outputAudioMixerGroup` is assigned as expected after any mixer restructure. +3. **Report delta:** State the before and after values for each setting changed. Do not assume the change was effective without reading back the applied importer values. +4. **Iterate limit:** Maximum 3 adjust-and-verify cycles before pausing to ask the user for feedback. + +## 6. Troubleshooting + +### Stereo clip on a 3D AudioSource — only left channel audible + +1. Confirm `audioSource.spatialBlend == 1`. +2. Confirm `audioSource.clip.channels == 2`. +3. Enable `forceToMono` in the AudioClip importer and reimport. Unity mixes both channels to mono during import, preserving level with `normalize = true` (keep on). +4. If the user does not want to reimport: set `audioSource.panStereo = 0` as a runtime workaround, but warn this does not recover stereo information. + +### Decompress On Load clip causes memory spike + +1. Confirm `clip.loadType == AudioClipLoadType.DecompressOnLoad` and `clip.length` is long (> 5 s). +2. Switch to `Streaming` if it is music or ambience, `CompressedInMemory` if played only occasionally. +3. If the clip is short but still large: check `clip.channels` (stereo wastes double the memory) and `clip.frequency` (high sample rate on a mobile target wastes memory). Apply Force To Mono and/or sample rate override. + +### AudioMixer CPU spike — DSP thread hot + +1. Confirm with the mixer-asset query recipe that the mixer graph exists. +2. Use `eval` to list all groups and their attached effects. Look for reverb, chorus, or EQ on high-level groups. +3. Move expensive effects down to leaf groups that are only active when sources are playing. +4. Use snapshots to bypass effect chains during gameplay states where they are not heard (e.g., bypass reverb during a menu). +5. If the DSP buffer is small (64 or 128 samples), raise it — see DSP Buffer Size Guidelines in [resources/platform-settings.md](resources/platform-settings.md). + +### Vorbis quality artifacts on dialogue + +1. Confirm `defaultSampleSettings.compressionFormat == AudioCompressionFormat.Vorbis`. +2. Confirm `defaultSampleSettings.quality` — default is 0.5, which is often audible on voice. Raise to 0.7–0.85. +3. On iOS: switch to AAC instead of Vorbis (hardware decode, better quality at equivalent bitrate). +4. Confirm the source file is lossless (WAV or AIFF). MP3 sources cannot recover quality lost before Unity's re-encode. + +### AudioListener count is not exactly one + +- **Zero listeners:** All audio will be silent. Use `eval` to add an `AudioListener` component to the main camera: `UnityEngine.Camera.main.gameObject.AddComponent()`. +- **Multiple listeners:** Unity uses the last enabled one, producing unpredictable spatialization. Use the scene-component query recipe for `UnityEngine.AudioListener` and disable all but the intended one. + +### `Load In Background` causes first-play silence + +This is expected behavior: the clip has not finished loading when `Play()` is first called. Mitigate with: +1. Preload the clip at scene start by calling `clip.LoadAudioData()` before it is needed. +2. Use `AudioSource.PlayScheduled()` with a slight delay to allow async load to complete. +3. For AudioSources that must play immediately: switch to `CompressedInMemory` (synchronous on first play) rather than `Streaming` with background load. + +## 7. Completion + +After finishing the audit or optimization: + +- Summarize every setting changed with before/after values. +- List any clips or groups that still need attention (e.g., clips that require on-device measurement to confirm savings). +- If the user needs runtime memory measurement, point them at the Memory Profiler package, which reports the largest AudioClips by runtime byte cost. +- If mixer CPU is still high after the audit, point them at the Unity Profiler's Audio module for DSP thread profiling. + +## Detailed References + +- **Platform settings, compression matrix, load types, sample rates:** [resources/platform-settings.md](resources/platform-settings.md) +- **AudioImporter API recipes and code patterns:** [resources/audio-import-api.md](resources/audio-import-api.md) + +## See Also + +- **Memory Profiler package** — finds the largest AudioClips by runtime byte cost. +- **Unity Profiler, Audio module** — DSP CPU markers and frame-time budget. +- `audio-setup-mixers` — creating mixers and routing Audio Sources into groups. diff --git a/skills/optimize-audio/resources/audio-import-api.md b/skills/optimize-audio/resources/audio-import-api.md new file mode 100644 index 0000000..6be4b53 --- /dev/null +++ b/skills/optimize-audio/resources/audio-import-api.md @@ -0,0 +1,146 @@ +# Audio Import API Recipes + +C# code recipes for `unity command eval --code ''`. All examples target the Unity 6 +AudioImporter API. + +`eval` compiles a statement block, so there are no `using` directives and every type is written +fully qualified. Each recipe `return`s its result as a string rather than calling `Debug.Log`, so +the value comes back on the CLI's stdout instead of only reaching the Editor console. + +## Read AudioClip Importer Settings + +```csharp +var path = UnityEditor.AssetDatabase.GetAssetPath(audioSource.clip); +var importer = (UnityEditor.AudioImporter)UnityEditor.AssetImporter.GetAtPath(path); +return $"forceToMono={importer.forceToMono}, loadType={importer.defaultSampleSettings.loadType}, " + + $"compressionFormat={importer.defaultSampleSettings.compressionFormat}, " + + $"quality={importer.defaultSampleSettings.quality}, " + + $"sampleRateSetting={importer.defaultSampleSettings.sampleRateSetting}, " + + $"sampleRateOverride={importer.defaultSampleSettings.sampleRateOverride}"); +``` + +## Force To Mono and Reimport + +```csharp +var path = UnityEditor.AssetDatabase.GetAssetPath(audioSource.clip); +var importer = (UnityEditor.AudioImporter)UnityEditor.AssetImporter.GetAtPath(path); +importer.forceToMono = true; +importer.SaveAndReimport(); +return $"Reimported {path} — channels now: {audioSource.clip.channels}"); +``` + +## Set Load Type + +```csharp +var path = UnityEditor.AssetDatabase.GetAssetPath(audioSource.clip); +var importer = (UnityEditor.AudioImporter)UnityEditor.AssetImporter.GetAtPath(path); +var settings = importer.defaultSampleSettings; +settings.loadType = UnityEngine.AudioClipLoadType.Streaming; // or CompressedInMemory, DecompressOnLoad +importer.defaultSampleSettings = settings; +importer.SaveAndReimport(); +``` + +## Enable Load In Background + +```csharp +var path = UnityEditor.AssetDatabase.GetAssetPath(audioSource.clip); +var importer = (UnityEditor.AudioImporter)UnityEditor.AssetImporter.GetAtPath(path); +importer.loadInBackground = true; +importer.SaveAndReimport(); +``` + +## Set Compression Format and Quality + +```csharp +var path = UnityEditor.AssetDatabase.GetAssetPath(audioSource.clip); +var importer = (UnityEditor.AudioImporter)UnityEditor.AssetImporter.GetAtPath(path); +var settings = importer.defaultSampleSettings; +settings.compressionFormat = UnityEngine.AudioCompressionFormat.Vorbis; +settings.quality = 0.7f; // 0.0–1.0; raise to 0.7–0.85 for dialogue +importer.defaultSampleSettings = settings; +importer.SaveAndReimport(); +``` + +## Override Sample Rate (Mobile) + +```csharp +var path = UnityEditor.AssetDatabase.GetAssetPath(audioSource.clip); +var importer = (UnityEditor.AudioImporter)UnityEditor.AssetImporter.GetAtPath(path); +var settings = importer.defaultSampleSettings; +settings.sampleRateSetting = UnityEditor.AudioSampleRateSetting.OverrideSampleRate; +settings.sampleRateOverride = 22050u; +importer.defaultSampleSettings = settings; +importer.SaveAndReimport(); +``` + +## Read AudioSource Properties (Batch) + +Read multiple properties in a single `eval` call: + +```csharp +var src = audioSource; +return $"clip={src.clip?.name}, loadType={src.clip?.loadType}, " + + $"channels={src.clip?.channels}, frequency={src.clip?.frequency}, " + + $"spatialBlend={src.spatialBlend}, rolloff={src.rolloffMode}, " + + $"mixerGroup={src.outputAudioMixerGroup?.name ?? "None"}, " + + $"bypassEffects={src.bypassEffects}"); +``` + +## Read DSP Buffer Size + +```csharp +UnityEngine.AudioSettings.GetDSPBufferSize(out int bufferLength, out int numBuffers); +return $"DSP buffer: {bufferLength} samples x {numBuffers} buffers"); +``` + +## Check Source File Format (Lossy Warning) + +```csharp +var path = UnityEditor.AssetDatabase.GetAssetPath(clip); +if (path.EndsWith(".mp3", System.StringComparison.OrdinalIgnoreCase)) + return $"'{clip.name}' is MP3 — lossy source quality is lost permanently after Unity re-encodes. Recommend WAV or AIFF sources."); +``` + +## Resolving `audioSource` / `clip` inside a snippet + +The recipes above are written against an `audioSource` or `clip` variable. `eval` runs each +snippet in a fresh scope, so nothing carries over between calls — resolve the object at the top of +the same snippet that uses it. + +By scene object: + +```csharp +var sources = UnityEngine.Object.FindObjectsByType( + UnityEngine.FindObjectsInactive.Include, UnityEngine.FindObjectsSortMode.None); +var audioSource = System.Array.Find(sources, s => s.gameObject.name == "TheGameObjectName"); +``` + +By asset path, when you already know the clip: + +```csharp +var clip = UnityEditor.AssetDatabase.LoadAssetAtPath("Assets/Audio/Foo.wav"); +``` + +## Enumerate scene components + +Substitute the component type (`UnityEngine.AudioSource`, `UnityEngine.AudioListener`). Inactive +objects are included deliberately — a disabled second listener still counts against the +one-listener rule. + +```csharp +var found = UnityEngine.Object.FindObjectsByType( + UnityEngine.FindObjectsInactive.Include, UnityEngine.FindObjectsSortMode.None); +var names = System.Linq.Enumerable.Select(found, c => c.gameObject.name); +return $"count={found.Length}: {string.Join(", ", names)}"; +``` + +## Enumerate mixer assets + +An `AudioMixer` is a project asset, not a scene object, so it is found through the asset database +rather than a scene query. + +```csharp +var guids = UnityEditor.AssetDatabase.FindAssets("t:AudioMixer"); +var paths = System.Linq.Enumerable.Select(guids, UnityEditor.AssetDatabase.GUIDToAssetPath); +return $"count={guids.Length}: {string.Join(", ", paths)}"; +``` diff --git a/skills/optimize-audio/resources/platform-settings.md b/skills/optimize-audio/resources/platform-settings.md new file mode 100644 index 0000000..bf4b785 --- /dev/null +++ b/skills/optimize-audio/resources/platform-settings.md @@ -0,0 +1,48 @@ +# Audio Platform Settings Reference + +## Compression Format Matrix + +| Platform | Recommended Format | Notes | +|---|---|---| +| PC / cross-platform | Vorbis, quality 0.5–0.7 | Raise to 0.7–0.85 for dialogue; default 0.5 often adds artifacts | +| iOS | AAC | Hardware decode; cheapest CPU | +| Android | Vorbis | Software decode | +| Xbox | XMA | Use platform override in import settings | +| PlayStation | ATRAC9 | Use platform override in import settings | +| Web | Vorbis | Browser handles decode | + +## Sample Rate Recommendations + +| Use Case | Recommended Rate | +|---|---| +| PC / console music and voice | 44100 Hz | +| PC / console SFX | 44100 Hz | +| Mobile SFX | 22050 Hz | +| Mobile dialogue | 22050 or 44100 Hz | +| UI clicks / blips | 22050 Hz | + +Halving the sample rate halves the PCM memory cost. Always report the estimated saving for each clip changed. + +## Load Type Decision Table + +| Load Type | Behavior | Use For | +|---|---|---| +| Decompress On Load | PCM in memory at load; zero per-play CPU | Short SFX < 200 KB (uncompressed) | +| Compressed In Memory | Stays compressed; decompresses on play | Medium clips played occasionally | +| Streaming | Streams from disk; minimal RAM, higher disk I/O | Music, long ambience, voice-overs | + +### Load Type Mismatch Flags + +- **Decompress On Load** on a clip > 1 MB bloats memory. +- **Streaming** on a clip that plays dozens of times simultaneously adds disk pressure. +- Always apply `Load In Background` for any Streaming clip to prevent the main thread stalling on first play. + +## DSP Buffer Size Guidelines + +| Setting | Buffer Size | Use Case | +|---|---|---| +| Best Latency | 256 | Rhythm games, real-time synthesis | +| Good Latency | 512 | General gameplay | +| Best Performance | 1024 | Ambient/cinematic, battery-saving | + +A very small buffer (64 or 128) costs more CPU per frame. If `bufferLength` is < 256, recommend increasing to "Good Latency" or "Best Performance" to trade latency for CPU stability. From 5dd951a48f0eb6aaa4c99a040c74a90f1b358e90 Mon Sep 17 00:00:00 2001 From: Ziyi Zhang Date: Wed, 12 Aug 2026 11:03:04 -0400 Subject: [PATCH 4/7] =?UTF-8?q?fix:=20pull=20asset-transformer-toolkit=20?= =?UTF-8?q?=E2=80=94=20its=20API=20lives=20in=20an=20Assistant-gated=20ass?= =?UTF-8?q?embly?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 1 - skills/asset-transformer-toolkit/SKILL.md | 28 --- .../api-docs/rule-api.md | 154 ------------ .../api-docs/ruleblock-api.md | 37 --- .../api-docs/ruleset-api.md | 138 ----------- .../references/create-importer.md | 121 ---------- .../references/lods.md | 61 ----- .../references/rulesets-and-actions.md | 227 ------------------ 8 files changed, 767 deletions(-) delete mode 100644 skills/asset-transformer-toolkit/SKILL.md delete mode 100644 skills/asset-transformer-toolkit/api-docs/rule-api.md delete mode 100644 skills/asset-transformer-toolkit/api-docs/ruleblock-api.md delete mode 100644 skills/asset-transformer-toolkit/api-docs/ruleset-api.md delete mode 100644 skills/asset-transformer-toolkit/references/create-importer.md delete mode 100644 skills/asset-transformer-toolkit/references/lods.md delete mode 100644 skills/asset-transformer-toolkit/references/rulesets-and-actions.md diff --git a/README.md b/README.md index 8b60a5c..34d2493 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,6 @@ npx skills add Unity-Technologies/skills | `optimize-audio` | Reduce audio memory and DSP CPU cost through import settings, load types, and mixer topology | | `setup-game-inputs` | Input System — action maps, bindings, control schemes, rebinding | | `android-add-adaptive-performance` | Android thermal and power signals mapped to dynamic quality tiers | -| `asset-transformer-toolkit` | 3D model and point-cloud import, RuleSets and Actions, LOD generation | ## Usage diff --git a/skills/asset-transformer-toolkit/SKILL.md b/skills/asset-transformer-toolkit/SKILL.md deleted file mode 100644 index a16f5ae..0000000 --- a/skills/asset-transformer-toolkit/SKILL.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -name: asset-transformer-toolkit -description: Imports 3D models and point clouds into Unity using Asset Transformer Toolkit (formerly Pixyz Plugin), and creates, modifies, and executes RuleSets and Actions for optimization and transformation. Use this skill for any task involving RuleSets (.asset files) or ImporterScriptableObjects, including explaining, inspecting, creating, or modifying rules and actions. It also handles LOD generation. This is the primary method to import 3D models and pointclouds. -required_packages: - com.unity.industry.toolkit: ">=4.0.0" ---- -### API Reference -Tool functions are provided by `Unity.Pixyz.Plugin4Unity.Editor.AI.ATTAssistantUtilities`. They are documented inline in each reference file below alongside the classes they operate on. - -### Technical Notes -Pixyz Plugin is the former name of Asset Transformer Toolkit. Prefer using the term 'Asset Transformer Toolkit' when addressing the user, unless the user is using the term 'Pixyz' -Most classes and code are in the Unity.Pixyz.Plugin4Unity.Editor assembly. -Asset Transformer Toolkit is NOT the same thing as Asset Transformer Studio/Pixyz Studio. NEVER rely on information about Asset Transformer Studio. - -### Importers -Asset Transformer Toolkit can import 3D file from outside the project. -When modifying Importers, NEVER assume it should be immediately followed by a reimport. The import process can be very long, so it must only be launched when the user requests it. -Modify fields in Importers using the ScriptableObject API. NEVER use reflection to access or modify Importer fields — reflection is not available. -To create a Pixyz/Asset Transformer Toolkit Importer for importing a file, or to reimport a model using an existing ImporterScriptableObject, read [references/create-importer](references/create-importer.md) - -### RuleSets and Actions -Read [references/rulesets-and-actions](references/rulesets-and-actions.md) -Read [api-docs/ruleset-api](api-docs/ruleset-api.md) when the RuleSet API is needed. -Read [api-docs/rule-api](api-docs/rule-api.md) when the Rule API is needed. -Read [api-docs/ruleblock-api](api-docs/ruleblock-api.md) when the RuleBlock API is needed, including `ActionBase.Id` for constructing `RuleBlock` instances. - -### Levels of Detail -Read [references/lods](references/lods.md) diff --git a/skills/asset-transformer-toolkit/api-docs/rule-api.md b/skills/asset-transformer-toolkit/api-docs/rule-api.md deleted file mode 100644 index 8c0fa7d..0000000 --- a/skills/asset-transformer-toolkit/api-docs/rule-api.md +++ /dev/null @@ -1,154 +0,0 @@ -## Contents -- [Rule constructors](#rule-constructors) — `Rule()` -- [Rule methods](#rule-methods) — `GetBlock`, `GetBlockIndex`, `RemoveBlockAt`, `RemoveBlock`, `AppendBlock`, `InsertBlock`, `IsLastBlock` -- [Rule properties](#rule-properties) — `Name`, `IsEnabled`, `BlocksCount`, `Blocks` - ---- - -## Rule constructors - -The Rule API reference contains the following constructors. - -### `Rule()` - -This constructor creates an empty Rule with no blocks. - -```csharp -public Rule() -``` - -## Rule methods - -The Rule API reference contains the following methods. - -### `GetBlock` - -This method retrieves the `RuleBlock` at the specified index. - -```csharp -public RuleBlock GetBlock(int i) -``` - -`GetBlock` accepts the following parameter. - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `i` | int | required | Zero-based index of the block to retrieve. | - -This method returns the `RuleBlock` at the given index. - -### `GetBlockIndex` - -This method returns the index of the given `RuleBlock` within the Rule. - -```csharp -public int GetBlockIndex(RuleBlock block) -``` - -`GetBlockIndex` accepts the following parameter. - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `block` | RuleBlock | required | The `RuleBlock` whose index to find. | - -This method returns an `int` index, or `-1` if the block is not found. - -### `RemoveBlockAt` - -This method removes the `RuleBlock` at the specified index. - -```csharp -public void RemoveBlockAt(int index) -``` - -`RemoveBlockAt` accepts the following parameter. - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `index` | int | required | Zero-based index of the block to remove. | - -### `RemoveBlock` - -This method removes a specific `RuleBlock` instance from the Rule. - -```csharp -public void RemoveBlock(RuleBlock block) -``` - -`RemoveBlock` accepts the following parameter. - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `block` | RuleBlock | required | The `RuleBlock` instance to remove. | - -### `AppendBlock` - -This method adds a `RuleBlock` to the end of the Rule and sets its back-reference to this Rule. - -```csharp -public void AppendBlock(RuleBlock block) -``` - -`AppendBlock` accepts the following parameter. - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `block` | RuleBlock | required | The `RuleBlock` instance to append. | - -### `InsertBlock` - -This method inserts a `RuleBlock` at the specified index, shifting subsequent blocks down. If `index` is beyond the last position, the block is appended instead. - -```csharp -public void InsertBlock(RuleBlock block, int index) -``` - -`InsertBlock` accepts the following parameters. - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `block` | RuleBlock | required | The `RuleBlock` instance to insert. | -| `index` | int | required | Zero-based position at which to insert the block. | - -## Rule properties - -The Rule API reference contains the following properties. - -### `Name` - -```csharp -public string Name { get; set; } -``` - -The display name of the Rule. Setting this property notifies the UI. - -> **Note:** C# property names are case-sensitive. Use `Name` (capital N) — `name` does not exist and will not compile. - -```csharp -rule.Name = "My Rule"; -string ruleName = rule.Name; -``` - -### `IsEnabled` - -```csharp -public bool IsEnabled { get; set; } -``` - -Controls whether the Rule is active within its RuleSet. When `false`, the Rule is skipped during execution. Setting this property notifies the UI. - -### `BlocksCount` - -```csharp -public int BlocksCount { get; } -``` - -The number of `RuleBlock` instances currently in the Rule. - -### `Blocks` - -```csharp -public IEnumerable Blocks { get; } -``` - -An enumerable over all `RuleBlock` instances in the Rule, in execution order. diff --git a/skills/asset-transformer-toolkit/api-docs/ruleblock-api.md b/skills/asset-transformer-toolkit/api-docs/ruleblock-api.md deleted file mode 100644 index 97cf80a..0000000 --- a/skills/asset-transformer-toolkit/api-docs/ruleblock-api.md +++ /dev/null @@ -1,37 +0,0 @@ -## ActionBase properties - -### `Id` - -```csharp -public abstract int Id { get; } -``` - -A unique integer identifier for an action type. Pass this to the `RuleBlock(int actionId)` constructor. - -## RuleBlock constructors - -The RuleBlock API reference contains the following constructors. - -### `RuleBlock(int actionId)` - -This constructor creates a RuleBlock that will execute the action identified by `actionId`. The action instance is created lazily on first access. - -```csharp -public RuleBlock(int actionId) -``` - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `actionId` | int | required | The ID of the action this RuleBlock will trigger. Use `ActionBase.Id` to obtain this value from an action instance. | - -## RuleBlock properties - -The RuleBlock API reference contains the following properties. - -### `IsEnabled` - -```csharp -public bool IsEnabled { get; set; } -``` - -Controls whether this block is active within its Rule. When `false`, the block is skipped during execution. diff --git a/skills/asset-transformer-toolkit/api-docs/ruleset-api.md b/skills/asset-transformer-toolkit/api-docs/ruleset-api.md deleted file mode 100644 index fbc40ee..0000000 --- a/skills/asset-transformer-toolkit/api-docs/ruleset-api.md +++ /dev/null @@ -1,138 +0,0 @@ -## Contents -- [RuleSet methods](#ruleset-methods) — `GetRule`, `GetRuleIndex`, `RemoveRuleAt`, `RemoveRule`, `InsertRule`, `AppendRule`, `IsValid` -- [RuleSet properties](#ruleset-properties) — `RulesCount` -- [RuleSet utility functions](#ruleset-utility-functions) — `RunRuleSet` - ---- - -## RuleSet methods - -The RuleSet API reference contains the following methods. - -### `GetRule` - -This method retrieves the `Rule` at the specified index. - -```csharp -public Rule GetRule(int i) -``` - -`GetRule` accepts the following parameter. - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `i` | int | required | Zero-based index of the rule to retrieve. | - -This method returns the `Rule` at the given index. - -### `GetRuleIndex` - -This method returns the index of the given `Rule` within the RuleSet. Returns `-1` and logs an error if the RuleSet is currently running. - -```csharp -public int GetRuleIndex(Rule rule) -``` - -`GetRuleIndex` accepts the following parameter. - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `rule` | Rule | required | The `Rule` whose index to find. | - -This method returns an `int` index, or `-1` if the RuleSet is running or the rule is not found. - -### `RemoveRuleAt` - -This method removes the rule at the specified index. Has no effect and logs an error if the RuleSet is currently running. - -```csharp -public void RemoveRuleAt(int index) -``` - -`RemoveRuleAt` accepts the following parameter. - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `index` | int | required | Zero-based index of the rule to remove. | - -### `RemoveRule` - -This method removes a specific `Rule` instance from the RuleSet. Has no effect and logs an error if the RuleSet is currently running. - -```csharp -public void RemoveRule(Rule rule) -``` - -`RemoveRule` accepts the following parameter. - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `rule` | Rule | required | The `Rule` instance to remove. | - -### `InsertRule` - -This method inserts a `Rule` at the specified index, shifting subsequent rules down. Has no effect and logs an error if the RuleSet is currently running. - -```csharp -public void InsertRule(int index, Rule rule, bool notify) -``` - -`InsertRule` accepts the following parameters. - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `index` | int | required | Zero-based position at which to insert the rule. | -| `rule` | Rule | required | The `Rule` instance to insert. | -| `notify` | bool | required | When `true`, notifies the UI that the RuleSet has changed. | - -### `AppendRule` - -This method adds a `Rule` to the end of the RuleSet. Has no effect and logs an error if the RuleSet is currently running. - -```csharp -public void AppendRule(Rule rule) -``` - -`AppendRule` accepts the following parameter. - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `rule` | Rule | required | The `Rule` instance to append. | - -### `IsValid` - -This method validates that every enabled action in the RuleSet has valid input. It skips disabled rules. - -```csharp -public bool IsValid() -``` - -This method accepts no parameters. It returns `true` if all enabled actions pass validation, or `false` if any action reports an error. - -## RuleSet properties - -The RuleSet API reference contains the following properties. - -### `RulesCount` - -```csharp -public int RulesCount { get; } -``` - -The number of `Rule` instances currently in the RuleSet. - -## RuleSet utility functions - -The following functions are from `Unity.Pixyz.Plugin4Unity.Editor.AI.ATTAssistantUtilities`. - -### `RunRuleSet` - -This function executes all rules in a RuleSet asset against the current scene selection, or the entire scene if nothing is selected. - -```csharp -public static void RunRuleSet(string rulesetPath) -``` - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `rulesetPath` | string | required | Project-relative path to the RuleSet asset to run. | diff --git a/skills/asset-transformer-toolkit/references/create-importer.md b/skills/asset-transformer-toolkit/references/create-importer.md deleted file mode 100644 index f0342d7..0000000 --- a/skills/asset-transformer-toolkit/references/create-importer.md +++ /dev/null @@ -1,121 +0,0 @@ -## Contents -- [Workflow steps](#workflow) — Pre-Flight, Create Importer, Configure Importer, Import -- [Importer utility functions](#importer-utility-functions) — `EnsureImporterSaveFolder`, `GetImporterTypes`, `GetImporterProperties`, `CreateImporter`, `CanImportFile`, `ImportFile` - ---- - -If a new Importer does not need to be created, skip to Step 3 - -### Step 1: Pre-Flight -- Check whether an importer asset already exists for this file. If it does, skip to Step 3. -- Verify the file to be imported exists. -- Verify the file type is supported by Pixyz/Asset Transformer Toolkit using `ATTAssistantUtilities.CanImportFile()`. -- Check what kind of file (eg. point cloud, CAD) is it. Find the Importer type that would best match it. - -### Step 2: Create Importer -- Create the appropriate Importer asset with the path to the file to be imported. The path must be relative to the Application.dataPath. -- If there are issues, fix and revalidate. Do not exceed 3 iterations. -Do not continue if this step cannot be completed successfully. - -### Step 3: Configure Importer -- If requested, change the importer's settings. -- If requested, assign the requested RuleSet to the Importer. -- If requested, make changes to the LOD generation. -Continue only if asked to also import the file. The import process can be very long, so NEVER assume you must import the file unless it was requested. - -### Step 4: Import -- Start the asynchronous import using `ATTAssistantUtilities.ImportFile()`. -- Report whether the import process started successfully. Remind the user this is a background process. - -## Importer utility functions - -The following functions are from `Unity.Pixyz.Plugin4Unity.Editor.AI.ATTAssistantUtilities`. - -### `EnsureImporterSaveFolder` - -Returns the project-relative asset save folder configured in Asset Transformer Toolkit Project Settings, creating it if it does not already exist. Use this as the destination path when creating a new `ImporterScriptableObject` asset. - -```csharp -public static string EnsureImporterSaveFolder() -``` - -Returns a `string` such as `"Assets/3DModels"`. - -### `GetImporterTypes` - -Returns the names of all `ImporterScriptableObject` types available in the project, including user-defined importers. Always call this before creating or referencing an importer type — never assume a type exists. - -```csharp -public static string[] GetImporterTypes() -``` - -Returns a `string[]` of unqualified type names (e.g. `"CADImporterScriptableObject"`). - -### `GetImporterProperties` - -Returns the public serialized fields of an `ImporterScriptableObject` type by name, including fields from intermediate base classes. Use this to discover what settings are available on any importer type — concrete importer classes may be internal or user-defined. - -```csharp -public static ImporterPropertyInfo[] GetImporterProperties(string typeName) -``` - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `typeName` | string | required | Unqualified type name as returned by `GetImporterTypes()`. | - -Returns an `ImporterPropertyInfo[]`. Each entry has the following fields: - -| Field | Type | Description | -|-------|------|-------------| -| `Name` | string | Human-readable property name. | -| `Type` | string | Unqualified type name of the field. | -| `SerializedPropertyPath` | string | The exact string to pass to `SerializedObject.FindProperty`. For auto-properties declared with `[field: SerializeField]`, this differs from `Name` (e.g. `k__BackingField`). Always use this field — never construct the path from `Name` yourself. | - -### `CreateImporter` - -Creates a new `ImporterScriptableObject` asset for the given file. Call `GetImporterTypes()` first to confirm the type name — never assume or invent one. Use `EnsureImporterSaveFolder()` to get the correct save path. - -```csharp -public static string CreateImporter(string filePath, string typeName, string savePath) -``` - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `filePath` | string | required | Absolute system path to the 3D file the importer will reference. | -| `typeName` | string | required | Unqualified importer type name as returned by `GetImporterTypes()`. | -| `savePath` | string | required | Project-relative folder path where the importer asset will be saved. | - -Returns a `string` with the project-relative path to the created asset (e.g. `"Assets/3DModels/MyModel.asset"`). Pass this path to `ImportFile` to trigger import. - -### `CanImportFile` - -Checks whether the Asset Transformer Toolkit supports a file format. Call this before `ImportFile` to avoid runtime errors. Throws if no file exists at `filePath`. - -```csharp -public static bool CanImportFile(string filePath) -``` - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `filePath` | string | required | Absolute system path to the 3D file to check. | - -Returns `true` if the file format is supported, `false` otherwise. - -### `ImportFile` - -Triggers import or re-import using an existing `ImporterScriptableObject` asset. Requires an `ImporterScriptableObject` to already exist at `importerPath`. - -```csharp -public static void ImportFile(string importerPath) -``` - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `importerPath` | string | required | Project-relative path to the `ImporterScriptableObject` asset. | - -### Technical Notes -- Because import is an asynchronous process, the imported files may not yet exist in the project by the time the prompt finishes execution. -- The imported model will always be saved to the same directory that the ImporterScriptableObject lives in. -- Importers are designed to be extended. Check which Importer types exist in the project using `ATTAssistantUtilities.GetImporterTypes()` instead of making assumptions like a point cloud file always maps to the PointCloudImporterScriptableObject. -- Only one import should be processing at a time. -- When creating a new ImporterScriptableObject, save it under the folder returned by `ATTAssistantUtilities.EnsureImporterSaveFolder()`. This reflects the user-configured save folder from Project Settings and defaults to `"Assets/3DModels"`. diff --git a/skills/asset-transformer-toolkit/references/lods.md b/skills/asset-transformer-toolkit/references/lods.md deleted file mode 100644 index 9e4a768..0000000 --- a/skills/asset-transformer-toolkit/references/lods.md +++ /dev/null @@ -1,61 +0,0 @@ -Choose a path depending on the task. - -### Path A: CADImporterScriptableObject -Choose this path if working with an object that is or inherits from CADImporterScriptableObject. -- Identify the types of LODs available in the project. Choose the one(s) most appropriate for your case. -- Find the LODGenerator property in the Importer. -- Execute the requested task. -- Verify the number of rules present in the Generator does not exceed 7. - -### Path B: Scene Object -Choose this path when asked to work with a scene asset directly. -- Add a UnityEngine.PixyzPlugin4Unity.Components.LODGeneratorComponent script to the GameObject if one doesn't exist anywhere in the hierarchy. -- Execute the given task. - -### Path C: PointCloudImporter -Choose this path when working with a PointCloudImporterScriptableObject object. -- The PointCloudImporter only permits enabling/disabling LOD generation and setting the number of LODs to be generated. Other actions like choosing the type of LODRule to apply is unsupported. -- Models imported by the PointCloudImporter will not have a LODGeneratorComponent script attached to them. -- If the task is permitted, execute it. - -### Path D: Other Importer -Choose this path if working with a different importer type. -- This is a user defined importer and not part of the base Asset Transformer Toolkit package. Precise instructions cannot be provided. -- Attempt to perform the requested task, but do not exceed three iterations. -- If the task cannot be performed successfully, inform the user working with this object is not currently supported by Assistant. - -## LOD utility functions - -The following functions are from `Unity.Pixyz.Plugin4Unity.Editor.AI.ATTAssistantUtilities`. - -### `GetLODRules` - -Returns all available `LODRule` implementations and their parameters. Use this before constructing or modifying a LOD configuration to discover valid types. - -```csharp -public static LODRuleDescription[] GetLODRules() -``` - -Returns a `LODRuleDescription[]`, each containing the type name, assembly-qualified name, and parameter list. - -### `LODRuleDescription` - -| Field | Type | Description | -|-------|------|-------------| -| `Name` | string | Unqualified class name of the LODRule type. | -| `QualifiedName` | string | Assembly-qualified name, suitable for `Type.GetType`. | -| `Parameters` | `LODParameterInfo[]` | Configurable properties on this LODRule type. | - -### `LODParameterInfo` - -| Field | Type | Description | -|-------|------|-------------| -| `Name` | string | Property name. | -| `Type` | string | Property type name. | - -### Technical Notes -- The user will need to refresh the Inspector to see changes applied by Assistant. -- In an Importer's settings for LOD generation, the number of LODs set to generate does not include LOD0. There will always be one more LOD than the setting says. For example, setting the PointCloudImporter's NumberofLODs setting to 1 will result in the model having two LODs: LOD0 and LOD1. -- While Unity supports up to 8 LODs including LOD0, the PointCloudImporter is a special case that only supports 7. -- With the exception of models imported by the PointCloudImporter, models that were imported with LODs will have a LODGeneratorComponent script. -- LODs have nothing to do with RuleSets and Actions. RuleSets and Actions will NOT help with any LOD-related task. diff --git a/skills/asset-transformer-toolkit/references/rulesets-and-actions.md b/skills/asset-transformer-toolkit/references/rulesets-and-actions.md deleted file mode 100644 index 834f21a..0000000 --- a/skills/asset-transformer-toolkit/references/rulesets-and-actions.md +++ /dev/null @@ -1,227 +0,0 @@ -## Contents -- [Actions](#actions) -- [RuleSets](#rulesets) - - Description - - Modifying RuleSets - - Creating RuleSets - - Setting Action Parameters - - Running RuleSets - - Validation Checklist -- [Action utility functions](#action-utility-functions) — `GetActionsList`, `GetActionDefinitions`, `SetActionParameter` -- [Action utility output types](#action-utility-output-types) — `ActionInfo`, `ActionDefinition`, `ActionParameterInfo`, `EnumInfo` - ---- - -## Actions - -Actions are classes derived from UnityEditor.PixyzPlugin4Unity.Actions.ActionBase that execute a task on a list of input objects. -It is expected that users will create additional Action classes to extend the Actions available in the base package. - -## RuleSets - -### Description -RuleSets are a SerializedObject containing a list of Rule instances, which contain a list of Action instances. RuleSets are used to ensure a set of Actions are executed in a specific order. -RuleSets are derived from ScriptableObject and must end with the '.asset' extension. -RuleSets support conversion to json. This will include information about the Actions they contain. - -### Modifying RuleSets - -#### Step 1: Load RuleSet -- Use AssetDatabase.LoadAssetAtPath to load the asset from memory. - -#### Step 2: Construct the script -- Read the API reference for the RuleSet class. Also read the API reference for the Rule and RuleBlock classes if required. -- Construct a script using the APIs from those files, to run as C# in a live Editor. Here is an - example that adds a Decimate Action to a RuleSet: - -```csharp -using UnityEngine; -using UnityEditor; -using UnityEditor.PixyzPlugin4Unity.RuleEngine; - -string path = "Assets/Rulesets/OptimizationRuleSet.asset"; -RuleSet ruleSet = AssetDatabase.LoadAssetAtPath(path); - -if (ruleSet == null) - throw new System.Exception($"RuleSet not found at {path}"); - -Undo.RecordObject(ruleSet, "Add Decimate action"); - -if (ruleSet.RulesCount == 0) - throw new System.Exception($"No rules found in RuleSet {path}"); - -Rule rule = ruleSet.GetRule(0); - -// Decimate Action ID is 277054868 -RuleBlock decimateBlock = new RuleBlock(277054868); - -rule.AppendBlock(decimateBlock); - -EditorUtility.SetDirty(ruleSet); -AssetDatabase.SaveAssets(); - -return $"Added Decimate action to {path} (Rule index 0)"; -``` - -A proper script for modifying a RuleSet has the following traits: -- Does not use the ScriptableObject API (this bypasses necessary event triggers). - -#### Step 3: Validation -- Run the script as C# in a live Editor. -- Validate the RuleSet against the validation checklist. - - -### Creating RuleSets - -#### Step 1: Create RuleSet -- Create the UnityEditor.PixyzPlugin4Unity.RuleEngine.RuleSet asset. -Continue to Step 2 if actions need to be added to the RuleSet. If not, add the GetContextGameObjects action and jump to Step 3. - -#### Step 2: Add Actions - -**Preflight** -- Ensure the RuleSet exists. -- Choose the combination of Actions that will best perform the requested procedure. NEVER create new Actions without explicit permission. Instead, use the Actions returned by `GetActionsList`. -- Divide Actions into Rules based on the GameObject they need to act upon. Each Rule initially executes on every GameObject unless the input is narrowed with a Filter action. Example: If only lights need to be disabled and only meshes with >10000 vertices need to be decimated, two rules will be needed as this is two different groups of GameObjects. - -**Adding Rules** -- Check whether the existing Rule(s) in the RuleSet is just the GetContextGameObjects action. If it is, append the group of actions to it rather than creating a new Rule. -- If a new Rule needs to be created, add it to the RuleSet. -- Add Actions to the Rules. -- Set action parameters if required — see Setting Action Parameters below. - -**Technical Notes** -- All Actions derive from the ActionBase class. -- Actions are located in the UnityEditor.PixyzPlugin4Unity.Actions namespace. - -#### Step 3: Validation -- Validate the RuleSet logic using the validation checklist. - - -### Setting Action Parameters - -#### Step 1: Gather data -- Gather any missing information needed to call `ATTAssistantUtilities.SetActionParameter`. If you need to retrieve a GlobalObjectId, first read the GlobalObjectId class to choose the correct function to call. -- Call `ATTAssistantUtilities.SetActionParameter` to set the parameter. -- If the result is false and not an exception, retry a maximum of three times. -- Follow a path based on the result. - -#### Path A: AITypeSecurityException -Follow these steps if `ATTAssistantUtilities.SetActionParameter` threw an AITypeSecurityException. -- Inform the user the parameter cannot be set programmatically for security reasons. -- Advise the user to manually set the parameter and what value to set it to. - -#### Path B: Exception -Follow these steps if `ATTAssistantUtilities.SetActionParameter` threw any other exception. -- Warn the user the property was unable to be set. -- Advise the user to manually set the parameter and what value to set it to. - -#### Path C: Success -Perform these steps if `ATTAssistantUtilities.SetActionParameter` returned true. -- If the property was set to a scene GameObject, NEVER validate it because it is not persistent. INSTEAD warn the user the property value is temporary and will be lost. -- Report the success to the user. - -#### Path D: Failure -Perform this step if `ATTAssistantUtilities.SetActionParameter` always returns false. -- Report the failure to the user and advise them to set the parameter manually. Tell them what the property should be set to. - -All paths are exclusive. - -**Technical Notes** -- For the Decimate action specifically, if mesh quality is going to be set to a preset, the Criterion parameter must also be set to Quality. -- Prefer using presets when possible rather than individually setting each value. -- If a preset is used, avoid changing values the preset changed unless requested otherwise. - -**Safety & Constraints** -1. **One-Strike Rule**: If `ATTAssistantUtilities.SetActionParameter` throws an AITypeSecurityException, you MUST TERMINATE the task immediately. Do NOT use raw C# execution, reflection, or any other method to bypass this. Follow the steps in Path A as your final actions. - - -### Running RuleSets -Use `ATTAssistantUtilities.RunRuleSet()` instead of the RuleSet's public API to run a RuleSet. -Only one RuleSet must be running at a time. -When running a RuleSet, remind the user it is a background task/asynchronous. - - -### Validation Checklist -- The first Action in each Rule is GetContextGameObjects or RunRules. -- If the RunRules Action is in a Rule, it is the only Action. -- Each Rule has at least one Action. - - -## Action utility functions - -The following functions are from `Unity.Pixyz.Plugin4Unity.Editor.AI.ATTAssistantUtilities`. - -### `GetActionsList` - -Returns all Rule Engine actions available in the project, including user-defined actions. Use this when you do not already know an action's ID. Pass the returned IDs to `GetActionDefinitions` to inspect parameters. - -```csharp -public static ActionInfo[] GetActionsList() -``` - -Returns an `ActionInfo[]` containing the name, tooltip, and ID of every available action. - -### `GetActionDefinitions` - -Returns parameter definitions for one or more actions by unqualified class name (e.g. `"Decimate"`, not `"UnityEditor.PixyzPlugin4Unity.Actions.Decimate"`). Use this before `SetActionParameter` to obtain correct parameter names and types. Throws if an action class name is not found. - -```csharp -public static ActionDefinition[] GetActionDefinitions(string[] actionClassNames) -``` - -Returns an `ActionDefinition[]`, each containing the action ID and its full parameter list. - -### `SetActionParameter` - -Sets a `UserParameter` field value on an action within a RuleSet. Use `GetActionDefinitions` first to obtain the correct parameter name. Returns `false` if the field was not found. - -```csharp -public static bool SetActionParameter(string ruleSetPath, int ruleIndex, int ruleblockIndex, string parameterName, string value) -``` - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `ruleSetPath` | string | required | Project-relative path to the RuleSet asset. | -| `ruleIndex` | int | required | Zero-based index of the rule containing the action. | -| `ruleblockIndex` | int | required | Zero-based index of the action block within the rule. | -| `parameterName` | string | required | The `ParameterPath` value from `GetActionDefinitions`. Must not be fully qualified. | -| `value` | string | required | String representation of the value to set. For Unity assets or scene objects, provide a `GlobalObjectId` string. For `LayerMask`, use layer names separated by `\|`. | - -Returns `true` if the parameter was set and the RuleSet saved, `false` if the field was not found. - -## Action utility output types - -### `ActionInfo` - -| Field | Type | Description | -|-------|------|-------------| -| `Name` | string | Fully qualified class name of the action. | -| `Description` | string | Tooltip text describing what the action does. | -| `ID` | int | Unique integer ID. Pass to `GetActionDefinitions` or use as `RuleBlock` action ID. | - -### `ActionDefinition` - -| Field | Type | Description | -|-------|------|-------------| -| `ID` | int | Unique integer ID of the action. | -| `Parameters` | `ActionParameterInfo[]` | All configurable `UserParameter` fields on the action. | - -### `ActionParameterInfo` - -| Field | Type | Description | -|-------|------|-------------| -| `Name` | string | Immediate field name. | -| `ParameterPath` | string | Full dot-separated path to pass as `parameterName` to `SetActionParameter` (e.g. `"advancedParametersQuality.surfacicTolerance"`). | -| `Type` | string | Fully qualified type name of the field. | -| `Description` | string | Tooltip describing the parameter. | -| `IsConditional` | bool | `true` if this parameter is only visible under certain conditions. | -| `PossibleEnumValues` | `EnumInfo[]` | Valid values if the parameter is an enum type. | -| `NestedParameters` | `ActionParameterInfo[]` | Child parameters for struct fields. | - -### `EnumInfo` - -| Field | Type | Description | -|-------|------|-------------| -| `Label` | string | Name of the enum value. | -| `Value` | Int64 | Underlying integer value of the enum member. | From ffa90f8616335cf57bef01273628d6773068d74f Mon Sep 17 00:00:00 2001 From: Ziyi Zhang Date: Wed, 12 Aug 2026 13:10:09 -0400 Subject: [PATCH 5/7] fix: hold android-add-adaptive-performance until its output is validated --- README.md | 1 - .../android-add-adaptive-performance/SKILL.md | 92 ------ .../resources/ADAPTIVE_PERFORMANCE_PLAN.md | 83 ----- .../AdaptivePerformanceSignalManager.cs | 269 ---------------- .../resources/AdaptiveQualityAdapter.cs | 297 ------------------ 5 files changed, 742 deletions(-) delete mode 100644 skills/android-add-adaptive-performance/SKILL.md delete mode 100644 skills/android-add-adaptive-performance/resources/ADAPTIVE_PERFORMANCE_PLAN.md delete mode 100644 skills/android-add-adaptive-performance/resources/AdaptivePerformanceSignalManager.cs delete mode 100644 skills/android-add-adaptive-performance/resources/AdaptiveQualityAdapter.cs diff --git a/README.md b/README.md index 34d2493..d6f36c4 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,6 @@ npx skills add Unity-Technologies/skills | `ui-imgui` | IMGUI editor tooling — EditorWindows, custom Inspectors, PropertyDrawers | | `optimize-audio` | Reduce audio memory and DSP CPU cost through import settings, load types, and mixer topology | | `setup-game-inputs` | Input System — action maps, bindings, control schemes, rebinding | -| `android-add-adaptive-performance` | Android thermal and power signals mapped to dynamic quality tiers | ## Usage diff --git a/skills/android-add-adaptive-performance/SKILL.md b/skills/android-add-adaptive-performance/SKILL.md deleted file mode 100644 index bdbd6ad..0000000 --- a/skills/android-add-adaptive-performance/SKILL.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -name: android-add-adaptive-performance -description: Implements Unity Adaptive Performance for Android (Unity >= 6.0 / 6000.0.0) by - handling hardware thermal/power signals and mapping them to quality tiers with dynamic - graphics and simulation quality adjustments. Use when the user asks about adaptive - performance, thermal throttling, dynamic quality scaling, FPS drops on Android, or - optimizing Android game performance. -required_editor_version: ">=6000.0.0" -required_packages: - com.unity.adaptiveperformance: ">=4.0.0" ---- - -## Quick Start - -Adds a full Adaptive Performance integration to a Unity Android project. Creates two MonoBehaviours (`AdaptivePerformanceSignalManager` and `AdaptiveQualityAdapter`), optionally configures URP quality settings, and bootstraps the system into the user's chosen scene. - -## Critical Rules - -- Do not make any subjective calls — ask the user when in doubt -- Follow steps in strict order; never jump ahead -- STOP at every `WAIT` checkpoint and await the user's response before continuing -- Do not install any packages -- Do not add the bootstrap GameObject until all scripts are written and in place -- Use `AdaptivePerformanceIntegration` as the namespace for all generated code -- Do not create placeholder files when creating folders - -## Workflow - -### Step 1: Gather Required Information - -Do not output any code until all answers are collected. - -1. Attempt to detect the graphics pipeline (URP / HDRP / Built-in). If there is any doubt or you cannot determine it, ask the user directly. -2. Ask the user how many quality tiers they want. Default is 3 (Best / Medium / Low). Determine appropriate tier names yourself based on their answer. -3. Ask whether quality should snap back to Best when conditions normalize, or use a sticky downgrade (stay at the lowest tier reached for the session, preventing oscillation). - -**WAIT for the user to answer all three questions before proceeding.** - -### Step 2: Create URP Quality Settings (skip entirely if not URP) - -- Create `mobile_adaptive` and `mobile_max` quality settings, configured to affect Android only. -- For each, create a matching URP Render Pipeline Asset under `Assets/AdaptivePerformanceManager/Settings/`: - - `urp_mobile_adaptive` — Enable Adaptive Performance: **ON** - - `urp_mobile_max` — Enable Adaptive Performance: **OFF** - -Tell the user what was created, then continue. - -### Step 3: Generate the Signal Handler Script - -Copy `AdaptivePerformanceSignalManager` from `resources/AdaptivePerformanceSignalManager.cs` into `Assets/AdaptivePerformanceManager/`. Adjust tier count and names to match the user's answers from Step 1. - -Tell the user the script has been added, then continue. - -### Step 4: Generate the Quality Adapter Script - -Copy `AdaptiveQualityAdapter` from `resources/AdaptiveQualityAdapter.cs` into `Assets/AdaptivePerformanceManager/`. Apply all scalers — never skip any. Add a `// HIGH VISUAL IMPACT` comment above any scaler line that may visually disrupt gameplay (leave the code intact). - -For FPS targets, use display Hz divisors: -- 60 Hz displays: 60, 30 FPS -- 90 Hz displays: 90, 45, 30 FPS -- 120 Hz displays: 120, 60, 40, 30 FPS - -See `resources/ADAPTIVE_PERFORMANCE_PLAN.md` for the full scaler table and URP-specific scalers. - -Tell the user both scripts are ready, then continue. - -### Step 5: Add Bootstrap GameObject - -Ask the user which scene to place the `AdaptivePerformanceSignalManager` GameObject in. - -**WAIT for the user to respond before continuing.** - -1. Add the GameObject to the specified scene. -2. Attach `AdaptivePerformanceSignalManager` and `AdaptiveQualityAdapter` components to it and configure all references. -3. Save the scene. - -### Step 6: Produce a Final Checklist - -List everything that was done, then list what the user must do manually: - -- Go to **Project Settings → Adaptive Performance** and enable "Enable Adaptive Performance" if not already checked. -- In the Providers section, check **Android Provider**. -- If new Quality Settings were added, review and adjust each one's settings and verify the URP Render Pipeline Assets are configured correctly. -- `AdaptivePerformanceSignalManager.cs` is the primary script to customize tier logic. -- Optional: `AdaptiveLayerCulling` via `Camera.main.layerCullDistances` is available but requires detailed per-project setup by the user. -- Optional: Post-processing `VolumeProfiles` in scene Volumes are not controlled by this integration and can be tuned manually for additional adaptive gains. - -## Detailed References - -- **Full implementation plan and scaler tables:** [resources/ADAPTIVE_PERFORMANCE_PLAN.md](resources/ADAPTIVE_PERFORMANCE_PLAN.md) -- **Signal handler template:** [resources/AdaptivePerformanceSignalManager.cs](resources/AdaptivePerformanceSignalManager.cs) -- **Quality adapter template:** [resources/AdaptiveQualityAdapter.cs](resources/AdaptiveQualityAdapter.cs) diff --git a/skills/android-add-adaptive-performance/resources/ADAPTIVE_PERFORMANCE_PLAN.md b/skills/android-add-adaptive-performance/resources/ADAPTIVE_PERFORMANCE_PLAN.md deleted file mode 100644 index aae7c80..0000000 --- a/skills/android-add-adaptive-performance/resources/ADAPTIVE_PERFORMANCE_PLAN.md +++ /dev/null @@ -1,83 +0,0 @@ -# Adaptive Performance Signal Handler Skill Plan -Target: Unity >=6.0 (6000.0.0), Android only -Constraints: -- Do not make any subjective calls -- Follow the full implementation plan strictly -- All steps should be done by you, unless it is explicitly mentioned that the user should do it -- Do not install any packages -- No flicker detection -- Stop at each step that says to ask the user a question and ask the question. You should not get all the way to the end and ask a lot of questions at once. This overwhelms the user. -- Do not add the bootstrap GameObject before adding and writing all necessary code -- Use AdaptivePerformanceIntegration as the namespace for any code -- Do not create any placeholder files when creating folders - -## Step 1: Gather minimum required information -Do not output code until all answers are collected. - -1. Attempt to detect graphics pipeline (URP/HDRP/Built-in) but if there is any doubt or inablity to do so, ask the user directly. -2. Tier structure: - - Default: 3 tiers (0..2) meaning Best/Medium/Low - - Ask the user if they want to change the number of tiers and determine good names to map them to on your own. -3. Hardware condition reversal behavior: - - Ask the user if they want to return to “Best” when conditions normalize or keep the game running at the lowest detected quality (sticky downgrade). - -## Step 2 (Skip if not using URP) Create quality settings and URP Render Pipeline assets -- Create one "mobile_adaptive" and one "mobile_max" quality setting. -- Each one should be set to affect Android only -- For each of the mobile quality settings, create and set a matching URP Render Pipeline Asset in Assets/AdaptivePerformanceManager/Settings/ named "urp_mobile_adaptive" and "urp_mobile_max" respectively. -- Turn on "Use Enable Adaptive Performance" for the URP Render Pipeline asset "urp_mobile_adaptive". -- Turn off "Use Enable Adaptive Performance" for the URP Render Pipeline asset "urp_mobile_max". - -## Step 3: Generate the “signal handling” script only -Add AdaptivePerformanceSignalManager from resources/AdaptivePerformanceSignalManager.cs to Assets/AdaptivePerformanceManager/ and adjust based on the decided tier structure - -## Step 4: Write an integration the user can customize -Here are the frame rates that each display type supports so you will want to try and determine the Hz of the display to know which FPS numbers you will be able to drop to. -60 Hz displays: 60 FPS, 30 FPS -90 Hz displays: 90 FPS, 45 FPS, 30 FPS -120 Hz displays: 120 FPS, 60 FPS, 40 FPS, 30 FPS - -Include all available scalers. Important: Do not skip any scaler. -Warn the user with a code comment in AdaptiveQualityAdapter if any particular scaler may negatively affect gameplay or high visual impact. - -For all pipelines: -General performance scalers (Do not skip ANY of these under any circumstances): -General scalers Min Scale Max Scale Max Level Visual Impact Target Setting scaled -AdaptiveLOD 0.4 1 3 High GPU QualitySettings.lodBias -AdaptiveResolution 0.5 1 9 Low GPU/FillRate AdaptivePerformanceRenderSettings.RenderScaleMultiplier and AdaptivePerformanceRenderSettings.ScalableBuffers -AdaptiveFramerate 15 60 45 High CPU/GPU/FillRate Application.targetFrameRate -AdaptiveViewDistance 50 1,000 40 High GPU Camera.main.farClipPlane -AdaptivePhysics 0.5 1 5 Low CPU Time.fixedDeltaTime - -Additional scalers for URP pipelines only: -Universal Render Pipeline (URP) scalers (Do not skip ANY of these under any circumstances) -URP scalers Min Scale Max Scale Max Level Visual Impact Target Setting scaled -AdaptiveBatching 0 1 1 Medium CPU AdaptivePerformanceRenderSettings.SkipDynamicBatching -AdaptiveLUT 0 1 1 Medium CPU/GPU AdaptivePerformanceRenderSettings.LutBias -AdaptiveMSAA 0 1 2 Medium GPU/FillRate AdaptivePerformanceRenderSettings.AntiAliasingQualityBias -AdaptiveShadowCascade 0 1 2 Medium CPU/GPU AdaptivePerformanceRenderSettings.MainLightShadowCascadesCountBias -AdaptiveShadowDistance 0.15 1 3 Low GPU AdaptivePerformanceRenderSettings.MaxShadowDistanceMultiplier -AdaptiveShadowQuality 0 1 3 High CPU/GPU AdaptivePerformanceRenderSettings.ShadowQualityBias -AdaptiveShadowmapResolution 0.15 1 3 Low GPU AdaptivePerformanceRenderSettings.MainLightShadowmapResolutionMultiplier -AdaptiveSorting 0 1 1 Medium CPU AdaptivePerformanceRenderSettings.SkipFrontToBackSorting -AdaptiveTransparency 0 1 1 High GPU AdaptivePerformanceRenderSettings.SkipTransparentObjects -AdaptiveDecals 0.01 1 20 Medium GPU AdaptivePerformanceRenderSettings.DecalsDrawDistance - -Add the example adapter script from resources/AdaptiveQualityAdapter.cs and place it in Assets/AdaptivePerformanceManager/. Add code comments above any lines for scalers that cause high visual impact (but leave the code intact). - -## Step 5: Add bootstrap GameObject -- Determine what scene the "AdaptivePerformanceSignalManager" bootstrap GameObject will be added to -- Add the "AdaptivePerformanceSignalManager" bootstrap GameObject to the scene -- Add the AdaptivePerformanceSignalManager and AdaptiveQualityAdapter components to the AdaptivePerformanceSignalManager GameObject and configure them -- Save the scene - -## Step 6: Produce a final checklist -- Include a checklist of what was done to the project and what steps the user should do on their own. -- Ask the user to go to Project Settings -> Adaptive Performance and check the "Enable Adaptive Performance" checkbox if it is not checked. -- Ask the user to check the "Android Provider" box in the Providers section in the Adaptive Performance settings. -- If new Quality settings were added, encourage the user to adjust the settings for each one and to check and adjust the URP Render Pipeline assets associated with each. - - The URP Render Pipeline asset for mobile_adaptive should have Enable Adaptive Performance turned on so ask the user to check. - - The URP Render Pipeline asset for mobile_max should have Enable Adaptive Performance turned off on so ask the user to check. -- Give a brief explanation of how "AdaptivePerformanceSignalManager.cs" is the main script they would edit to expand or change what was done. -- Let the user know they can also implement AdaptiveLayerCulling via Camera.main.layerCullDistances but it takes detailed. setup from the user -- Another thing that isn't really controlled in any of the settings we've talked about above is the postprocessing stack, the VolumeProfiles defined in Volumes in scene. Tell them this is something they can implement on their own for even better adaptive performance. \ No newline at end of file diff --git a/skills/android-add-adaptive-performance/resources/AdaptivePerformanceSignalManager.cs b/skills/android-add-adaptive-performance/resources/AdaptivePerformanceSignalManager.cs deleted file mode 100644 index ab5074f..0000000 --- a/skills/android-add-adaptive-performance/resources/AdaptivePerformanceSignalManager.cs +++ /dev/null @@ -1,269 +0,0 @@ -using System; -using UnityEngine; -using UnityEngine.AdaptivePerformance; - -namespace AdaptivePerformanceIntegration -{ - /// - /// Game-agnostic Adaptive Performance signal handler. - /// Reads thermal/performance state via the official API and reports the effective state index. - /// - /// This MonoBehaviour polls the Adaptive Performance subsystem at a configurable interval, - /// evaluates the current thermal warning level and temperature, and maps them to a simplified - /// integer state index (0–3). Downstream systems (e.g., quality adapters) subscribe to - /// to react to thermal changes without depending - /// on the Adaptive Performance API directly. - /// - /// Effective States: - /// 0: Normal (Cool) – No thermal concerns. - /// 1: Elevated (Pre-warning) – Temperature has risen past the preemptive threshold - /// but no official warning has been issued yet. - /// 2: Throttling Imminent – The subsystem reports . - /// 3: Throttling – The subsystem reports ; - /// the device is actively reducing clock speeds. - /// - public class AdaptivePerformanceSignalManager : MonoBehaviour - { - /// - /// Serializable policy block that controls how and when thermal state is evaluated. - /// Exposed in the Inspector so designers can tune thresholds per-project. - /// - [Serializable] - public class StatePolicy - { - /// - /// Normalized temperature level (0–1) above which the manager reports state 1 - /// ("Elevated") even when no official thermal warning has been raised. This lets - /// the game begin reducing load before the hardware signals a problem. - /// - [Tooltip("Temperature level (0-1) above which we report 'Elevated' (1) even if there is no official thermal warning.")] - [Range(0f, 1f)] - public float PreemptiveTemperatureThreshold = 0.5f; - - /// - /// How often (in seconds) the manager re-evaluates thermal state in Update(). - /// Lower values give faster reactions but cost more CPU. - /// - [Header("Polling interval (seconds)")] - [Min(0.1f)] - public float PollIntervalSeconds = 1.0f; - - /// - /// Master switch. When false, polling, evaluation, and event firing are all skipped. - /// - [Header("Enable/disable adaptation")] - public bool Enabled = true; - } - - /// - /// Inspector-exposed policy controlling thresholds, polling rate, and the enable flag. - /// - [Header("Policy")] - public StatePolicy Policy = new StatePolicy(); - - /// - /// When true, state transitions and subsystem acquisition are logged to the console - /// via . - /// - [Header("Debug")] - public bool VerboseLogging = true; - - /// - /// Fired whenever the effective hardware state index (0–3) changes. - /// Subscribers receive the new state index. - /// - public event Action HardwareStateChangedEvent; - - /// - /// Optional debug/telemetry event that carries the raw warning-level name and - /// normalized temperature (0–1) each time the state transitions. - /// - public event Action AdaptiveStateChangedEvent; - - /// - /// The most recently computed effective hardware state index (0–3). - /// Initialized to -1 to guarantee the first evaluation always triggers events. - /// - public int CurrentStateIndex { get; private set; } = -1; - - /// Cached reference to the Adaptive Performance subsystem instance. - private IAdaptivePerformance _ap; - - /// Unscaled timestamp of the next scheduled poll. - private float _nextPollTime; - - /// - /// Guards against double-subscribing to the thermal event if - /// is called more than once. - /// - private bool _subscribedToThermalEvent; - - /// - /// Marks this GameObject as persistent across scene loads so thermal monitoring - /// is never interrupted by scene transitions. - /// - private void Awake() - { - DontDestroyOnLoad(gameObject); - } - - /// - /// Attempts to acquire the Adaptive Performance subsystem on the first frame - /// and schedules the initial poll. - /// - private void Start() - { - if (!Policy.Enabled) return; - - TryAcquireInstance(); - _nextPollTime = Time.unscaledTime + Policy.PollIntervalSeconds; - } - - /// - /// Cleans up the thermal event subscription when this component is destroyed. - /// - private void OnDestroy() - { - UnsubscribeThermalEvent(); - } - - /// - /// Per-frame update. If the subsystem has not been acquired yet, retries at the - /// configured poll interval. Once acquired, evaluates thermal state on each poll tick. - /// Uses so polling is unaffected by time scale. - /// - private void Update() - { - if (!Policy.Enabled) return; - - // Subsystem not yet available – retry acquisition on the next poll tick. - if (_ap == null || !_ap.Active) - { - if (Time.unscaledTime >= _nextPollTime) - { - _nextPollTime = Time.unscaledTime + Policy.PollIntervalSeconds; - TryAcquireInstance(); - } - return; - } - - // Wait until the next scheduled poll. - if (Time.unscaledTime < _nextPollTime) return; - _nextPollTime = Time.unscaledTime + Policy.PollIntervalSeconds; - - EvaluateAndReport(); - } - - /// - /// Fetches the singleton instance from - /// . If successful, subscribes to thermal events - /// and performs an immediate evaluation so there is no gap until the first poll. - /// - private void TryAcquireInstance() - { - _ap = Holder.Instance; - if (_ap == null || !_ap.Active) return; - - Log($"Adaptive Performance acquired. Active={_ap.Active}"); - SubscribeThermalEvent(); - EvaluateAndReport(); - } - - /// - /// Subscribes to the subsystem's push-based - /// so the manager can react immediately when the device raises or clears a warning, - /// rather than waiting for the next poll tick. - /// - private void SubscribeThermalEvent() - { - if (_subscribedToThermalEvent || _ap?.ThermalStatus == null) return; - _ap.ThermalStatus.ThermalEvent += OnThermalEvent; - _subscribedToThermalEvent = true; - } - - /// - /// Removes the thermal event subscription, preventing callbacks after this - /// component has been destroyed or disabled. - /// - private void UnsubscribeThermalEvent() - { - if (!_subscribedToThermalEvent || _ap?.ThermalStatus == null) return; - _ap.ThermalStatus.ThermalEvent -= OnThermalEvent; - _subscribedToThermalEvent = false; - } - - /// - /// Callback invoked by the Adaptive Performance subsystem when thermal conditions - /// change. Delegates to . - /// - /// Latest thermal metrics snapshot from the subsystem. - private void OnThermalEvent(ThermalMetrics metrics) - { - if (!Policy.Enabled) return; - EvaluateAndReport(metrics); - } - - /// - /// Convenience overload that pulls the latest - /// from the cached subsystem reference and forwards to the evaluation logic. - /// - private void EvaluateAndReport() - { - if (_ap?.ThermalStatus == null) return; - EvaluateAndReport(_ap.ThermalStatus.ThermalMetrics); - } - - /// - /// Core evaluation logic. Maps the subsystem's and - /// normalized temperature to an integer state index (0–3). - /// - /// State mapping: - /// → 3 - /// → 2 - /// with temp ≥ threshold → 1 - /// Otherwise → 0 - /// - /// If the computed index differs from , both - /// and - /// are fired. - /// - /// Thermal metrics snapshot to evaluate. - private void EvaluateAndReport(ThermalMetrics metrics) - { - int stateIndex = 0; - - switch (metrics.WarningLevel) - { - case WarningLevel.Throttling: - stateIndex = 3; - break; - case WarningLevel.ThrottlingImminent: - stateIndex = 2; - break; - case WarningLevel.NoWarning: - default: - if (metrics.TemperatureLevel >= Policy.PreemptiveTemperatureThreshold) - stateIndex = 1; - break; - } - - if (stateIndex != CurrentStateIndex) - { - CurrentStateIndex = stateIndex; - Log($"Hardware State changed: {stateIndex} (warning={metrics.WarningLevel}, temp={metrics.TemperatureLevel:F2})"); - HardwareStateChangedEvent?.Invoke(CurrentStateIndex); - AdaptiveStateChangedEvent?.Invoke(metrics.WarningLevel.ToString(), metrics.TemperatureLevel); - } - } - - /// - /// Writes a timestamped debug message to the Unity console when - /// is enabled. - /// - /// Message to log, automatically prefixed with "[AdaptivePerformance]". - private void Log(string msg) - { - if (VerboseLogging) Debug.Log($"[AdaptivePerformance] {msg}"); - } - } -} diff --git a/skills/android-add-adaptive-performance/resources/AdaptiveQualityAdapter.cs b/skills/android-add-adaptive-performance/resources/AdaptiveQualityAdapter.cs deleted file mode 100644 index 0b84a9c..0000000 --- a/skills/android-add-adaptive-performance/resources/AdaptiveQualityAdapter.cs +++ /dev/null @@ -1,297 +0,0 @@ -using UnityEngine; -using UnityEngine.Rendering; -using UnityEngine.AdaptivePerformance; - -namespace AdaptivePerformanceIntegration -{ - /// - /// Dynamically adjusts visual quality settings in response to hardware thermal/performance - /// signals relayed by . - /// - /// The adapter maps discrete hardware states (Normal, Elevated, Throttling Imminent, Throttling) - /// to designer-defined quality tiers, then applies both general Unity quality knobs and - /// (optionally) URP-specific scalers for each tier. - /// - /// An optional "sticky downgrade" policy ensures that once the device has been stressed, - /// quality never climbs back up during the session—useful for preventing oscillation on - /// devices that hover near a thermal boundary. - /// - public class AdaptiveQualityAdapter : MonoBehaviour - { - /// - /// Defines the full set of rendering parameters for a single quality tier. - /// - [System.Serializable] - public struct QualityTierSettings - { - public string Name; - - // --- General Scalers --- - - [Header("General Scalers")] - - /// Multiplier layered on top of the base render scale. - public float RenderScaleMultiplier; - - /// Divisor applied to the device refresh rate to determine target FPS (e.g. 1, 2, 4). - public int FrameRateDivisor; - - /// LOD bias multiplier—lower values force lower-detail meshes earlier. - public float LODBias; - - /// Physics step interval; larger values reduce physics CPU cost at the expense of accuracy. - public float FixedDeltaTime; - - /// Main camera far clip plane; reducing it culls distant geometry. - public float FarClipPlane; - - // --- URP Adaptive Performance Scalers (Note: Only applies if using URP) --- - - [Header("URP Scalers (Inactive in Built-in)")] - - /// Multiplier for the maximum shadow draw distance inside URP. - public float MaxShadowDistanceMultiplier; - - /// Multiplier for the main directional light's shadow map resolution. - public float MainLightShadowmapResolutionMultiplier; - - /// Bias added to the shadow cascade count (negative values reduce cascades). - public int ShadowCascadesBias; - - /// Bias added to the anti-aliasing quality level (negative values lower AA quality). - public int AAQualityBias; - - /// Bias added to the shadow quality level (negative values lower shadow fidelity). - public int ShadowQualityBias; - - /// When true, dynamic batching is skipped to save CPU overhead. - public bool SkipDynamicBatching; - - /// When true, transparent objects are not rendered—a significant GPU savings. - public bool SkipTransparentObjects; - - /// When true, front-to-back sorting is skipped to reduce CPU sort time. - public bool SkipFrontToBackSorting; - - /// Draw distance multiplier for decal projectors. - public float DecalsDrawDistance; - - /// Bias for the color grading LUT resolution (lower = faster, less accurate color). - public float LutBias; - } - - [Header("Components")] - /// - /// Reference to the signal manager that translates raw Adaptive Performance data - /// into actionable hardware-state events this adapter subscribes to. - /// - public AdaptivePerformanceSignalManager SignalManager; - - [Header("Settings")] - /// - /// When enabled, the adapter only ever moves to a *lower* quality tier during the - /// session—it will never return to a higher one even if thermal conditions improve. - /// This prevents visual "popping" on devices that repeatedly cross a thermal boundary. - /// - [Tooltip("If true, once the game quality drops, it never returns to a higher tier.")] - public bool StickyDowngrade = true; - - /// - /// Ordered array of quality tiers from highest fidelity (index 0) to lowest. - /// - public QualityTierSettings[] Tiers = new QualityTierSettings[] - { - new QualityTierSettings - { - Name = "Best", - RenderScaleMultiplier = 1.0f, - FrameRateDivisor = 1, - LODBias = 2.0f, - FixedDeltaTime = 0.0166f, - FarClipPlane = 1000f, - MaxShadowDistanceMultiplier = 1.0f, - MainLightShadowmapResolutionMultiplier = 1.0f, - ShadowCascadesBias = 0, - AAQualityBias = 0, - ShadowQualityBias = 0, - SkipDynamicBatching = false, - SkipTransparentObjects = false, - SkipFrontToBackSorting = false, - DecalsDrawDistance = 1.0f, - LutBias = 1.0f - }, - new QualityTierSettings - { - Name = "Medium", - RenderScaleMultiplier = 0.85f, - FrameRateDivisor = 2, - LODBias = 1.5f, - FixedDeltaTime = 0.02f, - FarClipPlane = 800f, - MaxShadowDistanceMultiplier = 0.75f, - MainLightShadowmapResolutionMultiplier = 0.75f, - ShadowCascadesBias = -1, - AAQualityBias = -1, - ShadowQualityBias = -1, - SkipDynamicBatching = false, - SkipTransparentObjects = false, - SkipFrontToBackSorting = false, - DecalsDrawDistance = 0.75f, - LutBias = 0.75f - }, - new QualityTierSettings - { - Name = "Low", - RenderScaleMultiplier = 0.7f, - FrameRateDivisor = 4, - LODBias = 1.0f, - FixedDeltaTime = 0.0333f, - FarClipPlane = 500f, - MaxShadowDistanceMultiplier = 0.5f, - MainLightShadowmapResolutionMultiplier = 0.5f, - ShadowCascadesBias = -2, - AAQualityBias = -2, - ShadowQualityBias = -2, - SkipDynamicBatching = true, - SkipTransparentObjects = true, - SkipFrontToBackSorting = true, - DecalsDrawDistance = 0.5f, - LutBias = 0.5f - } - }; - - [Header("Hardware State Mapping")] - public HardwareStateMapping[] StateMappings = new HardwareStateMapping[] - { - new HardwareStateMapping { StateLabel = "Normal", TargetTierName = "Best" }, - new HardwareStateMapping { StateLabel = "Elevated", TargetTierName = "Medium" }, - new HardwareStateMapping { StateLabel = "Throttling Imminent", TargetTierName = "Low" }, - new HardwareStateMapping { StateLabel = "Throttling", TargetTierName = "Low" } - }; - - [System.Serializable] - public struct HardwareStateMapping - { - public string StateLabel; - public string TargetTierName; - } - - [Header("Runtime Info (Read Only)")] - public int MaxTierIndexReached = 0; - public int CurrentTierIndex = 0; - public int AppliedTierIndex = 0; - - private bool _urpEnabled = false; - - private void OnEnable() - { - if (SignalManager != null) - SignalManager.HardwareStateChangedEvent += OnHardwareStateChanged; - } - - private void OnDisable() - { - if (SignalManager != null) - SignalManager.HardwareStateChangedEvent -= OnHardwareStateChanged; - } - - private void Start() - { - // Detect URP without direct reference to its types to avoid compilation errors in Built-in projects - _urpEnabled = GraphicsSettings.currentRenderPipeline != null && - GraphicsSettings.currentRenderPipeline.GetType().Name.Contains("Universal"); - - Debug.Log("[AdaptiveQualityAdapter] Initialized and monitoring hardware signals."); - } - - private void OnHardwareStateChanged(int stateIndex) - { - int targetTierIndex = ResolveStateToTierIndex(stateIndex); - CurrentTierIndex = targetTierIndex; - - if (StickyDowngrade) - { - if (targetTierIndex > MaxTierIndexReached) - { - MaxTierIndexReached = targetTierIndex; - ApplyQuality(targetTierIndex); - AppliedTierIndex = targetTierIndex; - } - else - { - Debug.Log($"[AdaptiveQualityAdapter] Sticky policy: Ignoring return to tier index {targetTierIndex}. Current Max Tier index is {MaxTierIndexReached}."); - } - } - else - { - ApplyQuality(targetTierIndex); - AppliedTierIndex = targetTierIndex; - } - } - - private int ResolveStateToTierIndex(int stateIndex) - { - string targetName = (StateMappings != null && stateIndex < StateMappings.Length) - ? StateMappings[stateIndex].TargetTierName - : string.Empty; - - for (int i = 0; i < Tiers.Length; i++) - { - if (Tiers[i].Name == targetName) return i; - } - - return Mathf.Max(0, Tiers.Length - 1); - } - - private void ApplyQuality(int tierIndex) - { - if (tierIndex < 0 || tierIndex >= Tiers.Length) return; - - QualityTierSettings settings = Tiers[tierIndex]; - Debug.Log($"[AdaptiveQualityAdapter] Applied Quality Tier {tierIndex} ({settings.Name})"); - - // --- General Scalers --- - Application.targetFrameRate = CalculateTargetFrameRate(settings.FrameRateDivisor); - QualitySettings.lodBias = settings.LODBias; - Time.fixedDeltaTime = settings.FixedDeltaTime; - - if (Camera.main != null) - Camera.main.farClipPlane = settings.FarClipPlane; - - // AdaptivePerformanceRenderSettings applies generally if the package is present - AdaptivePerformanceRenderSettings.RenderScaleMultiplier = settings.RenderScaleMultiplier; - - if (!_urpEnabled) return; - - // --- URP Specific Scalers (Only if URP is active) --- - AdaptivePerformanceRenderSettings.MaxShadowDistanceMultiplier = settings.MaxShadowDistanceMultiplier; - AdaptivePerformanceRenderSettings.MainLightShadowmapResolutionMultiplier = settings.MainLightShadowmapResolutionMultiplier; - AdaptivePerformanceRenderSettings.MainLightShadowCascadesCountBias = settings.ShadowCascadesBias; - AdaptivePerformanceRenderSettings.AntiAliasingQualityBias = settings.AAQualityBias; - AdaptivePerformanceRenderSettings.ShadowQualityBias = settings.ShadowQualityBias; - AdaptivePerformanceRenderSettings.SkipDynamicBatching = settings.SkipDynamicBatching; - AdaptivePerformanceRenderSettings.SkipTransparentObjects = settings.SkipTransparentObjects; - AdaptivePerformanceRenderSettings.SkipFrontToBackSorting = settings.SkipFrontToBackSorting; - AdaptivePerformanceRenderSettings.DecalsDrawDistance = settings.DecalsDrawDistance; - AdaptivePerformanceRenderSettings.LutBias = settings.LutBias; - } - - private int CalculateTargetFrameRate(int divisor) - { - // Unity 6+ uses RefreshRateRatio for precise display frequencies - double refreshRate = Screen.currentResolution.refreshRateRatio.value; - - // On some platforms or when not yet available, fallback to a sensible default - if (refreshRate <= 0) refreshRate = 60.0; - - // Enforce "progressive halving" (1, 2, 4, 8...) by snapping the divisor to the next power of two. - // This ensures target FPS aligns with display VSync intervals for smooth pacing. - int effectiveDivisor = Mathf.NextPowerOfTwo(Mathf.Max(1, divisor)); - - int target = Mathf.RoundToInt((float)(refreshRate / effectiveDivisor)); - - // Requirement: Frame rate can't go below 30. - return Mathf.Max(30, target); - } - } -} From e84cc257c7afd301103901ffa9825a90d72c219e Mon Sep 17 00:00:00 2001 From: Ziyi Zhang Date: Wed, 12 Aug 2026 15:37:32 -0400 Subject: [PATCH 6/7] feat: add the optimize-web skill --- README.md | 1 + skills/optimize-web/SKILL.md | 393 ++++++++++++++++++ skills/optimize-web/resources/WebOptimizer.cs | 21 + .../optimize-web/resources/toktx-examples.sh | 11 + 4 files changed, 426 insertions(+) create mode 100644 skills/optimize-web/SKILL.md create mode 100644 skills/optimize-web/resources/WebOptimizer.cs create mode 100644 skills/optimize-web/resources/toktx-examples.sh diff --git a/README.md b/README.md index d6f36c4..775ec17 100644 --- a/README.md +++ b/README.md @@ -23,6 +23,7 @@ npx skills add Unity-Technologies/skills | `ui-ugui` | uGUI — Canvas hierarchies, RectTransform anchoring, Layout Groups, prefab UI | | `ui-imgui` | IMGUI editor tooling — EditorWindows, custom Inspectors, PropertyDrawers | | `optimize-audio` | Reduce audio memory and DSP CPU cost through import settings, load types, and mixer topology | +| `optimize-web` | Shrink and speed up WebGL/WebGPU builds — compression, stripping, memory, frame rate, KTX textures | | `setup-game-inputs` | Input System — action maps, bindings, control schemes, rebinding | ## Usage diff --git a/skills/optimize-web/SKILL.md b/skills/optimize-web/SKILL.md new file mode 100644 index 0000000..ccfe96e --- /dev/null +++ b/skills/optimize-web/SKILL.md @@ -0,0 +1,393 @@ +--- +name: optimize-web +description: Optimizes Unity 6 WebGL and WebGPU builds for smaller download size, faster initial load, and efficient browser runtime performance. Use when the user's web build is too large, stutters in a specific browser, consumes excessive battery, needs CDN/server compression configured, or needs guidance on resource stripping, shader variant reduction, KTX textures, quality settings, or web profiling. +--- +## Performance Notes +- Take your time to do this thoroughly. +- Quality is more important than speed. + +## Running C# in the Editor + +Every step below that reads or writes a Player Setting runs inside a live Editor through the Unity +CLI. **The `unity-cli` skill owns getting you there** — installing the CLI, confirming a connected +Editor, adding the project's `com.unity.pipeline` package, telling a genuinely absent Editor apart +from one stuck in Safe Mode, and discovering the Editor's command catalog. Follow it first; don't +re-derive any of it here. + +Two things it can't know for you: + +- **You need `eval` in particular**, not just a reachable Editor. Confirm it appears in the catalog. + Its presence depends on the Pipeline package version, not on the CLI, so a healthy install can + still lack it — if it's missing, say so and stop. +- **Player Settings can be read from `ProjectSettings/ProjectSettings.asset` in a pinch, but do not + write them that way.** The serialized names don't match the API names, several of these settings + are per-build-target, and a hand-edited value silently disagrees with what the build actually + uses. An unreachable Editor is a stop for the write steps. + +Run C# with `unity command eval --code ''`. `unity command` defaults to a 30 second +timeout. + +### Passing C# to `eval` + +`eval` compiles a **statement block, not a file**. Two consequences, both compile errors rather than +warnings: + +- **No `using` directives.** The compiler reads `using UnityEditor;` as a resource-disposal + statement and rejects it (`CS0210`). +- **Types must be fully qualified.** A bare `PlayerSettings` does not resolve (`CS0246`), and a bare + `Object` is ambiguous with `object` (`CS0104`). + +### Reading the settings this skill audits + +One call returns the whole Pre-Flight picture. Verified against Unity 6000.5.7f1: + +```csharp +var target = UnityEditor.Build.NamedBuildTarget.WebGL; +var w = new System.Collections.Generic.List(); +w.Add($"activeBuildTarget={UnityEditor.EditorUserBuildSettings.activeBuildTarget}"); +w.Add($"compressionFormat={UnityEditor.PlayerSettings.WebGL.compressionFormat}"); +w.Add($"decompressionFallback={UnityEditor.PlayerSettings.WebGL.decompressionFallback}"); +w.Add($"stripEngineCode={UnityEditor.PlayerSettings.stripEngineCode}"); +w.Add($"managedStrippingLevel={UnityEditor.PlayerSettings.GetManagedStrippingLevel(target)}"); +w.Add($"il2cppCodeGeneration={UnityEditor.PlayerSettings.GetIl2CppCodeGeneration(target)}"); +w.Add($"apiCompatibilityLevel={UnityEditor.PlayerSettings.GetApiCompatibilityLevel(target)}"); +w.Add($"exceptionSupport={UnityEditor.PlayerSettings.WebGL.exceptionSupport}"); +w.Add($"debugSymbolMode={UnityEditor.PlayerSettings.WebGL.debugSymbolMode}"); +w.Add($"dataCaching={UnityEditor.PlayerSettings.WebGL.dataCaching}"); +w.Add($"wasm2023={UnityEditor.PlayerSettings.WebGL.wasm2023}"); +w.Add($"initialMemorySize={UnityEditor.PlayerSettings.WebGL.initialMemorySize}"); +w.Add($"maximumMemorySize={UnityEditor.PlayerSettings.WebGL.maximumMemorySize}"); +w.Add($"memoryGrowthMode={UnityEditor.PlayerSettings.WebGL.memoryGrowthMode}"); +w.Add($"targetFrameRate={UnityEngine.Application.targetFrameRate}"); +w.Add($"vSyncCount={UnityEngine.QualitySettings.vSyncCount}"); +return string.Join("\n", w); +``` + +**Three API names to get right**, because the obvious spellings do not exist and fail to compile: + +| Setting | Correct form | Does NOT exist | +|---|---|---| +| Managed stripping level | `PlayerSettings.GetManagedStrippingLevel(NamedBuildTarget.WebGL)` | `PlayerSettings.managedStrippingLevel` | +| Wasm code optimization | `UnityEditor.WebGL.UserBuildSettings.codeOptimization` | `PlayerSettings.WebGL.codeOptimization`, `PlayerSettings.WebGL.optimizationLevel` | +| IL2CPP code generation | `PlayerSettings.GetIl2CppCodeGeneration(NamedBuildTarget.WebGL)` | a bare property | + +`UserBuildSettings` lives in the WebGL build-support module, so it only resolves when that module is +installed. Read it in a separate call from the rest, and treat a resolution failure as "the Web +module isn't installed" rather than as a bad snippet. + +### Applying the settings + +Most of the writes in this skill are a single batch, and +[resources/WebOptimizer.cs](resources/WebOptimizer.cs) already is that batch. It declares a class +with a `[MenuItem]`, so it is a **project file, not `eval` input** — a class declaration cannot be +flattened into a statement block. Save it under `Assets/Editor/`, let Unity compile, then invoke it +in one line: + +```csharp +UnityEditor.EditorApplication.ExecuteMenuItem("Tools/Apply Web Release Settings"); +``` + +Keep its `using` directives; they are correct in a file. For one-off changes — a single quality +level, a frame-rate flip — an inline `eval` statement is fine. + +## 0. Pre-Flight + +1. **Confirm Web build target:** Read, with the Pre-Flight snippet above, `EditorUserBuildSettings.activeBuildTarget` — must be `WebGL`; if not, warn the user. +2. **Read compression and stripping settings:** Read `compressionFormat`, `decompressionFallback`, `stripEngineCode` and the managed stripping level with the Pre-Flight snippet above. Note the stripping level is `PlayerSettings.GetManagedStrippingLevel(NamedBuildTarget.WebGL)` — there is no `PlayerSettings.managedStrippingLevel` property. +3. **Read exception and optimization settings:** Read `PlayerSettings.WebGL.exceptionSupport` from the Pre-Flight snippet above. For the wasm code optimization level use `UnityEditor.WebGL.UserBuildSettings.codeOptimization` — the `PlayerSettings.WebGL.codeOptimization` and `optimizationLevel` spellings do not exist and will not compile. +4. **Read frame rate settings:** Read, with the Pre-Flight snippet above, `Application.targetFrameRate` and `QualitySettings.vSyncCount`. +5. **Read additional player settings:** Read, with the Pre-Flight snippet above, `PlayerSettings.WebGL.dataCaching`, `PlayerSettings.WebGL.debugSymbolMode`, `PlayerSettings.WebGL.maximumMemorySize`, and `PlayerSettings.GetApiCompatibilityLevel`. +6. Proceed only after compression, stripping, frame rate, and player settings are confirmed. + +## 1. Assess Current State + +1. **Check Build Report:** Instruct the user to open `Window > General > Build Report` after a build and identify the largest asset and code size contributors. +2. **Verify server configuration:** Ask the user to confirm whether the hosting server sends `Content-Encoding: br` (Brotli) or `Content-Encoding: gzip` headers, and whether `Content-Type: application/wasm` is set for `.wasm` files. +3. **Check frame rate config:** Confirm, with the Pre-Flight snippet above, `Application.targetFrameRate` — should be `-1` for Web (let the browser drive). +4. **Check memory settings:** Read, with the Pre-Flight snippet above, `PlayerSettings.WebGL.initialMemorySize` and `PlayerSettings.WebGL.memoryGrowthMode`. +5. Report findings before making recommendations. + +## 2. Understand Request + +| User Says | Default Interpretation | +|-----------|----------------------| +| "build too large" / "download too slow" | Strip Engine Code on; Managed Stripping High; Disk Size + LTO; Brotli | +| "Decompression Fallback" / "slow startup" | Decompression Fallback off; fix server to send Content-Encoding | +| "stutter in Chrome" / "stutter in Safari" | Profile in browser DevTools; Safari caps at 60 fps | +| "excessive battery in browser" | `OnDemandRendering` on static screens; `targetFrameRate = -1` | +| "exceptions too large" | None for release; Wasm 2023 exceptions if browser baseline allows | +| "set up CDN" | Addressables remote groups + Brotli/Gzip on CDN | +| "WebAssembly 2023" | Enable when browser baseline supports it — smaller and faster | +| "memory growth slow" | Tune Initial Memory Size to peak estimate; use Geometric growth mode | +| "KTX" / "Basis Universal" / "texture formats unknown GPU" | KTX2 with Basis Universal; ETC1S for size, UASTC for quality | +| "strip unused code" / "remove unused packages" | Web Stripping Tool + remove unused packages + shader stripping | +| "quality settings for web" | Quality Level to Very Low or Low; lower quality = faster load | +| "shader variants too many" | Graphics settings: auto lightmap/fog modes; strip instancing + BRG variants; audit Always Included Shaders | +| "video not playing" / "audio issues" | Video: URL-only or StreamingAssets; Audio: no AudioEffects on Web, use Mono, compress | +| "profiler symbols" / "can't read Wasm stacks" | Embed profiling symbols via build processor or emscriptenArgs | +| "iOS crashes" / "Safari memory" | iOS memory limits; set Initial Memory Size high rather than growing; Gigacage 2GB limit pre-iOS 18 | + +## 3. Web Build Optimization Workflow + +### IMPORTANT: One-click optimization script + +**Always offer to generate this script for the user.** Unity's official web optimization docs provide a single editor menu script that applies all recommended release settings at once. Place in `Assets/Editor/WebOptimizer.cs` — see [resources/WebOptimizer.cs](resources/WebOptimizer.cs) for the template. + +Adapt the script to the user's project needs (e.g. keep exceptions if they use `try/catch`, switch Brotli to Gzip for HTTP hosting). This script is the single most impactful action for a new web project — it prevents settings from being missed. + +### Player Settings audit + +Verify and set these values through `eval`: + +| Setting | Release recommendation | +|---|---| +| **Compression Format** | **Brotli** (HTTPS hosting); Gzip for HTTP | +| **Decompression Fallback** | **Off** when server is correctly configured | +| **Strip Engine Code** | **On** | +| **Managed Stripping Level** | **High** (release) / Medium (dev) | +| **Code Optimization** | **Disk Size with LTO** (release) / Build Times (dev) | +| **WebAssembly Language Features** | **2023** if browser baseline allows | +| **Enable Exceptions** | **None** (smallest); Explicitly Thrown Only if `try/catch` required | +| **Initial Memory Size** | Tune to peak estimate; too small causes expensive growth | +| **Memory Growth Mode** | **Geometric** | +| **API Compatibility Level** | **.NET Standard 2.1** — smaller than .NET Framework | +| **IL2CPP Code Generation** | **Optimize Size** — smaller Wasm at slight runtime cost | +| **Debug Symbols** | **Off** for release; on for development builds only | +| **Data Caching** | **On** — caches asset data in browser IndexedDB for faster repeat loads | +| **Strip Unused Mesh Components** | **On** — removes unused vertex attributes | +| **Maximum Memory Size** | **2048 MB** default; up to 4096 for complex 3D (Firefox and Chrome < 119 have issues above 2048) | +| **vSyncCount** | 0 (browser handles pacing) | +| **targetFrameRate** | -1 (use `requestAnimationFrame`) | + +### Compression and server configuration + +| Compression | Use when | Notes | +|---|---|---| +| **Brotli** | HTTPS or localhost | Best ratio; browsers accept only over secure contexts | +| **Gzip** | HTTP delivery, legacy CDNs | Universal | +| **None** | Local dev / file:// | Largest payload; do not ship | + +Configure the server to: +- Serve `.br` files with `Content-Encoding: br`. +- Serve `.gz` files with `Content-Encoding: gzip`. +- Set `Content-Type: application/wasm` for `.wasm`, `application/javascript` for `.js`. +- Enable HTTP/2 or HTTP/3 to parallelize chunk fetches. + +If the host cannot inject `Content-Encoding`: set **Decompression Fallback = On** as a fallback only — it adds ~150 KB JS and slows startup. + +### Exception handling + +| Setting | Build size | Use | +|---|---|---| +| **None** | Smallest | Release builds where uncaught exceptions are acceptable | +| **Explicitly Thrown Only** | Modest | Default for projects that catch exceptions | +| **Full** | Largest, slowest | Rarely needed; avoid for release | + +Wasm 2023 introduces a cheaper exception model; switching from Explicitly Thrown Only (legacy) to Wasm exceptions reduces both size and cost when browser targets support it. + +### Remove unused resources + +Three categories to audit for build size reduction: + +**1. Unused packages** — Check `Packages/manifest.json` and the Package Manager **In Project** and **Built-in** views. Remove or disable packages the project does not use. The Input System package is a significant size contributor if unused. + +**2. Shader stripping** — Configure in `Edit > Project Settings > Graphics`: + +| Setting | Recommendation | +|---|---| +| **Lightmap Modes** | Automatic (strips unused lightmap shader variants) | +| **Fog Modes** | Automatic (strips unused fog shader variants) | +| **Instancing Variants** | Strip Unused | +| **Batch Renderer Group Variants** | Strip All (if BRGs are not used) | +| **Always Included Shaders** | Audit and remove any shaders the project does not reference | + +Test after stripping — ensure no referenced shaders were removed. + +**3. Web Stripping Tool** (`com.unity.web.stripping-tool`) — Analyzes the WebAssembly binary and identifies unused Unity engine submodules (e.g. 3D graphics in a 2D-only game). Install via Package Manager, profile the build, then configure which submodules to exclude. Can yield substantial size reductions beyond what Managed Stripping Level achieves alone. + +### Quality settings for Web + +Lower quality levels reduce load time and improve runtime performance. Set via `Edit > Project Settings > Quality`: + +- Use **Very Low** or **Low** as the default Web quality level. +- Set it with `eval`: `UnityEngine.QualitySettings.SetQualityLevel(0, true);` where 0 = Very Low. +- Consider creating a Web-specific quality level that disables features unnecessary in-browser (real-time shadows, post-processing effects, high particle counts). + +### Frame rate on Web + +- Set it with `eval`: `UnityEngine.Application.targetFrameRate = -1;` — let the browser use `requestAnimationFrame`. +- Note: **Safari caps at 60 fps** in WebGL; high-refresh targets do not apply. +- Use `OnDemandRendering.renderFrameInterval` to drop to 5–10 fps on static/idle screens to save battery. + +### KTX / Basis Universal textures + +KTX2 with Basis Universal supercompression ships a single texture file that transcodes at load time to the optimal GPU format for the browser's device (BC7 on desktop, ASTC on mobile, ETC2 on older Android). This avoids shipping separate texture variants for each GPU family — critical for Web where the target hardware is unknown. + +| Topic | Guidance | +|---|---| +| **Package** | Install `com.unity.cloud.ktx` (KtxUnity) via Package Manager | +| **When to use** | Runtime-loaded textures via Addressables or asset bundles served to unknown GPU targets | +| **When NOT to use** | Textures baked into the player build — Unity already selects the correct format at build time | +| **Supercompression** | Use **ETC1S** for smallest size (lossy, good for diffuse/albedo); **UASTC** for higher quality (near-lossless, better for normals/UI) | +| **Encoding** | Encode offline with `toktx` or `basisu` CLI; do not encode at runtime | +| **Linear data** | Set `--assign_oetf linear` when encoding normal maps, masks, or data textures to avoid incorrect sRGB conversion | +| **Mip maps** | Generate mips at encode time (`--genmipmap`) — browser-side mip generation is expensive | +| **Loading** | Use `KtxTexture.LoadFromStreamingAssets` or load bytes via UnityWebRequest and call `KtxTexture.LoadFromBytes` | +| **Memory** | Transcoded textures are standard GPU textures; memory cost equals the target format, not the KTX2 file size | +| **Orientation** | Always include `--lower_left_maps_to_s0t0` to match Unity's UV convention | + +**`toktx` CLI examples:** See [resources/toktx-examples.sh](resources/toktx-examples.sh) for commands covering albedo (ETC1S), normals/detail (UASTC), ICC profile errors, and linear data. + +### Streaming on Web + +- Use Addressables with **remote groups** hosted on a CDN with Brotli / Gzip. +- Avoid bundling the entire game into the initial download; stream levels on demand. +- Target < 30 MB initial download for "instant play"; level data follows. +- For streamed textures targeting mixed GPU hardware, prefer KTX2 bundles over per-platform variants — one bundle serves all browsers. + +### Profiling Web builds + +| Tool | Use | Notes | +|---|---|---| +| **Chrome DevTools > Performance** | CPU flamegraph; main-thread analysis | Default first stop for WebGL hitches; inspect Wasm call stacks | +| **Chrome DevTools > Memory** | Heap snapshot; allocation timeline | Find JS/Wasm memory leaks; compare snapshots before/after scene load | +| **Firefox Profiler** | Cross-platform; shareable URLs; native + Wasm view | Better Wasm symbolication than Chrome in some cases; shareable profile URLs for team review | +| **Safari Web Inspector** | iOS Safari and macOS Safari debugging | Required for Safari-specific issues; WebGL/Wasm runtime differs from Chromium | +| **Unity Profiler over WebSocket** | Connect to a development build; standard markers | Use for Unity-side markers (GC, rendering, scripts); does not capture browser-side overhead | + +**Symptom → tool quick reference:** + +| Symptom | First-line tool | Second-line tool | +|---|---|---| +| WebGL hitch / stutter | Chrome DevTools > Performance | Firefox Profiler | +| Memory climbing over time | Chrome DevTools > Memory | Unity Memory Profiler (WebSocket) | +| Slow initial load | Chrome DevTools > Network | Build Report Inspector | +| Safari-only rendering issue | Safari Web Inspector | Compare with Chrome DevTools | + +**Embedding profiling symbols** — browser profilers show mangled Wasm function names by default. To get readable C# method names in Chrome/Firefox flamegraphs, either enable `Player Settings > Publishing > Debug Symbols` for dev builds, or add a build processor: + +```csharp +using UnityEditor; +using UnityEditor.Build; +using UnityEditor.Build.Reporting; + +public class WebProfilingBuildProcessor : IPreprocessBuildWithReport +{ + public int callbackOrder => 0; + public void OnPreprocessBuild(BuildReport report) + { + PlayerSettings.SetAdditionalIl2CppArgs("--compiler-flags=--profiling-funcs"); + } +} +``` + +**Emscripten built-in profilers** — enable one at a time via `PlayerSettings.WebGL.emscriptenArgs`: + +| Flag | What it shows | +|---|---| +| `--cpuprofiler` | CPU profiler overlay in browser | +| `--memoryprofiler` | Visual memory map (white=allocated unused, pink=stack, blue=dynamic, green=fragmented) | +| `--threadprofiler` | Thread activity profiler | + +**GPU debugging** — No Frame Debugger support on Web. Use [Spector.js](https://spector.babylonjs.com/) as a browser-based alternative — it captures draw calls and WebGL state. + +**Firefox `about:memory`** — type `about:memory` as a URL in Firefox, click Measure to see per-tab breakdown: WASM code size, WASM heap, .data file, web audio. Watch for WASM heap > 300 MB (crash risk, especially on iOS Safari). + +Editor Play Mode does not represent browser runtime; always measure in browser. Chrome and Safari GC and JIT behavior differ — test both. + +### Web memory directives + +- Disable **Read/Write Enabled** on textures and meshes — it duplicates data into the WASM heap. +- Reduce `.data` file size by moving assets to Addressables or AssetBundles. +- Use compressed texture formats (KTX2/Basis) to reduce both download and decoded memory cost. + +### iOS Safari memory limits + +- **iOS < 18:** WebContent process limit ~1.5 GB. WASM memory (Gigacage) capped at 2 GB. Typed arrays share this pool. On iPhone X (iOS 16) heap growth caps at ~512 MB, but setting Initial Memory Size to 512 MB–1.5 GB upfront works. +- **iOS 18+:** Limits largely lifted; iPhone 11 can allocate ~4 GB. +- On iOS, set **Initial Memory Size** to the target peak rather than relying on growth — Safari handles large upfront allocations better than incremental growth. +- WASM heap > 300 MB risks crashes on older iOS; target < 200 MB for broad compatibility. + +### Video and audio on Web + +- **Video:** Playback only works from a URL (server with CORS enabled) or from StreamingAssets. On iOS the server must support HTTP range requests for streaming. Use browser-compatible formats (MP4/H.264). +- **Audio:** AudioEffects (mixer effects) require compute shaders — **not available on WebGL**. Mixers and MixerGroups work for volume control only. Set audio to **Mono** to improve loading. If `about:memory` shows web audio > 100 MB, audio is likely uncompressed — switch to Vorbis. + +### Canvas and DPI + +If the canvas is scaled up it takes the new resolution. Use `devicePixelRatio` in the web template to offset DPI scaling and avoid rendering at unnecessarily high resolution. + +## 4. Validation + +1. Re-read the Player Settings with the Pre-Flight snippet (compression, stripping, exceptions, targetFrameRate). +2. Rebuild the player and compare Build Report file sizes with baseline. +3. Verify in at least Chrome and Safari (GC and JIT behavior differ). +4. Max **3 iterations** before asking the user for feedback. + +## 5. Troubleshooting + +### Build still large after enabling Strip Engine Code + +1. Is **Managed Stripping Level** set to Medium or Low? → Set to High for release. +2. Are plug-ins using reflection to access engine modules that would otherwise be stripped? → Add a `link.xml` to preserve needed symbols. +3. Is **Exceptions** set to Full? → Full adds the largest code overhead; switch to None or Explicitly Thrown Only. + +### Brotli not working — Decompression Fallback required + +1. Is the server sending `Content-Encoding: br`? → Without this header the browser won't decompress; the fallback JS decompressor is then needed. +2. Is the build hosted over HTTP (not HTTPS)? → Brotli requires a secure context; degrade to Gzip for HTTP hosting. + +### Stutter in Safari but not Chrome + +1. Does the project set `Application.targetFrameRate = 60`? → On Safari WebGL this conflicts with browser pacing; set to `-1`. +2. Are there shaders that behave differently on Safari's WebGL implementation? → Test on device; Safari's WebGL/Wasm runtime differs from Chromium — some GLSL constructs are handled differently. + +### Memory growth slow path triggered + +1. Is **Initial Memory Size** too small for the project's peak? → Wasm memory growth requires a full buffer copy; set Initial Memory Size to a realistic peak estimate. +2. Is **Memory Growth Mode** set to Linear? → Switch to **Geometric** for saner growth curve. + +### Frame rate set to 60 but browser runs erratically + +1. Is `Application.targetFrameRate = 60` set in code? → On Web this conflicts with `requestAnimationFrame` browser pacing. Set to `-1`. +2. Is `vSyncCount` non-zero? → Set to 0; the browser handles pacing. + +### Firefox cache rejecting large files + +Firefox limits individual cache entries via `browser.cache.disk.max_entry_size`. If the build exceeds this (default ~50 MB), assets won't cache. Solution: use Addressables to split into bundles < 51 MB, or instruct users to increase the setting in `about:config`. + +### Local dev server setup + +For testing builds locally with proper MIME types: + +```bash +# Python (HTTP) +python -m http.server 55553 -d path/to/build + +# Node.js (install serve-handler) +npx serve path/to/build -l 3001 +``` + +For Brotli testing, use HTTPS — Brotli requires a secure context. Generate a self-signed cert with OpenSSL for local testing. + +## 6. Completion + +- Summarize: initial download size delta, settings changed (compression, stripping, exceptions, targetFrameRate), server configuration confirmed. +- List follow-up actions: CDN setup for Addressables remote groups, Safari testing, Wasm 2023 feature set upgrade when browser baseline allows. + +## See also + +These point at Unity tooling rather than other skills, because the topics they cover are not in +this plugin: + +- **Addressables package** — remote groups served over a CDN, when the download budget needs content + moved out of the initial payload. +- **Unity Profiler, connected to the browser** — the cross-platform profiling methodology. Section 3 + covers the Web-specific part of attaching it. +- **Shader variant stripping** (Graphics settings → Shader Stripping, and `ShaderVariantCollection`) + — variant count feeds directly into Wasm size, so it is worth checking when stripping alone hasn't + moved the number. +- **Project Settings → Player** — the same flags this skill reads, if the user would rather see them + in the inspector than have them reported. +- Mobile browser battery behaviour follows the same frame-rate and quality-level guidance in + Sections 3 and 4; there is no separate mobile path here. diff --git a/skills/optimize-web/resources/WebOptimizer.cs b/skills/optimize-web/resources/WebOptimizer.cs new file mode 100644 index 0000000..e17dc5f --- /dev/null +++ b/skills/optimize-web/resources/WebOptimizer.cs @@ -0,0 +1,21 @@ +using UnityEditor; +using UnityEditor.Build; + +public class WebOptimizer +{ + [MenuItem("Tools/Apply Web Release Settings")] + public static void Optimize() + { + var target = NamedBuildTarget.WebGL; + PlayerSettings.SetIl2CppCodeGeneration(target, Il2CppCodeGeneration.OptimizeSize); + PlayerSettings.SetManagedStrippingLevel(target, ManagedStrippingLevel.High); + PlayerSettings.stripUnusedMeshComponents = true; + PlayerSettings.WebGL.dataCaching = true; + PlayerSettings.WebGL.compressionFormat = WebGLCompressionFormat.Brotli; + PlayerSettings.WebGL.exceptionSupport = WebGLExceptionSupport.None; + PlayerSettings.WebGL.debugSymbolMode = WebGLDebugSymbolMode.Off; + PlayerSettings.WebGL.wasm2023 = true; + UnityEditor.WebGL.UserBuildSettings.codeOptimization = + UnityEditor.WebGL.WasmCodeOptimization.DiskSizeLTO; + } +} diff --git a/skills/optimize-web/resources/toktx-examples.sh b/skills/optimize-web/resources/toktx-examples.sh new file mode 100644 index 0000000..14e1698 --- /dev/null +++ b/skills/optimize-web/resources/toktx-examples.sh @@ -0,0 +1,11 @@ +# Albedo / diffuse (ETC1S, lossy, smallest) +toktx --bcmp --lower_left_maps_to_s0t0 output.ktx2 input.png + +# Normal / metallic / detail (UASTC, high fidelity) +toktx --encode uastc --uastc_quality 2 --t2 --lower_left_maps_to_s0t0 output.ktx2 input.png + +# Fix "ICC profile not found" errors +toktx --bcmp --assign_oetf srgb --lower_left_maps_to_s0t0 output.ktx2 input.png + +# Linear data (normal maps, masks) +toktx --bcmp --assign_oetf linear --lower_left_maps_to_s0t0 output.ktx2 input.png From 0f2876ececd20eddddb03361853bb8de3633a110 Mon Sep 17 00:00:00 2001 From: Ziyi Zhang Date: Thu, 13 Aug 2026 16:06:20 -0400 Subject: [PATCH 7/7] feat: drop setup-game-inputs from this batch --- skills/setup-game-inputs/SKILL.md | 32 - .../references/input-system.md | 924 ------------------ 2 files changed, 956 deletions(-) delete mode 100644 skills/setup-game-inputs/SKILL.md delete mode 100644 skills/setup-game-inputs/references/input-system.md diff --git a/skills/setup-game-inputs/SKILL.md b/skills/setup-game-inputs/SKILL.md deleted file mode 100644 index 1ffb1da..0000000 --- a/skills/setup-game-inputs/SKILL.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -name: setup-game-inputs -description: Handles game input setup and configuration — player controls, action maps, bindings, control schemes, and Input Actions assets. Use when setting up keyboard, mouse, or gamepad input, configuring PlayerInput or UI input, generating C# input wrappers, implementing rebinding, or working with the Input System package or Legacy Input Manager. ---- - -## Step 1: Determine Active Input Handler - -The system prompt context should provide the current active input system. If not check the project setting. - -- **Input System Package (New)** — new Input System only; `ENABLE_INPUT_SYSTEM` is defined. -- **Input Manager (Old)** — legacy Input Manager only; `ENABLE_LEGACY_INPUT_MANAGER` is defined. -- **Both** — both defines are set. - -Changing this setting requires an Editor restart. Read the matching path below before deep implementation. - -## Path A: Input System (new) - -**When:** Active Input Handler is **Input System Package (New)**. -Read [input-system.md](references/input-system.md) - -## Path B: Legacy Input Manager (old) - -**When:** Active Input Handler is **Input Manager (Old)** only. -Use **Project Settings > Input Manager** axes and `UnityEngine.Input` (`GetAxis`, `GetButton`, etc.). - - -## Path C: Both - -**When:** Active Input Handler is **Both**. - -Treat **Input System (new)** as the default. Read reference [input-system.md](references/input-system.md) - diff --git a/skills/setup-game-inputs/references/input-system.md b/skills/setup-game-inputs/references/input-system.md deleted file mode 100644 index 13abb6e..0000000 --- a/skills/setup-game-inputs/references/input-system.md +++ /dev/null @@ -1,924 +0,0 @@ -## Table of Contents -- [Performance Notes](#performance-notes) -- [0. Package Installation + Project Setting Check (Must Do First)](#0-package-installation-project-setting-check-must-do-first) -- [1. Pre-Flight Check (Crucial)](#1-pre-flight-check-crucial) -- [2. Gather Missing Information](#2-gather-missing-information) -- [3. Planning & Execution Steps](#3-planning-execution-steps) -- [4. Validation Checklist (Must Confirm)](#4-validation-checklist-must-confirm) -- [5. Final Confirmation Message (What reporting back)](#5-final-confirmation-message-what-reporting-back) -- [Important API notes](#important-api-notes) -- [Core Concepts Reference](#core-concepts-reference) -- [Responding to Actions Reference](#responding-to-actions-reference) -- [PlayerInput Component Reference](#playerinput-component-reference) -- [PlayerInputManager Reference (Multiplayer)](#playerinputmanager-reference-multiplayer) -- [UI Support Reference](#ui-support-reference) -- [Interactions Reference](#interactions-reference) -- [Composite Bindings Reference](#composite-bindings-reference) -- [Interactive Rebinding Reference](#interactive-rebinding-reference) -- [Processors Reference](#processors-reference) -- [Direct Device Access Reference (Prototyping Only)](#direct-device-access-reference-prototyping-only) -- [Migration from Legacy Input Manager](#migration-from-legacy-input-manager) -- [Common Mistakes to Avoid](#common-mistakes-to-avoid) - - -## Performance Notes -- Do this thoroughly. -- Quality is more important than speed. - -## 0. Package Installation + Project Setting Check (Must Do First) - -1. Package Installation Check -First of all **verify that the com.unity.inputsystem package is installed** -**Install if Missing:** add the package to the project manifest — `Packages/manifest.json`, -under `dependencies`: - -```json -"com.unity.inputsystem": "" -``` - -Don't invent the version string. Read the current one from the Unity registry — -`https://packages.unity.com/com.unity.inputsystem` lists every published version — or copy the version an -adjacent Unity package in this manifest already uses. A version that doesn't exist makes -Unity fail resolution **silently**, so a wrong guess looks like nothing happened. - -Unity resolves the new dependency the next time the Editor regains focus. This needs no -Editor connection, which is why it's the default route here. - -If you do have a live Editor to run C# in, the equivalent is: - -```csharp -using UnityEditor.PackageManager; - -var request = Client.Add("com.unity.inputsystem"); -UnityEngine.Debug.Log("Requested com.unity.inputsystem. Progress shows in the Package Manager window."); -``` -**Proceed:** Only continue to the next steps once InputSystem is confirmed to be installed. - -2. Active Input Handling Check -After verifying the package is installed, check the project's Active Input Handling setting: -- **Input System Package (New)** — only the new Input System is active. `ENABLE_INPUT_SYSTEM` is defined. -- **Input Manager (Old)** — only the legacy Input Manager is active. `ENABLE_LEGACY_INPUT_MANAGER` is defined. -- **Both** — both systems are active. Both defines are set. Use Input System (new) as default. - -Changing the Active Input Handling setting requires an Editor restart. - -## 1. Pre-Flight Check (Crucial) - -### Check Project-Wide Actions (CRITICAL — DO NOT GREP PROJECT FILES) - -**DO NOT** search or grep `ProjectSettings/ProjectSettings.asset`, `ProjectSettings/EditorBuildSettings.asset`, or any other project settings files to find the project-wide actions asset. The reference is stored internally via `EditorBuildSettings` config objects and is **not human-readable** in project files. Attempting to grep these files will fail and waste time. - -**The ONLY correct way** to check and manage project-wide actions is the C# API below, run in a live Editor: - -**To check if project-wide actions are assigned and inspect their contents:** - -Two routes. The file route needs no Editor: `.inputactions` assets are JSON on disk, so -glob for `*.inputactions` and read one directly to see its action maps, actions and -bindings. What a file cannot tell you is which asset is *assigned* project-wide — that -lives in project settings. - -With a live Editor to run C# in: - -```csharp -using UnityEngine; -using UnityEngine.InputSystem; - -var actions = InputSystem.actions; -if (actions == null) -{ - Debug.Log("No project-wide Input Actions asset is currently assigned."); - Debug.Log("To create one: Edit > Project Settings > Input System Package > Create a new project-wide Action Asset"); - - // Also check if any .inputactions assets exist in the project that could be assigned - var guids = UnityEditor.AssetDatabase.FindAssets("t:InputActionAsset"); - if (guids.Length > 0) - { - Debug.Log($"Found {guids.Length} InputActionAsset(s) in the project that could be assigned:"); - foreach (var guid in guids) - { - var assetPath = UnityEditor.AssetDatabase.GUIDToAssetPath(guid); - Debug.Log($" - {assetPath}"); - } - } - return "no project-wide actions assigned"; -} - -var path = UnityEditor.AssetDatabase.GetAssetPath(actions); -var report = new System.Text.StringBuilder(); -report.AppendLine($"Project-wide actions asset: {actions.name} ({path})"); -report.AppendLine($"Action Maps ({actions.actionMaps.Count}):"); -foreach (var map in actions.actionMaps) -{ - report.AppendLine($" - {map.name} ({map.actions.Count} actions)"); - foreach (var action in map.actions) - { - report.AppendLine($" {action.name} (Type: {action.type}, ExpectedControlType: {action.expectedControlType}, Bindings: {action.bindings.Count})"); - } -} -report.AppendLine($"Control Schemes ({actions.controlSchemes.Count}):"); -foreach (var scheme in actions.controlSchemes) -{ - report.AppendLine($" - {scheme.name}"); -} -return report.ToString(); -``` - -Return the report rather than only logging it: logs land in the Editor console, while the -returned value is what comes back to whoever ran the snippet. - -**To assign an existing .inputactions asset as project-wide:** -```csharp -var asset = UnityEditor.AssetDatabase.LoadAssetAtPath("Assets/MyActions.inputactions"); -if (asset != null) -{ - InputSystem.actions = asset; - UnityEngine.Debug.Log($"Assigned '{asset.name}' as project-wide actions."); -} -``` -Note: `InputSystem.actions` can only be assigned in Edit mode (not Play mode) and the asset must be a persistent file on disk inside the Assets folder. - -**To find all .inputactions assets in the project:** -```csharp -var guids = UnityEditor.AssetDatabase.FindAssets("t:InputActionAsset"); -foreach (var guid in guids) -{ - var assetPath = UnityEditor.AssetDatabase.GUIDToAssetPath(guid); - UnityEngine.Debug.Log($"Found: {assetPath}"); -} -``` - -### UI Input Module sanity (if the project uses UI) -If the user uses Unity UI (uGUI): -- Find (or create) an EventSystem. -- Ensure it has `InputSystemUIInputModule`. -- If `StandaloneInputModule` is present, remove it to avoid conflicts. - -For UI Toolkit (Unity 2023.2+): The UI actions defined in the default project-wide actions directly map to UI Toolkit input. `InputSystemUIInputModule` component is not needed. - -For UI Toolkit (pre-2023.2): `InputSystemUIInputModule` component must be used. - -## 2. Gather Missing Information -Before invoking tools, ensure the following details exist. If not, ask the user: - -### Core questions -* **Asset name/path:** Where input actions should be? (Default: Assets/Input/InputActions.inputactions) -* **Action maps:** e.g. Player, UI, Vehicle, Debug -* **Target GameObject(s):** which object gets PlayerInput / input scripts? (player prefab, character root, etc.) -* **Platforms/devices:** Keyboard&Mouse, Gamepad, Touch, XR? -* **Gameplay actions needed:** e.g. Move, Look, Jump, Sprint, Crouch, Interact, Fire, Aim, Pause, Navigate UI - -### Per-action details (important) -* **For each action, gather:** - * **Action Type:** Value / Button / PassThrough - * **Expected Control Type:** Vector2, Axis, Button, Delta, etc. - * **Bindings:** default keys/buttons, plus optional composites (2D Vector WASD, arrow keys) - * **Interactions/Processors:** Hold/Tap, Press behavior, Deadzone, Normalize, Invert Y, Sensitivity - -### Action Type Selection Guide - -| Action Type | Use When | Behavior | -|-------------|----------|----------| -| **Value** (default) | Continuous inputs: movement sticks, triggers, mouse delta | Tracks the most actuated control. Performs initial state check on enable. Conflict resolution picks highest magnitude. | -| **Button** | Discrete press actions: jump, fire, interact | Like Value but only binds to `ButtonControl`. No initial state check (avoids re-triggering held buttons on enable). | -| **PassThrough** | Multi-device monitoring, UI pointer actions, raw input | No conflict resolution. Every bound control change fires a callback. No single "driving" control. | - - -## 3. Planning & Execution Steps - -0. Identify existing input patterns and architecture in the project and follow them - -1. Create/Update the Input Actions asset -- Reuse existing Input Actions asset (or create new one) -- Create Input control schemes for required devices (or reuse existing). Don't skip this step, it is important for local multiplayer to have control schemes. -- Create required Action Maps (or reuse existing). -- Create Actions with correct types/control types. -- Add Bindings (including composites like WASD for Move). - -2. Generate a C# wrapper (Optional) -If the user wants strongly-typed code or it is the pattern in the project: -- Enable Generate C# Class on the .inputactions asset -- Set wrapper class name (e.g. GameInput) -- Ensure it regenerates when the asset changes - -3. Hook into gameplay -Base it on existing input patterns in the project. - -* **Option 1 - PlayerInput-based setup** -- Add PlayerInput to the player root (or ensure it exists). -- Assign the Input Actions asset to PlayerInput.actions. -- Set: - - Default Map (e.g. Player) - - Notification Behavior based on the project patterns or what the user specified (Prefer "Invoke CSharp Events" if no preference nor existing pattern exist): - * **Send Messages** when the PlayerInputs sends messages (Default) - - Implement or make sure there are functions in a script to take the messages - - Make sure the PlayerInput.NotificationBehaviour uses SendMessages - - Method signature: `public void OnActionName()` or `public void OnActionName(InputValue value)` - - The component must be on the same GameObject as PlayerInput - - `InputValue` is only valid during the callback; do not store it - * **Broadcast Messages** same as Send Messages but also sends to child GameObjects - - Method signature: same as Send Messages - - Component can be on the same or any child GameObject - * **Invoke CSharp Events** when a C# script subscribes for the PlayerInput - - Make a script to subscribe for the `onActionTriggered` event on the PlayerInput - - Make sure the PlayerInput.NotificationBehaviour uses InvokeCSharpEvents - - Method signature: receives `InputAction.CallbackContext` - * **Invoke Unity Events** when the PlayerInputs has set up unity events to call functions - - Implement or make sure there are functions in a script to take unity event - - Set up the unity events on the PlayerInput to call the functions - - Make sure the PlayerInput.NotificationBehaviour uses InvokeUnityEvents - - Method signature: `public void OnActionName(InputAction.CallbackContext context)` -- If using control schemes, set Default Control Scheme (optional). - -**CRITICAL PlayerInput rule:** When writing input code that works with PlayerInput, do NOT use `InputSystem.actions`. Use `playerInput.actions` instead. PlayerInput creates private copies of actions for device filtering in multiplayer. Using `InputSystem.actions` bypasses automatic device assignment. - -**Project-wide actions + PlayerInput caveat:** With project-wide actions, all action maps may be enabled by default. Disable `InputSystem.actions` and enable only the map PlayerInput should use: -```csharp -void Start() -{ - playerInput = GetComponent(); - InputSystem.actions.Disable(); - playerInput.currentActionMap?.Enable(); -} -``` - -* **Option 2 - InputAction asset reference** (Default) - - Create a script that owns an InputActionAsset / generated wrapper instance. - - Enable/disable maps in OnEnable/OnDisable. - - Subscribe to performed/canceled events. - -Example with generated C# wrapper: -```csharp -public class MyPlayerScript : MonoBehaviour, IGameplayActions -{ - MyPlayerControls controls; - - public void OnEnable() - { - if (controls == null) - { - controls = new MyPlayerControls(); - controls.gameplay.SetCallbacks(this); - } - controls.gameplay.Enable(); - } - - public void OnDisable() - { - controls.gameplay.Disable(); - } - - public void OnMove(InputAction.CallbackContext context) - { - var value = context.ReadValue(); - } - - public void OnJump(InputAction.CallbackContext context) { } -} -``` - -* **Option 3 - Project-Wide Actions** (Simplest) - - First check if a project-wide asset is assigned using the script from Section 1 "Check Project-Wide Actions". Do NOT grep ProjectSettings files. - - If no project-wide asset is assigned, either create one via the asset creation API (see API note 1) and assign it with `InputSystem.actions = asset;`, or instruct the user to go to Edit > Project Settings > Input System Package > Create a new project-wide Action Asset. - - Project-wide actions are enabled by default and ready to use. - - Hook actions into gameplay using `InputSystem.actions.FindAction("Move")`. Cache references in `Start()`, do NOT call `FindAction` every frame. - -Example: -```csharp -using UnityEngine; -using UnityEngine.InputSystem; - -public class PlayerController : MonoBehaviour -{ - InputAction moveAction; - InputAction jumpAction; - - void Start() - { - moveAction = InputSystem.actions.FindAction("Move"); - jumpAction = InputSystem.actions.FindAction("Jump"); - } - - void Update() - { - Vector2 moveValue = moveAction.ReadValue(); - - if (jumpAction.WasPressedThisFrame()) - { - // Jump logic - } - } -} -``` - -4. UI support (if requested) -- Ensure EventSystem + InputSystemUIInputModule exists. -- Ensure there's a UI action map (or use Unity's default UI actions pattern). -- Confirm the UI module references correct actions (depending on project setup). - -Required UI actions (names and types must match for UI Toolkit compatibility via Project-Wide Input Actions: `InputSystem.actions`): - -| Action | Action Type | Control Type | Description | -|--------|-------------|--------------|-------------| -| Navigate | PassThrough | Vector2 | D-pad / arrow key navigation | -| Submit | Button | Button | Confirm selection | -| Cancel | Button | Button | Exit interaction | -| Point | PassThrough | Vector2 | Cursor position | -| Click | PassThrough | Button | Primary click | -| RightClick | PassThrough | Button | Secondary click | -| MiddleClick | PassThrough | Button | Middle click | -| ScrollWheel | PassThrough | Vector2 | Scroll input | -| Tracked Device Position | PassThrough | Vector3 | XR position | -| Tracked Device Orientation | PassThrough | Quaternion | XR rotation | - -**IMPORTANT:** Pointer-type UI actions (Point, Click, RightClick, MiddleClick, ScrollWheel) MUST be set to PassThrough type so multiple devices can feed input without filtering. - -## 4. Validation Checklist (Must Confirm) -- Input System package installed -- Active Input Handling set correctly -- No UI module conflicts (StandaloneInputModule removed if necessary, InputSystemUIInputModule not required by UI Toolkit) -- The input action asset is not corrupted after the changes -- Modifications to input action assets do not result in missing input action references -- The input actions references assigned to the scripts where it is needed -- The input action asset has input control schemes -- Actions have correct types (Value for continuous, Button for discrete, PassThrough for multi-device) -- Composite bindings are correctly configured (2D Vector for WASD, 1D Axis for left/right, etc.) - -## 5. Final Confirmation Message (What reporting back) -Summarize what was created/changed: -- Input Actions asset path + action maps/actions -- Control schemes + bindings -- PlayerInput setup (target object, default map, notification behavior) -- UI EventSystem module state -- Any restart requirement (Active Input Handling change) - - -## Important API notes - -0. Never edit inputaction asset Json directly, always use the InputActionAsset API, run in a live Editor, to edit the asset. - -1. CreateAsset() should not be used to create a file of type 'inputactions'. -To create and save the '.inputaction' files use the next code example: -```csharp -InputActionAsset asset = ScriptableObject.CreateInstance(); - -string json = asset.ToJson(); -File.WriteAllText(path, json); -``` - -2. Do NOT use 'Input.' class to handle inputs, it might result with exceptions at runtime. Use `InputSystem.actions.FindAction()` or action references instead. - -3. To add or remove input control scheme use `asset.AddControlScheme(InputControlScheme)` - -4. To add an action to a map use the API example that follows `public static InputAction AddAction(this InputActionMap map, string name, InputActionType type = InputActionType.Value, string binding = null, string interactions = null, string processors = null, string groups = null, string expectedControlLayout = null)` - -5. To add a composite binding use `AddCompositeBinding`: -```csharp -moveAction.AddCompositeBinding("2DVector") - .With("Up", "/w") - .With("Down", "/s") - .With("Left", "/a") - .With("Right", "/d"); -``` - -6. To add a simple binding use `AddBinding`: -```csharp -fireAction.AddBinding("/leftButton"); -fireAction.AddBinding("/rightTrigger"); -``` - -7. Enable/Disable actions and maps: -```csharp -// Enable a single action -myAction.Enable(); - -// Enable an entire action map -gameplayMap.Enable(); - -// Disable -myAction.Disable(); -gameplayMap.Disable(); -``` -DO not change bindings while an action is enabled. Disable first, modify, then re-enable. - -8. To find an action in an asset or project-wide actions: -```csharp -// By action name (searches all maps) -var action = asset.FindAction("Jump"); - -// By map/action path (disambiguates if name collisions exist) -var action = asset.FindAction("Player/Jump"); -``` - -9. CRITICAL: Project-Wide Actions are NOT in ProjectSettings files. -**NEVER** search, grep, or read `ProjectSettings/ProjectSettings.asset`, `ProjectSettings/EditorBuildSettings.asset`, or any other settings files to find input actions. The project-wide actions reference is stored internally via `EditorBuildSettings` config objects (binary format, not greppable). Always use `InputSystem.actions` to read the current project-wide actions, and `InputSystem.actions = asset` to assign them. See Section 1 "Check Project-Wide Actions" for the complete script. - -10. CRITICAL: Correct Unity Input System API Names - -| WRONG (Hallucinated) | CORRECT | -|---------------------|---------| -| `InputSystem.GetDevice()` | `Keyboard.current` | -| `InputSystem.GetDevice()` | `Mouse.current` | -| `InputSystem.GetDevice()` | `Gamepad.current` | -| `Input.GetAxis("Horizontal")` | `InputSystem.actions.FindAction("Move").ReadValue().x` | -| `Input.GetButtonDown("Jump")` | `InputSystem.actions.FindAction("Jump").WasPressedThisFrame()` | -| `Input.GetButton("Jump")` | `InputSystem.actions.FindAction("Jump").IsPressed()` | -| `Input.GetButtonUp("Jump")` | `InputSystem.actions.FindAction("Jump").WasReleasedThisFrame()` | -| `Input.mousePosition` | `Mouse.current.position.ReadValue()` | -| `Input.GetMouseButtonDown(0)` | `Mouse.current.leftButton.wasPressedThisFrame` | -| `Input.GetKey(KeyCode.Space)` | `Keyboard.current.spaceKey.isPressed` | -| `Input.GetKeyDown(KeyCode.Space)` | `Keyboard.current.spaceKey.wasPressedThisFrame` | -| `InputActionMap.FromJson` creating an asset | `InputActionAsset.FromJson` for full assets | - -## Core Concepts Reference - -### Actions -Actions are named, game-meaningful inputs ("Jump", "Move") decoupled from hardware. They allow separating the purpose of an input from the device controls that perform it. - -Each action has: -- A **name** (unique within its action map) -- A unique **ID** (persists across renames) -- An **Action Type** (Value, Button, or PassThrough) -- An **Expected Control Type** (Vector2, Button, Axis, etc.) - -Actions are a runtime-only feature. Do NOT use them in Editor window code. - -### Action Maps -Action maps group actions for a context (e.g., "Player", "UI", "Vehicle"). Enable/disable entire maps as a unit to switch input contexts. - -### Input Action Assets -`.inputactions` files stored in JSON format containing action maps, actions, bindings, and control schemes. The recommended workflow is one asset assigned as project-wide actions. - -### Project-Wide Actions -One asset designated globally via **Edit > Project Settings > Input System Package**. Accessible as `InputSystem.actions`. Preloaded at startup. Actions are enabled by default. - -To create and assign default project-wide actions, go to **Edit > Project Settings > Input System Package** and click "Create a new project-wide Action Asset". This creates `InputSystem_Actions.inputactions` with default Player and UI action maps. - -### Control Schemes -Groups of bindings and devices (e.g., "Keyboard&Mouse", "Gamepad"). Used for: -- Enabling/disabling sets of bindings -- PlayerInput automatic device pairing -- UI device switching feedback - -### Bindings -Links an action to device control(s) via control paths. Types: -- **Normal binding**: direct path like `/leftStick` -- **Composite binding**: synthesizes a value from multiple part bindings (e.g., WASD → Vector2) - -Key binding properties: - -| Property | Description | -|----------|-------------| -| `path` | Control path identifying the control(s). Example: `"/leftStick"` | -| `overridePath` | Non-destructive override of `path`. Used for runtime rebinding. | -| `effectivePath` | Returns `overridePath` if set, otherwise `path`. | -| `action` | Name or ID of the action this binding triggers. | -| `groups` | Semicolon-separated binding groups (used for control schemes). Example: `"Keyboard&Mouse;Gamepad"` | -| `interactions` | Semicolon-separated interactions. Example: `"hold(duration=0.75)"` | -| `processors` | Semicolon-separated processors. Example: `"invertVector2(invertX=false)"` | -| `isComposite` | Whether this binding is a composite root. | -| `isPartOfComposite` | Whether this binding is a part of a composite. | - -Control path syntax: -- `/buttonSouth` — matches on any gamepad -- `/buttonSouth` — matches only PlayStation controllers -- `/button*` — wildcard matching -- `*/{Submit}` — matches any control with "Submit" usage on any device - -## Responding to Actions Reference - -### Polling (Recommended for Gameplay) -Read values in `Update()`. Cache action references in `Start()`. - -| Method | Description | -|--------|-------------| -| `ReadValue()` | Current value of the action. Type must match the bound control's value type. | -| `IsPressed()` | True if actuation is above press point and hasn't fallen to release threshold. | -| `WasPressedThisFrame()` | True if actuation crossed press point this frame. | -| `WasReleasedThisFrame()` | True if actuation fell from above press point to at/below release threshold this frame. | -| `WasPerformedThisFrame()` | True if the action's phase became Performed this frame (interaction-driven). | -| `WasCompletedThisFrame()` | True if the action's phase changed away from Performed this frame. | - -### Callbacks (Event-Driven) -Subscribe to action phase callbacks for sporadic or multi-listener setups. - -```csharp -action.started += ctx => { /* Interaction started */ }; -action.performed += ctx => { /* Interaction completed */ }; -action.canceled += ctx => { /* Interaction interrupted/released */ }; -``` - -`InputAction.CallbackContext` is only valid during the callback. Do not store it. - -**Action Phases:** - -| Phase | Description | -|-------|-------------| -| `Disabled` | Action is disabled and can't receive input. | -| `Waiting` | Action is enabled and waiting for input. | -| `Started` | Input has started an interaction with the action. | -| `Performed` | An interaction with the action has been completed. | -| `Canceled` | An interaction with the action has been interrupted. | - -### Default Interaction Behavior by Action Type - -| Callback | Value | Button | PassThrough | -|----------|-------|--------|-------------| -| `started` | Control changed away from default value | Button started being pressed | Not used | -| `performed` | Control changed value | Button crossed press threshold | Control changed value | -| `canceled` | Controls no longer actuated | Button released | Action disabled | - -### Other Callback Options -- `InputActionMap.actionTriggered` — single callback for all actions in a map (receives started, performed, canceled) -- `InputSystem.onActionChange` — global callback for all action-related changes - -## PlayerInput Component Reference - -The PlayerInput component provides: -- Configuring how Actions map to methods or callbacks -- Handling local multiplayer: device filtering, screen splitting - -### Configuration Properties - -| Property | Description | -|----------|-------------| -| **Actions** | The Input Actions asset (project-wide or standalone asset) | -| **Default Scheme** | Control scheme to enable by default | -| **Default Map** | Action map to enable by default. If None, no actions are enabled. | -| **Camera** | Player camera (only needed for split-screen) | -| **Behavior** | Notification method: Send Messages, Broadcast Messages, Invoke Unity Events, Invoke C# Events | - -### Notification Behaviors - -| Behavior | How it Works | Method Signature | -|----------|-------------|-----------------| -| **Send Messages** | `GameObject.SendMessage` on the PlayerInput's GameObject | `void OnActionName()` or `void OnActionName(InputValue value)` | -| **Broadcast Messages** | `GameObject.BroadcastMessage` down the hierarchy | Same as Send Messages | -| **Invoke Unity Events** | Separate UnityEvent per action, configurable in Inspector | `void OnActionName(InputAction.CallbackContext context)` | -| **Invoke C# Events** | Plain C# events: `onActionTriggered`, `onDeviceLost`, `onDeviceRegained` | `void Handler(InputAction.CallbackContext context)` | - -### Action Map Switching -```csharp -// Switch by name -playerInput.SwitchCurrentActionMap("UI"); - -// Check current -var currentMap = playerInput.currentActionMap; - -// Deactivate/Activate all input -playerInput.DeactivateInput(); -playerInput.ActivateInput(); // Re-enables default action map -``` - -### Device Lost/Regained -PlayerInput sends `DeviceLostMessage` and `DeviceRegainedMessage` notifications when devices disconnect/reconnect. - -### UI Integration -Assign an `InputSystemUIInputModule` reference to PlayerInput's `UI Input Module` field. Both must use the same Input Actions asset. PlayerInput will configure the UI module to use the same action/device configuration. - -For multiplayer UI, use `MultiplayerEventSystem` instead of `EventSystem`. Each player gets their own `MultiplayerEventSystem` + `InputSystemUIInputModule` + `PlayerInput`. - -## PlayerInputManager Reference (Multiplayer) - -Used alongside PlayerInput for local multiplayer. - -| Property | Description | -|----------|-------------| -| **Player Prefab** | Must have a PlayerInput component | -| **Join Behavior** | Join When Button Is Pressed / Join When Join Action Is Triggered / Manual | -| **Max Players** | Maximum player count (-1 = unlimited) | -| **Split Screen** | Enable/configure split-screen rendering | - -Each PlayerInput instance gets a private copy of actions with device filtering. Players are automatically paired to unique devices. - -## UI Support Reference - -### InputSystemUIInputModule -Required for Unity UI (uGUI). Replaces `StandaloneInputModule`. - -| Property | Description | -|----------|-------------| -| Move Repeat Delay | Initial delay before repeat navigation events | -| Move Repeat Rate | Interval between repeat navigation events | -| Actions Asset | Input Action Asset driving the UI | -| Deselect on Background Click | Clear selection when clicking empty space (default: true) | -| Pointer Behavior | How multiple pointers are handled | - -### Pointer Behaviors - -| Mode | Description | -|------|-------------| -| **Single Mouse or Pen But Multi Touch And Track** | Default. Mouse/pen unified; touch and tracked devices are separate. | -| **Single Unified Pointer** | All input unified into one pointer. | -| **All Pointers As Is** | Every device is its own pointer. | - -### UI Toolkit Compatibility - -| UI Solution | Compatible | UI Input Module Required | -|-------------|------------|-------------------------| -| UI Toolkit (2023.2+) | Yes | Not required | -| UI Toolkit (pre-2023.2) | Yes | Required | -| Unity UI (uGUI) | Yes | Required | -| IMGUI | No (use "Both" Active Input Handling for IMGUI + Input System coexistence) | - -## Interactions Reference - -Interactions are input patterns that drive action phase transitions. Applied to bindings or actions. - -### Built-in Interactions - -| Interaction | Description | Key Parameters | -|-------------|-------------|----------------| -| **Default** | Applied when no interaction is specified. Behavior varies by action type. | — | -| **Press** | Explicit button-press pattern. | `pressPoint`, `behavior` (PressOnly/ReleaseOnly/PressAndRelease) | -| **Hold** | Requires holding a control for a duration. | `duration` (default: `InputSettings.defaultHoldTime`), `pressPoint` | -| **Tap** | Press and release within a duration. | `duration` (default: `InputSettings.defaultTapTime`), `pressPoint` | -| **SlowTap** | Hold for minimum duration, then release to trigger. | `duration` (default: `InputSettings.defaultSlowTapTime`), `pressPoint` | -| **MultiTap** | Multiple taps in succession (e.g., double-click). | `tapCount` (default: 2), `tapTime`, `tapDelay`, `pressPoint` | - -### Interaction Phase Behavior - -**Hold:** -- `started` → control crosses press point -- `performed` → held above press point for >= duration -- `canceled` → released before duration elapsed - -**Tap:** -- `started` → control crosses press point -- `performed` → released before duration elapsed -- `canceled` → held too long (>= duration) - -**Adding Interactions:** -```csharp -// In code -action.AddBinding("/buttonSouth") - .WithInteractions("hold(duration=0.4)"); - -// On action directly -var action = new InputAction(interactions: "hold(duration=0.4)"); -``` - -Multiple interactions on a binding are processed in order. The first to trigger "consumes" the input. - -### Timeout Completion -```csharp -// Get progress of hold/tap interaction (0 to 1) -float progress = action.GetTimeoutCompletionPercentage(); -``` - -## Composite Bindings Reference - -Composites combine multiple controls into a single value. - -### Built-in Composites - -| Composite | Output Type | Parts | Usage | -|-----------|-------------|-------|-------| -| **1D Axis** | `float` | Positive, Negative | Left/Right, triggers | -| **2D Vector** (Dpad) | `Vector2` | Up, Down, Left, Right | WASD movement, D-pad | -| **3D Vector** | `Vector3` | Up, Down, Left, Right, Forward, Backward | 3D movement | -| **One Modifier** | Any | Modifier, Binding | SHIFT+Key shortcuts | -| **Two Modifiers** | Any | Modifier1, Modifier2, Binding | CTRL+SHIFT+Key | - -### Code Examples - -```csharp -// 1D Axis -myAction.AddCompositeBinding("1DAxis") - .With("Positive", "/d") - .With("Negative", "/a"); - -// 2D Vector (WASD) -myAction.AddCompositeBinding("2DVector") - .With("Up", "/w") - .With("Down", "/s") - .With("Left", "/a") - .With("Right", "/d"); - -// 2D Vector with mode -myAction.AddCompositeBinding("2DVector(mode=2)") // mode=2 is Analog - .With("Up", "/leftStick/up") - .With("Down", "/leftStick/down") - .With("Left", "/leftStick/left") - .With("Right", "/leftStick/right"); - -// One Modifier (SHIFT+1) -myAction.AddCompositeBinding("OneModifier") - .With("Binding", "/1") - .With("Modifier", "/ctrl"); - -// Two Modifiers (CTRL+SHIFT+1) -myAction.AddCompositeBinding("TwoModifiers") - .With("Button", "/1") - .With("Modifier1", "/leftCtrl") - .With("Modifier2", "/leftShift"); -``` - -### 2D Vector Mode Parameter - -| Mode | Value | Description | -|------|-------|-------------| -| DigitalNormalized | 0 | Default. Inputs treated as on/off, vector normalized (diamond-shaped range). | -| Digital | 1 | On/off but not normalized. Diagonals have magnitude > 1. | -| Analog | 2 | Full floating-point values. Down and Left inverted. | - -Each composite part can have multiple bindings (e.g., both WASD and arrow keys for the same 2D Vector). - -## Interactive Rebinding Reference - -Allow users to customize bindings at runtime. - -### Performing a Rebind -```csharp -void RemapButtonClicked(InputAction actionToRebind) -{ - var rebindOperation = actionToRebind - .PerformInteractiveRebinding() - .Start(); -} -``` -IMPORTANT: Dispose `RebindingOperation` instances via `Dispose()` to prevent memory leaks. - -### Configuration Options -- `WithExpectedControlType()` — filter by control type -- `WithControlsExcluding()` — exclude specific controls -- `WithCancelingThrough()` — set a cancel control -- `WithTargetBinding()` / `WithBindingGroup()` — target specific bindings - -### Save and Load Rebinds -```csharp -// Save -var rebinds = playerInput.actions.SaveBindingOverridesAsJson(); -PlayerPrefs.SetString("rebinds", rebinds); - -// Load (removes existing overrides by default) -var rebinds = PlayerPrefs.GetString("rebinds"); -playerInput.actions.LoadBindingOverridesFromJson(rebinds); -``` - -### Restore Defaults -```csharp -// Remove overrides from a single action -playerInput.actions["fire"].RemoveAllBindingOverrides(); - -// Remove all overrides from all actions -playerInput.actions.RemoveAllBindingOverrides(); -``` - -### Display Binding Strings -```csharp -// Get display string for an action -string displayStr = action.GetBindingDisplayString(); - -// Get display string for a specific binding index -string displayStr = action.GetBindingDisplayString(1); - -// Get with device/control info (for icon replacement) -string displayStr = action.GetBindingDisplayString(0, out string deviceLayout, out string controlPath); -``` - -### Apply Binding Overrides (Non-Interactive) -```csharp -// Override a binding path -playerInput.actions["fire"].ApplyBindingOverride("/leftTrigger"); - -// Override by binding index -var jumpAction = playerInput.actions["Jump"]; -var bindingIndex = jumpAction.GetBindingIndexForControl(Keyboard.current.spaceKey); -jumpAction.ApplyBindingOverride(bindingIndex, "/enter"); -``` - -Override properties (`overridePath`, `overrideProcessors`, `overrideInteractions`) are NOT saved with the asset JSON. Use `SaveBindingOverridesAsJson` / `LoadBindingOverridesFromJson` separately. - -## Processors Reference - -Processors transform input values. Applied to bindings or actions. Stack with processors on controls. - -Common processors: -- `invertVector2(invertX=true,invertY=true)` — invert axes -- `scaleVector2(x=1,y=1)` — scale axes -- `stickDeadzone(min=0.125,max=0.925)` — apply deadzone to stick input -- `axisDeadzone(min=0.125,max=0.925)` — apply deadzone to single axis -- `normalize(min=0,max=1,zero=0)` — normalize to range -- `clamp(min=0,max=1)` — clamp value -- `invert` — invert a single float value -- `scale(factor=1)` — scale a single float value - -```csharp -// In code -action.AddBinding("/leftStick") - .WithProcessors("stickDeadzone(min=0.2,max=0.9)"); - -// On action -var action = new InputAction(processors: "invertVector2(invertX=false)"); -``` - -### Parameter Overrides for Sensitivity -```csharp -// Adjust mouse sensitivity separately from gamepad -var look = new InputAction("look", type: InputActionType.Value); -look.AddBinding("/delta", groups: "KeyboardMouse", processors: "scaleVector2"); -look.AddBinding("/rightStick", groups: "Gamepad", processors: "scaleVector2"); - -look.ApplyParameterOverride("scaleVector2:x", 0.5f, InputBinding.MaskByGroup("KeyboardMouse")); -look.ApplyParameterOverride("scaleVector2:y", 0.5f, InputBinding.MaskByGroup("KeyboardMouse")); - -look.ApplyParameterOverride("scaleVector2:x", 2f, InputBinding.MaskByGroup("Gamepad")); -look.ApplyParameterOverride("scaleVector2:y", 2f, InputBinding.MaskByGroup("Gamepad")); -``` - -## Direct Device Access Reference (Prototyping Only) - -For quick prototyping or fixed-device scenarios. Less flexible than actions. - -```csharp -// Keyboard -if (Keyboard.current.spaceKey.wasPressedThisFrame) { } -if (Keyboard.current.wKey.isPressed) { } - -// Mouse -Vector2 mousePos = Mouse.current.position.ReadValue(); -if (Mouse.current.leftButton.wasPressedThisFrame) { } -Vector2 mouseDelta = Mouse.current.delta.ReadValue(); - -// Gamepad -var gamepad = Gamepad.current; -if (gamepad == null) return; // No gamepad connected -Vector2 move = gamepad.leftStick.ReadValue(); -if (gamepad.buttonSouth.wasPressedThisFrame) { } -if (gamepad.rightTrigger.wasPressedThisFrame) { } -``` - -Always null-check `*.current` as devices may not be connected. - -## Migration from Legacy Input Manager - -When migrating from old `Input` class to Input System: - -| Legacy (Old) | Input System (New) — Actions Approach | Input System (New) — Direct Approach | -|--------------|--------------------------------------|--------------------------------------| -| `Input.GetAxis("Horizontal")` | `moveAction.ReadValue().x` | `Keyboard.current.dKey.ReadValue() - Keyboard.current.aKey.ReadValue()` | -| `Input.GetButton("Fire1")` | `fireAction.IsPressed()` | `Mouse.current.leftButton.isPressed` | -| `Input.GetButtonDown("Jump")` | `jumpAction.WasPressedThisFrame()` | `Keyboard.current.spaceKey.wasPressedThisFrame` | -| `Input.GetButtonUp("Jump")` | `jumpAction.WasReleasedThisFrame()` | `Keyboard.current.spaceKey.wasReleasedThisFrame` | -| `Input.mousePosition` | `pointerAction.ReadValue()` | `Mouse.current.position.ReadValue()` | -| `Input.GetMouseButtonDown(0)` | `clickAction.WasPressedThisFrame()` | `Mouse.current.leftButton.wasPressedThisFrame` | -| `Input.GetKey(KeyCode.Space)` | `action.IsPressed()` | `Keyboard.current.spaceKey.isPressed` | -| `Input.touches` / `Input.touchCount` | Use `EnhancedTouchSupport` | `EnhancedTouch.Touch.activeTouches` | - -Preprocessor defines for conditional compilation: -- `#if ENABLE_INPUT_SYSTEM` — new Input System is active -- `#if ENABLE_LEGACY_INPUT_MANAGER` — old Input Manager is active -- Both can be true when Active Input Handling is set to "Both" - -Old API surface "Unity Input" resides in class `UnityEngine.Input`. Input System package API surface resides in root namespace `UnityEngine.InputSystem`. -The following exceptions exist and may be used regardless of Active Input Handling setting: -- `UnityEngine.Input.location` -- `UnityEngine.Input.stylusTouchSupported` -- `UnityEngine.Input.mousePresent` -- `UnityEngine.Input.multiTouchEnabled` - -## Common Mistakes to Avoid - -### 1. Using `Input.` Class with Input System -**Problem:** `Input.GetAxis`, `Input.GetButtonDown` etc. throw exceptions when only Input System (new) is active. -**Solution:** Use `InputSystem.actions.FindAction()` or direct device access (`Keyboard.current`, etc.). - -### 2. Calling FindAction Every Frame -**Problem:** `InputSystem.actions.FindAction("Move")` in `Update()` is wasteful. -**Solution:** Cache the `InputAction` reference in `Start()` or `Awake()`. - -### 3. Using InputSystem.actions with PlayerInput -**Problem:** `InputSystem.actions` is the singleton copy. PlayerInput creates private copies for device filtering. -**Solution:** Use `playerInput.actions` when working with PlayerInput. - -### 4. Not Disabling Actions Before Modifying Bindings -**Problem:** Changing bindings while actions are enabled causes temporary disable/re-enable of all actions. -**Solution:** Disable the action or map, make changes, then re-enable. - -### 5. Storing InputAction.CallbackContext -**Problem:** Context struct is only valid during the callback. -**Solution:** Read values during the callback; don't store the context for later use. - -### 6. Wrong Action Type for UI -**Problem:** Using Value type for UI pointer actions causes only one device to drive input. -**Solution:** UI pointer actions (Point, Click, ScrollWheel, etc.) must be PassThrough. - -### 7. Missing Control Schemes -**Problem:** Local multiplayer doesn't pair devices correctly. -**Solution:** Always create control schemes with required devices. PlayerInput uses these for automatic device pairing. - -### 8. Forgetting to Dispose RebindingOperation -**Problem:** `PerformInteractiveRebinding()` allocates unmanaged memory. -**Solution:** Always call `Dispose()` on the `RebindingOperation` when done. - -### 9. Not Saving Binding Overrides Separately -**Problem:** `overridePath` is not saved with `InputActionAsset.ToJson()`. -**Solution:** Use `SaveBindingOverridesAsJson()` / `LoadBindingOverridesFromJson()` and persist via PlayerPrefs or file. - -### 10. Editing .inputactions JSON Directly -**Problem:** Manual JSON edits can corrupt the asset, break binding IDs, or lose data. -**Solution:** Always modify assets programmatically through the InputActionAsset API, run in a live Editor. - -### 11. Searching ProjectSettings Files for Input Actions -**Problem:** Grepping or reading `ProjectSettings/ProjectSettings.asset` or other settings files to find the project-wide actions asset. The reference is stored via `EditorBuildSettings` config objects in binary format and is not searchable in text files. This always fails and wastes multiple tool calls. -**Solution:** Always use `InputSystem.actions` to check the current project-wide actions. See Section 1 "Check Project-Wide Actions" for the complete script.