Property Guide Algorithm#7
Conversation
There was a problem hiding this comment.
Pull request overview
Implements a “property guide” mechanism that lets the Android monkey/runtime report “precondition pages” to the native RL agents and persist that guidance separately from the existing reuse model.
Changes:
- Added a FlatBuffers-backed
.precondpersistence format and native load/save logic for precondition pages/templates. - Extended native agents (SARSA/Double SARSA/Curiosity via
AbstractAgent) to opportunistically pick “guide actions” and maintain episode-level counters/settlement. - Added Java + HTTP bridge (
/propertyFirstSatisfied) to dump the current UI hierarchy and synchronously forward it to native as a precondition signal; removed a large set of generated CMake build artifacts from version control.
Reviewed changes
Copilot reviewed 47 out of 105 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| android/native/storage/PreconditionModel.fbs | Defines FlatBuffers schema for .precond persistence. |
| android/native/storage/PreconditionModel_generated.h | Generated FlatBuffers bindings for the precondition model. |
| android/native/project/jni/fastbot_native.h | Adds JNI declaration for precondition reporting entrypoint. |
| android/native/project/jni/fastbot_native.cpp | Implements synchronous JNI bridge to parse XML and forward to Model::addCurrentPageAsPrecondition. |
| android/native/model/Model.h | Exposes Model::addCurrentPageAsPrecondition API. |
| android/native/model/Model.cpp | Builds state-only snapshot and forwards to agent precondition hook. |
| android/native/CMakeFiles/TargetDirectories.txt | Removes generated CMake build artifact. |
| android/native/CMakeFiles/progress.marks | Removes generated CMake build artifact. |
| android/native/CMakeFiles/Makefile2 | Removes generated CMake build artifact. |
| android/native/CMakeFiles/Makefile.cmake | Removes generated CMake build artifact. |
| android/native/CMakeFiles/InstallScripts.json | Removes generated CMake build artifact. |
| android/native/CMakeFiles/fastbot_native.dir/thirdpart/tinyxml2/tinyxml2.cpp.o.d | Removes generated dependency file. |
| android/native/CMakeFiles/fastbot_native.dir/progress.make | Removes generated CMake build artifact. |
| android/native/CMakeFiles/fastbot_native.dir/link.txt | Removes generated link command file. |
| android/native/CMakeFiles/fastbot_native.dir/link.d | Removes generated link depfile. |
| android/native/CMakeFiles/fastbot_native.dir/flags.make | Removes generated flags file. |
| android/native/CMakeFiles/fastbot_native.dir/DependInfo.cmake | Removes generated dependency info. |
| android/native/CMakeFiles/fastbot_native.dir/depend.make | Removes generated dependency stub. |
| android/native/CMakeFiles/fastbot_native.dir/compiler_depend.ts | Removes generated compiler dependency timestamp. |
| android/native/CMakeFiles/fastbot_native.dir/compiler_depend.make | Removes generated compiler dependency stub. |
| android/native/CMakeFiles/fastbot_native.dir/cmake_clean.cmake | Removes generated clean script. |
| android/native/CMakeFiles/CMakeDirectoryInformation.cmake | Removes generated directory info. |
| android/native/CMakeFiles/cmake.check_cache | Removes generated cache check file. |
| android/native/CMakeFiles/4.2.2/CompilerIdCXX/CMakeCXXCompilerId.cpp | Removes generated compiler-ID source. |
| android/native/CMakeFiles/4.2.2/CompilerIdC/CMakeCCompilerId.c | Removes generated compiler-ID source. |
| android/native/CMakeFiles/4.2.2/CMakeSystem.cmake | Removes generated system configuration file. |
| android/native/CMakeFiles/4.2.2/CMakeCXXCompiler.cmake | Removes generated compiler configuration file. |
| android/native/CMakeFiles/4.2.2/CMakeCCompiler.cmake | Removes generated compiler configuration file. |
| android/native/CMakeCache.txt | Removes generated CMake cache. |
| android/native/cmake_install.cmake | Modifies CMake install script content (currently machine-specific). |
| android/native/build_native.bat | Adds Windows batch build helper for multiple ABIs. |
| android/native/agent/SarsaAgent.cpp | Adds guide-action selection and .precond load/save wiring. |
| android/native/agent/ModelStorageConstants.h | Adds .precond extension constant. |
| android/native/agent/DoubleSarsaAgent.cpp | Adds guide-action selection and .precond load/save wiring. |
| android/native/agent/CuriosityAgent.cpp | Adds guide-action selection hook. |
| android/native/agent/AbstractAgent.h | Introduces guide/precondition data structures and APIs. |
| android/native/agent/AbstractAgent.cpp | Implements guide selection, settlement logic, template generation, and .precond FlatBuffers I/O (with legacy parsing). |
| android/monkey/src/main/java/com/bytedance/fastbot/AiClient.java | Adds Java-facing wrapper/native method for precondition sync reporting. |
| android/monkey/src/main/java/com/android/commands/monkey/utils/ProxyServer.java | Adds /propertyFirstSatisfied endpoint to trigger precondition reporting. |
| android/monkey/src/main/java/com/android/commands/monkey/source/MonkeySourceApeU2.java | Implements endpoint handler logic to dump hierarchy and call the new native method. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Install script for directory: D:/Mike/AndroidStudioProjects/Fastbot3/android/native | ||
|
|
||
| # Set the install prefix | ||
| if(NOT DEFINED CMAKE_INSTALL_PREFIX) | ||
| set(CMAKE_INSTALL_PREFIX "/usr/local") | ||
| set(CMAKE_INSTALL_PREFIX "C:/Program Files (x86)/fastbot_native") | ||
| endif() |
There was a problem hiding this comment.
cmake_install.cmake appears to be a generated build artifact and now contains machine-specific absolute paths (drive letters, SDK locations) and a hard-coded install prefix under Program Files. This will break installs/builds on other environments and should not be committed; remove this file from the repo (and add it to .gitignore) or revert it to a source-controlled, environment-agnostic script if it’s intentionally maintained.
| REM 设置 NDK 路径 | ||
| set NDK_TOOLCHAIN=D:\Android\Sdk\ndk\25.2.9519653\build\cmake\android.toolchain.cmake | ||
|
|
||
| REM 可选:清理旧构建 | ||
| if exist CMakeFiles rmdir /s /q CMakeFiles | ||
| if exist CMakeCache.txt del /q CMakeCache.txt | ||
|
|
||
| REM 编译 armeabi-v7a | ||
| echo ==== Building armeabi-v7a ==== | ||
| cmake -DCMAKE_TOOLCHAIN_FILE="%NDK_TOOLCHAIN%" -DANDROID_ABI=armeabi-v7a -DCMAKE_BUILD_TYPE=Release . | ||
| nmake | ||
| if exist CMakeFiles rmdir /s /q CMakeFiles |
There was a problem hiding this comment.
build_native.bat hard-codes a local NDK path and invokes nmake without selecting an appropriate CMake generator. This will fail for most Windows setups (missing NDK path, missing Developer Command Prompt, generator mismatch). Consider reading the NDK path from ANDROID_NDK_HOME/NDK_ROOT and using cmake -S . -B <buildDir> -G <generator> plus cmake --build <buildDir> to be portable.
| JNIEXPORT void JNICALL | ||
| Java_com_bytedance_fastbot_AiClient_addCurrentPageAsPreconditionSyncNative(JNIEnv *env, jobject, jstring activity, | ||
| jstring xmlDescOfGuiTree); |
There was a problem hiding this comment.
fastbot_native.h declares Java_com_bytedance_fastbot_AiClient_addCurrentPageAsPreconditionSyncNative(...), but there is no corresponding native method with that name/signature in AiClient.java, while AiClient does declare addCurrentPageAsPreconditionSync(String) returning int. Please align the JNI header declarations with the actual Java native methods to avoid confusion and accidental symbol drift (either add the missing declaration for the int method and/or remove the unused *Native variant).
| #include <cinttypes> | ||
| #include <cmath> | ||
| #include <cstdint> | ||
| #include <cstring> | ||
| #include <fstream> | ||
| #include <future> | ||
| #include "../storage/PreconditionModel_generated.h" |
There was a problem hiding this comment.
AbstractAgent.cpp uses std::min/std::max but does not include . It may compile via transitive includes on some toolchains, but it’s not guaranteed and can break with different standard library implementations. Please include explicitly here (or in a common header) where these functions are used.
| in.seekg(0, std::ios::end); | ||
| const std::streamsize sz = in.tellg(); | ||
| in.seekg(0, std::ios::beg); | ||
| if (sz <= 0) { | ||
| BLOGE("[GUIDE] load .precond failed: empty file %s", filepath.c_str()); | ||
| return false; | ||
| } | ||
| std::vector<uint8_t> buf(static_cast<size_t>(sz)); | ||
| in.read(reinterpret_cast<char *>(buf.data()), sz); | ||
| if (in.fail()) { |
There was a problem hiding this comment.
loadPreconditionPagesFromFile reads the entire .precond file into memory without any upper bound. A corrupted/oversized file could cause excessive memory use or OOM. Consider enforcing a maximum allowed .precond file size (similar to MaxModelFileSize used for .fbm) before allocating/reading, and fail gracefully if exceeded.
| GuideChoice best; | ||
| for (auto &kv : _preconditionPages) { | ||
| const uint64_t pageHash = kv.first; | ||
| PreconditionInfo &info = kv.second; | ||
| if (_coveredThisEpisode.count(pageHash) > 0) { | ||
| BLOG("[GUIDE] candidate page filtered: page=%" PRIu64 " reason=covered", pageHash); | ||
| continue; | ||
| } | ||
| if (info.score < GuideConstants::kMinCandidateScore) { | ||
| BLOG("[GUIDE] candidate page filtered: page=%" PRIu64 " reason=low-score score=%.4f", pageHash, info.score); | ||
| continue; |
There was a problem hiding this comment.
trySelectGuideAction emits multiple BLOG lines inside nested loops over all precondition pages/templates on every action selection, including per-candidate match logs. This can become a significant CPU/logging overhead as _preconditionPages grows and can flood logcat. Consider gating these logs behind a debug flag/log level or sampling, and avoid logging in the hot path by default.
| Logger.println("[MonkeySourceApeU2] Sent current page as precondition to native"); | ||
| Logger.println("[MonkeySourceApeU2] Precondition XML length: " + this.stringOfGuiTree.length()); | ||
| Logger.println("[MonkeySourceApeU2] Precondition XML start ---"); | ||
| Logger.println(this.stringOfGuiTree); | ||
| Logger.println("[MonkeySourceApeU2] Precondition XML end ---"); | ||
| return true; |
There was a problem hiding this comment.
sendCurrentPageAsPrecondition logs the entire GUI XML to logs (start/end + full content). This can be extremely large, impact performance, and may leak sensitive user data present in the UI hierarchy. Consider logging only length/hash and (optionally) a truncated prefix under a debug flag.
| return newFixedLengthResponse( | ||
| Response.Status.INTERNAL_ERROR, | ||
| "text/plain", | ||
| "{\"status\":\"error\",\"message\":\"eventSource not available\"}"); | ||
| } |
There was a problem hiding this comment.
The new /propertyFirstSatisfied endpoint returns JSON payloads but uses content-type "text/plain". For clients/proxies that depend on MIME types, this can cause parsing issues. Consider returning "application/json" for these responses.
No description provided.