Opensfm 370 - #2057
Draft
DodgySpaniard wants to merge 83 commits into
Draft
Conversation
A devcontainer is a small wrapper around Docker to make it easy to set up a clean development environment. Most IDE's and code editors support these now as well as many cloud environments. I've included a very simple environment to start with which we can extend as necessary later to streamline the build process.
This diff adds the pixi file and updates some SuperBuild scripts to compile. With this we can run `pixi install` and `pixi run build` and the compilation works.
Output report from before and after pixi are almost identical.
roughly a 6 minute build now
Ship the prod pixi conda env (.pixi/envs/prod) + SuperBuild/install + source instead of the old venv + embedded Python. win32env.bat now derives CONDA_PREFIX from the install dir and runs the env's conda activate.d scripts, so the bundle relocates to any install path without a fix-up step or pixi on the target. - innosetup.iss: bundle .pixi/envs/prod and all of SuperBuild/install; install winrun.bat as run.bat; drop winpostinstall. - scripts/dist.py + `pixi run dist`: port the installer-build half of configure.py (Inno Setup + VC redist download, CUDA runtime bundling for GPU SIFT parity, Azure signing, iscc). - publish-windows.yml: build/package via pixi (gpu env), add workflow_dispatch and a relocated-install smoke test. - Remove obsolete configure.py, vcpkg.json, vcpkg-requirements.txt, build_vcpkg_env.yaml, winpostinstall.bat.
The pixi migration deleted configure.sh and requirements.txt, which this macOS installer script invoked; native macOS builds now go through pixi (pixi install && pixi run build).
console.bat, win32env.bat and winrun.bat now live in scripts/ alongside
the other helper scripts. The installer still places them at the app root
(DestDir {app}), so the installed layout and their runtime cross-references
are unchanged.
PoissonRecon builds from a hand-written Makefile via `make`, which the minimal pixi container image does not provide (unlike the CI runners, which have a system make on PATH). Without it the Docker build failed at the poissonrecon step with 'make: command not found'.
This is a clunky solution but give we will want to upstream all of these patches anyway it seems ok for now. It means we can keep them as .patch files making that transition easier. In the future, we probably want to relook at how we handle these external cmake projects but for now I'm avoiding changing too much.
Shapely 2 no longer has `__getitem__` for MultiPolygon, so unary_union results were written whole instead of picking the largest polygon. Commit 75a1e1f only fixed one half of that mismatch by changing the schema to MultiPolygon; Fiona 1.9+ still rejects Polygon records against a MultiPolygon schema. Use .geoms to select the largest polygon and declare the GPKG layer as Polygon again.
It seems like we weren't setting the RPATH on MacOS correctly. This change should bake it into the binary matching how it works on Linux.
Windows uses a fundamentally different kind of dynamic library loading so we can't easily check it in smoke.py. Instead lets drop it for now.
Declare the standard OCI labels (revision, source, version, created) in the runtime stage of Dockerfile and gpu.Dockerfile via build args defaulting to unknown, and inject the values from the publish workflows so published images can be mapped back to their source commit.
Add the static title, description, url, documentation, licenses and vendor OCI image labels to the runtime stage of Dockerfile and gpu.Dockerfile.
Add a docker-build pixi task and scripts/docker-build.py that builds the runtime image with the GIT_COMMIT, BUILD_DATE and IMAGE_VERSION build args derived from the working tree, so local images carry the same OCI labels as CI. Point the README build instructions at the task.
Default the docker-build image tag to opendronemap/odm:<branch> when building off master (sanitising the branch name for Docker), keeping :latest and :gpu on master, so local branch builds don't clobber the latest tag.
- opendm/remote.py and stages/splitmerge.py, on windows this would fetch the host python env, patching to force the same interpreter. - opendm/osfm.py - we do not need win.bat anymore.
…uracy opensfm no longer supports planar algo, and applies anisotropic GPS weighting
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Current attempt to use latest our ODM/OpenSFM:370 which is synced with OpenSfM/OpenSfM:v1 in addition to our ODM changes.
This branch is sitting on top of pixi and pixi-windows-fix includes commits from Sam. It is currently a WIP, but should contain all require.
Disclaimer: AI was used in a tool capacity to speed up research. (As usual worth mentioning that the use of this tool has been rather helpful to handle all this random build system errors and likely save many hours of stack overflow scouting).