Try windows-latest gha platform - #16
Open
GregThain wants to merge 602 commits into
Open
Conversation
Fixup add_dependencies Benchmark process start
The condor-C tests (job_condorc_ab_van, job_condorc_abc_van) time out because the execution pool's condor_shadow.exe fast-fails on startup with STATUS_STACK_BUFFER_OVERRUN (0xC0000409) right after claim activation, so the job can never run. The fast-fail leaves no backtrace, so build with ASan to capture a symbolized stack-buffer-overflow report. Make the WITH_ADDRESS_SANITIZER cmake block MSVC-aware: use /fsanitize=address (auto-links the ASan runtime), strip the incompatible /RTC runtime checks from the Debug flags, and force /INCREMENTAL:NO. In the Windows CI workflow, enable -DWITH_ADDRESS_SANITIZER and scope the ctest run to just the two condor-C tests so the diagnostic run is fast and doesn't newly time out unrelated tests under ASan instrumentation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Back out the diagnostic-only workflow change that built with ASan and scoped ctest to the two condor-C tests. Restores the normal Windows CI run: full test suite, no sanitizer. The MSVC ASan support added to CondorConfigure.cmake is left in place, dormant (WITH_ADDRESS_SANITIZER is off), for future ASan runs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…t be sent OsProc::ShutdownGraceful() retried the soft kill after a blocking sleep(1) to paper over a Windows startup race (HTCONDOR-1093): a job removed just after it starts may not have created its window yet, so condor_softkill returns SOFTKILL_WINDOW_NOT_FOUND on the first try. That retry was a bad trade: sleep(1) blocks the DaemonCore event loop, the one-second delay is an arbitrary guess, and -- as the cmd_q_shows-sub failures show -- a delivered-but-ignored WM_CLOSE makes the retry report success while the job keeps running until the startd's distant kill timeout, so the test still hangs. Drop the retry and the sleep. If the graceful kill signal cannot be sent, escalate to a hard kill immediately via ShutdownFast() (virtual, so VanillaProc tears down the whole process family via Kill_Family). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Diagnostic change. The proprietary NMI build ships Release; GHA builds Debug, which enables the MSVC debug CRT and _ITERATOR_DEBUG_LEVEL=2 (checked iterators) plus /RTC. Those turn latent UB (e.g. invalidated iterators, invalid CRT args) into __fastfail aborts that surface as exit 0xC0000409 -- the crash signature seen in the condor_shadow and gridmanager failures that only reproduce in GHA. Switch the Windows build and ctest to RelWithDebInfo (Release semantics, _ITERATOR_DEBUG_LEVEL=0, but keeps PDBs) to partition the failures: any that disappear here are Debug-only latent-UB surfacing, matching what NMI actually ships. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…a WER Undo the RelWithDebInfo diagnostic and return to the Debug config. Add a Windows Error Reporting LocalDumps registry entry for condor_shadow.exe so the 0xC0000409 fast-fail in the condor-C tests produces a full post-mortem dump. The fastfail bypasses DaemonCore's own exception handler (no core is written), so WER is the way to capture it. Dumps are written under __build so the existing "Upload All on Failure" step ships them alongside the PDBs already copied into release_dir/bin. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Turn on -DWITH_ADDRESS_SANITIZER for the Windows build and run all ctests (no -R filter) under ASan. Lower test parallelism from -j 20 to -j 10 since ASan increases per-process memory use. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mirror ci-asan.yml so the Windows ASan debugging runs fire on push to the stress-test branch, not only on pull_request. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Swap the cmake config back to the non-ASan line (ASan line kept commented for easy toggling). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ure) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…down On Windows, ~SharedPortEndpoint()->StopListener() only waited up to 2 seconds for the named-pipe listener thread to exit and then returned regardless, letting ~DaemonCore free the object while the listener thread was still running. The thread re-reads member state (pipe_end) at the top of its loop via ConnectNamedPipe(), producing a heap-use-after-free that ASan flags as fatal (and is otherwise a latent race). This crashed condor_shadow.exe on essentially every job teardown, wedging tests such as job_condorc_ab_van into an endless evict/reschedule loop until they timed out. StopListener() now guarantees the listener thread has terminated before returning: it retries the wake-and-wait a few times and, as a last resort, TerminateThread()s and joins it so the thread can never touch the freed object. PipeListenerThread() also now honors thread_should_exit on the ConnectNamedPipe() failure path, so a persistently failing connect no longer spins the loop past the exit check (which is what made the old 2-second wait time out in the first place). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Turn WITH_ADDRESS_SANITIZER back on in the Windows cmake configuration step and drop ctest parallelism from -j 20 to -j 10 to reduce the memory pressure that produced allocator-OOM asan reports during the run. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The hosted Windows runner's machine name has no DNS record and no domain, so condor's FULL_HOSTNAME is a bare name that getaddrinfo() cannot resolve. Tools that resolve that name -- e.g. condor_drain in test_drain_policies -- fail with "unknown host <name>", erroring three of that test's cases (the proprietary CI passes because its DNS resolves the machine name). Add a hosts-file entry (with an FQDN) before the ctests step so the name resolves locally. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On Windows the job_dagman_abort-B .run script writes node .cmd files that use echo.bat and makeexit.bat as the executable, but neither bat was listed in the test's DEPENDS, so CMake never copied them into the run directory and the test failed. Add both to DEPENDS. They are harmless on Unix/Mac, which take the /bin/echo branch and never reference the bats. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
job_dagman_vars_prepend, lib_eventlog, and job_dagman_always_run_post-D run appendmsg.exe as their (vanilla-universe) job executable. That helper is built into the src/condor_tests build directory, and condor_submit resolves a relative executable against the job's Iwd, not $PATH. The number of directory levels between the job's Iwd and the build dir differs between harnesses -- batlab runs flat in src/condor_tests, while ctest adds an extra <test>_ctest dir -- so no single hardcoded relative path works in both, and the exe was unreachable under ctest (condor_submit "Can't access executable file appendmsg.exe", or the node job held with "Bad exe type"). Instead of hardcoding a path, copy appendmsg.exe into the job's Iwd from whichever parent level actually has it, and submit it by bare name. The copy lives in the Windows-only branch, so non-Windows platforms still use /bin/echo (or /bin/date) and appendmsg.exe is never named in CMake DEPENDS -- avoiding a missing-file break on Linux/Mac, where the target is not built. vars_prepend chdirs into per-check subdirs, so it copies into each check_N; the other two run in the run dir itself. always_run_post-D_A.cmd now names appendmsg.exe explicitly so the vanilla submit-time check matches the copied file. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
job_dagman_prepost, job_dagman_maxpostscripts, and job_dagman_maxprescripts generate Perl PRE/POST scripts that shell out to appendmsg.exe via system() on Windows. That helper is built into the src/condor_tests build directory, and DAGMan runs the PRE/POST scripts with the run dir as their cwd, where Windows resolves the bare command name. That build dir is the run dir under the batlab harness (so it worked there) but the parent dir under ctest, which adds an extra <test>_ctest level -- so appendmsg.exe was not in the scripts' cwd and the system() calls failed. Copy appendmsg.exe into the run dir from whichever parent level actually has it, matching the fix already applied to the tests that run it as a job executable. The copy is Windows-only and appendmsg.exe is never named in CMake DEPENDS, so non-Windows platforms (which use x_general_client.pl) are unaffected and the not-built-there target cannot break their configure. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Temporarily swap the Windows GHA test job off ASan and simplify the ctests step to a plain -j 20 run, to confirm the appendmsg.exe reachability fixes work independently of the ASan configuration. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ornithology tests submit sleep.cmd (generated from sleep.py) as the job executable. On the Windows runner these jobs went "executing" but never "terminated" -- ornithology wait() timed out in test_startup_limits, test_activation_metrics, test_dont_queue_small_sandboxes, test_spool_preserved_relative_paths, test_dagman_inline_desc, and others -- with empty job stdout. The old .cmd/.py polyglot prolog relied on `goto :EOF` to stop cmd.exe before the python body. When cmd fell through instead, it executed the body's `time.sleep(1)` line as the cmd built-in `time`, which prompts and blocks reading stdin (the starter's pipe), so the job hung until it was torn down. Replace the prolog with a single launcher line that runs python with -x (skip-first-line, the interpreter flag intended for batch/python polyglots) on the same file and then `exit /b`, so cmd never reaches the python body and fall-through is impossible. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The earlier hosts-file step (commit 85fc458) still left condor_drain failing with "unknown host <name>" -- getaddrinfo() never resolved the name, so the entry had no effect. Add-Content inserts no leading newline, and the stock hosts file's last line is a comment with no trailing newline, so the appended entry got glued onto that comment line and was ignored (the step still passed green). Harden the step: ensure the file ends in a newline before appending, write the entries on their own lines as ASCII (both name.localdomain/name and the bare name), flush the DNS cache, and then verify resolution with System.Net.Dns.GetHostAddresses -- failing the job if the name still does not resolve, so this surfaces here instead of as opaque "unknown host" failures deep in the test suite (test_drain_policies, cmd_drain, cmd_drain_scavenging_vacation, job_router_xform). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The push of d27f506 at 2026-08-06T22:29:08Z was received by GitHub (logged as a PushEvent; Travis and Advanced Security both reacted) but GitHub Actions created no check-suite for the stress-test push, so the CI Build and Test / ASan workflows never ran. Re-triggering. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…unner The hosted runner's machine name is bare and domain-less. condor's get_fqdn_from_hostname() only returns an FQDN when the resolved canonical name contains a dot (or DEFAULT_DOMAIN_NAME is set), so the drain tests' condor_drain <hostname> died with "unknown host <name>" before ever contacting the startd -- failing test_drain_policies, cmd_drain_scavenging_vacation, and timing out cmd_drain. The existing hosts-file entry is insufficient: its resolve check only requires the name to map to an address, not to yield a dotted canonical name. Setting _CONDOR_DEFAULT_DOMAIN_NAME=localdomain for the ctest step makes FULL_HOSTNAME <name>.localdomain (already dotted, so no DNS lookup needed, and matching the .localdomain hosts alias) for every condor tool the tests spawn. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On Windows several perl tests submit jobs whose executable is "sleep.exe" (with x_sleep.pl as the non-Windows fallback), e.g. cmd_status-transform and job_router_xform. sleep.exe/sleepw.exe are build artifacts produced into TEST_TARGET_DIR (the parent of the per-test rundir), not source files, so they can't be listed in the cross-platform DEPENDS and were never copied into the rundir. condor_submit then failed with "Can't access executable file sleep.exe: No such file or directory", no jobs ran, and the tests' output .ads files were never produced. Copy sleep.exe/sleepw.exe from rundir_parent into the rundir when they exist, mirroring how the perl helper modules are staged. No-op on platforms where those executables are not built. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ccb_proxy_bench.cpp is a fork()/poll() program with no Windows port; on WIN32 it compiles to a stub that prints "ccb_proxy_bench is not supported on this platform" and exits 1. The test was registered just outside the existing `if (NOT WINDOWS)` guard that already excludes the sibling test_ccb_streaming tests, so it ran on Windows and always failed. Move it inside that guard. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
<Insert PR description here, and leave checklist below for code review.>
HTCondor Pull Request Checklist for internal reviewers
After the above