GH-47583: [CI][Python] Enable PyArrow RelWithDebInfo build with assertions on Windows CI job

[raulcd]

Rationale for this change

Running a PyArrow Windows build with Assertions enhances our coverage, keeping debug information can helps us on future investigations.

What changes are included in this PR?

Run Windows CI job PyArrow build with RelWithDebInfo but remove NDEBUG so assertions are exercised.
We can't use DEBUG directly because otherwise we require linking with CPython debug build (Py_Debug).

Are these changes tested?

Yes on CI, I've validated the flags for building are correct.

Are there any user-facing changes?

No, this is only for our CI job.

• GitHub Issue: [CI][Python] Use Debug build on our Windows Python CI job #47583

[github-actions]

⚠️ GitHub issue #47583 has been automatically assigned in GitHub to PR creator.

[raulcd]

@pitrou the problem seems to be that Debug builds try to link python313_d.lib instead of python313.lib. This seems to be defined upstream on Python itself here (link for 3.16 but 3.13 is the same):
https://github.com/python/cpython/blob/2cd5b79284dd2331cbd9e11afabbfbf7e906103d/PC/pyconfig.h#L340-L342
And it fails with:

 LINK : fatal error LNK1104: cannot open file 'python313_d.lib'

On the Windows CI job we use actions/setup-python which do not allow to install CPython debug. I could try RelWithDebInfo instead of DEBUGotherwise I think we require installing CPython on Debug mode (manually?) which would not allow us to use the setup-python action.
Do you have other ideas?

[pitrou]

Ouch, I had missed that _DEBUG implied Py_DEBUG. This is annoying: we would like our own assertions (and the C++ stdlib's) without needing a special build of CPython :(

[pitrou]

See https://discuss.python.org/t/building-debug-version-of-extensions-on-windows-without-debug-binaries/105012

[pitrou]

According to the discussion above, perhaps we should use RelWithDebInfo (or even plain Release) and undefine the C preprocessor macro NDEBUG.

[pitrou]

I think we require installing CPython on Debug mode (manually?) which would not allow us to use the setup-python action

We can of course do this, assuming we find a readily available debug build of CPython on Windows, but then some dependencies might need to be recompiled from source and we'd rather avoid that.

[raulcd]

This seems to be good, on this PR:

[26/94] "C:\Program Files\Git\usr\bin\ccache.exe" C:\PROGRA1\MICROS2\2022\ENTERP1\VC\Tools\MSVC\14441.352\bin\Hostx64\x64\cl.exe /nologo /TP -DARROW_HAVE_RUNTIME_AVX2 -DARROW_HAVE_RUNTIME_AVX512 -DARROW_HAVE_RUNTIME_BMI2 -DARROW_HAVE_RUNTIME_SSE4_2 -DARROW_HAVE_SSE4_2 -DARROW_PYTHON_EXPORTING -D_CRT_SECURE_NO_WARNINGS -D__SSE2__ -D__SSE4_1__ -D__SSE4_2__ -Darrow_python_EXPORTS -ID:\a\arrow\arrow\python\pyarrow\src -IC:\Users\runneradmin\AppData\Local\Temp\tmpxly05kia\build\pyarrow\src -IC:\hostedtoolcache\windows\Python\3.13.14\x64\Include -I"C:\Program Files\Git\usr\include" -IC:\hostedtoolcache\windows\Python\3.13.14\x64\Lib\site-packages\numpy_core\include /EHsc /wd5105 /bigobj /utf-8 /W3 /wd4800 /wd4996 /wd4065 /arch:SSE4.2 /DWIN32 /D_WINDOWS /EHsc /D_SILENCE_TR1_NAMESPACE_DEPRECATION_WARNING /wd4190 /wd4293 /wd4800 /wd4551 /O2 /Ob1 -std:c++20 -MD -Zi /showIncludes /FoCMakeFiles\arrow_python.dir\pyarrow\src\arrow\python\arrow_to_pandas.cc.obj /FdCMakeFiles\arrow_python.dir\ /FS -c D:\a\arrow\arrow\python\pyarrow\src\arrow\python\arrow_to_pandas.cc

No /DNDEBUG and debug symbols present -Zi.

Previously for Release /DNDEBUG was present and -Zi was not:

[25/94] "C:\Program Files\Git\usr\bin\ccache.exe" C:\PROGRA1\MICROS2\2022\ENTERP1\VC\Tools\MSVC\14441.352\bin\Hostx64\x64\cl.exe /nologo /TP -DARROW_HAVE_RUNTIME_AVX2 -DARROW_HAVE_RUNTIME_AVX512 -DARROW_HAVE_RUNTIME_BMI2 -DARROW_HAVE_RUNTIME_SSE4_2 -DARROW_HAVE_SSE4_2 -DARROW_PYTHON_EXPORTING -D_CRT_SECURE_NO_WARNINGS -D__SSE2__ -D__SSE4_1__ -D__SSE4_2__ -Darrow_python_EXPORTS -ID:\a\arrow\arrow\python\pyarrow\src -IC:\Users\runneradmin\AppData\Local\Temp\tmpgjv_4rcf\build\pyarrow\src -IC:\hostedtoolcache\windows\Python\3.13.14\x64\Include -I"C:\Program Files\Git\usr\include" -IC:\hostedtoolcache\windows\Python\3.13.14\x64\Lib\site-packages\numpy_core\include /EHsc /wd5105 /bigobj /utf-8 /W3 /wd4800 /wd4996 /wd4065 /arch:SSE4.2 /DWIN32 /D_WINDOWS /EHsc /D_SILENCE_TR1_NAMESPACE_DEPRECATION_WARNING /wd4190 /wd4293 /wd4800 /wd4551 /O2 /Ob2 /DNDEBUG -std:c++20 -MD /showIncludes /FoCMakeFiles\arrow_python.dir\pyarrow\src\arrow\python\arrow_to_pandas.cc.obj /FdCMakeFiles\arrow_python.dir\ /FS -c D:\a\arrow\arrow\python\pyarrow\src\arrow\python\arrow_to_pandas.cc

[Copilot]

Pull request overview

This PR updates the Windows PyArrow CI build configuration to use RelWithDebInfo while ensuring assertions remain enabled (avoiding a full Debug build that would require linking against CPython debug libraries).

Changes:

• Switch Windows C++ build type from Release to RelWithDebInfo.
• Override MSVC RelWithDebInfo compile flags to avoid defining NDEBUG (so assertions are exercised).
• Pass the same CMake build type and flag overrides through the pip install build configuration for PyArrow.