Repository navigation
Add portable IoTDB ODBC driver with Session TLS and application compatibility tests - #18878
hongzhi-gao wants to merge 17 commits into
Conversation
# Conflicts: # .github/workflows/multi-language-client.yml
Caideyipi
left a comment
There was a problem hiding this comment.
I found the following correctness and robustness issues in the ODBC implementation:
-
[P1]
SQLDescribeColcan dereference a nullDataTypePtr(driver.cpp:1664)DataTypePtris optional, but the column-size path unconditionally evaluates*dataType. For example,SQLDescribeCol(stmt, 1, nullptr, 0, nullptr, nullptr, &columnSize, nullptr, nullptr)causes an access violation. Please obtain the type in a local variable whenever the column size is requested, or avoid dereferencing the optional pointer. -
[P1]
SQLColAttributewrites the SQL type through the wrong pointer type (driver.cpp:1078)numericAttributeis anSQLLEN*, but it is reinterpreted as anSQLSMALLINT*before callinggetColType. On 64-bit builds this returns negative type codes as large positive values (for example,SQL_WVARCHARbecomes65527instead of-9). Please write to anSQLSMALLINTlocal and then assign that value to theSQLLENoutput. -
[P1/P2]
SQL_C_NUMERICsilently truncates floating-point values (ODBCField.cpp:761-770)The FLOAT/DOUBLE conversion stores only the integer part, forces
scaleto zero, and still returns success. A value such as1.5is therefore returned as1; setting a descriptor scale does not preserve the fractional part either. Please honor the requested precision/scale or reject unsupported conversions, and report the required fractional-truncation diagnostic (01S07) when applicable. -
[P1/P2]
INT64_MINcannot be converted correctly toSQL_C_NUMERIC(ODBCField.cpp:738)std::abs(INT64_MIN)cannot be represented as anint64_t; the current loop consequently leaves the numeric value as zero while returning success. Please calculate the magnitude in an unsigned type or handleINT64_MINexplicitly. -
[P2] Disconnect/free can leave child handles with dangling connection pointers (driver.cpp:1707, driver.cpp:2809, StatementHandle.h:104-108)
SQLDisconnectonly closes the session, andSQLFreeHandle(SQL_HANDLE_DBC, ...)deletes the connection while statements and implicit descriptors still retain raw pointers to it. A sequence such as allocating a statement, disconnecting, freeing the DBC, and then freeing the statement makes the statement destructor access freed memory. Please centrally close/delete/detach all child statements and descriptors before disconnecting or freeing the connection. -
[P2]
SQLColAttribute(SQL_DESC_LABEL)does not support length-only calls (driver.cpp:1037-1044)getColNameis called only whencharacterAttributeis non-null. A valid length-only call withCharacterAttributePtr == nullptrtherefore leavesStringLengthPtrat zero instead of returning the label length. Please callgetColNameunconditionally forSQL_DESC_LABEL. -
[P2] A negative
bufferLengthcan cause an invalidmemset(driver.cpp:1020)The output buffer is cleared before validating
bufferLength. Passing-1with a non-null character buffer converts that value to a hugesize_tformemsetand causes an access violation. Please validate negative lengths first and returnHY090without touching the buffer. -
[P2] Column-name truncation returns
SQL_SUCCESS(driver.cpp:471-485)getColNamelogs truncation but always returnsSQL_SUCCESS. ODBC callers needSQL_SUCCESS_WITH_INFOplus the01004truncation diagnostic so thatSQLDescribeColandSQLColAttributecan report the shortened result correctly. -
[P2] Tree-model
SQL_IDENTIFIER_QUOTE_CHARreturns a space (driver.cpp:3948-3955)The comment says the tree model uses backticks, but the implementation returns
" ". This makes metadata-driven SQL generators quote tree identifiers incorrectly. Please return`for the tree model.
I reproduced the crashes and metadata/value mismatches with direct ABI probes against the Windows driver artifact, and the existing ODBC compatibility and parameter tests do not cover these cases.
|
@Caideyipi Addressed the nine items from your review in d227562:
Validation completed locally: Windows and manylinux CTest 4/4 each, including new unit regressions and direct exported-ABI probes against the built DLL/shared library; Windows/Linux CPack; Linux GLIBC <= 2.28/dependency checks; Spotless and RAT. A real disposable server also passed the plain-mode Session checks plus Windows System.Data.Odbc and MSDASQL/ADO consumers. This is not a claim of an Excel/Power Query UI run. The fork package/TLS/application CI and upstream PR ODBC workflow now both pass all eight jobs for d227562, including extracted-package real-server TLS/mTLS and Windows .NET/ADO tests. The earlier DataNode baseline failures are now fixed upstream by a7b88b1 (master Unit-Test run: all four jobs passed). Integrated that baseline in merge commit 112f064; the ODBC/C++ SDK sources and native workflows are unchanged by this merge. The unchanged driver passed local Windows CTest 4/4 again. Fork ODBC CI and upstream ODBC CI now both pass all eight jobs for 112f064 against the updated server baseline, including extracted-package TLS/mTLS and Windows .NET/ADO consumers. All current upstream PR checks have now completed: 54 passed and two optional analysis jobs (codecov/sonar) were skipped, with no failures or pending jobs. This includes the fresh ARM C++ build, Java tests, and AINode integration checks. No unrelated Java fixes were authored in this PR. |
Description
Add a portable ODBC client for IoTDB's tree and table models, backed by the in-tree C++ Session SDK. The default transport is Session RPC, including verified TLS and mutual TLS; the existing HTTP REST transport remains available through
RESTFUL=1.Driver behavior and application compatibility
The driver provides the ODBC 3.8 ABI, forward-only/read-only cursors, column and rowset binding, explicit/implicit descriptors, prepared input parameters, parameter arrays, data-at-execution input, diagnostics, and capability discovery. Unsupported transactions, stored procedures, output parameters, bookmarks and asynchronous execution return explicit SQLSTATEs and are not advertised as supported.
The implementation includes the following compatibility fixes:
Wentry points for connection strings, SQL, catalog arguments, descriptions, descriptors and diagnostics. STRING/TEXT metadata usesSQL_WVARCHAR/SQL_WLONGVARCHAR, so .NET and MSDASQL/ADO request UTF-16 data instead of interpreting UTF-8 with the Windows ANSI code page.timeprojections and dotted aliases in table queries. Correct DATE sizes, legacy numeric C-type binding used by ADO, empty-string reads, type-info cursor reuse, and type descriptions for empty/all-NULL catalog results.expressions, retain declared REST types, convert DATE/BLOB values, and keep non-query responses separate from result sets.TLS configuration accepts
SSL,SSLCA,SSLCERTandSSLKEYin connection strings, DSNs and Windows setup. CA verification and server identity checks are enabled; incomplete client credentials, certificate options with SSL disabled and TLS options combined with REST are rejected. Connection strings support braced values and escaped closing braces.SQL_C_NUMERIC output currently supports exact integer/epoch values with precision/scale handling; FLOAT/DOUBLE conversion returns 07006 without changing the output. The README documents defaults and supported scale limits. Disconnect/free centrally drops connection-owned statement and descriptor resources before closing the Session; stale cleanup calls are checked against the live-handle registry.
Native build and packaging
with-odbcenables the Maven module; CMake/CPack can also build it directly. Release scope is Windows x64 and Linux x86_64 with a glibc 2.28 baseline. The driver links Session, Thrift, OpenSSL and curl statically, ships dependency licenses/NOTICE and build provenance, and does not require a separate C++ SDK, OpenSSL installation or VC redistributable. OS ODBC manager/installer libraries remain prerequisites.The shared C++ build adds scoped static-link/CRT options for ODBC while retaining standalone SDK defaults. Linux releases use the shared manylinux/GCC baseline and check imported libraries and maximum GLIBC symbol versions. Windows packages check the PE architecture and imported DLLs. Windows source builds use valid generated batch-file line endings and also reject negative process exit codes. SDK Windows packaging uses Boost source headers rather than toolchain-specific binary installers.
GNU/Linux binaries incorporate GCC runtime object code under GPL-3.0-or-later WITH GCC-exception-3.1. No GCC runtime source or GPL license text is vendored in this source tree. Only GNU/Linux install/packaging obtains the upstream notices, verifies pinned SHA-256 hashes, and includes them under licenses/gcc with a binary LICENSE disclosure. Windows/MSVC packages contain neither notice. The README documents the ASF GCC Runtime Library exception policy and explicitly leaves the required inclusion review to the project PMC.
Validation
Current head 112f064 integrates upstream master a7b88b1, including the deletion-test flush-control fix; master Unit-Test CI passed all four jobs. The merge leaves ODBC/C++ SDK source and native workflows unchanged. Fork package/server CI and upstream ODBC CI both passed all eight jobs for this updated baseline, including real-server TLS/mTLS and Windows .NET/ADO consumers. All current upstream PR checks have completed: 54 passed, two optional analysis jobs (codecov/sonar) were skipped, and none failed or remain pending. The successful checks include C++ ARM packaging, Java tests, and AINode integration. The completed d227562 runs below verify the same native driver before this server-baseline update.
mvn clean package -pl distribution -am -DskipTests.System.Data.Odbcand MSDASQL/ADO consumers: Unicode SQL/data, parameterized inserts, long text/emoji, NULL/empty strings, DATE/BOOLEAN, catalog browsing for both models, empty searches, and database changes. Tests use unique disposable databases and user DSNs, preserve normal registry-parent behavior, and stop only processes they launched.Office verification scope: .NET and Office's MSDASQL/ADO data path were exercised. A Microsoft Excel/Power Query UI run has not been performed: the local
Excel.Applicationregistration points to WPS, and hosted CI runners do not include Excel. The README includes 64-bit Excel installation/import/refresh instructions and records this distinction. TIMESTAMP query columns retain epoch-integer semantics; DATE usesSQL_TYPE_DATE.This PR has:
Key packages:
iotdb-client/odbc, shared native build configuration underiotdb-client/client-cpp/cmake, and native package workflows/scripts.