Skip to content

[DISCUSSION] import sqlpp23 to support sqlcatalog #135

Closed

Description

@zhjwpku

SQLCatalog can be a valuable component for users to test the functionality of iceberg-cpp, as it can maintain some state persisted in a database. Initially, we didn't implement the SQLCatalog because we were not aware of a suitable C++ ORM framework. However, I recently came across sqlpp23 [1], a domain-specific language for SQL queries, which seems like a good fit for implementing SQLCatalog.

One consideration with sqlpp23 is that it requires C++23, whereas iceberg-cpp currently targets C++20. There is an alternative, sqlpp11 [2], from the same author, required C++11, however, according to its README, the author recommends migrating to sqlpp23. Therefore, I'd like to propose updating iceberg-cpp to C++23.

One immediate benefit of moving to C++23 is that we can leverage std::expected and eliminate the dependency on zeus-cpp/expected.

I've reviewed some potential users of iceberg-cpp: ClickHouse requires C++23, whereas DuckDB requires C++11 🤪

I'd like to hear your thoughts on two points:

  • Should we implement support for SQLCatalog?
  • Should we update the project to require C++23?

What are your opinions? @wgtmac @lidavidm @gty404 @mapleFU @Fokko @lishuxu

[1] https://github.com/rbock/sqlpp23
[2] https://github.com/rbock/sqlpp11

Activity

  1. lidavidm commented on Jul 3, 2025

    @lidavidm
    Member

    I've used sqlpp11 before; it worked OK, but occasionally it was difficult to figure out the exact right syntax it wanted, and some things I ended up having to embed raw SQL. IMO, we'd have to evaluate whether the kinds of queries we want to write would actually work with the ORM, because if we do have to embed raw SQL, then I'd question why we don't just deal with raw SQL directly.

    For DuckDB, this would end up as an extension right? Do extensions have to use the same C++ version as DuckDB?

  2. wgtmac commented on Jul 3, 2025

    @wgtmac
    Member

    I'm not sure if sqlpp23 has solved the issue that you've experienced with sqlpp11. Perhaps AI can make it easier these days? If we end up with writing raw SQL, ORM framework still help the integration with some popular database backends, right?

    I'm not sure whether DuckDB will pick up iceberg-cpp or continue their development in the Iceberg extension. I think that iceberg-cpp still takes a long time to achieve the 1.0.0 release so bumping to C++23 is not a bad idea.

  3. lidavidm commented on Jul 3, 2025

    @lidavidm
    Member

    It's not clear to me that there is enough training data for the library. At least I don't think Copilot back then was able to give useful completions. I didn't try with Claude Code or similar.

    It might help handle just wrangling different database clients. On the other hand we should be sure that it actually supports all the backends we want.

  4. zhjwpku commented on Jul 3, 2025

    @zhjwpku
    MemberAuthor

    I've used sqlpp11 before; it worked OK, but occasionally it was difficult to figure out the exact right syntax it wanted, and some things I ended up having to embed raw SQL. IMO, we'd have to evaluate whether the kinds of queries we want to write would actually work with the ORM, because if we do have to embed raw SQL, then I'd question why we don't just deal with raw SQL directly.

    I just took a look at iceberg-rust's SQL catalog implementation, it utilizes sqlx to support multiple backends. The use cases appear to be quite straightforward, just some prepared statements and bind execution.

    I also checked sqlpp23, it supports mysql/postgres/sqlite with prepared statements and bind, which seems sufficient.

    For DuckDB, this would end up as an extension right? Do extensions have to use the same C++ version as DuckDB?

    Not sure, duckdb extension template doesn't specify cxx standard[1], so I guess it's ok to use a different C++ version.

    [1] https://github.com/duckdb/extension-template/blob/main/CMakeLists.txt

  5. zhjwpku commented on Jul 3, 2025

    @zhjwpku
    MemberAuthor

    I'm not sure if sqlpp23 has solved the issue that you've experienced with sqlpp11. Perhaps AI can make it easier these days? If we end up with writing raw SQL, ORM framework still help the integration with some popular database backends, right?

    I'm not sure whether DuckDB will pick up iceberg-cpp or continue their development in the Iceberg extension. I think that iceberg-cpp still takes a long time to achieve the 1.0.0 release so bumping to C++23 is not a bad idea.

    Ok, I will create a PR to bump to C++23 this weekend if there are no objections.

  6. lidavidm commented on Jul 3, 2025

    @lidavidm
    Member

    Ok, sounds good to me. Thanks for checking.

  7. self-assigned this
    on Jul 8, 2025
  8. zhjwpku commented on Jun 5, 2026

    @zhjwpku
    MemberAuthor

    closed via #693

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions