Skip to content

Conversation

@JoshuaSBrown
Copy link
Collaborator

@JoshuaSBrown JoshuaSBrown commented Jan 29, 2026

Ticket

#1835

Description

Tasks

  • - A description of the PR has been provided, and a diagram included if it is a new feature.
  • - Formatter has been run
  • - CHANGELOG comment has been added
  • - Labels have been assigned to the pr
  • - A reviwer has been added
  • - A user has been assigned to work on the pr
  • - If new feature a unit test has been added

Summary by Sourcery

Introduce a new proto3-based common protocol module and library for DataFed.

New Features:

  • Add a proto3 common protocol directory structure with enums, messages, auth, anon, and envelope definition files.
  • Add a CMake target to generate C++ code from proto3 files and build the datafed-protobuf library.

@JoshuaSBrown JoshuaSBrown self-assigned this Jan 29, 2026
@sourcery-ai
Copy link
Contributor

sourcery-ai bot commented Jan 29, 2026

Reviewer's Guide

Introduces a new proto3-based common protocol module by adding a dedicated CMake target that compiles all proto3 definitions into a C++ library, along with the full set of protobuf3 message, enum, and request/response definition files organized under anon, auth, messages, and enums namespaces.

Class diagram for proto3 protocol module organization

classDiagram
    class common_proto3_common {
    }

    class enums {
        <<namespace>>
        access_token_type
        dependency_dir
        dependency_type
        encryption
        error_code
        execution_method
        note_state
        note_type
        project_role
        search_mode
        service_status
        sort_option
        task_command
        task_status
        task_type
    }

    class messages {
        <<namespace>>
        acl_rule
        alloc_data
        alloc_stats_data
        coll_data
        dependency_data
        dependency_spec_data
        group_data
        listing_data
        note_comment
        note_data
        path_data
        project_data
        record_data
        record_data_location
        record_data_size
        repo_data
        repo_record_data_locations
        schema_data
        tag_data
        task_data
        topic_data
        user_data
    }

    class anon {
        <<namespace>>
        ack_reply
        auth_status_reply
        authenticate_by_password_request
        authenticate_by_token_request
        daily_message_reply
        daily_message_request
        get_auth_status_request
        nack_reply
        version_reply
        version_request
    }

    class auth {
        <<namespace>>
        acl_data_reply
        acl_shared_list_items_request
        acl_shared_list_request
        acl_update_request
        acl_view_request
        check_perms_reply
        check_perms_request
        coll_create_request
        coll_data_reply
        coll_delete_request
        coll_get_offset_reply
        coll_get_offset_request
        coll_get_parents_request
        coll_list_published_request
        coll_move_request
        coll_path_reply
        coll_read_request
        coll_update_request
        coll_view_request
        coll_write_request
        data_delete_request
        data_get_reply
        data_get_request
        data_path_reply
        data_path_request
        data_put_reply
        data_put_request
        generate_credentials_reply
        generate_credentials_request
        get_perms_reply
        get_perms_request
        group_create_request
        group_data_reply
        group_delete_request
        group_list_request
        group_update_request
        group_view_request
        listing_reply
        metadata_validate_reply
        metadata_validate_request
        note_comment_edit_request
        note_create_request
        note_data_reply
        note_list_by_subject_request
        note_update_request
        note_view_request
        project_create_request
        project_data_reply
        project_delete_request
        project_get_role_reply
        project_get_role_request
        project_list_request
        project_search_request
        project_update_request
        project_view_request
        query_create_request
        query_data_reply
        query_delete_request
        query_exec_request
        query_list_request
        query_update_request
        query_view_request
        record_alloc_change_reply
        record_alloc_change_request
        record_create_batch_request
        record_create_request
        record_data_reply
        record_delete_request
        record_export_reply
        record_export_request
        record_get_dependency_graph_request
        record_list_by_alloc_request
        record_lock_request
        record_owner_change_reply
        record_owner_change_request
        record_update_batch_request
        record_update_request
        record_view_request
        repo_allocation_create_request
        repo_allocation_create_response
        repo_allocation_delete_request
        repo_allocation_set_default_request
        repo_allocation_set_request
        repo_allocation_stats_reply
        repo_allocation_stats_request
        repo_allocations_reply
        repo_authz_request
        repo_calc_size_reply
        repo_calc_size_request
        repo_create_request
        repo_data_delete_request
        repo_data_get_size_request
        repo_data_reply
        repo_data_size_reply
        repo_delete_request
        repo_list_allocations_request
        repo_list_object_allocations_request
        repo_list_request
        repo_list_subject_allocations_request
        repo_path_create_request
        repo_path_delete_request
        repo_update_request
        repo_view_allocation_request
        repo_view_request
        revoke_credentials_request
        schema_create_request
        schema_data_reply
        schema_delete_request
        schema_revise_request
        schema_search_request
        schema_update_request
        schema_view_request
        search_request
        tag_data_reply
        tag_list_by_count_request
        tag_search_request
        task_data_reply
        task_list_request
        task_view_request
        topic_data_reply
        topic_list_topics_request
        topic_search_request
        topic_view_request
        user_access_token_reply
        user_create_request
        user_data_reply
        user_find_by_name_uid_request
        user_find_by_uuids_request
        user_get_access_token_request
        user_get_recent_ep_reply
        user_get_recent_ep_request
        user_list_all_request
        user_list_collab_request
        user_set_access_token_request
        user_set_recent_ep_request
        user_update_request
        user_view_request
    }

    class envelope {
        <<message>>
        envelope
    }

    common_proto3_common o-- enums
    common_proto3_common o-- messages
    common_proto3_common o-- anon
    common_proto3_common o-- auth
    common_proto3_common o-- envelope
Loading

File-Level Changes

Change Details Files
Add CMake build configuration to generate a C++ protobuf library from the new proto3 schema set.
  • Collect all proto3 files from enums, messages, anon, auth, and the envelope file using GLOB_RECURSE.
  • Invoke protobuf_generate with the new proto3 files and current source directory as import path, capturing generated outputs.
  • Create a shared/static datafed-protobuf library from the generated C++ sources, honoring BUILD_SHARED_LIBS and common protocol version variables.
  • Set target properties including PIC, SOVERSION, and VERSION, and link against protobuf::libprotobuf.
  • Expose include directories for generated headers from the binary dir and an interface include pointing at the project common/proto output.
common/proto3/common/CMakeLists.txt
Introduce the proto3 replacement schema set for the common protocol, structured into anon, auth, messages, enums, and envelope definitions.
  • Add anon request/response proto files for unauthenticated operations such as auth status, authentication, daily messages, version, and simple acks/nacks.
  • Add auth request/response proto files covering ACLs, collections, data operations, credentials, permissions, groups, notes, projects, queries, records, repositories, schemas, search, tags, tasks, topics, and user management.
  • Add enums proto files defining shared enum types such as access tokens, dependency characteristics, encryption, error codes, execution methods, note attributes, project roles, search modes, service status, sort options, and task attributes.
  • Add messages proto files for shared data structures such as ACL rules, allocations, collections, dependencies, groups, listings, notes, paths, projects, records, repositories, schemas, tags, tasks, topics, and users.
  • Add a top-level envelope.proto that likely wraps request/response messages for transport.
common/proto3/common/anon/ack_reply.proto
common/proto3/common/anon/auth_status_reply.proto
common/proto3/common/anon/authenticate_by_password_request.proto
common/proto3/common/anon/authenticate_by_token_request.proto
common/proto3/common/anon/daily_message_reply.proto
common/proto3/common/anon/daily_message_request.proto
common/proto3/common/anon/get_auth_status_request.proto
common/proto3/common/anon/nack_reply.proto
common/proto3/common/anon/version_reply.proto
common/proto3/common/anon/version_request.proto
common/proto3/common/auth/acl_data_reply.proto
common/proto3/common/auth/acl_shared_list_items_request.proto
common/proto3/common/auth/acl_shared_list_request.proto
common/proto3/common/auth/acl_update_request.proto
common/proto3/common/auth/acl_view_request.proto
common/proto3/common/auth/check_perms_reply.proto
common/proto3/common/auth/check_perms_request.proto
common/proto3/common/auth/coll_create_request.proto
common/proto3/common/auth/coll_data_reply.proto
common/proto3/common/auth/coll_delete_request.proto
common/proto3/common/auth/coll_get_offset_reply.proto
common/proto3/common/auth/coll_get_offset_request.proto
common/proto3/common/auth/coll_get_parents_request.proto
common/proto3/common/auth/coll_list_published_request.proto
common/proto3/common/auth/coll_move_request.proto
common/proto3/common/auth/coll_path_reply.proto
common/proto3/common/auth/coll_read_request.proto
common/proto3/common/auth/coll_update_request.proto
common/proto3/common/auth/coll_view_request.proto
common/proto3/common/auth/coll_write_request.proto
common/proto3/common/auth/data_delete_request.proto
common/proto3/common/auth/data_get_reply.proto
common/proto3/common/auth/data_get_request.proto
common/proto3/common/auth/data_path_reply.proto
common/proto3/common/auth/data_path_request.proto
common/proto3/common/auth/data_put_reply.proto
common/proto3/common/auth/data_put_request.proto
common/proto3/common/auth/generate_credentials_reply.proto
common/proto3/common/auth/generate_credentials_request.proto
common/proto3/common/auth/get_perms_reply.proto
common/proto3/common/auth/get_perms_request.proto
common/proto3/common/auth/group_create_request.proto
common/proto3/common/auth/group_data_reply.proto
common/proto3/common/auth/group_delete_request.proto
common/proto3/common/auth/group_list_request.proto
common/proto3/common/auth/group_update_request.proto
common/proto3/common/auth/group_view_request.proto
common/proto3/common/auth/listing_reply.proto
common/proto3/common/auth/metadata_validate_reply.proto
common/proto3/common/auth/metadata_validate_request.proto
common/proto3/common/auth/note_comment_edit_request.proto
common/proto3/common/auth/note_create_request.proto
common/proto3/common/auth/note_data_reply.proto
common/proto3/common/auth/note_list_by_subject_request.proto
common/proto3/common/auth/note_update_request.proto
common/proto3/common/auth/note_view_request.proto
common/proto3/common/auth/project_create_request.proto
common/proto3/common/auth/project_data_reply.proto
common/proto3/common/auth/project_delete_request.proto
common/proto3/common/auth/project_get_role_reply.proto
common/proto3/common/auth/project_get_role_request.proto
common/proto3/common/auth/project_list_request.proto
common/proto3/common/auth/project_search_request.proto
common/proto3/common/auth/project_update_request.proto
common/proto3/common/auth/project_view_request.proto
common/proto3/common/auth/query_create_request.proto
common/proto3/common/auth/query_data_reply.proto
common/proto3/common/auth/query_delete_request.proto
common/proto3/common/auth/query_exec_request.proto
common/proto3/common/auth/query_list_request.proto
common/proto3/common/auth/query_update_request.proto
common/proto3/common/auth/query_view_request.proto
common/proto3/common/auth/record_alloc_change_reply.proto
common/proto3/common/auth/record_alloc_change_request.proto
common/proto3/common/auth/record_create_batch_request.proto
common/proto3/common/auth/record_create_request.proto
common/proto3/common/auth/record_data_reply.proto
common/proto3/common/auth/record_delete_request.proto
common/proto3/common/auth/record_export_reply.proto
common/proto3/common/auth/record_export_request.proto
common/proto3/common/auth/record_get_dependency_graph_request.proto
common/proto3/common/auth/record_list_by_alloc_request.proto
common/proto3/common/auth/record_lock_request.proto
common/proto3/common/auth/record_owner_change_reply.proto
common/proto3/common/auth/record_owner_change_request.proto
common/proto3/common/auth/record_update_batch_request.proto
common/proto3/common/auth/record_update_request.proto
common/proto3/common/auth/record_view_request.proto
common/proto3/common/auth/repo_allocation_create_request.proto
common/proto3/common/auth/repo_allocation_create_response.proto
common/proto3/common/auth/repo_allocation_delete_request.proto
common/proto3/common/auth/repo_allocation_set_default_request.proto
common/proto3/common/auth/repo_allocation_set_request.proto
common/proto3/common/auth/repo_allocation_stats_reply.proto
common/proto3/common/auth/repo_allocation_stats_request.proto
common/proto3/common/auth/repo_allocations_reply.proto
common/proto3/common/auth/repo_authz_request.proto
common/proto3/common/auth/repo_calc_size_reply.proto
common/proto3/common/auth/repo_calc_size_request.proto
common/proto3/common/auth/repo_create_request.proto
common/proto3/common/auth/repo_data_delete_request.proto
common/proto3/common/auth/repo_data_get_size_request.proto
common/proto3/common/auth/repo_data_reply.proto
common/proto3/common/auth/repo_data_size_reply.proto
common/proto3/common/auth/repo_delete_request.proto
common/proto3/common/auth/repo_list_allocations_request.proto
common/proto3/common/auth/repo_list_object_allocations_request.proto
common/proto3/common/auth/repo_list_request.proto
common/proto3/common/auth/repo_list_subject_allocations_request.proto
common/proto3/common/auth/repo_path_create_request.proto
common/proto3/common/auth/repo_path_delete_request.proto
common/proto3/common/auth/repo_update_request.proto
common/proto3/common/auth/repo_view_allocation_request.proto
common/proto3/common/auth/repo_view_request.proto
common/proto3/common/auth/revoke_credentials_request.proto
common/proto3/common/auth/schema_create_request.proto
common/proto3/common/auth/schema_data_reply.proto
common/proto3/common/auth/schema_delete_request.proto
common/proto3/common/auth/schema_revise_request.proto
common/proto3/common/auth/schema_search_request.proto
common/proto3/common/auth/schema_update_request.proto
common/proto3/common/auth/schema_view_request.proto
common/proto3/common/auth/search_request.proto
common/proto3/common/auth/tag_data_reply.proto
common/proto3/common/auth/tag_list_by_count_request.proto
common/proto3/common/auth/tag_search_request.proto
common/proto3/common/auth/task_data_reply.proto
common/proto3/common/auth/task_list_request.proto
common/proto3/common/auth/task_view_request.proto
common/proto3/common/auth/topic_data_reply.proto
common/proto3/common/auth/topic_list_topics_request.proto
common/proto3/common/auth/topic_search_request.proto
common/proto3/common/auth/topic_view_request.proto
common/proto3/common/auth/user_access_token_reply.proto
common/proto3/common/auth/user_create_request.proto
common/proto3/common/auth/user_data_reply.proto
common/proto3/common/auth/user_find_by_name_uid_request.proto
common/proto3/common/auth/user_find_by_uuids_request.proto
common/proto3/common/auth/user_get_access_token_request.proto
common/proto3/common/auth/user_get_recent_ep_reply.proto
common/proto3/common/auth/user_get_recent_ep_request.proto
common/proto3/common/auth/user_list_all_request.proto
common/proto3/common/auth/user_list_collab_request.proto
common/proto3/common/auth/user_set_access_token_request.proto
common/proto3/common/auth/user_set_recent_ep_request.proto
common/proto3/common/auth/user_update_request.proto
common/proto3/common/auth/user_view_request.proto
common/proto3/common/enums/access_token_type.proto
common/proto3/common/enums/dependency_dir.proto
common/proto3/common/enums/dependency_type.proto
common/proto3/common/enums/encryption.proto
common/proto3/common/enums/error_code.proto
common/proto3/common/enums/execution_method.proto
common/proto3/common/enums/note_state.proto
common/proto3/common/enums/note_type.proto
common/proto3/common/enums/project_role.proto
common/proto3/common/enums/search_mode.proto
common/proto3/common/enums/service_status.proto
common/proto3/common/enums/sort_option.proto
common/proto3/common/enums/task_command.proto
common/proto3/common/enums/task_status.proto
common/proto3/common/enums/task_type.proto
common/proto3/common/envelope.proto
common/proto3/common/messages/acl_rule.proto
common/proto3/common/messages/alloc_data.proto
common/proto3/common/messages/alloc_stats_data.proto
common/proto3/common/messages/coll_data.proto
common/proto3/common/messages/dependency_data.proto
common/proto3/common/messages/dependency_spec_data.proto
common/proto3/common/messages/group_data.proto
common/proto3/common/messages/listing_data.proto
common/proto3/common/messages/note_comment.proto
common/proto3/common/messages/note_data.proto
common/proto3/common/messages/path_data.proto
common/proto3/common/messages/project_data.proto
common/proto3/common/messages/record_data.proto
common/proto3/common/messages/record_data_location.proto
common/proto3/common/messages/record_data_size.proto
common/proto3/common/messages/repo_data.proto
common/proto3/common/messages/repo_record_data_locations.proto
common/proto3/common/messages/schema_data.proto
common/proto3/common/messages/tag_data.proto
common/proto3/common/messages/task_data.proto
common/proto3/common/messages/topic_data.proto
common/proto3/common/messages/user_data.proto

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@JoshuaSBrown JoshuaSBrown added Type: Refactor Imlplementation change, same functionality Type: New Feature New or enhanced feature Type: Update Update a dependency or some other package Priority: High Highest priority labels Jan 29, 2026
Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • Using file(GLOB_RECURSE ...) for ProtoFiles can make builds non-deterministic when proto files are added/removed without touching CMake; consider listing the proto files explicitly or at least constraining the glob by directory/extension pattern to what you expect long term.
  • The target_include_directories(datafed-protobuf INTERFACE ${PROJECT_BINARY_DIR}/common/proto) path looks like it still points to the old proto location; double-check that consumers of this target see the correct include directory for the new proto3-generated headers (likely ${PROJECT_BINARY_DIR}/common/proto3/common or ${CMAKE_CURRENT_BINARY_DIR}).
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- Using `file(GLOB_RECURSE ...)` for `ProtoFiles` can make builds non-deterministic when proto files are added/removed without touching CMake; consider listing the proto files explicitly or at least constraining the glob by directory/extension pattern to what you expect long term.
- The `target_include_directories(datafed-protobuf INTERFACE ${PROJECT_BINARY_DIR}/common/proto)` path looks like it still points to the old proto location; double-check that consumers of this target see the correct include directory for the new proto3-generated headers (likely `${PROJECT_BINARY_DIR}/common/proto3/common` or `${CMAKE_CURRENT_BINARY_DIR}`).

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@JoshuaSBrown JoshuaSBrown added Component: Web UI Relates to web appp user interface Component: Core Relates to core service Component: Repository Relates to repository service Component: GridFTP Auth Module Relates to GridFTP authorization library Component: Python API Relates to Python API labels Jan 29, 2026
@JoshuaSBrown JoshuaSBrown requested a review from nedvedba January 29, 2026 18:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Component: Core Relates to core service Component: GridFTP Auth Module Relates to GridFTP authorization library Component: Python API Relates to Python API Component: Repository Relates to repository service Component: Web UI Relates to web appp user interface Priority: High Highest priority Type: New Feature New or enhanced feature Type: Refactor Imlplementation change, same functionality Type: Update Update a dependency or some other package

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Refactor] - Rework Protobuf files following protobuf design Principles

2 participants