Skip to content

discoveryengine: add enable_llm_layout_parsing and enable_get_processed_document to data store#18239

Merged
BBBmau merged 3 commits into
GoogleCloudPlatform:mainfrom
JonathanvanWestendorp:discoveryengine-datastore-layout-parsing
Jul 10, 2026
Merged

discoveryengine: add enable_llm_layout_parsing and enable_get_processed_document to data store#18239
BBBmau merged 3 commits into
GoogleCloudPlatform:mainfrom
JonathanvanWestendorp:discoveryengine-datastore-layout-parsing

Conversation

@JonathanvanWestendorp

Copy link
Copy Markdown
Contributor

Adds the two remaining GA layout_parsing_config fields to google_discovery_engine_data_store so the block matches the v1 API:

  • enable_llm_layout_parsing — if true, the PDF layout is refined using an LLM.
  • enable_get_processed_document — if true, the processed document is made available for the GetProcessedDocument API.

Both are added to default_parsing_config.layout_parsing_config and the per-file-type parsing_config_overrides block, and exercised in the document_processing_config_layout_full example. Both fields exist in the v1 (GA) discovery doc.

discoveryengine: added `enable_llm_layout_parsing` and `enable_get_processed_document` fields to `google_discovery_engine_data_store`

@modular-magician modular-magician added the awaiting-approval Pull requests that need reviewer's approval to run presubmit tests label Jul 9, 2026
@JonathanvanWestendorp JonathanvanWestendorp marked this pull request as ready for review July 9, 2026 13:46
@github-actions github-actions Bot requested a review from BBBmau July 9, 2026 13:46
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

Googlers: For automatic test runs see go/terraform-auto-test-runs.

@BBBmau, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

@modular-magician modular-magician added service/discoveryengine and removed awaiting-approval Pull requests that need reviewer's approval to run presubmit tests labels Jul 9, 2026
@modular-magician

modular-magician commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes for commit 5f76e7e:

Diff report

Your PR generated the following diffs in downstream repositories:

Repository Diff Link Changes
google provider View Diff 4 files changed, 118 insertions(+), 6 deletions(-)
google-beta provider View Diff 4 files changed, 118 insertions(+), 6 deletions(-)
terraform-google-conversion View Diff 1 file changed, 44 insertions(+)
Open in Cloud Shell View Diff 1 file changed, 8 insertions(+), 6 deletions(-)

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_discovery_engine_data_store (54 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_discovery_engine_data_store" "primary" {
  document_processing_config {
    parsing_config_overrides {
      layout_parsing_config {
        enable_get_processed_document = # value needed
        enable_llm_layout_parsing     = # value needed
      }
    }
  }
}

Missing doc report (experimental)

The following resources have fields missing in documents.

  • google_discovery_engine_data_store
    • Expected Document Path: /website/docs/r/discovery_engine_data_store.html.markdown
    • Fields: [document_processing_config.parsing_config_overrides.layout_parsing_config.enable_get_processed_document document_processing_config.parsing_config_overrides.layout_parsing_config.enable_llm_layout_parsing]

Test report

Analytics

Total Tests Passed Skipped Affected
43 37 4 2
Affected Service Packages
  • discoveryengine

Learn how VCR tests work


Step 1: Replaying Mode

Action taken

Found 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit.

Click here to see the affected tests
  • TestAccDiscoveryEngineDataStore_discoveryengineDatastoreDocumentProcessingConfigLayoutFullExample
  • TestAccDiscoveryEngineDataStore_discoveryengineDatastoreKmsKeyNameExample

View the replaying VCR build log


Step 2: Recording Mode

Recording Mode Replaying Rerun Test Name
✅ Log TestAccDiscoveryEngineDataStore_discoveryengineDatastoreDocumentProcessingConfigLayoutFullExample
❌ Error · Log - TestAccDiscoveryEngineDataStore_discoveryengineDatastoreKmsKeyNameExample

Caution

Issues requiring attention before PR completion

🔴 Initial Recording Failed: Some tests failed during the recording step. See the table above for details.

Please address these issues to complete your PR. If you believe these detections are incorrect or unrelated to your change, please raise the concern with your reviewer.

View the recording VCR build log or the debug logs folder for detailed results.

@JonathanvanWestendorp, @BBBmau VCR tests complete for 5f76e7e!

@modular-magician modular-magician added awaiting-approval Pull requests that need reviewer's approval to run presubmit tests and removed awaiting-approval Pull requests that need reviewer's approval to run presubmit tests labels Jul 10, 2026
@modular-magician

modular-magician commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes for commit d4c7c65:

Diff report

Your PR generated the following diffs in downstream repositories:

Repository Diff Link Changes
google provider View Diff 4 files changed, 127 insertions(+), 6 deletions(-)
google-beta provider View Diff 4 files changed, 127 insertions(+), 6 deletions(-)
terraform-google-conversion View Diff 1 file changed, 44 insertions(+)
Open in Cloud Shell View Diff 1 file changed, 17 insertions(+), 6 deletions(-)

Missing doc report (experimental)

The following resources have fields missing in documents.

  • google_discovery_engine_data_store
    • Expected Document Path: /website/docs/r/discovery_engine_data_store.html.markdown
    • Fields: [document_processing_config.parsing_config_overrides.layout_parsing_config.enable_get_processed_document document_processing_config.parsing_config_overrides.layout_parsing_config.enable_llm_layout_parsing]

Test report

Analytics

Total Tests Passed Skipped Affected
43 37 4 2
Affected Service Packages
  • discoveryengine

Learn how VCR tests work


Step 1: Replaying Mode

Action taken

Found 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit.

Click here to see the affected tests
  • TestAccDiscoveryEngineDataStore_discoveryengineDatastoreDocumentProcessingConfigLayoutFullExample
  • TestAccDiscoveryEngineDataStore_discoveryengineDatastoreKmsKeyNameExample

View the replaying VCR build log


Step 2: Recording Mode

Recording Mode Replaying Rerun Test Name
✅ Log TestAccDiscoveryEngineDataStore_discoveryengineDatastoreDocumentProcessingConfigLayoutFullExample
❌ Error · Log - TestAccDiscoveryEngineDataStore_discoveryengineDatastoreKmsKeyNameExample

Caution

Issues requiring attention before PR completion

🔴 Initial Recording Failed: Some tests failed during the recording step. See the table above for details.

Please address these issues to complete your PR. If you believe these detections are incorrect or unrelated to your change, please raise the concern with your reviewer.

View the recording VCR build log or the debug logs folder for detailed results.

@JonathanvanWestendorp, @BBBmau VCR tests complete for d4c7c65!

@BBBmau BBBmau left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM just address the comment i have and i can approve and merge.

The failing test is urnelated as well as it failing for quite a while based on the nightly-test history

type: Boolean
description: |
If true, the pdf layout will be refined using an LLM.
required: false

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this doesn't need to be set since by default fields are set to be optional

@modular-magician modular-magician added the awaiting-approval Pull requests that need reviewer's approval to run presubmit tests label Jul 10, 2026
@github-actions github-actions Bot requested a review from BBBmau July 10, 2026 18:58
@modular-magician modular-magician removed the awaiting-approval Pull requests that need reviewer's approval to run presubmit tests label Jul 10, 2026
@modular-magician

modular-magician commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes for commit bd63e46:

Diff report

Your PR generated the following diffs in downstream repositories:

Repository Diff Link Changes
google provider View Diff 4 files changed, 127 insertions(+), 6 deletions(-)
google-beta provider View Diff 4 files changed, 127 insertions(+), 6 deletions(-)
terraform-google-conversion View Diff 1 file changed, 44 insertions(+)
Open in Cloud Shell View Diff 1 file changed, 17 insertions(+), 6 deletions(-)

Missing doc report (experimental)

The following resources have fields missing in documents.

  • google_discovery_engine_data_store
    • Expected Document Path: /website/docs/r/discovery_engine_data_store.html.markdown
    • Fields: [document_processing_config.parsing_config_overrides.layout_parsing_config.enable_get_processed_document document_processing_config.parsing_config_overrides.layout_parsing_config.enable_llm_layout_parsing]

Test report

Analytics

Total Tests Passed Skipped Affected
43 38 4 1
Affected Service Packages
  • discoveryengine

Learn how VCR tests work


Step 1: Replaying Mode

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit.

Click here to see the affected tests
  • TestAccDiscoveryEngineDataStore_discoveryengineDatastoreKmsKeyNameExample

View the replaying VCR build log


Step 2: Recording Mode

Recording Mode Replaying Rerun Test Name
❌ Error · Log - TestAccDiscoveryEngineDataStore_discoveryengineDatastoreKmsKeyNameExample

Caution

Issues requiring attention before PR completion

🔴 Initial Recording Failed: Some tests failed during the recording step. See the table above for details.

Please address these issues to complete your PR. If you believe these detections are incorrect or unrelated to your change, please raise the concern with your reviewer.

View the recording VCR build log or the debug logs folder for detailed results.

@JonathanvanWestendorp, @BBBmau VCR tests complete for bd63e46!

@BBBmau BBBmau left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! missing report is wrong, if you check the generated provider result we do get documentation added

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants