Skip to content

remove database backend #489#507

Merged
joshdimanteto merged 36 commits intodevelopfrom
remove-database-backend-#489
Mar 16, 2026
Merged

remove database backend #489#507
joshdimanteto merged 36 commits intodevelopfrom
remove-database-backend-#489

Conversation

@joshdimanteto
Copy link
Copy Markdown
Contributor

@joshdimanteto joshdimanteto commented Nov 26, 2024

This PR will close #489

Description

Remove the database backend functionality from the DataGateway API. It includes significant changes such as deleting database-related files and functionality, updating tests to remove database dependencies, and modifying the configuration to align with the changes.

Testing Instructions

Add a set up instructions describing how the reviewer should test the code

  • Review code
  • Check GitHub Actions build
  • If icatdb Generator Script Consistency Test CI job fails, is this because of a deliberate change made to the script to change generated data (which isn't actually a problem) or is here an underlying issue with the changes made?
  • Review changes to test coverage
  • Does this change mean a new patch, minor or major version should be made? If so, does one of the commit messages feature fix:, feat: or BREAKING CHANGE: so a release is automatically made via GitHub Actions upon merge?
  • {more steps here}

Agile Board Tracking

Connect to #489


notes

The icat models still use SQLAlchemy, this is going to be update in the pydantic V2 upgrade to use pydantic

@joshdimanteto joshdimanteto changed the title refactor: remove database backend #469 remove database backend #469 Nov 26, 2024
@joshdimanteto joshdimanteto changed the title remove database backend #469 remove database backend #489 Nov 26, 2024
@joshdimanteto joshdimanteto force-pushed the remove-database-backend-#489 branch from 5e5d6c4 to 743b34b Compare November 26, 2024 16:57
@joshdimanteto joshdimanteto force-pushed the remove-database-backend-#489 branch 4 times, most recently from 86a347f to c3a28c4 Compare December 2, 2024 13:56
@joshdimanteto joshdimanteto force-pushed the remove-database-backend-#489 branch from c3a28c4 to 2939010 Compare December 2, 2024 14:17
@joshdimanteto joshdimanteto force-pushed the remove-database-backend-#489 branch from e1afd0a to 34d2603 Compare January 27, 2025 14:02
@joshdimanteto joshdimanteto force-pushed the remove-database-backend-#489 branch from 94aad25 to e280471 Compare November 17, 2025 12:15
Comment thread .github/workflows/ci-build.yml Outdated
@joshdimanteto joshdimanteto force-pushed the remove-database-backend-#489 branch from 9ac04a6 to 84b9f59 Compare November 17, 2025 15:50
@joshdimanteto joshdimanteto marked this pull request as ready for review January 14, 2026 15:57
@joshdimanteto joshdimanteto requested a review from VKTB January 15, 2026 15:38
Comment thread test/integration/conftest.py Outdated
@@ -45,11 +42,10 @@ def flask_test_app():
def flask_test_app_db():
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This name should change. I have changed in #524 where i had to refactored the tests

Comment thread .github/workflows/ci-build.yml Outdated
Comment thread datagateway_api/src/common/config.py
Comment thread datagateway_api/src/common/config.py Outdated
@joshdimanteto joshdimanteto force-pushed the remove-database-backend-#489 branch from a68b552 to 9fae76f Compare February 12, 2026 10:27
@joshdimanteto joshdimanteto requested a review from VKTB February 12, 2026 10:45
Copy link
Copy Markdown
Contributor

@VKTB VKTB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the changes look fine, I tried my best. I have not tested it locally as I don't have a working stack to point it it to so I just trust that it works as expected. Might be worth asking someone else to have a look too. Approving.

Copy link
Copy Markdown

@patrick-austin patrick-austin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Functionally I think it's OK (I can run the API and get results back).

There's a few whitespace changes I would make (as a result of strings being moved onto one line) and one or two possible code changes.

Comment thread .github/workflows/ci-build.yml Outdated
Comment thread .github/workflows/ci-build.yml Outdated
Comment thread datagateway_api/src/common/config.py Outdated
Comment thread test/integration/datagateway_api/icat/test_query.py Outdated
Comment thread test/integration/datagateway_api/icat/test_query.py Outdated
Comment thread test/integration/datagateway_api/icat/test_query.py Outdated
Comment thread README.md Outdated
Comment thread README.md Outdated
Comment thread README.md Outdated
Comment thread README.md Outdated
- remove unnecessary argument "validate_assignment = True" in config
- format strings better in test_query for title
- update README
- remove installation of java this should installed by the ansible
  script
Copy link
Copy Markdown

@patrick-austin patrick-austin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed over Teams, there are some instances where the models do not strictly represent the types/fields required by the API (for example required fields when POSTing or id when PATCHing).

However, given that further changes are in progress with the FastAPI refactor and the primary use case for the API is DataGateway's read operations, I'm happy that the use of Optional lets things work for now, and it doesn't matter so much that someone referring to the Swagger wouldn't know id is required for PATCH (as who's looking at the Swagger in the first place?)

@joshdimanteto joshdimanteto force-pushed the remove-older-python-versions-#517 branch from 5a831a1 to 8b29a8b Compare March 11, 2026 09:52
@joshdimanteto joshdimanteto changed the base branch from remove-older-python-versions-#517 to develop March 11, 2026 12:18
@joshdimanteto joshdimanteto merged commit 109ccc5 into develop Mar 16, 2026
8 checks passed
@joshdimanteto joshdimanteto deleted the remove-database-backend-#489 branch March 16, 2026 09:56
@patrick-austin patrick-austin mentioned this pull request Apr 30, 2026
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove Database Backend

3 participants