Add test for conn.getObjects('Annotations', opts={...}) by will-moore · Pull Request #6458 · ome/openmicroscopy

will-moore · 2026-06-30T15:22:33Z

What this PR does

All tests are passing: https://merge-ci.openmicroscopy.org/jenkins/job/OMERO-test-integration/68/testReport/

New test passing now: https://merge-ci.openmicroscopy.org/jenkins/job/OMERO-test-integration/lastCompletedBuild/testReport/OmeroPy.test.integration.gatewaytest.test_get_objects/TestGetObject/testGetObjectsAnnotation/history/

pwalczysko

The testGetObjectsAnnotation(self) added in this PR is consistently failing on merge-ci. It might be a case of the migration (of merge-ci) to NFS, but I have not seen it passing on the old merge-ci either. Could you please check @will-moore ? Especialy please comment on workflows which need I/O on the disk -> these might be influenced by the NFS migration.

Edit: The NFS migration has nothing to do with this. The test is failing locally for me as well (non-FS local test setup).

pwalczysko

Please comment on the failing https://merge-ci.openmicroscopy.org/jenkins/job/OMERO-test-integration/lastCompletedBuild/testReport/OmeroPy.test.integration.gatewaytest.test_get_objects/TestGetObject/testGetObjectsAnnotation/

pwalczysko · 2026-07-17T13:58:14Z

Note, the test is failing for me locally (I have only added some printouts so that I know that indeed an new user, group and 0 annotations are the starting point) with following error:

pytest test/integration/gatewaytest/test_get_objects.py::TestGetObject::testGetObjectsAnnotation -vv -s
================================================================================ test session starts ================================================================================
platform linux -- Python 3.9.18, pytest-8.4.2, pluggy-1.6.0 -- /opt/omero/server/venv3/bin/python3
cachedir: .pytest_cache
rootdir: /openmicroscopy/components/tools
configfile: pytest.ini
collected 1 item                                                                                                                                                                    

test/integration/gatewaytest/test_get_objects.py::TestGetObject::testGetObjectsAnnotation User ID: 9
Username: 1106a046-9b37-430a-b3eb-1f5fc6e3ed1c
Group ID: 10
Initial annotations: 0
FAILED

===================================================================================== FAILURES ======================================================================================
______________________________________________________________________ TestGetObject.testGetObjectsAnnotation _______________________________________________________________________

self = <test.integration.gatewaytest.test_get_objects.TestGetObject object at 0xffffb9a8ba30>

    def testGetObjectsAnnotation(self):
    
        # create new group and user - don't rely on existing test user/group
    
        group = self.new_group(perms='rwra--')
        client, user = self.new_client_and_user(group=group)
    
        conn = BlitzGateway(client_obj=client)
        update = conn.getUpdateService()
    
        print("User ID:", conn.getUserId())
        print("Username:", conn.getUser().getName())
        print("Group ID:", conn.getGroupFromContext().id)
    
        anns = list(conn.getObjects("Annotation"))
    
        print("Initial annotations:", len(anns))
        for a in anns:
            print(
                a.id,
                type(a).__name__,
                a.getOwner().id,
                a.getNs() if hasattr(a, "getNs") else None,
            )
    
        ns = "test_get_objects_annotation_comment"
        ns_tag = "test_get_objects_annotation_tag"
        ns_file = "test_get_objects_annotation_file"
    
        def create_dataset_with_annotations(name, dtype="Dataset"):
            if dtype == "Dataset":
                obj = DatasetI()
                obj.name = rstring(name)
                obj = update.saveAndReturnObject(obj)
                wrapper = omero.gateway.DatasetWrapper(conn, obj)
            elif dtype == "Project":
                obj = ProjectI()
                obj.name = rstring(name)
                obj = update.saveAndReturnObject(obj)
                wrapper = omero.gateway.ProjectWrapper(conn, obj)
    
            # create Comment
            ann = omero.gateway.CommentAnnotationWrapper(conn)
            ann.setNs(ns)
            ann.setValue("Test Comment: " + name)
            ann = wrapper.linkAnnotation(ann)
            # create Tag
            tag = omero.gateway.TagAnnotationWrapper(conn)
            tag.setNs(ns_tag)
            tag.setValue("Test Tag: " + name)
            wrapper.linkAnnotation(tag)
            # create FileAnnotation
            fileAnn = omero.gateway.FileAnnotationWrapper(conn)
            fileObj = omero.model.OriginalFileI()
            fileObj = omero.gateway.OriginalFileWrapper(conn, fileObj)
            fileObj.setName(omero.rtypes.rstring('fileName'))
            fileObj.setPath(omero.rtypes.rstring('path/to/file'))
            fileObj.setHash(omero.rtypes.rstring('a'))
            fileObj.setSize(omero.rtypes.rlong(0))
            fileObj.save()
            fileAnn.setFile(fileObj)
            fileAnn.setNs(ns_file)
            fileAnn.save()
            wrapper.linkAnnotation(fileAnn)
    
            return wrapper
    
        dataset1 = create_dataset_with_annotations("Dataset 1")
        dataset2 = create_dataset_with_annotations("Dataset 2")
        create_dataset_with_annotations("Project 1", dtype="Project")
    
        # get all annotations on one Dataset
        annGen = conn.getObjects("Annotation",
                                 opts={'parent_type': "dataset",
                                       'parent_ids': [dataset1.id]})
>       assert len(list(annGen)) == 3
E       assert 9 == 3
E        +  where 9 = len([<FileAnnotationWrapper id=36>, <TagAnnotationWrapper id=35>, <CommentAnnotationWrapper id=34>, <FileAnnotationWrapper id=33>, <TagAnnotationWrapper id=32>, <CommentAnnotationWrapper id=31>, <FileAnnotationWrapper id=30>, <TagAnnotationWrapper id=29>, <CommentAnnotationWrapper id=28>])
E        +    where [<FileAnnotationWrapper id=36>, <TagAnnotationWrapper id=35>, <CommentAnnotationWrapper id=34>, <FileAnnotationWrapper id=33>, <TagAnnotationWrapper id=32>, <CommentAnnotationWrapper id=31>, <FileAnnotationWrapper id=30>, <TagAnnotationWrapper id=29>, <CommentAnnotationWrapper id=28>] = list(<generator object _BlitzGateway.getObjects at 0xffffb9a76e40>)

test/integration/gatewaytest/test_get_objects.py:715: AssertionError

The test is not ready imho. Adding Exclude label.

will-moore · 2026-07-19T05:52:22Z

@pwalczysko Thanks for looking at this...

This PR adds tests for ome/omero-py#489 so without that PR these tests will fail.

I don't know if that PR got merged when the tests were failing above (not sure which build that was) but the PR seems to be merged in latest at https://merge-ci.openmicroscopy.org/jenkins/job/OMERO-python-superbuild-push/12/console

So hopefully the exclude isn't needed, but fine to leave it till I'm back.

pwalczysko · 2026-07-19T19:32:51Z

don't know if that PR got merged when the tests were failing above (not sure which build that was) but the PR seems to be merged in latest at https://merge-ci.openmicroscopy.org/jenkins/job/OMERO-python-superbuild-push/12/console

This is easy to see - that test never passed (on the new merge-ci). These are 12 runs which are encompassing the one which you and I am citing :), namely https://merge-ci.openmicroscopy.org/jenkins/job/OMERO-test-integration/12/#showFailuresLink - this is the testintegration build 12 which matches the python-superbuild 12 which you are citing. There IS a problem.

will-moore · 2026-07-20T04:49:59Z

Ah, OK, I see that in that failure (run 12), the test fails at

        # get ALL annotations
        anns = list(conn.getObjects("Annotation"))
>       assert len(anns) == 9
E       assert 11 == 9
E        +  where 11 = len([<FileAnnotationWrapper id=365>, <FileAnnotationWrapper id=366>, <CommentAnnotationWrapper id=5328>, <TagAnnotationWrapper id=5329>, <FileAnnotationWrapper id=5330>, <CommentAnnotationWrapper id=5331>, ...])

and this seems to be due to a couple of older FileAnnotations (IDs 365 and 366) that are created by earlier tests, possibly in the user group, and these are getting retrieved even in the new group and user created in this test.
I can add a fix for that.

pwalczysko · 2026-07-21T11:02:32Z

Ah, OK, I see that in that failure (run 12), the test fails at
        # get ALL annotations
        anns = list(conn.getObjects("Annotation"))
>       assert len(anns) == 9
E       assert 11 == 9
E        +  where 11 = len([<FileAnnotationWrapper id=365>, <FileAnnotationWrapper id=366>, <CommentAnnotationWrapper id=5328>, <TagAnnotationWrapper id=5329>, <FileAnnotationWrapper id=5330>, <CommentAnnotationWrapper id=5331>, ...])
and this seems to be due to a couple of older FileAnnotations (IDs 365 and 366) that are created by earlier tests, possibly in the user group, and these are getting retrieved even in the new group and user created in this test. I can add a fix for that.

Yes, @will-moore and please when you are at it, could you have a look also at https://merge-ci.openmicroscopy.org/jenkins/job/OMERO-test-integration/16/testReport/ ? I have summarized the problem in #6468.
This is similar case imho. I investigated a bit, and now I am inclined to rather think that there is a race. In the case which I am pinging here, there is a getAllObjects() call which is calling whilst the server-side chgrp is still in progress - am I right ? So the test only relies on the fact that the chgrp will be fast ? That will have following phenotype then: (and this is what is happening in the case I am talking about): the test will sometimes pass, sometimes fail in the same apparent conditions, as long as the conditions are not optimal (such as running it on slower NFS, with many other tests etc.)
I would like to have your thoughts on this please, thanks.

pwalczysko · 2026-07-21T19:44:19Z

@will-moore Please see also #6470

will-moore · 2026-07-22T21:36:08Z

These tests are passing now:
https://merge-ci.openmicroscopy.org/jenkins/job/OMERO-test-integration/19/testReport/junit/OmeroPy.test.integration.gatewaytest.test_get_objects/TestGetObject/testGetObjectsAnnotation/

sbesson

See ome/omero-py#489 (review)

pwalczysko · 2026-09-22T11:58:44Z

+        tagAnns = list(tag.listAnnotations())
+        assert len(tagAnns) == 2
+        # Since obj._loadAnnotationLinks() doesn't load child annotations
+        # for AnootationAnnotationLink ?? (unexpected) the anns are


really not sure about the clarity of this comment -> do we have an issue if there is such suprisingly unexpected behaviour of the API ?

In case the lack of loading of child annotations is less shocking/surprising than the comment is suggesting, then I would actually remove the comment and also the commented out assert ?

pwalczysko

00:28:11 2026-09-21 23:28:11,651 [   scc.merge] INFO  Merged PRs:
00:28:11 2026-09-21 23:28:11,651 [   scc.merge] INFO    - PR 6423 jburel 'Use \d'
00:28:11 2026-09-21 23:28:11,651 [   scc.merge] INFO    - PR 6458 will-moore 'Add test for conn.getObjects('Annotations', opts={...})'
00:28:11 2026-09-21 23:28:11,651 [   scc.merge] INFO    - PR 6459 will-moore 'JSON API annotations tests'
00:28:11 2026-09-21 23:28:11,651 [   scc.merge] INFO    - PR 6478 dominikl 'Add omero-zarr-pixel-buffer'

Tests from the first file edited here https://github.com/will-moore/openmicroscopy/blob/7cdea5403c719700bd3f7d60ac74e41b071d0ca8/components/tools/OmeroPy/test/integration/gatewaytest/test_annotation.py passing, see https://merge-ci.openmicroscopy.org/jenkins/job/OMERO-test-integration/96/testReport/OmeroPy.test.integration.gatewaytest/test_annotation/.

Tests from the second file edited here https://github.com/will-moore/openmicroscopy/blob/7cdea5403c719700bd3f7d60ac74e41b071d0ca8/components/tools/OmeroPy/test/integration/gatewaytest/test_get_objects.py are also passing, see https://merge-ci.openmicroscopy.org/jenkins/job/OMERO-test-integration/96/testReport/OmeroPy.test.integration.gatewaytest.test_get_objects/.

lgtm except for my comment Re: comment inside the test #6458 (comment) (I guess this could be better formulated or, better, removed ?

Add test for conn.getObjects('Annotations', opts={...})

3e1ea54

will-moore mentioned this pull request Jun 30, 2026

Add more opts support for getObjects('Annotation') ome/omero-py#489

Merged

will-moore added 3 commits July 1, 2026 09:57

Test getObjects() with FileAnnotations

102e3a2

flake8 fixes

5fa410d

Test ann Files loaded without ann_type:file

d7a8b2b

pwalczysko reviewed Jul 17, 2026

View reviewed changes

pwalczysko requested changes Jul 17, 2026

View reviewed changes

pwalczysko added the exclude label Jul 17, 2026

will-moore removed the exclude label Jul 22, 2026

Ignore any 'user' group annotations

c34a89d

will-moore force-pushed the get_objects_annotations_tests branch from ef27199 to c34a89d Compare July 22, 2026 11:37

will-moore added 10 commits July 30, 2026 17:42

testGetObjectsAnnotation() tests all annotation types

51c71e7

Test parent_type case insensitive

e196a0e

Test parent_type 'experimentergroup' mapping to ExperimenterGroup

8834cfe

Add test for conn.getAnnotationLinks() on an Annotation

2084d18

flake8 fixes

0c391a6

Add test for conn.countAnnotations()

a43223b

Use group.linkAnnotation(tag)

76e5514

Add testAnnotationAnnotationLinks()

ab93ce0

flake8 fixes

75c94e5

Add group.listAnnotations() to test

1adbba8

will-moore mentioned this pull request Aug 27, 2026

Wrapper improvements for ROIs ome/omero-py#459

Open

test ann.getParentLinks()

dc500f5

will-moore added 3 commits September 18, 2026 14:52

Add annotations test for ExperimenterWrapper

6c29a67

Remove testing of Channel annotation

29fc583

flake8 fix

7cdea54

sbesson approved these changes Sep 22, 2026

View reviewed changes

sbesson requested a review from pwalczysko September 22, 2026 08:57

pwalczysko reviewed Sep 22, 2026

View reviewed changes

pwalczysko requested changes Sep 22, 2026

View reviewed changes

Remove 'unexpected' comment

b16d1ca

pwalczysko approved these changes Sep 22, 2026

View reviewed changes

Navigation Menu

Add test for conn.getObjects('Annotations', opts={...}) - #6458

Add test for conn.getObjects('Annotations', opts={...})#6458
will-moore wants to merge 20 commits into
ome:developfrom
will-moore:get_objects_annotations_tests

will-moore commented Jun 30, 2026 •

edited

Loading

Uh oh!

pwalczysko left a comment •

edited

Loading

Uh oh!

pwalczysko left a comment

Uh oh!

pwalczysko commented Jul 17, 2026

Uh oh!

will-moore commented Jul 19, 2026

Uh oh!

pwalczysko commented Jul 19, 2026

Uh oh!

will-moore commented Jul 20, 2026

Uh oh!

pwalczysko commented Jul 21, 2026 •

edited

Loading

Uh oh!

pwalczysko commented Jul 21, 2026

Uh oh!

will-moore commented Jul 22, 2026

Uh oh!

sbesson left a comment

Uh oh!

pwalczysko Sep 22, 2026

Uh oh!

pwalczysko Sep 22, 2026

Uh oh!

pwalczysko left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

will-moore commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does

Uh oh!

pwalczysko left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pwalczysko left a comment

Choose a reason for hiding this comment

Uh oh!

pwalczysko commented Jul 17, 2026

Uh oh!

will-moore commented Jul 19, 2026

Uh oh!

pwalczysko commented Jul 19, 2026

Uh oh!

will-moore commented Jul 20, 2026

Uh oh!

pwalczysko commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pwalczysko commented Jul 21, 2026

Uh oh!

will-moore commented Jul 22, 2026

Uh oh!

sbesson left a comment

Choose a reason for hiding this comment

Uh oh!

pwalczysko Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

pwalczysko Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

pwalczysko left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

will-moore commented Jun 30, 2026 •

edited

Loading

pwalczysko left a comment •

edited

Loading

pwalczysko commented Jul 21, 2026 •

edited

Loading