Skip to content

REST pool mover queue data #8188

Description

@calestyo

Hey.

This is rather a nice-to-have but not really that important

The REST gives us something like the following:

$ curl -s http://localhost:3880/api/v1/pools/lcg-lrz-dc01_0/usage | jq '.["poolData"]["detailsData"]["costData"]' 
{
  "label": "Pool Cost Info",
  "mover": {
    "active": 0,
    "maxActive": 513,
    "queued": 0,
    "readers": 0,
    "writers": 0
  },
  "store": {
    "active": 0,
    "maxActive": 0,
    "queued": 0,
    "readers": 0,
    "writers": 0
  },
  "restore": {
    "active": 0,
    "maxActive": 0,
    "queued": 0,
    "readers": 0,
    "writers": 0
  },
  "p2p": {
    "active": 0,
    "maxActive": 10,
    "queued": 0,
    "readers": 0,
    "writers": 0
  },
  "p2pClient": {
    "active": 0,
    "maxActive": 0,
    "queued": 0,
    "readers": 0,
    "writers": 0
  },
  "extendedMoverHash": {
    "dcap": {
      "name": "dcap",
      "active": 0,
      "maxActive": 5,
      "queued": 0,
      "readers": 0,
      "writers": 0
    },
    "webdav.tls": {
      "name": "webdav.tls",
      "active": 0,
      "maxActive": 20,
      "queued": 0,
      "readers": 0,
      "writers": 0
    },
    "ftp": {
      "name": "ftp",
      "active": 0,
      "maxActive": 5,
      "queued": 0,
      "readers": 0,
      "writers": 0
    },
    "dcap.plain": {
      "name": "dcap.plain",
      "active": 0,
      "maxActive": 8,
      "queued": 0,
      "readers": 0,
      "writers": 0
    },
    "webdav.plain": {
      "name": "webdav.plain",
      "active": 0,
      "maxActive": 20,
      "queued": 0,
      "readers": 0,
      "writers": 0
    },
    "root": {
      "name": "root",
      "active": 0,
      "maxActive": 100,
      "queued": 0,
      "readers": 0,
      "writers": 0
    },
    "dcap.gsi": {
      "name": "dcap.gsi",
      "active": 0,
      "maxActive": 15,
      "queued": 0,
      "readers": 0,
      "writers": 0
    },
    "nfs": {
      "name": "nfs",
      "active": 0,
      "maxActive": 200,
      "queued": 0,
      "readers": 0,
      "writers": 0
    },
    "xrootd": {
      "name": "xrootd",
      "active": 0,
      "maxActive": 100,
      "queued": 0,
      "readers": 0,
      "writers": 0
    },
    "regular": {
      "name": "regular",
      "active": 0,
      "maxActive": 20,
      "queued": 0,
      "readers": 0,
      "writers": 0
    },
    "ftp.gsi": {
      "name": "ftp.gsi",
      "active": 0,
      "maxActive": 20,
      "queued": 0,
      "readers": 0,
      "writers": 0
    }
  },
  "defaultQueueName": "regular",
  "space": {
    "total": 34952002178938,
    "free": 4373845000691,
    "precious": 0,
    "removable": 0,
    "lru": 1787746521,
    "gap": 53687091200,
    "breakEven": 0.7
  },
  "moverCostFactor": 0.5
}

AFAIU from the code:

  • mover is simply an aggregation over all the "custom"/named queues, i.e. those from extendedMoverHash
  • queued + active = the number of total movers (per queue)
  • readers + writers = the number of total movers (per queue)
  • thus queued + active = readers + writers

Does that sound right?

Wearing my dCache admin hat, I think it would be interesting to get the detailed data:

  • queued.readers
  • queued.writers
  • active.readers
  • active.writers
    I had a quick glance over the code and this information (i.e. queued, active, readers + writers) seems to be rather deeply rooted, so I guess making this more detailed would be rather difficult.

What would have also been nice:

  • If the built-in queues (except mover) would also have been in their own JSON array. Currently they're in the same array as movers (which is however only an aggregation), extendedMoverHash, defaultQueueName, space, ...
    This makes it less convenient to simply loop over the elements to get all queues. Right now one can of course hard-code it one or he other way, bu it's at least thinkable, that other new built-in queues or non-queue elements get added to that array.
    Problem is of course that this change would break the API, so probably too late.

Cheers,
Chris.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions