Skip to content

Add pagination to admin #34

@OLamelas

Description

@OLamelas

Some layers seem to retrieve enormous amounts of data when accessed from admin, in some cases producing OOM errors. We've noticed it after migrating production data.

Error happens in specific layers (i guess usually due to lots of availabilities) when trying to modify them from the administrator.

I think using pagination would be a great addition to the admin, avoiding these errors and improving all-round performance and usability. We've found the most problematic requests to be:

  • /api/cartographies/{layer_id}/availabilities?projection=view
  • /api/cartographies/{layer_id}/permissions?projection=view
  • /api/cartographies/{layer_id}/treeNodes?projection=view

In this video you can see the usual behaviour for a non-problematic layer, and then the error happening on another layer (remember that the console and requests are visible in jam, feel free to skip load time on second layer): https://jam.dev/c/3b434bce-0824-48e5-b073-734544f4d77a

Error logs in backend core can be seen in added file: OOM_SITMUN3_CARTO.txt

Is adding pagination a possibility? Could it potentially break other usages of these requests?

I could patch it temporarily using these env vars in backend service:

  • JAVA_OPTS=-Xms2g -Xmx6g -XX:+UseG1GC -XX:MaxGCPauseMillis=100 -XX:InitiatingHeapOccupancyPercent=35
  • SPRING_DATASOURCE_URL=${DATABASE_URL:-jdbc:postgresql://postgres:5432/}${DATABASE:-sitmun3}?defaultRowFetchSize=100

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions