Some layers seem to retrieve enormous amounts of data when accessed from admin, in some cases producing OOM errors. We've noticed it after migrating production data.
Error happens in specific layers (i guess usually due to lots of availabilities) when trying to modify them from the administrator.
I think using pagination would be a great addition to the admin, avoiding these errors and improving all-round performance and usability. We've found the most problematic requests to be:
- /api/cartographies/{layer_id}/availabilities?projection=view
- /api/cartographies/{layer_id}/permissions?projection=view
- /api/cartographies/{layer_id}/treeNodes?projection=view
In this video you can see the usual behaviour for a non-problematic layer, and then the error happening on another layer (remember that the console and requests are visible in jam, feel free to skip load time on second layer): https://jam.dev/c/3b434bce-0824-48e5-b073-734544f4d77a
Error logs in backend core can be seen in added file: OOM_SITMUN3_CARTO.txt
Is adding pagination a possibility? Could it potentially break other usages of these requests?
I could patch it temporarily using these env vars in backend service:
- JAVA_OPTS=-Xms2g -Xmx6g -XX:+UseG1GC -XX:MaxGCPauseMillis=100 -XX:InitiatingHeapOccupancyPercent=35
- SPRING_DATASOURCE_URL=${DATABASE_URL:-jdbc:postgresql://postgres:5432/}${DATABASE:-sitmun3}?defaultRowFetchSize=100
Some layers seem to retrieve enormous amounts of data when accessed from admin, in some cases producing OOM errors. We've noticed it after migrating production data.
Error happens in specific layers (i guess usually due to lots of availabilities) when trying to modify them from the administrator.
I think using pagination would be a great addition to the admin, avoiding these errors and improving all-round performance and usability. We've found the most problematic requests to be:
In this video you can see the usual behaviour for a non-problematic layer, and then the error happening on another layer (remember that the console and requests are visible in jam, feel free to skip load time on second layer): https://jam.dev/c/3b434bce-0824-48e5-b073-734544f4d77a
Error logs in backend core can be seen in added file: OOM_SITMUN3_CARTO.txt
Is adding pagination a possibility? Could it potentially break other usages of these requests?
I could patch it temporarily using these env vars in backend service: