How to Log events on remote side?

[0asys]

Hi,

I'm developing currently an application for managing a borg backup server based on django. it's inspired from borgbackupserver and borgbase, but it should be open source. I wanted to make something to manage the borgbackups at home for my devices and servers in a central place with the option of selfhosting and satisfy my needs to stay informed if all my devices are backuped 😄
it will be open source when its usuable. Currently, the account/repo creation with access over ssh works. some features are missing, like logs/events, storage informations and notifications.

For this to happen: I would like to display/record when a repository is last backuped (or archive created), and display some historized storage usage informations.
If possible, also the logs of interaction with the "serve" and if some backups are failed.

is there a way to get these informations from borg serve or from the repository itself?
The only option I'm playing around is with a borg logging config, but with low yields. Accessing the repository with borg info requires a password in repokey mode, as every thing is encrypted.

a simple but not very efficient way is to regurarly du the repository for its size and get the last access or backup time from the last modifications of index.x or hints.x

Any ideas?

[ThomasWaldmann]

Hi @bpereto!

The problem is the fundamental mismatch here between:

• borg doing most stuff client side and not trusting the server
• you wanting to do stuff server side

You can see some stuff on the fs level of course, as you noted.

You can't see success status as that is only generated/emitted client side. Even there, it is not always as clear as "success" or "error", "warning" can be rather frequent and requires somebody having a look into the log output, which is not available at the server side anyway.

[0asys]

Yes, I see the mismatch here. Possibly, borgbackup can be extended, or usage of some post hooks like in borgmatic to send the Logs to an API (https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/).

At the moment I can detect the following:

• Borg Repository open: lock.exclusive is created
• Borg Repository close: lock.exclusive is removed
• Last Access: Modification Time of repository directory (because of create/delete of lock file)
• Last Update: Modification Time of index.*
• Backup Usage: Usage on Filesystem

Last Update indicates some change, but there is no easy evidence of creating/deleting archives. Aditionally, the deduplication/compression information is missing.

I like that borg distrust the server, but there are situations where you possibly want those informations. The previous mentioned monitoring hooks of borgmatic looks interesting to me to send additional information.

[ThomasWaldmann]

I started experimental work on this in borg2, PR #9788.

It will use the untrusted repository as a communication relay / storage point in between the borg client and the borg monitoring.

To ensure trust and not expose any potentially confidential information, it will use public key cryptography:

• borg client has the secret signing key and the public encryption key of the monitoring.
• borg monitoring has the public key of the client to check signatures and the secret key to decrypt the monitoring information.
• key material is safely derived from the borg key on the client.