Skip to content

Add Export/Import backup functionality to the admin panel Setup page#715

Draft
Copilot wants to merge 7 commits intomasterfrom
copilot/add-backup-restore-functionality
Draft

Add Export/Import backup functionality to the admin panel Setup page#715
Copilot wants to merge 7 commits intomasterfrom
copilot/add-backup-restore-functionality

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 20, 2026

  • Create IBackupService interface in src/Persistence/
  • Create BackupService in src/Persistence/ — no EF dependencies, uses IPersistenceContextProvider.CreateNewContext() and IContext.GetAsync<T>() via the repository pattern
  • Update InMemoryBackupService to extend BackupService (inherits export) and only override RestoreBackupAsync
  • Delete src/Persistence/EntityFramework/BackupService.cs
  • Register backup services in DI (Startup/Program.cs and Dapr/Common/Extensions.cs)
  • Create BackupController API controller in src/Web/AdminPanel/API/
  • Update Setup.razor and Setup.razor.cs for Export/Import UI
  • Add resource strings to Resources.resx and Resources.Designer.cs
  • Fix build errors: disambiguate type references, add missing using directives, fix ambiguous InputFile component
  • IConvertibleTo<BasicModel.ChatServerDefinition> and IConvertibleTo<BasicModel.GameServerDefinition> in ExtendedTypes.Custom.cs
  • Add LocalizedStringJsonConverter to JsonObjectSerializer and JsonObjectDeserializer options to fix runtime export error
  • Add IElement member ignore rule to MapsterConfigurator to prevent Mapster compile exception during export
  • Fix import NotSupportedException: copy BrowserFileStream to MemoryStream (pre-sized with file.Size) before passing to RestoreBackupAsync, since ZipArchive uses synchronous reads internally
  • Show Bootstrap spinner during import to give clear visual feedback while the operation runs
Original prompt

This section details on the original issue you should resolve

<issue_title>Backup/Restore over the admin panel</issue_title>
<issue_description>Is your feature request related to a problem? Please describe.
For backup and debugging purposes it would be nice to have a simple way to backup and restore the configuration and selected accounts. Not only GameConfiguration needs to be considered, but all other configuration types (not in its composition root), too.

Describe the solution you'd like
The admin panel could get a Export and Import functionality on the Setup page of the admin panel. As data format, we could use jsons in zip archive, since we have a stable json serializer for that which supports circular references and other nasty stuff. The single configuration files and account files could be packed into one zip archive.

See also JsonDownloadController.
</issue_description>

Comments on the Issue (you are @copilot in this section)


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI changed the title [WIP] Add backup and restore functionality over the admin panel Add Export/Import backup functionality to the admin panel Setup page Mar 20, 2026
Copilot AI requested a review from sven-n March 20, 2026 23:27
Copy link
Copy Markdown
Member

@sven-n sven-n left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. There are some build errors.
  2. Can we move/refactor the EfBackupService to the common MUnique.OpenMU.Persistence project, rename to BackupService and use the available repositories? Additionally, you can use the JsonObjectLoader for all types, you just have to pass the commonly used classes into it: new JsonObjectLoader(new JsonQueryBuilder(), new JsonObjectDeserializer(), new CachingReferenceHandler()).

…jectLoader for all types, fix ambiguous type references

Co-authored-by: sven-n <5238610+sven-n@users.noreply.github.com>
Agent-Logs-Url: https://github.com/MUnique/OpenMU/sessions/f54d8be1-3e2f-429c-8ae6-779e5215fe8e
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 21, 2026

  1. There are some build errors.
  2. Can we move/refactor the EfBackupService to the common MUnique.OpenMU.Persistence project, rename to BackupService and use the available repositories? Additionally, ...

Fixed in 145e33c:

  1. Build errors resolved — added missing using directives (System.IO, System.Threading, Microsoft.EntityFrameworkCore), disambiguated JsonObjectDeserializer with Json. prefix, qualified InputFile as Microsoft.AspNetCore.Components.Forms.InputFile, and regenerated Resources.Designer.cs entries.

  2. EfBackupService renamed to BackupService (file and class) in Persistence.EntityFramework.

  3. JsonObjectLoader used uniformly — all types now use new JsonObjectLoader(new JsonQueryBuilder(), new Json.JsonObjectDeserializer(), new CachingReferenceHandler()) via a shared generic helper ExportByLoaderAsync<TEfModel, TBasicModel>. GameConfigurationJsonObjectLoader is kept for GameConfiguration since it uses the special GameConfigurationJsonQueryBuilder that ensures maps (which depend on all other data) come last.

  4. Reflection-based method dispatch removed from DeserializeAsync — replaced with a direct typed dispatch (one if per known BasicModel type).

Copilot AI requested a review from sven-n March 21, 2026 15:45
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move the backup service to the Persistence project, without dependencies to the entity framework.
It should work for all persistence implementations, which may come up later. Therefore, use the repositories which are available.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 39ebd06. BackupService is now in src/Persistence/BackupService.cs with no EntityFramework dependencies. It uses IPersistenceContextProvider.CreateNewContext() and IContext.GetAsync<T>() to load objects via the available repositories, making it backend-agnostic. InMemoryBackupService now extends BackupService (inheriting the export path) and only overrides RestoreBackupAsync to throw NotSupportedException.

… IContext.GetAsync

Co-authored-by: sven-n <5238610+sven-n@users.noreply.github.com>
Agent-Logs-Url: https://github.com/MUnique/OpenMU/sessions/cf6f3fc0-2445-4600-a2e6-11cd40e6cd5d
Copilot AI requested a review from sven-n March 21, 2026 16:12
@sven-n
Copy link
Copy Markdown
Member

sven-n commented Mar 21, 2026

@copilot I get an error when I try to do an export:

System.InvalidOperationException: The type 'System.ReadOnlySpan`1[System.Char]' of property 'ValueInNeutralLanguageAsSpan' on type 'MUnique.OpenMU.Interfaces.LocalizedString' is invalid for serialization or deserialization because it is a pointer type, is a ref struct, or contains generic parameters that have not been replaced by specific types.
   at System.Text.Json.ThrowHelper.ThrowInvalidOperationException_CannotSerializeInvalidType(Type typeToConvert, Type declaringType, MemberInfo memberInfo)
   at System.Text.Json.Serialization.Metadata.DefaultJsonTypeInfoResolver.CreatePropertyInfo(JsonTypeInfo typeInfo, Type typeToConvert, MemberInfo memberInfo, NullabilityInfoContext nullabilityCtx, JsonSerializerOptions options, Boolean shouldCheckForRequiredKeyword, Boolean hasJsonIncludeAttribute)
   at System.Text.Json.Serialization.Metadata.DefaultJsonTypeInfoResolver.AddMembersDeclaredBySuperType(JsonTypeInfo typeInfo, Type currentType, NullabilityInfoContext nullabilityCtx, Boolean constructorHasSetsRequiredMembersAttribute, PropertyHierarchyResolutionState& state)
   at System.Text.Json.Serialization.Metadata.DefaultJsonTypeInfoResolver.PopulateProperties(JsonTypeInfo typeInfo, NullabilityInfoContext nullabilityCtx)
   at System.Text.Json.Serialization.Metadata.DefaultJsonTypeInfoResolver.CreateTypeInfoCore(Type type, JsonConverter converter, JsonSerializerOptions options)
   at System.Text.Json.JsonSerializerOptions.CachingContext.CreateCacheEntry(Type type, CachingContext context)

…and JsonObjectDeserializer

Co-authored-by: sven-n <5238610+sven-n@users.noreply.github.com>
Agent-Logs-Url: https://github.com/MUnique/OpenMU/sessions/96be83dd-97c3-41c5-908e-be6c736acec6
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 21, 2026

@copilot I get an error when I try to do an export:

System.InvalidOperationException: The type 'System.ReadOnlySpan`1[System.Char]' of property 'ValueInNeutralLanguageAsSpan' o...

Fixed in 64e7860. LocalizedString is a readonly struct with a ReadOnlySpan<char> property (ValueInNeutralLanguageAsSpan) that System.Text.Json can't reflect over. Added the existing LocalizedStringJsonConverter (from MUnique.OpenMU.Interfaces) to both JsonObjectSerializer and JsonObjectDeserializer options so LocalizedString is handled as a plain string during backup export and import.

@sven-n
Copy link
Copy Markdown
Member

sven-n commented Mar 21, 2026

@copilot next issue when clicking the backup button:

System.TypeInitializationException: The type initializer for 'Mapster.TypeAdapter`2' threw an exception.
 ---> Mapster.CompileException: Error while compiling
source=MUnique.OpenMU.AttributeSystem.IElement
destination=MUnique.OpenMU.AttributeSystem.IElement
type=Map
 ---> System.TypeLoadException: Method 'add_ValueChanged' in type 'GeneratedType_1' from assembly 'Mapster.Dynamic, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' does not have an implementation.
   at System.Reflection.Emit.RuntimeTypeBuilder.CreateTypeNoLock()
   at System.Reflection.Emit.RuntimeTypeBuilder.CreateTypeInfoImpl()
   at Mapster.Utils.DynamicTypeGenerator.CreateTypeForInterface(Type interfaceType)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Mapster.Utils.DynamicTypeGenerator.GetTypeForInterface(Type interfaceType)
   at Mapster.Utils.DynamicTypeGenerator.GetTypeForInterface(Type interfaceType, Boolean ignoreError)
   at Mapster.Adapters.RecordTypeAdapter.CreateInstantiationExpression(Expression source, Expression destination, CompileArgument arg)
   at Mapster.Adapters.BaseAdapter.CreateBlockExpressionBody(Expression source, Expression destination, CompileArgument arg)
   at Mapster.Adapters.BaseAdapter.CreateExpressionBody(Expression source, Expression destination, CompileArgument arg)
   at Mapster.Adapters.BaseAdapter.CreateAdaptFunc(CompileArgument arg)
   at Mapster.TypeAdapterConfig.CreateMapExpression(CompileArgument arg)
   --- End of inner exception stack trace ---
   at Mapster.TypeAdapterConfig.CreateMapExpression(CompileArgument arg)
   at Mapster.TypeAdapterConfig.CreateMapExpression(TypeTuple tuple, MapType mapType)
   at Mapster.TypeAdapterConfig.<GetMapFunction>b__59_0(TypeTuple tuple)
   at Mapster.TypeAdapterConfig.<>c__DisplayClass56_0`1.<AddToHash>b__0(TypeTuple types)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Mapster.TypeAdapterConfig.AddToHash[T](ConcurrentDictionary`2 hash, TypeTuple key, Func`2 func)
   at Mapster.TypeAdapterConfig.GetMapFunction(Type sourceType, Type destinationType)
   at Mapster.TypeAdapterConfig.GetMapFunction[TSource,TDestination]()
   at Mapster.TypeAdapter`2..cctor()
   --- End of inner exception stack trace ---
   at lambda_method9411(Closure, SkillEntry)
   at lambda_method9410(Closure, SkillEntry)
   at lambda_method9408(Closure, ICollection`1)
   at lambda_method9403(Closure, Character)
   at lambda_method9402(Closure, Character)
   at lambda_method9401(Closure, ICollection`1)
   at lambda_method9400(Closure, Object)
   at Mapster.TypeAdapter.Adapt[TDestination](Object source, TypeAdapterConfig config)
   at Mapster.TypeAdapter.Adapt[TDestination](Object source)
   at MUnique.OpenMU.Persistence.EntityFramework.Model.Account.Convert() in C:\Users\Sven\Documents\GitHub\OpenMU\src\Persistence\EntityFramework\Model\ExtendedTypes.Custom.cs:line 205
   at MUnique.OpenMU.Persistence.BackupService.ExportAsync[TData,TBasic](ZipArchive archive, String filePrefix, IContext context, IdReferenceHandler sharedHandler, CancellationToken cancellationToken) in C:\Users\Sven\Documents\GitHub\OpenMU\src\Persistence\BackupService.cs:line 125
   at MUnique.OpenMU.Persistence.BackupService.CreateBackupAsync(Stream outputStream, CancellationToken cancellationToken) in C:\Users\Sven\Documents\GitHub\OpenMU\src\Persistence\BackupService.cs:line 57
   at MUnique.OpenMU.Web.AdminPanel.API.BackupController.DownloadBackupAsync(CancellationToken cancellationToken) in C:\Users\Sven\Documents\GitHub\OpenMU\src\Web\AdminPanel\API\BackupController.cs:line 42
   at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.TaskOfIActionResultExecutor.Execute(ActionContext actionContext, IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Logged|12_1(ControllerActionInvoker invoker)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
   at Microsoft.AspNetCore.Localization.RequestLocalizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)

I think it tries to map some transient properties (marked with TransientAttribute) of the SkillEntry class, which can be ignored in the backup process.

…s in MapsterConfigurator

Co-authored-by: sven-n <5238610+sven-n@users.noreply.github.com>
Agent-Logs-Url: https://github.com/MUnique/OpenMU/sessions/f52ad4aa-9751-4aca-8632-70f3363c04c7
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 21, 2026

@copilot next issue when clicking the backup button:

System.TypeInitializationException: The type initializer for 'Mapster.TypeAdapter`2' threw an exception.

...

Fixed in cabe60c. SkillEntry has IElement-typed properties (PowerUpDuration, PowerUps, etc.) that are runtime-computed values not stored in the DB. Mapster tried to map them and failed because IElement has an add_ValueChanged event method it couldn't implement dynamically.

Added a global Mapster ignore rule in MapsterConfigurator.EnsureConfigured() that uses a recursive ContainsIElement helper to skip any member whose type is, derives from, or contains IElement (covers plain IElement, arrays, generic collections, and nested combinations). Also updated the source generator template so future regenerations include the same rule.

@sven-n
Copy link
Copy Markdown
Member

sven-n commented Mar 22, 2026

@copilot The export works now, now I'm stuck at the import.
When I select the zip file for import, I get the exception:

System.NotSupportedException: Synchronous reads are not supported.
   at Microsoft.AspNetCore.Components.Forms.BrowserFileStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Stream.CopyTo(Stream destination, Int32 bufferSize)
   at System.IO.Compression.ZipArchive..ctor(Stream stream, ZipArchiveMode mode, Boolean leaveOpen, Encoding entryNameEncoding)
   at System.IO.Compression.ZipArchive..ctor(Stream stream, ZipArchiveMode mode, Boolean leaveOpen)
   at MUnique.OpenMU.Persistence.BackupService.RestoreBackupAsync(Stream inputStream, CancellationToken cancellationToken) in C:\Users\Sven\Documents\GitHub\OpenMU\src\Persistence\BackupService.cs:line 63
   at MUnique.OpenMU.Web.AdminPanel.Services.SetupService.CreateDatabaseAsync(Func`1 dataInitialization) in C:\Users\Sven\Documents\GitHub\OpenMU\src\Web\AdminPanel\Services\SetupService.cs:line 132
   at MUnique.OpenMU.Web.AdminPanel.Pages.Setup.OnImportFileChangeAsync(InputFileChangeEventArgs e) in C:\Users\Sven\Documents\GitHub\OpenMU\src\Web\AdminPanel\Pages\Setup.razor.cs:line 93
   at MUnique.OpenMU.Web.AdminPanel.Pages.Setup.OnImportFileChangeAsync(InputFileChangeEventArgs e) in C:\Users\Sven\Documents\GitHub\OpenMU\src\Web\AdminPanel\Pages\Setup.razor.cs:line 96

Additionally, a progress overlay should be shown during the import to the user, because otherwise it's not clear what (or that anything) is happening.

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 22, 2026

@copilot The export works now, now I'm stuck at the import.
When I select the zip file for import, I get the exception:

...

Fixed in 08bc360:

  1. Sync-read error: BrowserFileStream only supports async reads, but ZipArchive internally uses sync reads. The fix copies the browser stream into a pre-sized MemoryStream asynchronously before passing it to RestoreBackupAsync.

  2. Progress indicator: Replaced the plain text message with a Bootstrap spinner-border + label while import is in progress, consistent with the spinner pattern used elsewhere in the admin panel (e.g. EditConfigGrid.razor).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Backup/Restore over the admin panel

2 participants