Deserialization of generic types fails due to inability to find property setters that use init

[AArnott]

Bug description

A generic type containing properties with an init accessor fails to serialize.

Repro steps

This type fails to serialize with the DynamicObjectResolver:

        [MessagePackObject]
        public class GenericPerson<T>
        {
            [Key(0)]
            public string Name { get; init; }
        }

This test fails:

        [Fact]
        public void RoundtripGenericClass()
        {
            var person = new GenericPerson<int> { Name = "bob" };
            byte[] msgpack = MessagePackSerializer.Serialize(person, MessagePackSerializerOptions.Standard);
            var deserialized = MessagePackSerializer.Deserialize<GenericPerson<int>>(msgpack, MessagePackSerializerOptions.Standard);
            Assert.Equal(person.Name, deserialized.Name);
        }

Expected behavior

Successful pass.

Actual behavior

  Message: 
    MessagePack.MessagePackSerializationException : Failed to deserialize MessagePack.Tests.DynamicObjectResolverTests+GenericPerson`1[[System.Int32, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]] value.
    ---- System.MissingMethodException : Method not found: 'Void GenericPerson`1.set_Data(!0)'.
  Stack Trace: 
    MessagePackSerializer.Deserialize[T](MessagePackReader& reader, MessagePackSerializerOptions options) line 251
    MessagePackSerializer.Deserialize[T](ReadOnlyMemory`1 buffer, MessagePackSerializerOptions options, CancellationToken cancellationToken) line 270
    DynamicObjectResolverTests.RoundtripGenericClass() line 321
    ----- Inner Stack Trace -----
       at MessagePack.Formatters.MessagePack_Tests_DynamicObjectResolverTests\.GenericPerson`1\[\[System_Int32\, System_Private_CoreLib\]\]Formatter1.Deserialize(MessagePackReader& reader, MessagePackSerializerOptions options)
    MessagePackSerializer.Deserialize[T](MessagePackReader& reader, MessagePackSerializerOptions options) line 246

[AArnott]

Based on this doc it seems likely that our IL code gen has to be updated to include modreq([System.Runtime]System.Runtime.CompilerServices.IsExternalInit) in the called method signature somehow. It seems all C# compiler generated calls to init methods require this, and it's evident in decompilers but not in our code-generated calls to it.

As to why it only fails in generic classes may be a bug/detail of the .NET 5 runtime.

[AArnott]

See linked dotnet/runtime bug for more info. I'm pretty certain this is a bug in .NET.

[AArnott]

This is a bug in .NET. .NET 6.0 has the fix, which may be ported to 5.x.
For now, a workaround is to use DynamicMethod to make the call instead. That translates in MessagePack to folks using the StandardResolverAllowPrivate resolver instead of StandardResolver.

[simonziegler]

@AArnott thanks for moving #1147 to here. I probably cannot use StandardResolverAllowPrivate because of side effects through my code base. For now I am selectively using set; rather than init;. To help me chart the best path forwards pending .NET 6, can you let me know if you expect some sort of resolution in MessagePack, and if so how long you think it may take before that resolution is in place? I'll then orientate my medium term serialization strategy around that advice. Thanks!

[AArnott]

I'd like to make it "just work" with either dynamic resolver, perhaps by falling back to reflection when the bug is present. Hopefully by end of year.

[simonziegler]

Hi @AArnott - since you closed this, what is the status pls?

[AArnott]

The PR with the fix has merged and will be included in our 2.3 release.
You can try it out prior to its release on nuget by following these instructions.

[simonziegler]

Thanks @AArnott. I just returned to working to integrate MessagePack into my data model. I see that 2.3 is in Alpha. Do you anticipate any issues for me using this release?

Many thanks, Simon

[AArnott]

@simonziegler no, there's no known instability in messagepack 2.3-alpha.

[simonziegler]

Hey @AArnott I was going to drop by here this week to let you know that 2.3-alpha is working great and without any issues regarding init/set. It all just works. Many thanks!

[SapiensAnatis]

Hey, I see this has been closed and a fix was included in 2.3, but I am on 2.4.35 and I just got an exception linking me to this issue while using the ContractlessStandardResolver (that's supposed to include the dynamic resolver, right?):

InitAccessorInGenericClassNotSupportedException: `init` property accessor [...] data_headers found in generic type, which is not supported with the DynamicObjectResolver. Use the AllowPrivate variety of the resolver instead. See https://github.com/neuecc/MessagePack-CSharp/issues/1134 for details.

I am having some trouble putting together a minimum reproducible example though, and my existing code probably wouldn't be much help because it's in an ASP.NET Core MVC app. I was able to fix it by including the [MessagePackObject(true)] decorator above my class, but I didn't think I would have to do that with the contractless resolver.

Is the ContractlessStandardResolver supposed to throw this error?

[AArnott]

@SapiensAnatis did you notice my comment #1134 (comment)? What runtime are you on? Anything less than .NET 6 may give you trouble unless you use an AllowPrivate resolver. But contractless+allowprivate is just terrible (it serializes fields and the properties that expose them).

[SapiensAnatis]

@AArnott I hadn’t seen that comment, but I’m on 6.0 as this is a new project. I won’t have access to a development environment for a week or so, but I would like to try putting together a minimal example using ASP.NET since a simple console app was unable to reproduce the issue. Will let you know what I find.

[martinvaner]

Hey, I am facing this issue after upgrading to .NET 8 while using the same code as in .NET 7.
I am using Microsoft.AspNetCore.SignalR.Protocol.MessagePackHubProtocol.SignalRResolver. Any advise how to fix this?

aspnetcore issue link

[AArnott]

@martinvaner What does your linked issue have to do with failing deserialization due to init accessors?

[martinvaner]

@AArnott Well, on server side I got exception linking me to this issue, therefore I assume the underlying problem might be something similar. On client side (.NET SignalR client) I got HubException described in the linked issue.

[martinvaner]

Hi, this problem clearly remains in .NET 8.
MessagePack in version 2.5.168 (and lower) does not work with SignalR. Given workaround does not work because SignalRResolver uses DynamicEnumAsStringResolver and ContractlessStandardResolver resolvers, thus I cannot change that.
I am getting following exception:

Failed writing message. Aborting connection. <s:Microsoft.AspNetCore.SignalR.HubConnectionContext>
MessagePack.MessagePackSerializationException: Failed to serialize System.Collections.Generic.List`1[[XXX.RevisionedValue`1[[XXX.DTO, XXX, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]], XXX, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]] value.
 ---> System.TypeInitializationException: The type initializer for 'FormatterCache`1' threw an exception.
 ---> System.TypeInitializationException: The type initializer for 'FormatterCache`1' threw an exception.
 ---> MessagePack.Internal.InitAccessorInGenericClassNotSupportedException: `init` property accessor XXX.RevisionedValue`1[[XXX.DTO, XXX, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].Value found in generic type, which is not supported with the DynamicObjectResolver. Use the AllowPrivate variety of the resolver instead. See https://github.com/neuecc/MessagePack-CSharp/issues/1134 for details.
   at MessagePack.Internal.ObjectSerializationInfo.EmittableMember.ThrowIfNotWritable()
   at MessagePack.Internal.ObjectSerializationInfo.CreateOrNull(Type type, Boolean forceStringKey, Boolean contractless, Boolean allowPrivate, Boolean dynamicMethod)
   at MessagePack.Internal.DynamicObjectTypeBuilder.BuildType(DynamicAssembly assembly, Type type, Boolean forceStringKey, Boolean contractless)
   at MessagePack.Resolvers.DynamicContractlessObjectResolver.FormatterCache`1..cctor()
   --- End of inner exception stack trace ---
   at MessagePack.Resolvers.DynamicContractlessObjectResolver.GetFormatter[T]()
   at MessagePack.Resolvers.ContractlessStandardResolver.FormatterCache`1..cctor()
   --- End of inner exception stack trace ---
   at MessagePack.Resolvers.ContractlessStandardResolver.GetFormatter[T]()
   at Microsoft.AspNetCore.SignalR.Protocol.MessagePackHubProtocol.SignalRResolver.Cache`1.ResolveFormatter()
   at Microsoft.AspNetCore.SignalR.Protocol.MessagePackHubProtocol.SignalRResolver.Cache`1..cctor()
--- End of stack trace from previous location ---
   at MessagePack.FormatterResolverExtensions.Throw(TypeInitializationException ex)
   at MessagePack.Formatters.ListFormatter`1.Serialize(MessagePackWriter& writer, List`1 value, MessagePackSerializerOptions options)
   at MessagePack.MessagePackSerializer.Serialize[T](MessagePackWriter& writer, T value, MessagePackSerializerOptions options)
   --- End of inner exception stack trace ---
   at MessagePack.MessagePackSerializer.Serialize[T](MessagePackWriter& writer, T value, MessagePackSerializerOptions options)
   at MessagePack.MessagePackSerializer.SerializeSemiGeneric[T](MessagePackWriter& writer, Object valueObject, MessagePackSerializerOptions options)
   at MessagePack.MessagePackSerializer.Serialize(Type type, MessagePackWriter& writer, Object obj, MessagePackSerializerOptions options)
   at Microsoft.AspNetCore.SignalR.Protocol.DefaultMessagePackHubProtocolWorker.Serialize(MessagePackWriter& writer, Type type, Object value)
   at Microsoft.AspNetCore.SignalR.Protocol.MessagePackHubProtocolWorker.WriteArgument(Object argument, MessagePackWriter& writer)
   at Microsoft.AspNetCore.SignalR.Protocol.MessagePackHubProtocolWorker.WriteCompletionMessage(CompletionMessage message, MessagePackWriter& writer)
   at Microsoft.AspNetCore.SignalR.Protocol.MessagePackHubProtocolWorker.WriteMessageCore(HubMessage message, MessagePackWriter& writer)
   at Microsoft.AspNetCore.SignalR.Protocol.MessagePackHubProtocolWorker.WriteMessage(HubMessage message, IBufferWriter`1 output)
   at Microsoft.AspNetCore.SignalR.Protocol.MessagePackHubProtocol.WriteMessage(HubMessage message, IBufferWriter`1 output)
   at Microsoft.AspNetCore.SignalR.HubConnectionContext.WriteCore(HubMessage message, CancellationToken cancellationToken)

[AArnott]

Ok, from #1137's description:

When .NET 6 ships with the fix, we can add a .NET 6 target that re-allows setting init property setters from the DynamicObjectResolver.

It sounds like there was some follow-up that perhaps never happened. I'll check it out.

[AArnott]

Check out the fix in 2.5.171