Repository navigation
Add complex type management with a new specialized IComplexTypeConverterFactory - #446
Merged
masesdevelopers merged 17 commits intoFeb 27, 2026
Conversation
Introduce support for complex/strong-typed properties and raw byte[] values across Avro and Protobuf serialization. Changes include: updated Avro schemas and generated classes to include "bytes" for property values; extended AvroValueContainer to accept complexProperties, complexPropertyValues and an IComplexTypeConverterFactory, with a BuildPropertyValue helper and ConvertBack handling. Protobuf: extended GenericValue.proto with ComplexType message and bytes/complextype variants, updated generated Protobuf classes and added ComplexType constructors/GetContent conversion helpers that use IComplexTypeConverterFactory. New interfaces and factory (IComplexTypeConverter, IComplexTypeConverterFactory, ComplexTypeConverterFactory) were added and related SerDes and helper types updated (IValueContainer, DefaultValueContainer, KEFCore SerDes helpers, etc.). Also updated several storage/Kafka-related internal classes to accommodate the new value handling. These changes enable converting complex CLR types to storable representations and back when reading data.
masesdevelopers
added
documentation
masesdevelopers
added
KEFCore
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
âĶe-and-in-serializers
âĶ a property with the same name of the containing Entity
Introduce PreferredConversionType (Text/Binary) to indicate serializer preference and update IComplexTypeConverter signatures to accept it. Adjusted all call sites and implementations to pass Text or Binary as appropriate (Avro/Protobuf use Binary; DefaultValueContainer uses Text). Made Properties nullable in DefaultValueContainer and fixed conversion direction calls (Convert vs ConvertBack). Updated example converter and docs (split char, exception logging) and removed an unused using in Protobuf file.
Collaborator
Author
|
Some tests fail for this reasons:
The PR can be merged |
masesdevelopers
deleted the
445-manage-complextype-both-into-the-engine-and-in-serializers
branch
February 27, 2026 00:50
masesdevelopers
added a commit
to masesgroup/KNet
that referenced
this pull request
Feb 28, 2026
Applies the fix of masesgroup/KEFCore#446 to avoid errors when executing some tests
masesdevelopers
added a commit
to masesgroup/KNet
that referenced
this pull request
Feb 28, 2026
* Make RecordMetadata.DateTime nullable Return a nullable DateTime for RecordMetadata.DateTime and guard conversion with HasTimestamp(). This avoids producing a DateTime for records without a timestamp (returns null instead). Also updated the XML doc comment to reflect the nullable return and changed the conversion logic to only call Timestamp() when HasTimestamp() is true. * Apply the fix for test execution on GitHub Applies the fix of masesgroup/KEFCore#446 to avoid errors when executing some tests
masesdevelopers
added a commit
to masesgroup/KNet
that referenced
this pull request
Feb 28, 2026
* Make RecordMetadata.DateTime nullable Return a nullable DateTime for RecordMetadata.DateTime and guard conversion with HasTimestamp(). This avoids producing a DateTime for records without a timestamp (returns null instead). Also updated the XML doc comment to reflect the nullable return and changed the conversion logic to only call Timestamp() when HasTimestamp() is true. * Apply the fix for test execution on GitHub Applies the fix of masesgroup/KEFCore#446 to avoid errors when executing some tests
5 of 9 tasks
masesdevelopers
added a commit
to masesgroup/KNet
that referenced
this pull request
Feb 28, 2026
* Make RecordMetadata.DateTime nullable Return a nullable DateTime for RecordMetadata.DateTime and guard conversion with HasTimestamp(). This avoids producing a DateTime for records without a timestamp (returns null instead). Also updated the XML doc comment to reflect the nullable return and changed the conversion logic to only call Timestamp() when HasTimestamp() is true. * Apply the fix for test execution on GitHub Applies the fix of masesgroup/KEFCore#446 to avoid errors when executing some tests
masesdevelopers
commented
Mar 2, 2026
masesdevelopers
left a comment
masesdevelopers
left a comment
Collaborator
Author
There was a problem hiding this comment.
Update Avro and Protobuf serializers
| { | ||
| foreach (var item in complexProperties) | ||
| { | ||
| int index = item.GetIndex(); |
Collaborator
Author
There was a problem hiding this comment.
Do not use GetIndex because complexPropertyValues are stored based on the same index of complexProperties. See DefaultValueContainer of Json
| { | ||
| foreach (var item in complexProperties) | ||
| { | ||
| int index = item.GetIndex(); |
Collaborator
Author
There was a problem hiding this comment.
| @@ -53,41 +57,78 @@ public AvroValueContainer(IEntityType tName, IProperty[]? properties, object[] r | |||
| { | |||
| int index = item.GetIndex(); | |||
Collaborator
Author
There was a problem hiding this comment.
Same of https://github.com/masesgroup/KEFCore/pull/446/changes#r2873168382 is valid for properties and propertyValues
This was referenced Mar 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add complex type and bytes support to SerDes
Introduce support for complex/strong-typed properties and raw byte[] values across Avro and Protobuf serialization. Changes include: updated Avro schemas and generated classes to include "bytes" for property values; extended AvroValueContainer to accept complexProperties, complexPropertyValues and an IComplexTypeConverterFactory, with a BuildPropertyValue helper and ConvertBack handling. Protobuf: extended GenericValue.proto with ComplexType message and bytes/complextype variants, updated generated Protobuf classes and added ComplexType constructors/GetContent conversion helpers that use IComplexTypeConverterFactory. New interfaces and factory (IComplexTypeConverter, IComplexTypeConverterFactory, ComplexTypeConverterFactory) were added and related SerDes and helper types updated (IValueContainer, DefaultValueContainer, KEFCore SerDes helpers, etc.). Also updated several storage/Kafka-related internal classes to accommodate the new value handling. These changes enable converting complex CLR types to storable representations and back when reading data.
Add PreferredConversionType and update converters
Introduce PreferredConversionType (Text/Binary) to indicate serializer preference and update IComplexTypeConverter signatures to accept it. Adjusted all call sites and implementations to pass Text or Binary as appropriate (Avro/Protobuf use Binary; DefaultValueContainer uses Text). Made Properties nullable in DefaultValueContainer and fixed conversion direction calls (Convert vs ConvertBack). Updated example converter and docs (split char, exception logging) and removed an unused using in Protobuf file.
Related Issue
closed #445
Motivation and Context
How Has This Been Tested?
Screenshots (if appropriate):
Types of changes
Checklist: