Possible mergeable classes for merging code bases #1261
Comments
DavoudEshtehari
added
the
➕ Code Health
|
I think
Some classes like TdsParser, TdsParserStateObject, SqlDataReader, SqlCommand are very large and I don't think we can do in a single PR, it would be impossible to review. I believe the way to do those types is a many small PR's which take identical functionality from both into a new shared version of the file slowly reducing each project specific file to the parts which are different and then those can be thoroughly reviewed. |
|
Thanks for the comments. |
|
Looking at some of the merged from @lcheunglci I thought it might be useful to write down some of my knowledge and methodology used in my merges. From what I've gathered: The nextfx codebase was forked at some point and that fork had a managed implementation of the network interface written so that it could be used on linux for netcore. This means that this codebase now has managed and native versions of the Sql Network Interface (SNI) in the netcore codebase and they are switched out depending on the platform that you restore the nuget package for. You can also force the use of managed network in windows with an appcontext switch. This mostly makes a difference in the core of TdsParser and TdsParserStateObject. After netcore was forked the codebase was cleaned up a bit. So in general netcore is newer than netfx. However, changes and fixes may have been made to the netcore and netfx codebases independently so there's no guarantee that either was one better than the other they can both contain better versions of any particular function. The netfx codebase is very very old, it was in the original 1.0 release of netfx and it hasn't always been updated to use the latest features. An example of this is the pool key derivation mechanism on netfx, it uses direct api calls to windows security apis because the managed equivalents were not available or not know to be usable at the time of writing. This means that any change can be old or new and come from any version of the source code. To merge them you have to try to work out if they are functionally identical or equivalent enough that the difference cannot be observed by users up to and including timing and ordering of operations. This can get complicated and time consuming. Style between the two codebases is highly variable. In general when merging a file it's a good idea to turn on the compiler Info messages and look through all the possible refactoring options is gives you. You don't have to take all the suggestions just take a look at them and make the code feel like it has the same style as some of the already merged components. The objective is to work towards a coherently styled single codebase where naming and formatting are easy to understand between most files. |
|
Thanks @Wraith2 for your insight. I'll keep that in mind when I'm doing the code base merges. I'm currently starting small particularly focusing on the simpler code merges with the least conflicts before I look at the bigger ones. Your feedback and suggestions are much appreciated. |
|
While I was looking into |
Code merge for dotnet#1261 Issue for SmiXetterAccessMap class. Added the common code to .Common.cs class.
|
@DavoudEshtehari SqlEnums.cs was moved to shared in #1317 |
|
The PR referenced above (#2369) covers the merge of
This'd need some additional testing, but I think this would enable Always Encrypted support across Windows and Linux. If I get time, I might be able to look at DbConnectionPool, DbConnectionClosed, DbConnectionInternal, DbConnectionFactory, which are also very similar between the two projects. |
|
When I worked on the encryption portions of the codebases I found that they could not be merged because netstandard does not have one of the required crypto primitives. This forces the use of platform specific apis and ifdefs. There may be a comment in the code but i'm sure if you look up the PR's around it you should find the description in my PR. |
|
@Wraith2 So removing netstandard support will improve the abaility to merge code? |
|
I think I see the PR you're talking about @Wraith2 (#1022), thanks for that. Nowadays, the main problem lies in the Oddly, .NET Framework 4.6.1 (which implements .NET Standard 2.0) has this class. Other frameworks naturally implement it, so we could potentially have one code path for .NET Standard 2.1 and .NET Framework, then a PlatformNotSupportedException for everything else. This means that it wouldn't support the platform versions below:
I personally think it'd be justifiable not to explicitly add support for a platform that the vendors don't support themselves, but not adding anything for UWP is a pity. I don't think there's a way to include it though. Naturally if SqlClient as a whole drops support for .NET Standard (2.0 or more broadly) then this problem becomes academic! |
it definitely will do and specifically NetStandard 2.0 |
|
Just as an update on this: PRs #2369, #2376, #2383 and #2390 will merge SqlClientFactory, AlwaysEncryptedHelperClasses, TdsParserHelperClasses, AAsyncCallContext and DbConnectionPoolIdentity. They also lay the groundwork for the merge of SqlDataReader. I think I can also safely merge DbConnectionFactory, DbReferenceCollection, DbConnectionClosed, SqlAuthenticationProviderManager and SqlFileStream without raising too many eyebrows - the implementations are identical (or so close to identical that it barely matters.) I can see the open issue and PR to terminate .NET Standard support, and I'll wait for this to be merged before dealing with the various encryption/enclave providers. In one situation, specific functionality will need to be locked behind conditional compilation: the .NET Framework's MDS supports the This leaves the more complicated cases, where functionality and support have diverged. So far I've come across a few of these:
Once there's the .NET Standard PR is implemented and there's nothing more I can merge, I'll take another look at these situations and see if they're as difficult as they look. I'm not planning to look at TdsParser, SqlCommand or SqlConnection any time soon. These probably need someone with a much better understanding of the library's history than me. |
|
@DavoudEshtehari In the wake of that removal, #2501 merges three of the four The remaining derived class is Once #2501 and the extra PR (edit: #2521) have been merged, there'll be room for Unix support for Always Encrypted. |
|
I've reached a stopping point with the merge, so it's now just a matter of triage. To summarise the table I've been using to keep track: Files covered by PRs
SQLCLR, code access security typesAll of these depend upon #2862. There's not much code to merge here, just a lot of files. I think there'll probably be 3-4 PRs for these.
Metrics, loggingThese use two different mechanisms between .NET Framework and .NET Core, and they also record metrics at slightly different times. It might be better to merge the API surface and verify the metric timings as part of any future OpenTelemetry work. The issue tracking that is #2210 and #2211.
Native SNIThis'll change the public-facing dependencies for the NuGet packages. Barring any objections, I'm planning to change the .NET Framework package to reference Microsoft.Data.SqlClient.SNI.runtime. This'll mean that there's one architecture-dependent DLL file in the resultant builds, not three DLLs which are toggled between at runtime.
TDS parserI'm pretty confident that the majority of the logic is identical, it's just obscured by the fact that only one codebase was refactored. There are probably some methods which can be merged as-is, but for the vast majority of cases I think we'll just need to repeat that refactor piecemeal. I'm not planning to start until #2714 is merged, given how deeply any refactor work will touch the state machine. |
|
The work to merge the two codebases has been going on for a long time. The problem is always testing and review. |
|
I've definitely seen that - there's a lot of history here. A lot of the problem seems to be that we've got a mixture of different types of changes:
Reviewing and testing PRs which contain any combination of those is always going to be tricky; many of the remaining classes have so much of the first two types of change that it's obscuring the changes which need the most review. I've seen a few "style change" PRs which touch large parts of both codebases. Those are disruptive when merged and their size makes them harder to review, but they might be a safer way to fix those first type of change and free up some bandwidth to review the more important changes. |
The main aim of this ticket is to clarify the project progress and having contribution with who is interested in it.
At the first step, we've prepared a list of possible mergeable files with their types which maybe need some pruning. Then, priority could be declared before starting to merge.
Mergable files list
The text was updated successfully, but these errors were encountered: