NTLM 401 macOS (Apple sillicon)

[montella1507]

Description

NTLM Authentication on macOS doesn't work correctly.

I have found this pull request:
#54101
but it doesnt seem to be working.

We are using ExchangeService from Microsoft.Exchange.WebServices and .NET Core 6+

However, it returns 401. Only on macOS.

Reproduction Steps

Any NTLM based authentication against windows server (exchange for the instance)

Expected behavior

It should work correctly.

Actual behavior

401 Unauthroized is returned - header is not present.

Regression?

I think so:
#54101

Known Workarounds

None

Configuration

No response

Other information

No response

[ghost]

Tagging subscribers to this area: @dotnet/ncl, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

NTLM Authentication on macOS doesn't work correctly.

I have found this pull request:
#54101
but it doesnt seem to be working.

We are using ExchangeService from Microsoft.Exchange.WebServices and .NET Core 6+

However, it returns 401. Only on macOS.

Reproduction Steps

Any NTLM based authentication against windows server (exchange for the instance)

Expected behavior

It should work correctly.

Actual behavior

401 Unauthroized is returned - header is not present.

Regression?

I think so:
#54101

Known Workarounds

None

Configuration

No response

Other information

No response

Author:	montella1507
Assignees:	-
Labels:	area-System.Net.Security, untriaged
Milestone:	-

[wfurt]

Can you get packet captures or traces @montella1507. It will be very difficult to troubleshoot without some more details.
And did you try it from plain Intel based macOS? And I assume you use SocketsHttpHandler ?

[montella1507]

Hi. I am using ExchangeService from Microsoft.Exchange.WebServices package.

I dont know how should i debug this issue. I have got here just by googling it is issue with NTLM on macOS when connecting windows machine...

https://stackoverflow.com/questions/65939196/ews-exchange-asmx-return-error-code-401-on-macos
#46247
#887

I may be able to provide traces...

Can you suggest a tool to trace network communication from .NET to remote server, for Apple Sillicon?

I prefer UI Based if possible.

[wfurt]

If you can make it work via HTTP, Wireshark is great tool. With some more effort, you can try to use https://github.com/wfurt/PcapStream. This is tool that allows you to capture inner unencrypted data. What we really want to see is exchange of authorization headers.

Reading this again, it is not clear to me if you tested .NET 7.0. I would strongly encourage that before doing anything else.

Alternatively, https://github.com/wfurt/Ntlm experiment had some debug parts that may be useful. While that is different implementation it can still help us to figure out what the server expects.

cc: @filipnavara for any other recommendations.

[filipnavara]

We actually do run the exact same EWS scenario, and we know it works. In fact, I have a permanent VM with Exchange and AD set up in the cloud, with unit tests hitting it and testing this scenario.

That said, there are some complex setups where the same credentials that work on Windows don't work on macOS. I don't remember the exact details but it's when the NTLM is relied from the Exchange server to the AD server the domain has to match what the AD expects (IIRC). In other terms, one of the fallbacks in the NTLM specification with relaxed domain name is not working, but there is always a set of credentials that do work.

Alternatively, NSUrlSessionHandler can be used for the HTTP handler, and that handles even the odd NTLM cases. It requires modifications to the EWS library but they are fairly trivial.

[filipnavara]

I found my notes about the specific scenario that works differently on macOS than on Windows.

Suppose you have a an AD server named CONTOSOAD, and Exchange server hosting domain server.com that authenticates against that AD. Then you have a client application on Windows with the user name set to user@server.com and password passWORD.

The actual protocol operates on a triple of Domain, User name, and Password. There are two ways to interpret the user name on the input. Either it's treated as a) Domain = "server.com", User = "user", or b) Domain = (empty), User = "user@server.com". .NET actually gives you control over this in the NetworkCredential class so you can specify either of them. Often the SERVER.COM\user syntax is translated to option a), and the "user@server.com" syntax to option b). However, GSSAPI on macOS always translates the option b) to option a) internally. There's no way to avoid that behavior.

The way the authentication with option b) works is that the Exchange server "server.com" relies the NTLM authentication to the AD server. When the AD server verifies the NTLM exchange it may know about the "user@server.com" alias, and that's the happy path. However, there are also cases where the AD doesn't know about the alias. NTLM has a quirk where the AD server strips the domain name, and tries to find the user again. Effectively it looks up user CONTOSOAD\user and if it exists and the password matches, it says that it's okay. So the authentication succeeds even if the domain/user name combination is not entirely correct (ie. you specify domain name of the Exchange server and not the one of the AD server).

For option a) the domain has to exactly match what the AD controller expects. In this case, the authentication would succeed with Domain = "CONTOSOAD", User = "user", Password = "passWORD". It would fail with Domain = "server.com", User = "user", Password = "passWORD". Often the name of the domain controller is not told to the end user, unfortunately. It is possible to get it from the Wireshark trace of HTTP communication though.

[montella1507]

I found my notes about the specific scenario that works differently on macOS than on Windows.

Suppose you have a an AD server named CONTOSOAD, and Exchange server hosting domain server.com that authenticates against that AD. Then you have a client application on Windows with the user name set to user@server.com and password passWORD.

The actual protocol operates on a triple of Domain, User name, and Password. There are two ways to interpret the user name on the input. Either it's treated as a) Domain = "server.com", User = "user", or b) Domain = (empty), User = "user@server.com". .NET actually gives you control over this in the NetworkCredential class so you can specify either of them. Often the SERVER.COM\user syntax is translated to option a), and the "user@server.com" syntax to option b). However, GSSAPI on macOS always translates the option b) to option a) internally. There's no way to avoid that behavior.

The way the authentication with option b) works is that the Exchange server "server.com" relies the NTLM authentication to the AD server. When the AD server verifies the NTLM exchange it may know about the "user@server.com" alias, and that's the happy path. However, there are also cases where the AD doesn't know about the alias. NTLM has a quirk where the AD server strips the domain name, and tries to find the user again. Effectively it looks up user CONTOSOAD\user and if it exists and the password matches, it says that it's okay. So the authentication succeeds even if the domain/user name combination is not entirely correct (ie. you specify domain name of the Exchange server and not the one of the AD server).

For option a) the domain has to exactly match what the AD controller expects. In this case, the authentication would succeed with Domain = "CONTOSOAD", User = "user", Password = "passWORD". It would fail with Domain = "server.com", User = "user", Password = "passWORD". Often the name of the domain controller is not told to the end user, unfortunately. It is possible to get it from the Wireshark trace of HTTP communication though.

I think that may be our case! Will check with our IT and will come back.

[ghost]

This issue has been marked needs-author-action and may be missing some important information.

[montella1507]

Hi @filipnavara

we have probably found the real cause of this issue. There may be issue with the length of the username. I was told, there is limitation in "login-mode" where syntax domain/username is used. The limit is set to 20 characters max for the username.

Our username is 21 characters long...

[montella1507]

Okay, i can confirm now.

The problem was too long username - [21-chars-long]@ipcue.com works, but domain/[21-chars-long] doesn't on windows AD. And because of the behaviour explained by @filipnavara on macOS - it doesn't work on macOS with [21-chars-long]@ipcue.com because it is translated to "domain/username" format.

Shall i close the issue?

[filipnavara]

Thanks for analyzing the issue, I really appreciate it. It's a scenario that I didn't encounter before. I don't know if there's anything actionable (either as possible workaround on the protocol side, something that could be reported to Apple, or something that could be tested as edge case in a unit test).

[montella1507]

Well.. i think only actionable thing here is to intercept the process and let the developer know, there is a problem with username handling on macOS, all actions may be "hacky" though - like throw the "argument exception" when the username has too many characters?

It is not a solution, it is something to help others in the future.

[wfurt]

https://stackoverflow.com/questions/704891/windows-username-maximum-length suggest the boundary is 20 characters. I did not find any specific limits in NTLM spec. It may be worth of getting capture and see if/how the name is truncated on the wire.

[filipnavara]

Adding a user with a long name looks like this:

You always get two user names assigned - one in the long form, and another shortened one. The shortened one will work with macOS since it uses the DOMAIN\user form. The long one is, unfortunately, not possible to send through GSSAPI since the system library always rewrites the user@DOMAIN syntax into DOMAIN\user equivalent.