Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Runtime developer can run Networking tests without Loopback server capability (on mobile/browser platforms) #42852

Closed
7 tasks done
Tracked by #44314
lewing opened this issue Sep 29, 2020 · 34 comments
Closed
7 tasks done
Tracked by #44314
Labels
area-System.Net.Http Bottom Up Work Not part of a theme, epic, or user story Priority:2 Work that is important, but not critical for the release Team:Libraries test-enhancement Improvements of test source code User Story A single user-facing feature. Can be grouped under an epic.
Milestone

Comments

@lewing
Copy link
Member

lewing commented Sep 29, 2020

Some platforms don't support listening to incoming requests but do support making outgoing requests. Most of the existing http functional tests expect to be able to start up a loopback server to respond to incoming request which means the tests cannot run on the more restricted platforms.


Plan:

@Dotnet-GitSync-Bot Dotnet-GitSync-Bot added area-System.Net.Http untriaged New issue has not been triaged by the area owner labels Sep 29, 2020
@ghost
Copy link

ghost commented Sep 29, 2020

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

@akoeplinger
Copy link
Member

Some of the http cert and websocket tests use an echo server hosted in Azure, presumably we could do the same for the basic http scenarios:

public static string EchoClientCertificateRemoteServer => GetValue("DOTNET_TEST_HTTPHOST_ECHOCLIENTCERT", "https://corefx-net-tls.azurewebsites.net/EchoClientCertificate.ashx");

@wfurt
Copy link
Member

wfurt commented Sep 29, 2020

If the tests can run under platform emulator using containers may be better - like we do for stress and enterprise authentication. Dependency on external servers was problematic in the past and all the tests using them are marked as Outerloop.

@stephentoub
Copy link
Member

stephentoub commented Sep 29, 2020

Some of the http cert and websocket tests use an echo server hosted in Azure

Some, yes. Two problems:

  1. Going off-box results in significantly more instability of the tests, which is why most of our networking tests that do so have been relegated to outerloop, and why we've pushed to move any tests that don't require going remote to not do so. Even so, there have been discussions in the past about making it configurable.
  2. Lots of test use shared in-memory state to coordinate between the client and server, in some cases just as a way to pass data back and forth, but in other cases actually synchronizing execution to ensure proper ordering for the what the test is validating. That's much harder (if not impossible) to do well with a remote server.

@wfurt
Copy link
Member

wfurt commented Sep 29, 2020

for the state, can we use the new connection hook and use transport other than TCP socket? Pipe, memory stream or something else?

@halter73
Copy link
Member

I'm not sure what these tests are using for the server-side, but I've started using the new SocketsHttpHandler connection hook and an in-memory stream (based on System.IO.Pipelines) in some Kestrel tests.

https://github.com/dotnet/aspnetcore/blob/925926c6156e56c6414ad004e5f1e54ced8c29c4/src/Servers/Kestrel/test/InMemory.FunctionalTests/Http2/Http2EndToEndTests.cs#L49-L61

@stephentoub
Copy link
Member

Yes, we could use the connection callback for SocketsHttpHandler to use something other than sockets for testing... but that would only apply to SocketsHttpHandler. It wouldn't address the browser-based handler that is currently the crux of @lewing's issue. If we were to expose similar callbacks on HttpClientHandler (which wraps the browser-based handler), then maybe we could, but I don't know if the browser-based handler would even be able to support it... I suspect not... but maybe in that case supplying such a connection callback would actually opt you out of the browser-based handler? I'm waving my hands.

@stephentoub stephentoub added test-enhancement Improvements of test source code and removed untriaged New issue has not been triaged by the area owner labels Sep 30, 2020
@stephentoub stephentoub added this to the 6.0.0 milestone Sep 30, 2020
@lewing
Copy link
Member Author

lewing commented Nov 12, 2020

We've resolved some other blocking issues for testing the networking side in CI for Browser but we still need some way to test actual network requests even if they are going to a local endpoint.

@karelz karelz added the Bottom Up Work Not part of a theme, epic, or user story label Dec 7, 2020
@karelz karelz added Team:Libraries User Story A single user-facing feature. Can be grouped under an epic. Priority:2 Work that is important, but not critical for the release labels Jan 4, 2021
@karelz karelz changed the title Many functional http test require a loopback server. As Runtime developer, I want to run Networking tests on mobile platforms (without loopback server) Jan 4, 2021
@karelz karelz changed the title As Runtime developer, I want to run Networking tests on mobile platforms (without loopback server) As Runtime developer, I want to run Networking tests without loopback server capability (mobile/browser platforms) Jan 12, 2021
@karelz karelz changed the title As Runtime developer, I want to run Networking tests without loopback server capability (mobile/browser platforms) Runtime developer can run Networking tests without Loopback server capability (on mobile/browser platforms) Jan 12, 2021
@geoffkizer
Copy link
Contributor

I don't really see any alternative here other than remote server tests. These platforms don't support doing loopback and they don't support connection-level hooks. They basically only support talking to a remote server.

As mentioned above, we do have some remote server tests, but in general they are not factored in a way that would make it easy to run only these tests (and not the loopback tests). One thing that would help here would be to refactor these tests so remote server tests are separated from loopback tests; see also #30205

The remote server tests we have today are fairly limited, but there's no reason we couldn't add some more here if we wanted. I don't think we need anywhere near the coverage for mobile/browser scenarios that we want for SocketsHttpHandler, because we are mainly relying on the platform HTTP implementation in these cases, whereas SocketsHttpHandler is implementing HTTP itself and thus we want to do extensive on-the-wire validation (which, as @stephentoub pointed out above, is only possible when using a local, in-memory server).

@wfurt
Copy link
Member

wfurt commented Jan 22, 2021

The "loopback" can possibly be implemented externally IMHO. Let say we have external helper outside of emulator proxying connection. The "loopback" can connect to it and tell client what port to connect to. We would end-up with two stream both in process so I think most of the current logic would work.

public LoopbackServer(Options options = null)
{
_options = options ??= new Options();
try
{
_listenSocket = new Socket(options.Address.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
_listenSocket.Bind(new IPEndPoint(options.Address, 0));
_listenSocket.Listen(options.ListenBacklog);
var localEndPoint = (IPEndPoint)_listenSocket.LocalEndPoint;
string host = options.Address.AddressFamily == AddressFamily.InterNetworkV6 ?
$"[{localEndPoint.Address}]" :
localEndPoint.Address.ToString();
string scheme = options.UseSsl ? "https" : "http";
if (options.WebSocketEndpoint)
{
scheme = options.UseSsl ? "wss" : "ws";
}
_uri = new Uri($"{scheme}://{host}:{localEndPoint.Port}/");

All the helper would need to do is to pick a port, give it back to the code above and splice any data to/from the first connection to it.

Is UnixDomainSocket (or something similar) viable option @lewing? We could hijack the actual TCP and we could steer it to "loopback" via other means. Once again we would end up with test process having both sides locally available. I don't know if the platforms handlers could work on UDS but curl can AFAIK so there is at least some precedent.

@geoffkizer
Copy link
Contributor

I'm not quite understanding what you are proposing, and I'm not sure how it addresses the "close coordination" issue. Maybe some code examples would help?

@wfurt
Copy link
Member

wfurt commented Jan 24, 2021

I put together crude prototype @geoffkizer
I made small changes to Loopback: https://github.com/dotnet/runtime/compare/main...wfurt:remoteLoop?expand=1

I pick test that uses LoopbackServer.CreateClientAndServerAsync(...)
before running the test I set HELPER=192.168.0.106:10000
I started the helper on 192.168.0.106.
The loopback can run successfully without any changes and without listening and incoming socket.

With that, we should be able to run existing tests without modifications. Timing my be different but if the helper runs on Helix machine where the emulator runs, everything will be done without hitting any real network. It would be up to the logic starting the emulator to also start the loopback helper and set the environment (or let tests know via some other means.

Let me know if that make some more sense.

BTW here is the simplified helper I used for testing

helper.cs
using System;
using System.Net;
using System.Net.Sockets;
using System.Threading.Tasks;

namespace helper
{
    class Program
    {
         static async Task relayData(Socket s1, Socket s2)
        {
            using (NetworkStream ns1 = new NetworkStream(s1, ownsSocket: true))
            using (NetworkStream ns2 = new NetworkStream(s2, ownsSocket: true))
            { 

                Task t1 = ns1.CopyToAsync(ns2);
                Task t2 = ns2.CopyToAsync(ns1);
                // TBD errors and propagate half-close
                Task.WaitAll(t1, t2);
                Console.WriteLine("relayData all done");
            }
        }

        static void Main(string[] args)
        {
            byte[] port = new byte[2];
            int listenPort = 10000;

            if (args.Length > 0)
            {
                listenPort = int.Parse(args[0]);
            }
            var listener = new Socket(AddressFamily.InterNetworkV6, SocketType.Stream, ProtocolType.Tcp);
            listener.DualMode = true;
            listener.Bind(new IPEndPoint(IPAddress.IPv6Any, listenPort));
            listener.Listen(1);

            while ( true )
            {
                Socket s1 = listener.Accept();
                Console.WriteLine("Got new connection from Loopback {0} {1}", s1.LocalEndPoint, s1.RemoteEndPoint);

                var l  = new Socket(s1.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
                if (((IPEndPoint)s1.LocalEndPoint).Address.IsIPv4MappedToIPv6)
                {
                    l.DualMode = true;
                }

                l.Bind(new IPEndPoint(((IPEndPoint)s1.LocalEndPoint).Address, 0));
                l.Listen(1);
                // Since we binded, the new listener has port assign.
                int p = ((IPEndPoint)l.LocalEndPoint).Port;
                port[0] = (byte)(((IPEndPoint)l.LocalEndPoint).Port & 0xff);
                port[1] = (byte)(((IPEndPoint)l.LocalEndPoint).Port >> 8);

                // send it to loopback so client knows where to connect to.
                s1.Send(port);

                // This will block until HttpClient connects 
                Socket s2 = l.Accept();
                l.Close();

                // Send one byte to signal presense of connected client.
                // We may send whole RemoteEndPoint if that is ever interesting.
                s1.Send(port, 0, 1, SocketFlags.None);
                Task.Run(() => relayData(s1, s2));
            }
        }
    }
}

@pavelsavara
Copy link
Member

pavelsavara commented May 12, 2021

Hi @wfurt, @karelz ,

I started working on solving this problem for the browser/mobile.
In the first phase I'm not solving generic loopback socket, just the echo services.
The idea is to host the echo server in the xharness process.

I have very simple draft, in which I re-implemented 2 endpoints.
It works fine but it has two problems:

  • the echo server code would move away from the tested code, into xharness repo. It makes network improvements into multi-repo PRs.
  • the server code would be duplicated

I would like to be able to reuse the endpoint handler code and keep it in the runtime repo.
I thought that I would extend xharness to be able to consume assembly with the echo handlers as binary assembly via commandline switch.

To be able to do that I would have to upgrade code of the existing CoreFxNetCloudService from ASP.NET 4.5.1 to .NET core/kestrel. Unfortunately the current project doesn't even compile for me.

Right now I'm thinking that I would create new kestrel library project side by side with WebServer and make it work with xharness.

As it would eventually become functionally 1:1 with the old implementation, you could take over the new code and deploy it to Azure instead of the current one.

Does that work for you ? Do you have any suggestions ?

@karelz
Copy link
Member

karelz commented May 12, 2021

@pavelsavara I don't think the source code you found is used for our Azure endpoints.
I believe the latest version of the source code now lives in https://github.com/davidsh/corefx-net-endpoint which we have longer-term plan to move into dotnet org (I don't think it happened yet).

Can you check that source code? If it is usable for your efforts, we can move it into Runtime repo. That may save you some effort ...

@wfurt
Copy link
Member

wfurt commented May 12, 2021

I'm not sure if the echo belongs to xharness. Many tests still need close coordination and as far as I understand, outbound connections work so using the Azure endpoint (or what ever else) should work.
What we discussed with @lewing was idea of plumbing socket proxy to xharness so we can sort of emulate inbound connections. Perhaps we can chat on Teams to get in sync @pavelsavara ?

@premun
Copy link
Member

premun commented May 13, 2021

I am not sure how much you are talking about mobile devices here and how much about WASM only, but please bare in mind that on Apple devices we have in Helix (iOS 14), local network access is not possible. You cannot open any connection to a local IP. This started with iOS 14 this year, so it's something new too.

We go around this by creating a TCP tunnel through the USB cabel and then XHarness and the phone connect to a port on their localhost. I don't think there is such limitation for public network.

Maybe I am off as I don't understand the whole scenario but just something to keep in mind ^^

@pavelsavara
Copy link
Member

pavelsavara commented May 13, 2021

We spoke with @wfurt yesterday and clarified that echo in xharness is just first step. It would unblock wasm developers to be able to create new unit tests. To to create necessary server side for testing wasm specific websocket scenarios locally, without Azure deployment.

My #52642 doesn't cover full scope of the original CoreFxNetCloudService, but I don't need it now. So I created #52693 to describe the gaps I know about.

@premun we briefly spoke about iOS limitations on Mono team meeting yesterday. @marek-safar suggested that Wasm and Android platforms are good first steps. As we discussed offline with @premun, we could enable TCP tunnel as later step for iOS.

About @wfurt idea of "asking external server to create the loopback", that could work with both Azure and xharness hosted server. For wasm, the unit test is running in the browser, where we don't have full socket, nor pipe. So we would have to implement the "ask"/coordination part with websockets. Again, this is later step from my perspective.

@karelz
Copy link
Member

karelz commented May 13, 2021

@pavelsavara I am a bit lost in the overall plan. What happens when and who does it?
Would it make se to agree on a plan and then keep up-to-date version in the top post?

@pavelsavara
Copy link
Member

@karelz I updated the issue description with my plan. I will talk to @lewing to confirm priorities. Thanks for your feedback.

@wfurt
Copy link
Member

wfurt commented May 13, 2021

What exactly do you mean by 'local network' @premun? Does it mean that you can access remote networks but not directly connected one? I assume that is some limitation of the emulator, right?

@premun
Copy link
Member

premun commented May 13, 2021

@wfurt this is for iOS devices. Simulators seem fine but I didn't try it much. I haven't tried remote networks. For those you might need some permission which you can put in the application's manifest.

Details on the local network access:
https://support.apple.com/en-mk/HT211870
https://developer.apple.com/videos/play/wwdc2020/10110/

You cannot connect to things that "don't look localhost" but I don't understand how Apple tells it's "local". We were originally having a direct connection and we were trying to connect to things like these (which ended up in the dialog):

  • fe80::1%1
  • 127.0.0.1
  • 10.0.0.14
  • fe80::f9:699f:80e1:33a2%6
  • 2a00:1028:838a:5156:1889:420c:e658:43cf
  • 2a00:1028:838a:5156:8194:4691:1156:90ed
  • fe80::6c5f:97ff:fe0c:597c%11
  • fe80::6c5f:97ff:fe0c:597c%12
  • fe80::c127:785a:da15:3304%13
  • fe80::563e:5f37:bc04:aba7%14

The original intention was so that you cannot scan devices around and figure out location or fingerprint people's home...

@wfurt
Copy link
Member

wfurt commented May 13, 2021

good to know @premun. I think the work @pavelsavara is doing can still work. Aside from external server, he is going to do simple prototype with "Loopback", Once that ready you can perhaps check if that would work on iOS devices and provide more feedback.

@pavelsavara
Copy link
Member

Both loopback and echo server are now also implemented as kestrel middleware for Xhareness process.
Many tests are now enabled for WASM platform.

@mdh1418 is looking at Android.

@mdh1418 mdh1418 assigned mdh1418 and unassigned mdh1418 Jun 23, 2021
@mdh1418
Copy link
Member

mdh1418 commented Jun 23, 2021

Closing this issue as the wasm side is now complete.
Separating mobile work into another issue #54626

@mdh1418 mdh1418 closed this as completed Jun 23, 2021
@ghost ghost locked as resolved and limited conversation to collaborators Jul 23, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-System.Net.Http Bottom Up Work Not part of a theme, epic, or user story Priority:2 Work that is important, but not critical for the release Team:Libraries test-enhancement Improvements of test source code User Story A single user-facing feature. Can be grouped under an epic.
Projects
None yet
Development

No branches or pull requests