net: Allow custom DNS resolvers

[chrisdickinson]

Per @bnoordhuis's suggestion in #2868:

[...] a change that makes the DNS resolver configurable in the call to net.connect() might be acceptable. [...]
The API I have in mind is something like this:

net.connect({
 host: "nodejs.org",
 port: 80,
 resolver: function(host, type, callback) {
   return dns.resolve(host, type, callback);
 }
});

It's up for debate if the 'type' argument should be called 'family' instead and be a number, like how dns.lookup() works. I'm open to well-reasoned suggestions.

The PR should be against master -- this would be a good first feature for someone looking to contribute (be sure to include tests, doc updates, and to follow the guidelines for contributing).

[trevnorris]

I like it. It'll allow to more easily tool things like creating a host lookup cache to reduce dns resolution time.

[cdlewis]

I thought I'd have a go at doing this as a first time contributor. I basically gave the user the option to substitute a function for dns.lookup, which is what is current used by net.connect. I also added some documentation and a test.

Hopefully I'm on the right track :)

#8564

[tjfontaine]

What is it exactly we're trying to solve for here?

Keep in mind that we have two kinds of name to ip resolution models in Node.js. First we have dns.lookup and second we have dns.resolve.

The contract of the interface for dns.lookup uses the system's resolution mechanism to associate a name to an ip. This can be managed by the system to provide what you might consider custom results for specific names. For instance, your /etc/nsswitch.conf has the entries files dns which means that first it tries to read /etc/hosts and if the name can't be found there it will then try and ask the internet for the result. But it doesn't have to do either of those things, and in fact may do something else entirely and do some automatic hash generation, or some other custom module the environment has designed for the system.

The second interface of dns.resolve and its related methods will only ever communicate with the internet via dns to associate a name to an ip.

net.connect uses [unless provided an ip] dns.lookup to resolve, which is the way the majority of all other platforms resolve names for you.

So what are we trying to solve for? What use case are we trying to satisfy?

A generic way to override the IP for a specific request? I could use my "custom" resolver before calling net.connect

Override the way all future net.connects work? You could wrap the socket with my own and proxy the lookup. You could do unfortunate monkey patching for net.connect or dns.lookup.

Are you trying to avoid having dns.lookup use the thread pool? This seems warranted, there are multiple ways to change this behavior:

• Node could overcome its static threadpool size and have something that scales
  • work to be done in libuv
• Node could have a threadpool for dns.lookup that doesn't share a threadpool with queued work or filesystem calls
  • internal to node's C++ layer
• Node could have a mechanism that switches the default for dns.lookup to only be dns.resolve
  • all in JS, but requires lots of thinking about the ramifications.

I'm not going to say this isn't a necessarily an easy change for someone to make, but before we make a change we need to make sure we understand what we're trying to solve for, and who our consumers will be.

Food for thought.

[trevnorris]

The idea of the change is to prevent us from needing to think of/maintain the "appropriate" way to handle this issue. Create an appropriate API to allow users to figure out the best solution for their needs.

[sam-github]

I'm a bit lost as to purpose, too.

Users can resolve the name any way they think best, and then pass the IP to net.connect, instead of passing a function to net.connect which it then calls in order to do the resolution you didn't.

[bnoordhuis]

I can't find it anymore but there was an issue around the same time where someone asked for a way to time the DNS queries that net.connect() and http.request() make.

Yes, the user can separate the query from the connection - but as net.connect() already conflates the two, you might as well make it pluggable. Done right, it should also clean up the connection logic in lib/net.js.

[trevnorris]

As @bnoordhuis explained, that is my hope for introducing these types of hooks.

[jasnell]

@chrisdickinson @trevnorris @bnoordhuis ... any further thoughts on this one?

[bnoordhuis]

It's been added to io.js. It should be possible to back-port to joyent/node if someone wants to take that on. If not, I suggest closing.

[jasnell]

I'll mark it as iojs-backport and defer but will also close it. I'm happy just to pick it up downstream