projectdiscovery / fastdialer

Currently, the default state for HTTPX/Naabu/Nuclei is to use a set of default resolvers for DNS as defined in the DefaultResolvers variable within options.go in fastdialer.

// DefaultResolvers trusted
var DefaultResolvers = []string{
	"1.1.1.1:53",
	"1.0.0.1:53",
	"8.8.8.8:53",
	"8.8.4.4:53",
}

This is a deviation from the expected behavior, which is to use the host systems DNS configuration as a default. This is, for example, the way curl works.

There are a few significant drawbacks to doing this.

When doing an internal scan, some organizations block outgoing DNS requests that aren't going to their own DNS servers.
When conducting an internal red team style engagement where stealth is a factor, seeing a large number of DNS requests to 8.8.8.8, 1.1.1.1, .etc, may be a red flag for network defenders in an environment where DNS servers are explicitly set.

This is a deviation from the expected behavior, which is to use the host systems DNS configuration as a default. This is, for example, the way curl works.

@liquidsec this is actually intentional, using DefaultResolvers instead of system resolver in all the PD tools as default and optionally support using system resolver when required, the reason being, unlike curl, these tools dnsx/naabu/httpx/nuclei/tlsx meant to used and required to perform large scale dns resolution and relying on system resolvers resulted into multiple failures initially mainly because system resolvers are not sufficient to work with large inputs.

Thanks for replying. I can understand the decision, it makes sense that the public DNS servers can better handle the load. It might be helpful to explicitly state this in the documentation for each of the tools.

Is there any chance at at all that this had anything to do with initial failures? We noticed it in the first place being in an environment where 8.8.8.8,1.1.1.1, etc were suddenly blocked and the local system resolver wasn't very forgiving.

@liquidsec Thanks for opening this issue. By default, fastdialer, unless the user specifies otherwise, builds a list of base resolvers consisting of the public ones you provided (1.1.1.1:53, 1.0.0.1:53, 8.8.8.8:53, 8.8.4.4:53) along with the default system resolvers. As you can see from the default options, by default, we parse the system resolver file (ref

fastdialer/fastdialer/options.go

Line 58 in 1250d5e

ResolversFile: true,

) and the hosts' file (ref

fastdialer/fastdialer/options.go

Line 57 in 1250d5e

HostsFile: true,

). The default number of retries is 5, this ensures that after hitting the public resolvers, at least one system resolver is used.
I suspect that projectdiscovery/httpx#695 might be related to a different context, as the full URL doesn't match the one piped to the httpx tool, but the issue is still under investigation. Hopefully, with more details, it will be possible to determine the root cause.
Please let me know if I can provide further information or if you have any possible improvements/suggestions. Thanks!

The code seems to cover already the case mentioned - Converting to discussion

Default DNS resolvers should pull from system settings