When we enter the name of the server or the domain name of the site in the browser, perform a ping or launch any remote application, the operating system must convert the specified names into an IP address. This process is called domain name resolution. At first glance, it may seem very transparent, but a multi-layered mechanism is hidden behind it.
This article is the beginning of a series devoted to the low-level architecture of name resolution. Let's talk about how this process is organized in Linux at the kernel level, various C libraries and system calls.
Many people know that the name resolution process in Linux is not just a "DNS call", but a chain of libraries, configuration records and calls that depend on the implementation of a specific application, the types of libraries used and system settings.
However, engineers still have questions. For example, does the application need to be restarted after the DNS server address has changed? In addition, in order to diagnose errors, timeouts, and other problems with name resolution in the application and in the system, it is important to understand how this entire chain works — from getaddrinfo() to resolv.conf. In this part, we will try to analyze everything layer by layer and collect some fundamental base in a short and accessible form.
The tip of the iceberg
Almost all modern Linux applications, from curl to systemd, use the getaddrinfo() function from the standard C library (glibc or musl). It is she who performs the main work of translating the domain name into IP addresses (A, AAAA-records) depending on the settings and the request.
At the same time, it not only performs DNS queries, but also processes other types of data, such as service names, for example, converts the name of the network service "http" to port 80 using /etc/services. This makes it a universal tool for network applications.
The getaddrinfo() function returns a list of addrinfo structures, each of which contains an IP address, socket type, protocol, and other parameters. This allows applications to choose the most appropriate address to connect to.
Example of using getaddrinfo() in pseudocode:
struct addrinfo hints, *res;
zero_memory(hints);
hints.ai_family = ANY_FAMILY;
hints.ai_socktype = TCP;
err = getaddrinfo("example.com", "http", hints, &res);
if (err == 0) {
for each addr in res:
use(addr)
freeaddrinfo(res);
} else {
print(gai_strerror(err));
}
At the same time, getaddrinfo() is the tip of the iceberg. To obtain an IP address, it calls a chain of internal mechanisms prescribed in the configuration data of the system. One of these mechanisms is NSS (Name Service Switch).
NSS
NSS is implemented on the basis of loadable modules — dynamic libraries corresponding to the glibc API, such as libnss_dns.so, libnss_files.so, libnss_myhostname.so and others. They function as plugins and are loaded by the glibc library at runtime, responsible for specific methods of resolving IP addresses. The order and set of sources used for name resolution is specified in the /etc/nsswitch.conf configuration file.
An example of the content of nsswitch.conf:
# /etc/nsswitch.conf
passwd: files systemd
group: files systemd
shadow: files
gshadow: files
hosts: files dns myhostname
networks: files
protocols: db files
services: db files
ethers: db files
rpc: db files
netgroup: nis
For example, a line in modules with the content ``hosts: files dns'' says that the local /etc/hosts file is first searched for a match, and if the files module returns a result, then subsequent modules such as dns (which makes a DNS query) will not be called.
Accordingly, if the hosts line in nsswitch.conf does not include a mention of the dns module, the configuration file resolv.conf, which contains the settings for accessing DNS sources, will be ignored, and the DNS request will not be generated.
The mdns (for Zeroconf/Avahi), nis (in old systems) and myhostname modules can also be used in NSS.
The myhostname module is part of systemd and is used to resolve the local hostname. It is not always present in minimalistic systems such as Alpine Linux.
Libraries
The following libraries of the Linux ecosystem are key and provide applications with a certain set of functions, including domain name resolution.
Glibc is the most common implementation of the C standard library, which implements high-level functions such as getaddrinfo(). It interacts with NSS (Name Service Switch) to determine name resolution sources (for example, /etc/hosts, DNS) and uses the libresolv library to perform DNS queries.
Glibc can use system calls such as sendto and recvfrom to send and receive DNS queries over UDP or TCP. Widely distributed in most Linux distributions (Ubuntu, Debian, Fedora, etc.)
Musl is an alternative C standard library designed with minimalism, performance, and POSIX compatibility in mind. It is used in lightweight distributions such as Alpine Linux.
Musl implements domain name resolution directly, without using NSS, independently reads /etc/hosts and /etc/resolv.conf and sends DNS requests without using external libraries like libresolv. However, musl has limitations in supporting some resolv.conf parameters, such as rotate or complex search.
Libresolv.so is a part of glibc that performs low-level DNS work by performing requests such as res_query() and res_send(), but can be used independently in some applications such as nslookup (which allows DNS queries to be performed directly, bypassing standard name resolution mechanisms).
Libresolv uses glibc to perform DNS queries when NSS indicates that DNS needs to be addressed. It reads /etc/resolv.conf, forms DNS packets and sends them to specified servers via UDP or TCP.
It is worth noting that some applications, for example, written in Go, can completely bypass glibc/musl and use their own DNS resolvers.
How resolv.conf is processed
The file /etc/resolv.conf contains the basic settings of the DNS client, namely: list of servers, parameters, search domains. For example:
nameserver 192.168.1.1
search dev.local
options timeout:2 attempts:3
Glibc and libresolv parse it manually if necessary.
Important points and limitations:
options such as rotate, ndots, timeout and attempts affect the behavior of the request;
the rotate option is used for cyclic selection of servers from the nameserver list, but it is not supported in musl;
search is used for autocompletion, for example, if the name db01 is not an FQDN, domains from the search directive will be substituted for it in turn.
It is important to note that the resolv.conf file can be dynamically changed by the DHCP client, NetworkManager, or the resolvconf utility, which can cause confusion when solving DNS problems. We will talk about this in one of the following parts.
What does res_query() do
This is a libresolv function called internally during name resolution. It forms a DNS packet manually and sends it to the DNS servers specified in resolv.conf. It is used by utilities like nslookup, as well as some programs that bypass getaddrinfo().
The function sends DNS requests using res_send() over UDP, and if necessary, for example, when receiving responses exceeding 512 bytes, it switches to TCP.
Important: when using res_query() you will not get information from /etc/hosts, NSS or other sources. This is a pure DNS request. Therefore, dig or nslookup can get one result, and, for example, ping or curl - a completely different one.
Res_query() is considered a deprecated function, it is not recommended to use it. For more convenient and safe work with DNS, it is better to give preference to getaddrinfo() or such libraries as c-ares or libdns.
c-ares is a lightweight library for asynchronous DNS queries, often used in highly loaded applications (for example, curl and Node.js)
libunbound (from the Unbound project) is a more powerful library with DNSSEC support and flexible request settings.
Order of implementation of requests and priorities
Here is a typical order of name resolution in Linux when using glibc and NSS:
the application calls getaddrinfo();
getaddrinfo() addresses the NSS system and follows the order specified in nsswitch.conf;
if the files module is specified first, the name is searched in the /etc/hosts file;
if the dns module is enabled, NSS calls libnss_dns.so, which calls functions from libresolv;
libresolv forms a DNS query using res_query() and sends it using res_send() to the addresses of DNS servers specified in resolv.conf, then receives and returns IP addresses.
Important: if the name is found in one of the steps, for example, in hosts, subsequent sources are not used.
On minimalistic systems, such as Alpine Linux with musl, the order may be different, since musl does not use NSS and performs DNS queries directly by reading /etc/hosts and resolv.conf on its own.
Some applications and languages (for example, Go, Java, Node.js) can use their own DNS resolvers, completely ignoring system settings.
For example, let's analyze the work of the curl utility.
Team:
strace -f -e trace=network curl -s download.astralinux.ru > /dev/null
strace output:
socket(AF_INET6, SOCK_DGRAM, IPPROTO_IP) = 3
socketpair(AF_UNIX, SOCK_STREAM, 0, [3, 4]) = 0
socketpair(AF_UNIX, SOCK_STREAM, 0, [5, 6]) = 0
trace: Process 283163 attached
[pid 283163] socket(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 7
[pid 283163] connect(7, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
[pid 283163] socket(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 7
[pid 283163] connect(7, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
[pid 283163] socket(AF_INET, SOCK_DGRAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP) = 7
[pid 283163] connect(7, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("172.24.31.107")}, 16) = 0
[pid 283163] sendmmsg(7, [{msg_hdr={msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\250\207\1\0\0\1\0\0\0\0\0\0\10download\nastralinux"..., iov_len=40}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, msg_len=40}, {msg_hdr={msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\240\215\1\0\0\1\0\0\0\0\0\0\10download\nastralinux"..., iov_len=40}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, msg_len=40}], 2, MSG_NOSIGNAL) = 2
[pid 283163] recvfrom(7, "\250\207\201\200\0\1\0\1\0\0\0\0\10download\nastralinux"..., 2048, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("172.24.31.107")}, [28->16]) = 56
[pid 283163] recvfrom(7, "\240\215\201\200\0\1\0\0\0\1\0\0\10download\nastralinux"..., 65536, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("172.24.31.107")}, [28->16]) = 114
[pid 283163] sendto(6, "\1", 1, MSG_NOSIGNAL, NULL, 0) = 1
[pid 283163] +++ exited with 0 +++
socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 5
setsockopt(5, SOL_TCP, TCP_NODELAY, [1], 4) = 0
setsockopt(5, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0
setsockopt(5, SOL_TCP, TCP_KEEPIDLE, [60], 4) = 0
setsockopt(5, SOL_TCP, TCP_KEEPINTVL, [60], 4) = 0
connect(5, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("130.193.50.59")}, 16) = -1 EINPROGRESS (The operation is currently in progress)
getsockopt(5, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
getpeername(5, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("130.193.50.59")}, [128->16]) = 0
getsockname(5, {sa_family=AF_INET, sin_port=htons(48488), sin_addr=inet_addr("172.24.31.241")}, [128->16]) = 0
sendto(5, "GET / HTTP/1.1\r\nHost: download.a"..., 86, MSG_NOSIGNAL, NULL, 0) = 86
recvfrom(5, "HTTP/1.1 200 OK\r\nServer: nginx/1"..., 102400, 0, NULL, NULL) = 1617
What we see in this strace
- Trying to use NSCD (Name Service Cache Daemon)
connect(..., "/var/run/nscd/socket", ...) = -1 ENOENT
This means that glibc first tries to use the name cache from NSCD if it is running. There is no ego in the system, and the request goes further.
- Call socket() and connect() to the DNS server
socket(AF_INET, SOCK_DGRAM|..., IPPROTO_IP) = 7
connect(7, ..., sin_addr=inet_addr("172.24.31.107")...)
Here, a UDP socket is created to access the DNS server specified in /etc/resolv.conf.
- Call sendmmsg() — sending DNS requests
sendmmsg(7, [ { "download.astralinux.ru" }, { "download.astralinux.ru" } ], ...)
Requests for name resolution are sent here.
- Response from DNS
recvfrom(...) = 56
recvfrom(...) = 114
Now the IP address is known.
56 is the size of the DNS response in bytes containing the A record (IPv4 address)
114 - the size of additional data, for example, CNAME, or authoritative servers in the case of a recursive request.
- TCP connection over IP
connect(5, ..., sin_addr=inet_addr("130.193.50.59"))
Here, curl itself establishes a TCP connection to the IP address returned by getaddrinfo().
So when we call curl, we don't see the DNS queries directly - they are made by the glibc library inside the getaddrinfo() call. But strace allows you to see indirect signs:
Among the calls will be a request to connect to nscd, a connect() call to the DNS server, sending a UDP packet via sendmmsg(), and then a standard TCP connection over IP:
connect(7, {AF_INET, 172.24.31.107:53}) = 0
sendmmsg(7, [{ "download.astralinux.ru" }]) = 2
recvfrom(7, ...) = ...
connect(5, {130.193.50.59:80}) = 0
It is important to note that the behavior of getaddrinfo() may depend on the libc implementation. For example, in glibc, the results can be cached, which affects the performance and relevance of the data.
Brief summary and key points
A DNS query in Linux is not necessarily a query to a DNS server. The reverse chain can include hosts, NSS, glibc and other sources.
NSS and nsswitch.conf define the order and sources of name resolution.
glibc uses NSS and can cache results; musl implements DNS resolving directly with limited support for resolv.conf options.
Resolv.conf controls resolver settings, but can be changed dynamically.
Getaddrinfo() is the main interface for name resolution, handles both DNS and other sources.
Different programming languages (Go, Java, Python with dns.resolver, Node.js) can use their own DNS query mechanisms.
In the next part, we will supplement the picture with a general idea of how DNS records are cached — a key mechanism that directly affects the performance, reliability, and behavior of applications when changing IP addresses.