ping and traceroute Unable to Resolve Hostnames While Other Programs Work Fine

I was chatting with a fellow OpenBSD user on IRC. He was having some very strange problems with name resolution on his OpenBSD machine.

ping www.openbsd.org
ping: no address associated with name

Name resolution as his non-root user worked fine with host or nslookup. Most programs would work without a hitch. However, he was unable to ping or traceroute any hosts on the internet. Both programs would return that there was no address associated with the name (any name).

We went through and checked his resolv.conf file for accuracy. We checked his /etc/hosts file. He was using a local nameserver, which he could query directly with no problems using tools like host or nslookup. He could query 8.8.8.8 directly as well; so no firewall issues.

However, if he put 8.8.8.8 in /etc/resolv.conf it would fail again.

I had him try it as root to try to rule out permissions issues. It still failed as root. So you would think no permissions issues. The programs ping and traceroute are both setuid root anyway.

Finally on a hunch I had him check the permissions on /etc/resolv.conf

It was like this:

-rw-r-----  1 root  wheel  66 Aug 22 02:10 /etc/resolv.conf

Hopefully you are screaming at the screen that you know the issue… That /etc/resolv.conf is only owner (root) and group (wheel) readable. No user outside the group wheel can read it.

So why would this make host and nslookup work fine and ping and traceroute fail?

Let’s think about that for a moment. If the gentleman I was chatting with on IRC’s local user was in group wheel (it was) then any program run as HIS user would work fine because /etc/resolv.conf was group wheel and was group readable.

Then why wouldn’t ping and traceroute work? They are setuid root. So root should be able to read resolv.conf…

They are setuid when they start, but the programmers are smart enough to change the uid after they get the raw socket needed for sending ICMP. (At least on OpenBSD ping they are.)

How can we see this? Try to ping some website. Start ping in the background as root:

ping www.OpenBSD.org &

Now search the process list for ping to see what user is running ping:

 ps auxww | grep ping
_ping    47674  0.0  0.0   668   596 p5  Sp      1:51AM    0:00.04 ping www.OpenBSD.org

WTF? How is ping now running as the user _ping? The ping program revokes its root privileges early by setting the user id to _ping. So even when run as root the ping program is only running as root for a short time to get the socket, and the changing over to _ping.

Don’t forget to kill that ping process you started in the background.

Well, the user _ping is not in the group wheel! So _ping cannot read /etc/resolv.conf ! And hence our program cannot resolve names. The program traceroute works in basically the same way. So all of this applies to it as well.

His solution was simple (as root):

chmod 644 /etc/resolv.conf

Now /etc/resolv.conf is world readable (other readable) as it should be, and everything works fine because the _ping user can access it, and all the other users can too, like the should be able to.

Problem solved.

What would have been another (possibly more efficient) approach to solving this?

ktrace of course! If we trace our system calls with ktrace we can see the problem our program is facing. In this case we need to run it as root since ping is setuid root.

# ktrace -f ping.out ping openbsd.org
ping: no address associated with name
# kdump -f ping.out | grep -n1 resolv.conf
365- 20730 ping     CALL  stat(0x1069c5f03a6d,0x7f7ffffcbef0)
366: 20730 ping     NAMI  "/etc/resolv.conf"
367- 20730 ping     STRU  struct stat { dev=0, ino=103983, mode=-rw-r----- , nlink=1, uid=0<"root">, gid=0<"wheel">, rdev=420253, atime=1598683868<"Aug 29 01:51:08 2020">.621464705, mtime=1598080210<"Aug 22 02:10:10 2020">.244384888, ctime=1598684275<"Aug 29 01:57:55 2020">.564346572, size=66, blocks=4, blksize=16384, flags=0x0, gen=0x0 }
--
375- 20730 ping     CALL  open(0x1069c5f03a6d,0x10000<O_RDONLY|O_CLOEXEC>)
376: 20730 ping     NAMI  "/etc/resolv.conf"
377- 20730 ping     RET   open -1 errno 13 Permission denied

The first occurrence of /etc/resolv.conf is just to stat() the file to make sure it exists and check permissions, etc. You can even see the bad permissions listed right there in the “STRU struct stat” line:

mode=-rw-r-----

The second occurrence of /etc/resolv.conf is where ping tries to open it. Now running as the user _ping, this has the return (RET) from the open() call:

RET   open -1 errno 13 Permission denied

It’s right there in black and white (green and black?)

So we could have used ktrace to diagnose this (possibly) a little faster. (Took about 10 mins of trying and checking various things.) It’s a great troubleshooting tool!

Hopefully you have learned a little something, or just figured out why ping or traceroute keep returning:

ping: no address associated with name

or

traceroute: no address associated with name

while other programs work fine.