Fun with FreeIPA and a slightly more complex DNS setup


The Plan

+---------+    +------------------------+    +---------------------------+
| FreeIPA | -> | upstream hidden master | -> | public facing dns servers |
+---------+    +------------------------+    +---------------------------+

Sounds simple enough right? Well …

The Fun

Let’s get right too it … FreeIPA only sends out notifications to the
NS records listed in the zone. But our hidden master is not reachable
from the outside and should not be listed as an NS.

‘But bind has “also-notify” just use that.’ you might say now. Which
is correct. So a quick check on the ldap scheme reveals there is no
setting in the LDAP tree for it. Ok… the nice solution is dead.

Let’s try the ugly one.

Notify settings in the global scope. It means we will notify our hidden
master also about zones it should not handle, but that should not cause
any problems besides maybe some unneeded log messages. Not too bad.

Added the settings. Fired up tshark again. Restarted named-pkcs11.

And guess what … it just notified the NS servers again. The also-notify
setting only caused notifications for the zones defined in the named.conf.

This smells like a regression in FreeIPA (lack of notify settings per zone)
and bind-dyndb-ldap (ignoring the global notify settings).

This is documented in Bug #6791.

The solution

+---------+    +------------------+    +------------------------+    +---------------------------+
| FreeIPA | -> | PDNS with script | -> | upstream hidden master | -> | public facing dns servers |
+---------+    +------------------+    +------------------------+    +---------------------------+

Well the workaround. I remembered that powerdns has lua scripting in basically
all their daemons nowadays. A quick check revealed there was also a hook for AXFR.
We fired up a new VM and installed pdns with sqlite3 on it.

The config:

# grep -vE '^(#.*|\s*)$' /etc/pdns/pdns.conf
allow-axfr-ips=<ip of hidden master>/32,,::1
also-notify=<ip of hidden master>
only-notify=<ip of hidden master>/32

And register the zone with:

pdnsutil create-slave-zone <ip of freeipa>
# verify it works with:
dig -t AXFR @<ip of freeipa>
dig -t AXFR @

Now to the lua script. The documentation for the axfrfilter is here.
So we copy that example and replace most of the stuff with our NS conditional.

--- /etc/pdns/filter-internal-ns.lua v1
function axfrfilter(remoteip, zone, qname, qtype, ttl, prio, content)
  if qtype == pdns.NS and content == "" then
    -- skip this record
    resp = {}
    return 0, resp

To enable the script we insert the following record in our little sqlite DB:

INSERT INTO domainmetadata (domain_id, kind, content)
       VALUES (3, "LUA-AXFR-SCRIPT", "/etc/pdns/filter-internal-ns.lua");

Did some dummy change in the zone and suddenly our pdns instance did not feel
responsible for the zone anymore. What happened?

During cleanup I deleted a little bit too much:

--- /etc/pdns/filter-internal-ns.lua v2
function axfrfilter(remoteip, zone, qname, qtype, ttl, prio, content)
  resp = {}
  if qtype == pdns.NS and qname == "" then
    -- skip this record
    return 0, resp
  -- preserve all others
  return -1, resp

This looked better. Our pdns felt authorative again for the zone. Though the NS
record was still there. A short pondering later … comparing “qname” to the content
of the NS record was wrong. We should compare “content”! Changed the line. And now
the script reported trying to compare nil values. A few debug prints later …
our value was in “prio”. Wait what? Maybe a better signature would be:

function axfrfilter(remoteip, zone, qname, qtype, ttl, field1, field2)

Anyway … v3:

--- /etc/pdns/filter-internal-ns.lua v3
function axfrfilter(remoteip, zone, qname, qtype, ttl, prio, content)
   resp = {}
   if qtype == pdns.NS and prio == "" then
      -- skip this record
      return 0, resp
   -- preserve all others
   return -1, resp
dig -t AXFR @

Now reported 1 NS record less than our freeipa. The logs (and tshark) confirmed
that our upstream hidden master received the changes and shortly after also our
public servers.

Time to call it a day.

A little extra fun with capability sets

Ok need to mention one more thing before. … Of course pdns did not start up
right away in my setup. I knew the pdns.service file had some security measures
enabled. In a first run i just copied pdns.service to /etc/systemd/system and
disabled them all. PrivateTmp, PrivateDevices, CapabilityBoundingSet, NoNewPrivileges,
ProtectSystem, ProtectHome all gone. “systemctl daemon-reload” and i could focus
on my real problem before fighting with this.

Later on i wanted a better solution.

  1. use /etc/pdns/pdns.service.d/
  2. maybe upstream the needed fix

So I reenabled all options and looked at our permissions. An upstream dev told me
it should actually work if the DB is in /var/lib/pdns/. This lead me to look at
the permissions again. They looked fine too:

drwxr-x--- 2 pdns pdns 6 Nov 16 01:33 /var/lib/pdns

And there it was … root cant read the /var/lib/pdns … what if pdns tries to open
the DB file as root? From fun with apparmor I knew that root reading such files requires
the DAC override capability. Quickly creating a config file for it and daemon-reload systemd:

# /etc/systemd/system/pdns.service.d/capset.conf

Voila … my pdns starts up again. The package in server:dns/pdns already has the fix and we will
upstream the patch.

And now it is time for the cinema.