Onboarding
Registration
- Contact info@dnstapir.se (?)
- Provide User/Organisation, contact person, contact information (email/phone)
- Provide PGP key, Signal user handle or other trusted out-of-band channel.
Contract
- In the test phase of DNS TAPIR all users need a contract with the TAPIR Core operations partner (The Swedish Internet Foundation, IIS)
- Consumer only might not need this in the future.
Enrollment Key
- After the contract is signed, the organisation/user will receive enrollments credentials from a DNS TAPIR representative on a trusted out-of-band channel.
Consumer only
- Not available in first phases
Offboarding
Sad to see you leave but when all other options are not available this might happen
- Terminate the contract with the operations provider
- Keys will be de-activated and prevented from renewal. This will be matched with the termination date of contract.
- Uninstall TAPIR EDM, TAPIR POP, TAPIR-CLI
- Data will no longer be sent to TAPIR Core
Overview
In DNS TAPIR Edge, measures are taken to avoid the transmission of sensitive data from Edge to Core. We specifically want to avoid transmitting two things: i) IP addresses of end-users and ii) sets of queries that are known to stem from a small number of end-users. This document describes how we achieve this.
The Privacy-unfriendly Way
To understand how the DNS TAPIR does this, let's first study a privacy-unfriendly implementation. In such an implementation the data sent from Edge to Core could look something like the following:
{
"measurement_interval": {
"start": 2026-07-27 12:50:00+02:00,
"end": 2026-07-27 12:55:00+02:00
},
"domains": [
"example.com": {
"a_count": 1000,
"mx_count": 100,
"other_rtype_count": 200,
"nx_count": 0,
"ok_count": 1300,
"other_rcode_count": 0,
"clients": [
"192.0.2.1",
"192.0.2.10",
"192.0.2.100"
]
},
"example.org": {
"a_count": 200,
"mx_count": 20,
"other_rtype_count": 40,
"nx_count": 0,
"ok_count": 260,
"other_rcode_count": 0,
"clients": [
"192.0.2.10",
"198.51.100.100"
]
}
]
}
The data sent was collected during a 5-minute interval. Most of it is just counters, which do not harm the privacy of the end-users, and they can be aggregated over, say, the course of a day or a week if more long-term trends are being studied. Unfortunately, relying solely on counters for analysing DNS data is a bit meager. Therefore, this privacy-unfriendly example also transmits the IP-addresses of the end-users that asked the questions. While this allows for a detailed study of how individual end-users are behaving over time, it also violates their privacy and as such, it is NOT how DNS TAPIR does things.
A Naive "Solution"
A naive approach of mitigating the privacy violation might be to simply hash the IP addresses of the clients. That avoids transmitting the IP addresses, but it might still harm end-user privacy. Consider the following chunk of data:
{
"measurement_interval": {
"start": 2026-07-27 12:50:00+02:00,
"end": 2026-07-27 12:55:00+02:00
},
"domains": [
"gnesta.se": {
# counters omitted for brevity
"clients": [
"123abc",
"8d3e01"
]
},
"ssa.se": {
# counters omitted for brevity
"clients": [
"123abc",
"fa3957",
"492041",
"859a38"
]
},
"smogon.com": {
# counters omitted for brevity
"clients": [
"123abc",
"1930d8",
"9c958a"
]
},
"aa.org": {
# counters omitted for brevity
"clients": [
"123abc"
]
}
]
}
No IP addresses are being transmitted, but we can still study
end-user behavior with the same amount of detail. Unfortunately, being
able to do so might still reveal sensitive information about individual
end-users. From the data in the example above, one might infer that the
end-user whose IP address hashes to 123abc is someone who lives in
the small city of Gnesta, Sweden, is engaged in the Swedish amateur
radio community via ssa.se and the Pokemon community via smogon.com
and has problems with alcohol, as suggested by aa.org. Arguably, not
a lot of people fit a description like that and in the DNS TAPIR
project, we consider this a privacy violation as well.
A "Solution" that is Too Lossy
A data model that respects privacy might look like this:
{
"measurement_interval": {
"start": 2026-07-27 12:50:00+02:00,
"end": 2026-07-27 12:55:00+02:00
},
"domains": [
"example.com": {
"a_count": 1000,
"mx_count": 100,
"other_rtype_count": 200,
"nx_count": 0,
"ok_count": 1300,
"other_rcode_count": 0,
"client_count": 3
},
"example.org": {
"a_count": 200,
"mx_count": 20,
"other_rtype_count": 40,
"nx_count": 0,
"ok_count": 260,
"other_rcode_count": 0,
"client_count": 2
}
]
}
In this model, the "clients" field was replaced with a simple counter
that keeps track of the number of unique end-users seen querying for a
particular domain name during a particular interval. This approach
comes with a major limitation: it does not allow the aggregation of
data over multiple domain names or time intervals. For example, it
not allow us to answer fundamental questions like "how many unique
end-users were asking for example.com in the last week?" or "how many
unique end-users that asked for example.com were also asking for
example.org?". Moreover, if implemented naively, this model will
still require the same amount of memory on an Edge node as the previous
two examples. And if the number of end-users is high, that might be a
lot.
A Compromise: HyperLogLog
To be able to aggregate end-user counts over time or across domains, DNS TAPIR employs a so-called HyperLogLog data structure. HyperLogLog is mainly used for its memory-efficiency but it also allows us to store less information about the end-users. Specifically, all we to store about an end-users IP is a few bits of its hash (typically, a MurmurHash) and a count of the starting number of zeroes from a given offset for a subset of all end-users hashed IPs.
The act of storing the IP addresses 192.0.2.1, 192.0.2.10, and
192.0.2.100 in a HyperLogLog structure is as follows:
We start by hashing 192.0.2.1 and let's say it hashes to 00011 for
the purpose of this tutorial. From the hash value we extract two
properties: the first p bits (p=2 in this case) and the number
of leading zeroes of the remaining bits (plus one). The first p bits
selects a "bucket" in which we store the value 2 since there is one
leading zero, which we increment by one.
graph LR
ip(192.0.2.1)
hasher
extract
subgraph bucket00 [Bucket 00]
contents00[2]
end
subgraph bucket01 [Bucket 01]
contents01[0]
end
subgraph bucket10 [Bucket 10]
contents10[0]
end
subgraph bucket11 [Bucket 11]
contents11[0]
end
ip-->hasher
hasher-->|00011|extract
extract -.00.-bucket00
extract --1+1-->contents00
Next, we hash 192.0.2.10 and get the result 10111. This selects
bucket 10 and since there were no leading zeroes we store the value
1 in this bucket.
---
title:
---
graph LR
ip(192.0.2.10)
hasher
extract
subgraph bucket00 [Bucket 00]
contents00[2]
end
subgraph bucket01 [Bucket 01]
contents01[0]
end
subgraph bucket10 [Bucket 10]
contents10[1]
end
subgraph bucket11 [Bucket 11]
contents11[0]
end
ip-->hasher
hasher-->|10111|extract
extract -.10.-bucket10
extract --0+1-->contents10
Lastly, we hash 192.0.2.100 and get 00100. Again, this selects
bucket 00 but this time we DO NOT update the value of the bucket. We
only do that when the value we are trying to add is greater than what
is already in the bucket.
---
title:
---
graph LR
ip(192.0.2.100)
hasher
extract
subgraph bucket00 [Bucket 00]
contents00[2]
end
subgraph bucket01 [Bucket 01]
contents01[0]
end
subgraph bucket10 [Bucket 10]
contents10[1]
end
subgraph bucket11 [Bucket 11]
contents11[0]
end
ip-->hasher
hasher-->|00100|extract
extract -.00.-bucket00
extract --0+1--xcontents00
After adding 192.0.2.1, 192.0.2.10 and 192.0.2.100 to the
buckets we have the following structure, called a "sketch" of the
set {192.0.2.1, 192.0.2.10, 192.0.2.100}:
---
title:
---
graph LR
subgraph bucket00 [Bucket 00]
contents00[2]
end
subgraph bucket01 [Bucket 01]
contents01[0]
end
subgraph bucket10 [Bucket 10]
contents10[1]
end
subgraph bucket11 [Bucket 11]
contents11[0]
end
This is then typically encoded with some compact binary representation.
For instance, we could encode the buckets as a single 32-bit integer.
This is more compact than representing the set
{192.0.2.1, 192.0.2.10, 192.0.2.100} as an array of 32-bit integers,
where each integer represents an IP-address. Sketches can be merged
and have elements added to them, just like the set they are
representing, even though the sketch doesn't actually contain the set
elements.
When we want to count the number of distinct elements in our sketch, we do this by taking the Harmonic Mean of the bucket values and multiplying that with a constant.
An example using HyperLogLog
Using the 32-bit integer encoding mentioned above, a model with HyperLogLog could look something like the following:
{
"measurement_interval": {
"start": 2026-07-27 12:50:00+02:00,
"end": 2026-07-27 12:55:00+02:00
},
"domains": [
"example.com": {
"a_count": 1000,
"mx_count": 100,
"other_rtype_count": 200,
"nx_count": 0,
"ok_count": 1300,
"other_rcode_count": 0,
"clients_hll": 65538
},
"example.org": {
"a_count": 200,
"mx_count": 20,
"other_rtype_count": 40,
"nx_count": 0,
"ok_count": 260,
"other_rcode_count": 0,
"clients_hll": 66560
]
}
]
}
Although in reality, the binary encoding of the HyperLogLog is more complicated to make it even more space-efficient. But its data contents are no different. Also, in DNS TAPIR we use separate sketches for IPv4 and IPv6.
What Information Can a HyperLogLog Sketch Leak?
Consider the following state of the buckets:
{"bucket 00": 0, "bucket 01": 4, "bucket 10": 1, "bucket 11": 0}.
We can ask ourselves, is 192.0.2.1 in this sketch? If we know what
hashing function the sketch uses (which we typically do), we might be
able to answer the question. As in the previous example, 192.0.2.1
hashes to 00011. We calculate this hash, which tells us that we
should inspect Bucket 00. Since Bucket 00 contains the value zero, we
know for sure that 192.0.2.1 is not in this sketch. Otherwise, the
bucket couldn't possibly hold a value of 0. That is, that end-user
did not visit "example.org" within the time interval that the sketch
represents.
We can also ask ourselves, is 192.0.2.10 in the sketch? Again, we
hash it to 10111 and see that Bucket 10 holds a value that doesn't
exclude the possibility that 192.0.2.10 is in the sketch (i.e. that
it visited "example.org" in the given time interval). But we cannot say
for sure, perhaps it was an end-user whose IP adress hashed to 10100?
Or 10101? We cannot say for sure.
Lastly, we can ask ourselves, is 198.51.100.100 in the sketch? Let's
imagine that that IP adress hashes to 01000. We check Bucket 10 and
see that it value does not exclude this possibility. In fact, there is
only one hash that could've caused Bucket 10 to hold the value 4,
and that is the hash of 198.51.100.100. Therefore, in this specific
case we can say for certain that the end-user whose IP address hashes
to 01000 visited "example.org" within the given time frame.
Introduction Edge Services
In addition to a recursive resolver, an Edge installation consists of three services as described below. This document guides you through the installation of these three services. Additionally it gives you an example of configuration options for unbound at the end.
dnstapir-pop
The dnstapir-pop service is provided by the dnstapir-pop package. It
consists of a daemon process that communicates with a DNS TAPIR Core
instance over MQTT and with a recursive resolver using zone transfers.
dnstapir-pop in itself is actually a DNS server that receives its
observations from DNS TAPIR Core as described above and, based on a locally
configured policy, produces an RPZ and initiates a zone transfer to your
selected recursive resolver using DNS NOTIFY messages.
dnstapir-renew
The dnstapir-renew service is installed by the dnstapir-cli package. It
automates the process of renewing mTLS certificates (used to secure the
MQTT connection) by issuing dnstapir-cli commands on a systemd timer.
dnstapir-edm
The dnstapir-edm service is installed by the dnstapir-edm package. It
consists of a daemon process that communicates with a recursive
resolver using DNSTAP and a DNS TAPIR Core instance over MQTT and
HTTPS. It receives DNSTAP data from the resolver, which it anonymizes
and sends to the Core instance in aggregates using HTTPS. Certain
events, such as domain names being encountered for the first time, is
sent over MQTT to the same Core instance.
dnstapir-reloader
The dnstapir-reloader service is also installed by the dnstapir-edm
package. It is a small workaround that makes sure that dnstapir-edm
gets a SIGHUP after certificates have been renewed to trigger a
reload.
Installation footprint
A typical DNS TAPIR Edge installation will have the following footprint.
Users and Groups
Users:
dnstapir-pop, systemd service userdnstapir-edm, systemd service userdnstapir-renew, systemd service user
Groups:
dnstapir, common group for service users and sysadmin account
Files and Folders
Files:
/usr/bin/dnstapir-pop, executable for DNS TAPIR POP/usr/bin/dnstapir-edm, executable for DNS TAPIR EDM/usr/bin/dnstapir-cli, executable for POP management and certificate renewal service/usr/lib/systemd/system/dnstapir-pop.service, service unit for DNS TAPIR POP/usr/lib/systemd/system/dnstapir-edm.service, service unit for DNS TAPIR EDM/usr/lib/systemd/system/dnstapir-reloader.service, service unit for triggering a cert and config reload in DNSTAPIR EDM/usr/lib/systemd/system/dnstapir-renew.service, service unit for certificate renewal/usr/lib/systemd/system/dnstapir-renew.timer, timer unit for certificate renewal
Folders:
/etc/dnstapir, for configuration/var/log/dnstapir, for DNS TAPIR POP logging/var/lib/dnstapir/edm, for DNS TAPIR EDM runtime state
Getting the Packages
Debian-based
Prerequisites
Make sure you have all the build requirements installed:
sudo apt install git make golang build-essential
Building the Package
We currently do not provide any official debian builds. However, it is possible to build debian packages from a cloned repo. To build debian packages for an edge installation, start by cloning these repositories:
git clone https://github.com/dnstapir/pop
git clone https://github.com/dnstapir/cli
git clone https://github.com/dnstapir/edm
For each cloned repo, build the corresponding package by issuing:
make deb
Then install it with:
sudo dpkg -i path/to/package
RPM-based
DNS TAPIR provides three rpm packages for an Edge installation:
dnstapir-pop, dnstapir-edm and dnstapir-cli. They are built using Fedora's
public Copr instance with the @dnstapir group.
Currently, packages are being built for EPEL 10, EPEL 9, Fedora 42,43 and
OpenSUSE Leap 15.6.
Packages in the @dnstapir/edge-testing repo are signed with this PGP key:
07FC 9787 0134 6ED4 522A 17E7 2C4D 4FAC 02CF 0AC2
Packaging code lives side-by-side with the source code in the respective repos:
- https://github.com/dnstapir/pop/tree/main/rpm
- https://github.com/dnstapir/edm/tree/main/rpm
- https://github.com/dnstapir/cli/tree/main/rpm
Enable the repositories in your package manager:
dnf
sudo dnf copr enable @dnstapir/edge-testing
zypper
sudo zypper ar https://copr.fedorainfracloud.org/coprs/g/dnstapir/edge-testing/repo/opensuse-leap-15.6/group_dnstapir-edge-testing-opensuse-leap-15.6.repo
And install them:
dnf
sudo dnf install dnstapir-pop dnstapir-cli dnstapir-edm
zypper
sudo zypper in dnstapir-pop dnstapir-cli dnstapir-edm
Building an RPM Package Locally
It is also possible to build and RPM package locally. This requires
make, golang, rpmbuild and git. To build it, start by cloning
the desired repo. This example will use pop
git clone https://github.com/dnstapir/pop
Then, change into the newly cloned repo and issue
make rpm
This will build an RPM package that can then be installed with your preferred package manager.
Managing permissions
Three system users, dnstapir-pop, dnstapir-edm and dnstapir-renew,
and a group, dnstapir, will have been created. Add your administrator
user to this group for easier bootstrapping and maintenance:
sudo usermod -a -G dnstapir <USERNAME>
Log out and back in and make sure the new group membership is in effect before proceeding.
Enrolling with the DNS TAPIR Node Manager
To connect with DNS TAPIR Core, an Edge node needs to be enrolled. You should have received enrollments credentials from a DNS TAPIR representative on a trusted out-of-band channel. They will look something like the following:
{
"name":"enroll-example.test.dnstapir.se",
"key":{
"kty":"OKP",
"kid":"123456789012345678901234",
"alg":"EdDSA",
"crv":"Ed25519",
"x":"ABCD_EFGHIJKLMNO_PQRSTUVW_XYZ123456789_12345",
"d":"abcdefghijklmno_pqrstuvwxyz_123456789012345"
},
"nodeman_url":"https://nodeman.test.dnstapir.se/"
}
Store the credentials in a file on the node that is to be enrolled. Then run:
sudo -g dnstapir dnstapir-cli --standalone enroll --enroll-credentials <PATH TO ENROLL CREDS>
The reason for running with sudo -g dnstapir is that, apart from
exchanging cryptographic material with DNS TAPIR Core, the above
enrollment command also generates a number of config files under
/etc/dnstapir (by default). They need to have dnstapir as the
group owner so that they can be used by the three system users
mentioned before.
Upon success this command will generate a file of the form
nodeman-resp-[UNIX TIME].json in the current working directory. If you in the
future would like to re-generate the configuration boilerplate without
enrolling, as described in the following section, you need to save this file.
Regenerating Configuration Without Enrolling
Sometimes it might be desirable to re-generate the configuration boilerplate without actually enrolling. This can be done by issuing:
sudo -g dnstapir dnstapir-cli --standalone enroll --enroll-credentials <PATH TO ENROLL CREDS> --local-response <PATH TO ENROLLMENT RESPONSE>
or shortly:
sudo -g dnstapir dnstapir-cli --standalone enroll -c <PATH TO ENROLL CREDS> -L <PATH TO ENROLLMENT RESPONSE>
Where <PATH TO ENROLL CREDS> is the same credentials file as above, and
<PATH TO ENROLLMENT RESPONSE> is the file that the above command generates
with a name of the form nodeman-resp-[UNIX TIME].json. This will in other
words use the locally stored information from a previous enrollment to generate
the configuration boilerplate.
Edits to the Configuration Boilerplate
The configuration generated in the enrollment step contains sensible
defaults for most deployments. However, some final touches need to be
made before it can be properly integrated with a recursive resolver and
with DNS TAPIR Core. By default, these files will be generated under
/etc/dnstapir. This guide uses the default.
pop-outputs.yaml
Edit this file where annotated (1 location) with the destination to which POP will be sending DNS NOTIFY messages about changes to the RPZ zone it has generated based on the observations from Core and on the local policies.
dnstapir-pop.yaml
Edit this file where annotated (1 location) with the interface on which POP will listen to incoming zone transfer requests for the RPZ zone it has generated.
dnstapir-edm.toml
Edit this file where annotated (2 locations) with a strong secret/password and a DNSTAP interface. The secret is used when pseudonymizing the recursive traffic with Crypto-PAn. The DNSTAP interface is the IP + port where EDM listens for DNSTAP traffic from the recursive resolver.
Start the services
sudo systemctl start dnstapir-pop.service
sudo systemctl start dnstapir-edm.service
sudo systemctl start dnstapir-renew.timer
Enable the services
sudo systemctl enable dnstapir-pop.service
sudo systemctl enable dnstapir-edm.service
sudo systemctl enable dnstapir-reload.service
sudo systemctl enable dnstapir-renew.timer
dnstapir-renew is not a daemon that keeps running, it's a program
that terminates, but which needs to be run on a regular basis. Hence
the .timer suffix.
dnstapir-reload is triggered whenever dnstapir-renew.service finishes
and only needs to be enabled, not started.
Sample Config for Resolvers
Unbound
Note that for unbound to support DNSTAP, the flag --enable-dnstap
must be passed at compile time. Make sure your unbound package is
doing this. After doing so, edit unbound.conf to contain the following:
rpz:
name: dnstapir
primary: <IP>@<PORT> # Must match config in dnstapir-pop.yaml
zonefile: "/var/run/unbound/dnstapir.zone"
rpz-log: yes
rpz-log-name: dnstapir
dnstap:
dnstap-enable: yes
dnstap-ip: <IP>@<PORT> # Must match config in dnstapir-edm.yaml
dnstap-tls: no
dnstap-log-client-query-messages: yes
dnstap-log-client-response-messages: yes
dnstap-send-identity: yes
dnstap-send-version: yes
server:
module-config: "respip validator iterator" # "respip" module needed for RPZ
Post-installation
Check out the post-installation docs for info on how to to basic connectivity checks and verify that your Edge is running as intended.
Maintaining the "Well-known Domains"-filter
DNS TAPIR EDM uses a compressed list of domain patterns (we often refer to it as a "DAWG-file" file or just "DAWG") to determine whether a domain it sees should be processed in aggregate and submitted to DNS TAPIR Core as part of a histogram or submitted as an event for more qualitative analysis in Core. If a domain matches a pattern in the list (the DAWG-file), it is selected for aggregate processing. If not, qualitative processing.
Checking if You Have the Default Filter
Check if you have the default filter using md5sum:
md5sum path/to/well-known-domains.dawg
<DIGEST> path/to/well-known-domains.dawg
The current default filter has the md5 digest: 58a5de0534bfd127ed20b39b97e9a0db
Replacing the Default Filter
By default, DNS TAPIR EDM ships with a small DAWG-file that mostly
selects qualitative processing (this is bound to change in the near
future). Operators should manually replace the default filter. The most
recent filter is available at
https://public.test.dnstapir.se/well-known-domains.dawg.
The most recent filter has the md5 digest: a99b0adaa5ea091015b9f5bdc5951302
Replacing the filter can be done as such, for instance:
# Get the file and check if its ok
cd /tmp
wget https://public.test.dnstapir.se/well-known-domains.dawg
md5sum well-known-domains.dawg
# check if digest matches the docs at https://dnstapir.github.io/techdocs/postinstall.html#replacing-the-default-filter
# Move the file into place and reload EDM config
cd /etc/dnstapir/edm
sudo mv well-known-domains.dawg well-known-domains.dawg.old
sudo mv /tmp/well-known-domains.dawg .
sudo systemctl reload dnstapir-edm # Make EDM reload its configuration
Your EDM is now running with the latest filter. Verify by checking its log output:
sudo journalctl -u dnstapir-edm
If the file was reloaded successfully, dnstapir-edm will have
mentioned it.
Looking up a domain in a filter
See if a domain is in a filter by issuing:
dnstapir-cli --standalone dawg --dawg path/to/dawgfile lookup --name example.com
(The --standalone flag can typically be omitted if the command is
run on a deployed Edge node.)
Looking up a suffix in a filter
See if a suffix is in a filter by issuing:
dnstapir-cli --standalone dawg --dawg path/to/dawgfile lookup --name .example.com
If a suffix is in the filter, the EDM daemon process will consider it a
match if the suffix of a domain it sees is stored in its DAWG. For
example, it will consider these.labels.are.garbage.example.com to be
a match if the DAWG file contains the suffix .example.com.
Listing All the Contents of a Filter
This might be costly for a big filter but you can list its contents by issuing:
dnstapir-cli --standalone dawg --dawg path/to/dawgfile list
Checking connectivity
The DNS TAPIR Looptest Domain
As part of our test deployment of DNS TAPIR Core, we have set up a
special domain for testing connectivity, looptest.dnstapir.se..
It currently has two uses.
The "Ticker" Test
To ensure that a POP receives observations from Core, Core will
periodically send out observations with
observation encoding flag 1024 set. By
issuing dnstapir-cli filterlists on a running system, you should be
able to see the following:
operator@edge $ dnstapir-cli filterlists
Domain |Source |Src Fmt |Filter |Flags
---------------------------------------------------------------------------------------------------------------------------------------
# ...snip...
epoch-1761739157.ticker.looptest.dnstapir.se. |dns-tapir |tapir-msg-v1 |doubt |1024
epoch-1761737417.ticker.looptest.dnstapir.se. |dns-tapir |tapir-msg-v1 |doubt |1024
epoch-1761738377.ticker.looptest.dnstapir.se. |dns-tapir |tapir-msg-v1 |doubt |1024
epoch-1761738677.ticker.looptest.dnstapir.se. |dns-tapir |tapir-msg-v1 |doubt |1024
# ...snip...
The presence of the ticker.looptest.dnstapir.se. observations
indicates that connectivity from Core to your POP is working.
The "From-edge" Test
To ensure that your resolver can connect to your EDM, that your EDM can
connect to Core and that Core can connect to your POP, queries that
follow a specific pattern will cause a corresponding observation to be
sent out by Core, again using the 1024 flag.
From a machine that can connect with the resolver on your Edge, run:
dig @<your resolver> <unique label>.from-edge.looptest.dnstapir.se
If the qname is something your EDM sees for the first time, it will send an event to Core. Core will recognize the domain as our looptest domain, flag it and then send out an observation to all Edges.
You should be able to see that the "loop is closed" by issuing
dnstapir-cli filterlists on your Edge system:
operator@edge $ dnstapir-cli filterlists
Domain |Source |Src Fmt |Filter |Flags
---------------------------------------------------------------------------------------------------------------------------------------
# ...snip...
<unique-label>.from-edge.looptest.dnstapir.se. |dns-tapir |tapir-msg-v1 |doubt |1024
# ...snip...
It might be helpful to grep for your query since there may be a lot
of other domains listed in the output. Seeing your query in the output
indicates that your resolver is communicating with your EDM, your EDM
is communicating with Core and Core is communicating with your POP.
Note that other DNS TAPIR users will be able to see your looptests since observations are being sent out to all enrolled POPs. No profanity!
Check services are active
sudo systemctl status dnstapir-pop
sudo systemctl status dnstapir-edm
Verify that TAPIR Core receives histograms from TAPIR EDM: ....
TODO
Verify that TAPIR POP receives observations from TAPIR Core: ....
TODO
TAPIR-POP: DNS TAPIR Policy Processor
The DNS TAPIR Policy Processor, TAPIR-POP, is the component that processes the intelligence data from the TAPIR-Core (and possibly other sources) and applies local policy to reach a filtering decision.
It is the connection between TAPIR Core and the Edge platform. It manages local configurations and gets updates from TAPIR Core with alerts and config changes.
TAPIR-POP is responsible for the task of integrating all intelligence sources into a single Response Policy Zone (RPZ) that is as compact as possible. The RPZ file is used by the DNS resolver to implement blocklists and other policy-related functions.
A unified single RPZ zone instead of multiple sources
TAPIR-POP presents a single output with all conflicts resolved, rather than feeding the resolver multiple sources of data from which to look for policy guidance, where sources can even be conflicting (eg. a domainname may be flagged by one source but allowlisted by another).
The result is smaller, as no allowlisting information is needed for the resolver.
TAPIR-POP supports a local policy configuration
TAPIR-POP is able to apply further policy to the intelligence data, based on a local policy configuration. To enable the resolver operator to design a suitable threat policy TAPIR-POP uses a number of concepts:
-
lists: there are three types of lists of domain names:
- allowlists (names that must not be blocked)
- denylists (names that must be blocked)
- doubtlists (names that should perhaps be blocked)
-
observations: these are attributes of a suspicious domain name. In reality whether a particular domain name should be blocked or not is not an absolute, it is a question of propabilities. Therefore, rather than a binary directive, "this name must be blocked", some intelligence sources, including DNS TAPIR, present the resolver operator with observed attributes of the name. Examples include:
- the name has only been observed on the Internet for a short time
- the name draws huge query traffic
- the name resolves to an IP address known to host bad things, etc.
-
sources: TAPIR-POP supports the following types of sources for intelligence data:
- RPZ: imported via AXFR or IXFR. TAPIR-POP understands DNS NOTIFY.
- MQTT: DNS TAPIR Core Analyser sends out rapid updates for small numbers of names via an MQTT message bus infrastructure.
- DAWG: Directed Acyclic Word Graphs are extremely compact data structures. TAPIR-POP is able to mmap very large lists in DAWG format which is used for large allowlists.
- CSV Files: Text files on local disk, either with just domain names, or in CSV format are supported.
- HTTPS: To bootstrap an intelligence feed that only distributes deltas (like DNS TAPIR, over MQTT), TAPIR-POP can bootstrap the current state of the complete feed via HTTPS.
-
outputs: TAPIR-POP outputs RPZ zones to one or several recipients. Both AXFR and IXFR is supported.
Overview of the TAPIR-POP policy
The resulting policy has the following structure (in order of precedence):
- no allowlisted name is ever included.
- blocklisted names are always included, together with a configurable RPZ action.
- doubtlisted names that have particular tags that the resolver operator chooses are included, together with a configurable RPZ action.
- the same doubtlisted name that appear in N distinct intelligence feeds is included, where N is configureable, as is the RPZ action.
- a doubtlisted name that has M or more tags is included, where both M and the action are configurable.
Logging
Logging is done by writing either to stdout or stderr and letting
systemd handle it. Logging will consist of four verbosity levels:
Debug, Info, Warning and Error. Each component is responsible for its own
logging.
Telemetry
Telemetry is done by publishing on a dedicated MQTT topic. Each component has its own topic and is responsible for publishing its own data. Most of the data will be aggregated statistics such as packet counters.
"Exceptional" Events
Some events are of special interest, both to the operator of a particular Edge deployment and to the Core operator. An example of such an event would be failure to renew a TLS certificate. As such, those events are written BOTH to the syslog and published over the MQTT telemetry topic by the component that observes the event. The component also assigns an identifier that is visible both in the log and in the telemetry packet so that the two can be associated.
Versioning Scheme
DNS TAPIR Edge software uses semantic versioning for releases.
The Edge components are released independently, but should all adhere
to the same scheme, that is X.Y.Z. Development builds, nightly builds
and other unofficial builds should have a version of 0.0.0.
Versioning Scheme for Debian-packaged Edge Components
Debian-packaged Edge software should adhere to the same scheme as the
upstream component. That is, X.Y.Z for releases and 0.0.0 for
unofficial packages. Additionally, for unofficial packages, a snapshot
string will be used to identify when the build was made and what
revision of the upstream code was used. For Edge components built from
non-release revisions of the code, the versioning scheme will be
0.0.0+local20251118.<SHORT SHA>. Officially released packages will
not have the snapshot part, i.e. it will just be X.Y.Z.
Versioning Scheme for RPM-packaged Edge Components
RPM-packaged Edge software follows similar rules as Debian-packaged
software. The version format will be slightly different for
unofficial builds: 0.0.0^20251118.<SHORT SHA>-1.
Embedding Versioning Information in an Executable Binary
All Edge binaries should have version and source code revision
information stored in them. This can be done, for example by
using the -ldflags option at build-time.
For example:
go build -ldflags "-X 'main.version=0.0.0' -X 'main.commit=12345678'"
This will create two variables with versioning information in
main.go. Ideally, they should be printed when early on the binary is
invoked. To make it clear when a binary has been built in a bad way,
they can be declared like this in main.go:
# in main.go
var version = "BAD-VERSION"
var commit = "BAD-COMMIT"
When printed, it will be clear from the logs that the binary has not been built in the intended way and should be replaced.
Building In-repo vs. from Extracted Tarball
During development, one will typically build the binary by issuing
make while standing in the root of a checked-out git repo. However,
sometimes the binary will be built from an extracted tarball. In either
case, versioning information should be embedded in the built binary.
This means that the git sha and any tag information that is relevant
must be included when making the tarball. For instance, it could be
put into two files, VERSION and COMMIT, that are included in the
tarball as it is created.
Example of multi-new observation
sequenceDiagram
participant NATS Server
participant microservice
participant NATS KV-Store
microservice->>+NATS Server: Subscribe(EVENT_NEW_QNAME)
note over microservice,NATS Server: ...some time passes...
NATS Server->>microservice: Publish(EVENT_NEW_QNAME, "new.example.com.")
microservice->>+NATS KV-Store: Request("new.example.com.")
note over microservice,NATS KV-Store: ...gets collected data about the domain...
microservice->>microservice: Has "new.example.com." been observed as new by other resolvers recently?
microservice->>NATS Server: Publish(OBSERVATION_MULTI_NEW, "new.example.com.")
Example of ramp observation
sequenceDiagram
participant NATS Server
participant Data Loader
participant S3
participant microservice
NATS Server->>Data Loader: Publish(EVENT_NEW_AGGREGATE)
Data Loader->>S3: Get(NEW_AGGREGATE)
Data Loader->>Data Loader: Create histogram
Data Loader->>S3: Post(Histogram)
S3-->>microservice: Publish(EVENT_NEW_HIST)
microservice->>microservice: For domain in hist, hasRamp?
microservice->>NATS Server: Publish(OBSERVATION_RAMP, "evil.hula.se")
note over microservice: ...Publish ramping domains...
microservice->>NATS Server: Publish(OBSERVATION_RAMP, "z5.nu")
Communication pattern for "NOT well-known" domains
flowchart
EDM-->|2, EVENT_NEW_QNAME, MQTT, mTLS, RFC 7515|bridge
bridge-->|7, OBSERVATION_RAMP, MQTT, mTLS, RFC 7515|POP
subgraph CORE
bridge-->|3, EVENT_NEW_QNAME|NATS_Server
NATS_Server-->|4, EVENT_NEW_QNAME|MULTINEW_microservice
MULTINEW_microservice-->|5, OBSERVATION_MULTI_NEW|NATS_Server
NATS_Server-->|6, OBSERVATION_MULTI_NEW|bridge
end
subgraph EDGE
POP-->|8, RPZ XFR|RecResolver
RecResolver-->|1, DNSTAP|EDM
end
Communication pattern for "well-known" domains
There is some discrepancy between this image and the sequence diagram for the ramp observation has the data loader component has not been taken into account here.
flowchart
EDM-->|2, one-minute aggregate, HTTPS, mTLS, RFC 7515|aggrec
bridge-->|7, OBSERVATION_RAMP, MQTT, mTLS, RFC 7515|POP
subgraph CORE
aggrec-->|3, Publish EVENT_NEW_AGGREGATE|NATS_Server
NATS_Server-->|4, Publish EVENT_NEW_AGGREGATE|RAMP_microservice
RAMP_microservice-->|5, Publish OBSERVATION_RAMP|NATS_Server
NATS_Server-->|6, OBSERVATION_RAMP|bridge
end
subgraph EDGE
POP-->|8, RPZ XFR|RecResolver
RecResolver-->|1, DNSTAP|EDM
end
Observation Encodings
Observations sent out from DNS TAPIR Core are packed in a 32-bit word with the following interpretations:
| Encoding | Observation | Interpretation |
|---|---|---|
| 1 | GLOBALLY_NEW | Previously unseen by any Edge |
| 2 | TODO | TODO |
| 4 | NEWLY_REGISTERED | Domain was created very recently |
| 8 | REGISTRY_INVESTIGATION | Domain is under investigation by its registry |
| 16 | DROPCATCH | Domain was registered via dropcatching |
| 32 | TODO | TODO |
| 64 | TODO | TODO |
| 128 | TODO | TODO |
| 256 | TODO | TODO |
| 512 | TODO | TODO |
| 1024 | LOOPTEST | Observation for testing use only |
| others | TODO | TODO |
Detailed Explanations
GLOBALLY_NEW
This observation flag is set for a domain the first time any Edge resolver in a DNS TAPIR system receives a query for it. It is set regardless of what the response was. For example, it will be set even for NXDOMAIN and NODATA responses.
Some domains will not cause a corresponding GLOBALLY_NEW observation to be sent out. Those are the so-called "well-known" domains that are configured as part of installing a DNS TAPIR Edge system. For those domains, only aggregated statistics such as counters will be collected.
NEWLY_REGISTERED
This observation flag is set for a domain if it was newly registered when it was first seen by an Edge resolver. I.e. when the domain is no older than one week when an Edge resolver first encountered it.
REGISTRY_INVESTIGATION
This observation flag is set for a domain if it under investigation by its registry for some reason. A registry can launch an investigation into a domain for numerous reasons. For example, the registry might be trying to verify registrant data for a domain, a domain might be involved in a legal dispute or there has been complaints about a domain being involved in some kind of abuse.
DROPCATCH
This observation flag is set for a domain that was registered shortly after its expiration and release (typically within 5-10 minutes). The practice of registering newly released domains is known as dropcatching and is sometimes used by threat actors to obtain domains with a good (or at least neutral) reputation that can subsequently be used in phishing campains or otherwise abusive behavior.
LOOPTEST
This observation flag is set for domains that are used for testing purposes. More documentation on how a looptest can be conducted can be found here.
Developing a New Microanalyst
Getting Started
It is recommended to start the development of a new microanalyst from our cookiecutter. For more info about using cookiecutters, visit the docs for the cookiecutter project.
Also, please be aware of the checklists for introducing new analysts and new observations. They must be completely ticked off before your new analyst can be deployed.
Developing
After generating the microanalyst boilerplate, you can start making changes to the code.
Compiling
Compile your newly written code by issuing make.
Building a Container
Microanalysts are typically run as containerized microservices. A
container image can be built locally using ko.
Build an image by issuing make ko. Local ko images can be viewed
by issuing docker image ls | grep ko.local.
Working with tapir-analyse-lib
Analysts rely heavily on the shared library
github.com/dnstapir/tapir-analyse-lib. It contains common code such as
datatypes, manipulating domain names, interfacing with NATS and schema
validation. If changes are being made to this library, it is recommended
to use a Go workspace. The
following file structure is suitable:
my-tapir-repos/
├── go.work
├── observation-encoder/
├── tapir-analyse-lib/
├── tapir-analyse-listchecker/
├── tapir-analyse-new-qname/
└── tapir-analyse-my-new-analyst/
And the contents of go.work should be something like:
go 1.26.1
use (
./observation-encoder
./tapir-analyse-listchecker
./tapir-analyse-new-qname
./tapir-analyse-my-new-analyst
)
replace github.com/dnstapir/tapir-analyse-lib => ./tapir-analyse-lib
With a setup like this, the local version of tapir-analyse-lib will
be used whenever you are building your new analyst.
Testing
Currently, our cookiecutter does not provide a lot of boilerplate to facilitate unit testing, unfortunately. However, there is a small integration test suite that can be used to simplify manual testing. It is also encourage to write new tests whenever a new analyst is being integrated.
Downloading and Running the Integration Tests As Is
Python, make and docker-compose is required to run the tests.
They can be run as follows:
# Get the sources
git clone https://github.com/dnstapir/core-integration-test.git
cd core-integration-test
# Install the Python dependencies in a virtual environment
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
# Run the tests
make
Downloading and Running the Integration Tests With Local Containers
By default, the integration test uses the latest container images from
our github container registry. However, during development one probably
wants to use local images. This can be controlled using environment
variables. core-integration-test/misc/local_containers.env shows how
the environment should be set to enable local container images in the
test. If one does not want to use local images for all components in
the test, only set the variables that correspond to the desired images.
Note that the test suite does not handle building the local images, that has to be done manually as described above.
An example using only local images
# Get the source code and install the deps, as describe above
# From the integration test repo, set the environment variables
source ./misc/
# Run the tests
make
Adding a New Container to the Integration Test
Add your new analyst's conf to sut/tapir-analyse-my-new-analyst
and add an entry for it in sut/docker-compose.yaml. This ensures that
the service will be spun up during the integration tests.
Using the Integration Test Setup for Manual Tests
If one wants to interact manually with the integration test targets,
the targets can be started with misc/start_containers.sh. Then, one
can issue new_qname events with the script mist/send_new_qname.sh
to trigger events for the microanalysts to process. For example:
./misc/send_new_qname.sh example.dnstapir.se core-integration-test.events.new_qname fake-resolver-id
Will emulate an event arriving in NATS which is then forwarded to the microanalysts. If you have the NATS cli installed, outgoing observations from the analysts can be subscribed to by issuing:
nats sub core-integration-test.out
This provides a basic way of manually interacting with a small DNS TAPIR Core.
New Microanalyst Checklist
This needs to be ticked of when developing a new microanalyst.
- Create a public repository under
github.com/dnstapir. - Add the new microanalyst service to the integration test compose file
- Add a configuration directory for the microanalyst systems-under-test.
- Add a new environment variable to the integration test local containers.
- Make sure your new image can be easily pulled by someone running the integration tests.
- Write some new integration tests
- If the analyst is creating NATS buckets for data that might be of interest to other services, update the NATS bucket docs.
New Observation Checklist
This needs to be ticked of when introducing a new type of observation. New microanalysts do not necessarily introduce new observation types, so this list is not always applicable.
- Add a corresponding row to the table of observation encodings.
- Add a description of the observation here.
- Add its encoded value and textual value to the shared Go library.
- Configure the observation encoder deployment to make sure a new bucket for storing the observation is created.
- Update the NATS bucket docs.
Important Concepts
The DNS TAPIR Core relies on a mix of custom concepts and concepts introduced by NATS. At a very high level, Core uses NATS for storing data and for communicating between different services. Important concepts for understanding how DNS TAPIR Core uses NATS are subjects, buckets, service identifiers and thumbprints.
Subjects
The most important concept is that of a "subject". A service can receive, send read or store data under a specific subject.
public.to-edge.observations
Used to send observation JSON blobs to Edge. The mqtt-bridge service
listens on this subject, signs the incoming blobs and then transmits
them to Edge over MQTT. It is not used for persistent storage, only for
messaging between observation-encoder and mqtt-bridge.
internal.observations.*.>
Namespace used to keep track of which observation flags are active for which domain. It is used for persistent storage so anyone with NATS access can look up which observations are active for which domains. The value behind a given subject is not used, the presence/absence of a subject holds all the necessary info. All subjects in this namespace have a limited time-to-live and thus, an observation will only be valid for a limited amount of time.
Examples
internal.observations.looptest.xa.foo.www
Presence of this key indicates that www.foo.xa has the looptest
observation flag set.
internal.observations.globally_new.xa.bar
Presence of this key indicates that bar.xa has the globally_new
observation flag set.
internal.seen-domains.>
Namespace used to keep track of which domains have been encountered. It is used for persistent storage and all contained subjects will have a very long time-to-live. The value will be a JSON dict with thumbprints as keys and timestamps as values, indicating when it has been observed by different Edge nodes. It is possible for a domain to be "re-discovered" if it re-appears after not being seen for a long period.
Examples
internal.seen-domains.xa.foo.mail
Presence of this key indicates that the domain mail.foo.xa has been
seen at least once. If the corresponding value is
{"tp1": 123, "tp2": 234}, it means that it was first seen by an Edge
using data signing key with thumbprint "tp1" at time 123 and later by
and Edge with thumbprint "tp2" at time 234.
internal.service.{{SERVICE IDENTIFIER}}.>
Namespace used by a given service for arbitrary purposes.
Examples
internal.service.tapir-analyse-dummy
Namespace used by service with identifier "tapir-analyse-dummy".
Buckets
NATS has a concept of "buckets", which is essentially key-value stores
whose backends may be separate. For instance, two different buckets
might use different files/folders on disk for persistence. The choice
of bucket for publishing/storing a value has no effect on what its
subject/key can be. For instance, two subjects/keys foo.bar and
foo.bar.baz may live in separate buckets even though one is part of
the other's namespace.
globally_new_bucket
Bucket containing all subjects/keys for domains that currently have the
globally_new observation flag set, i.e. those matching
internal.observations.globally_new.>. Provisioned by
observation-encoder.
looptest_bucket
Bucket containing all subjects/keys for domains that currently have the
looptest observation flag set, i.e. those matching
internal.observations.looptest.>. Provisioned by
observation-encoder.
dropcatch_bucket
Bucket containing all subjects/keys for domains that currently have the
dropcatch observation flag set, i.e. those matching
internal.observations.dropcatch.>. Provisioned by
observation-encoder.
seen_domains_bucket
Bucket containing all subjects/keys that indicate whether a domain has
been seen before, i.e. those matching internal.seen-domains.>.
Provisioned by service
tapir-analyse-new-qname.
tapir-analyse-new-qname_bucket
Bucket for internal use by tapir-analyse-new-qname. Provisioned by
the same service.
Service Identifiers
Services in Core have a unique identifier. The identifier should be stable across different versions of the software. A suitable choice of identifier is the repository name of the service's source-code. The main use of the service identifier is to allocate resources in NATS. For example, if a service wants to create a NATS bucket for its own personal use, the bucket name should contain the identifier to indicate which service uses it.
observation-encoder
Service identifier of the service that is used to aggregate all observations that are stored in NATS into a JSON blob that is sent to Edge.
mqtt-bridge
Service identifier of the service that bridges messages between NATS and the MQTT bus that Edge nodes use for communication with Core.
tapir-analyse-looptest
Service identifier of the service that is used for Core-Edge connectivity tests.
tapir-analyse-new-qname
Service identifier of the service that is used to keep track of which
Edge nodes have seen which domains and also responsible for setting
the globally_new observation.
Thumbprints
A thumbprint is related to identifying from which Edge node a certain
message originated. When an Edge node signs data, it uses a key that it
received during enrollment and that is associated with its identity.
When the mqtt-bridge has successfuly validated an incoming message
from an Edge node, it publishes the message in NATS with a NATS header
containing the signing key thumbprint. Core services can use this
thumbprint as they wish, for example to keep track of how many
different Edge nodes has seen a certain domain.
Overview
DNS TAPIR analysts are typically implemented as simple microservices written in the Go programming language. A template project exists at https://github.com/dnstapir/tapir-analyse-go-cookiecutter that can be used with https://www.cookiecutter.io/ to bootstrap an implementation.
Observation Encodings
Perhaps the most important task of an analyst is to produce observations that are consumed by Edge nodes. Whether observations affect the resolution process of an Edge node is up to each Edge operator to decide for themselves via configuration of their DNS TAPIR Policy Processor. Available observations are documented in the Observation Encodings Section.
Design Patterns
Some design patterns that analysts should follow to ensure a smooth deployment include:
Logging
Analysts should write their logs to stdout in JSON format. There is
currently no schema for the logging format.
Source Code Version Traceability
During startup, analysts should produce a log entry containing the git sha from which it was built.
CLI Args Should Not Be Required
It should be possible to invoke the analyst without any command line arguments. That means configuration should either go in a file or be read from the environment. A file is preferred in the general case while environment variables are preferred for secrets such as URLs containing credentials.
Configuration File Handling
Analysts should by default look for a configuration file relative to
the working directory. The name of the configuration file should be
config.toml and it should use the TOML format: https://toml.io/.
Environment Variables
Analysts should be able to read potentially sensitive configuration
options from environment variables. Some examples include URLs if they
have a userinfo field and API tokens. If, during runtime, an
environment variable overrides a configuration option read from a CLI
argument or a file, this should be noted in the logs, typically during
startup. The name of the overriding environment variable should be
stated but not the value since sensitive info should not appear in the
log.