Maintaining the "Well-known Domains"-filter
DNS TAPIR EDM uses a compressed list of domain patterns (we often refer to it as a "DAWG-file" file or just "DAWG") to determine whether a domain it sees should be processed in aggregate and submitted to DNS TAPIR Core as part of a histogram or submitted as an event for more qualitative analysis in Core. If a domain matches a pattern in the list (the DAWG-file), it is selected for aggregate processing. If not, qualitative processing.
Checking if You Have the Default Filter
Check if you have the default filter using md5sum:
md5sum path/to/well-known-domains.dawg
<DIGEST> path/to/well-known-domains.dawg
The current default filter has the md5 digest: 58a5de0534bfd127ed20b39b97e9a0db
Replacing the Default Filter
By default, DNS TAPIR EDM ships with a small DAWG-file that mostly
selects qualitative processing (this is bound to change in the near
future). Operators should manually replace the default filter. The most
recent filter is available at
https://public.test.dnstapir.se/well-known-domains.dawg.
The most recent filter has the md5 digest: a99b0adaa5ea091015b9f5bdc5951302
Replacing the filter can be done as such, for instance:
# Get the file and check if its ok
cd /tmp
wget https://public.test.dnstapir.se/well-known-domains.dawg
md5sum well-known-domains.dawg
# check if digest matches the docs at https://dnstapir.github.io/techdocs/postinstall.html#replacing-the-default-filter
# Move the file into place and reload EDM config
cd /etc/dnstapir/edm
sudo mv well-known-domains.dawg well-known-domains.dawg.old
sudo mv /tmp/well-known-domains.dawg .
sudo systemctl reload dnstapir-edm # Make EDM reload its configuration
Your EDM is now running with the latest filter. Verify by checking its log output:
sudo journalctl -u dnstapir-edm
If the file was reloaded successfully, dnstapir-edm will have
mentioned it.
Looking up a domain in a filter
See if a domain is in a filter by issuing:
dnstapir-cli --standalone dawg --dawg path/to/dawgfile lookup --name example.com
(The --standalone flag can typically be omitted if the command is
run on a deployed Edge node.)
Looking up a suffix in a filter
See if a suffix is in a filter by issuing:
dnstapir-cli --standalone dawg --dawg path/to/dawgfile lookup --name .example.com
If a suffix is in the filter, the EDM daemon process will consider it a
match if the suffix of a domain it sees is stored in its DAWG. For
example, it will consider these.labels.are.garbage.example.com to be
a match if the DAWG file contains the suffix .example.com.
Listing All the Contents of a Filter
This might be costly for a big filter but you can list its contents by issuing:
dnstapir-cli --standalone dawg --dawg path/to/dawgfile list
Checking connectivity
The DNS TAPIR Looptest Domain
As part of our test deployment of DNS TAPIR Core, we have set up a
special domain for testing connectivity, looptest.dnstapir.se..
It currently has two uses.
The "Ticker" Test
To ensure that a POP receives observations from Core, Core will
periodically send out observations with
observation encoding flag 1024 set. By
issuing dnstapir-cli filterlists on a running system, you should be
able to see the following:
operator@edge $ dnstapir-cli filterlists
Domain |Source |Src Fmt |Filter |Flags
---------------------------------------------------------------------------------------------------------------------------------------
# ...snip...
epoch-1761739157.ticker.looptest.dnstapir.se. |dns-tapir |tapir-msg-v1 |doubt |1024
epoch-1761737417.ticker.looptest.dnstapir.se. |dns-tapir |tapir-msg-v1 |doubt |1024
epoch-1761738377.ticker.looptest.dnstapir.se. |dns-tapir |tapir-msg-v1 |doubt |1024
epoch-1761738677.ticker.looptest.dnstapir.se. |dns-tapir |tapir-msg-v1 |doubt |1024
# ...snip...
The presence of the ticker.looptest.dnstapir.se. observations
indicates that connectivity from Core to your POP is working.
The "From-edge" Test
To ensure that your resolver can connect to your EDM, that your EDM can
connect to Core and that Core can connect to your POP, queries that
follow a specific pattern will cause a corresponding observation to be
sent out by Core, again using the 1024 flag.
From a machine that can connect with the resolver on your Edge, run:
dig @<your resolver> <unique label>.from-edge.looptest.dnstapir.se
If the qname is something your EDM sees for the first time, it will send an event to Core. Core will recognize the domain as our looptest domain, flag it and then send out an observation to all Edges.
You should be able to see that the "loop is closed" by issuing
dnstapir-cli filterlists on your Edge system:
operator@edge $ dnstapir-cli filterlists
Domain |Source |Src Fmt |Filter |Flags
---------------------------------------------------------------------------------------------------------------------------------------
# ...snip...
<unique-label>.from-edge.looptest.dnstapir.se. |dns-tapir |tapir-msg-v1 |doubt |1024
# ...snip...
It might be helpful to grep for your query since there may be a lot
of other domains listed in the output. Seeing your query in the output
indicates that your resolver is communicating with your EDM, your EDM
is communicating with Core and Core is communicating with your POP.
Note that other DNS TAPIR users will be able to see your looptests since observations are being sent out to all enrolled POPs. No profanity!
Check services are active
sudo systemctl status dnstapir-pop
sudo systemctl status dnstapir-edm
Verify that TAPIR Core receives histograms from TAPIR EDM: ....
TODO
Verify that TAPIR POP receives observations from TAPIR Core: ....
TODO