rescached - yet another DNS/resolver cache daemon

ms · 2010-08-21 15:37:16

Hello Archers,

In the spirit of scratch-my-own-itch, I created another DNS cache tool called rescached.

Rescached is a daemon that act as a DNS server, serve the clients queries, but also caching all that queries for later use. Unlike any other name-service daemon, rescached save all of cache objects when program exit and loaded it back to memory when started back.

History.

I have a slow and unstable connection. Back then, I try anything to minimize all outgoing data by caching and tweaking all my traffic (squid, privoxy, web browser caches, blocking images, etc.). Until I found about djbdnsd and I cannot install it. So, long story short I created my own DNS cache. This was a few weeks before I know about pdnsd, and my program already half way finished so I just continue it, beside that I learn many thing about DNS protocol.

Since Archer always want to know what the system do, if you want to view what your system request each time, edit file "/etc/rescached/rescached.cfg" and set option "debug" to "1".

For non-technical explanation you can read it here [1].
For user documentation you can read it here [2].
AUR package is here [3].
Source is here [4].

Quick installation,

Install package from AUR.

Change your name server,

$ sudo cp /etc/resolv.conf /etc/resolv.conf.org
$ sudo echo "nameserver 127.0.0.1" > /etc/resolv.conf

Restart rescached
```
$ sudo systemctl restart rescached
```
Do not forget to restart any application (browser, email, etc) to make it read new /etc/resolv.conf.

All feedbacks is welcome.

Thank you all,

ms

--
[1] http://www.kilabit.info/journal/2009/12 … ed_is_here
[2] http://www.kilabit.info/projects/rescached/doc/user/
[3] https://aur4.archlinux.org/packages/rescached-git/
[4] https://github.com/shuLhan/rescached

UPDATE:
2015.07.06

Update hosts ads.
New config option "server.parent.connection". This allow rescached connect to parent server using UDP or TCP.
Fixes on TCP request handle.
Startup and logs.

2014.12

Many major fix (memory leaks, DNS type for AAAA, multiple answers, serving /etc/hosts)
Ads blocking by hostname (Credit to http://pgl.yoyo.org for providing the list).

2010

fix link #1 in this post.
rescached updated to version 2010.11.18.2123.

Last edited by ms (2015-07-19 01:37:24)

superchango · 2010-08-27 03:22:17

Sound great, i'll tet it in my system now.

Thnxs!!

gtklocker · 2010-08-28 10:38:07

Awesome program. It improved my browsing speed very much.

ms · 2010-08-31 23:25:28

Well, thank you all, especially to superchango and gtklocker for direct replies, for already trying my program.

It's really feel greats to create something that can be useful for others.

Just a quick note, in case some one missed it from the last section in documentation,

There is a problem with DNS caching. Sometimes domain name change their IP address with a new IP address. Since old IP address is already cached in rescached program, and your program still use an old IP address, program cannot connect to the specific domain name. You can notice this happen when the program staled (loading infinitely) or failed to connect.
There is only two solution.
First, stop rescached program, delete the cache file and start the rescached program back. This will start the program back with an empty cache.
Second, stop rescached program, edit cache file, delete specific record based on the domain name that have the problem, and restart the rescached program back. Be careful when doing this, make sure you really delete one record (you can see in cache file that each record start with "domain.name" and end with character "|").

Currently there is no "easy" way (I.e. automatic detection) and simple solution to this problem yet. But, if anyone have an idea or solution I will listen to it.

Last edited by ms (2010-08-31 23:27:38)

faelar · 2010-09-01 00:39:27

Can you do the following ? :

- There is something cached, let's call it "olddata".
- Things have changed since last time so in reality what we need here is "newdata".

1. The user make a request.
2. rescached answers with "olddata".
3. rescached start a query.
4. cache is updated with "newdata".

It doesn't resolve the problem for the first query, but on a second time thing will be ok without any user intervention.

yejun · 2010-09-01 05:20:32

Have you tried unbound which support prefetching and dnssec.

And glibc also comes with a dns caching /etc/rc.d/nscd

Most modern browsers have buildin dns prefetching and caching.

Edit:
Bind and dnsmasq are also worth trying.

Last edited by yejun (2010-09-01 13:47:32)

ms · 2010-09-01 10:59:19

@faelar,

If rescached do that for every request from client, then it break the main
purpose of rescached it self: to minimize traffic to outside network.

What make rescached different with others DNS cache daemon is the caches is
permanent, once the host-name "A" is already cached there is no way the same
query go outside of the network again. I tried to keep it that way. That is
why rescached is best used only for personal use, when DNS traffic data only
about host-name and email address. [1]

The only solution that come to my mind right now is by using a custom Time To
Live (TTL) value. In example refresh "old-data" if older than 1 day, or 1 week,
which can be customed by user.

@yeyun,

I have not tried unbound yet, but thanks for the information, I will leave it
to another Archers for trying it.

NSCD, yes I *knew* it, and I rather not to talk about it.

- - -

[1] In conclusion, I can say that rescached is a very lite version of pdnsd,
where it keep fast, simple configuration, and minimal resources as possible.

ms · 2010-11-24 07:14:32

This is an update to rescached.

There is a new option for a new update: cache.mode (in /etc/rescached/rescached.cfg).

if cache.mode is set to 1, all DNS record will be keep forever, no possible query or renewed DNS record happen until the specific DNS record removed from cache.

If cache.mode is set to 2 (default), all DNS record will be keep only after the specific time, based on maximum TTL value in DNS record. If TTL is reached, hostname will be queried again to the DNS parent server, and old cache and TTL will be replaced with a new one.

In non-technical explanation, you do not need to worry again about the problem that I mention above, rescached will handle it automatically. But, there is a consequence: old cache file does not work anymore because the change in cache format file.

A new update is already in AUR.

Thanks,

ms

Last edited by ms (2010-11-24 07:17:23)

herve18 · 2011-05-22 11:55:38

Hi, I used rescached for about a month and I was glad because I could still surf the Web while downloading at full speed, though I'm on a asthmatic ADSL line.

But I was getting strange behaviour lately: there was several seconds before pacman downloaded a package list from a mirror (I tried several ones), surfing was stalled while downloading, etc.

Now I've installed dnsmaq and it works out of the box. And then I can surf without even noticing I am downloading at the same time. I still need to "fix" those f*cking OpenDNS servers that reply with their own website on unknown addresses.

Thanks anyway for rescached solving that long-time issue in the first place.

essence-of-foo · 2011-05-23 07:40:10

I've been using it for a few days and it's really nice

For unknown reasons some DNS lookups fail (which is probably not the fault of rescached). Whenever this happens, I need to stop rescached, edit/delete the cache file and start it again.

Therefore I propose the following feature: Introduce an articifial TLD like *.new which will force rescached to refresh the domain that's written before .new

example.org --> unreachable
example.org.new --> let rescached check the domain again and return the result

Last edited by essence-of-foo (2011-05-23 07:42:26)

ms · 2011-05-23 15:02:23

@herve18, @essence-of-foo,

Not trying to be biased here, I have been using the same rescached source, even build and use the same package from AUR, and cannot recall the problem that you had. But, I will try to make more robust test and double check the code next time. Like someone said, network program is a hell of beast, you cannot predict how others use their network.

@essence-of-foo, I can see your point. Technically, if rescached fail at querying to DNS server (i.e: server unreachable or time-out) it will returned nothing. It will just removed from queue and waiting for the next query from client. Your application will see this as normal time-out. Adding a new TLD will just make a lot of work (checking, parsing, etc; if you know what I mean).

Thanks for trying it anyway.

PS: This comment remind me to add IPv6 support to rescached, which I planned to finished before June 8, 2011. I hope I can make it in time.

kofrad · 2011-05-24 02:32:54

Just installed this and got it running, so far everything looks great. My quick little benchmark saw dns lookup times from 35->3 and 136->3 for microsoft.com and epcot.com. I think the feature that sold me over on pdnsd was the way the cache will purge all entries less than a certain threshold once it fills. I'm a little uncertain what optimum values would be for those fields but I'm using 100,000/25 for the max/threshold, if anyone wants to get a collection of config setups going. I haven't had time to test this on a day-to-day basis yet, but everything seems to be working fine with no errors so far. Great work, would love to see continued development on this!

Thorsten Reinbold · 2012-12-07 12:35:39

Could it be that rescached isn't compatible with up-to-date Arch Systems using systemd?

I get this, when trying to start:

sudo /usr/sbin/rcrescached start
/usr/sbin/rcrescached: Line 3: /etc/rc.conf: File or Directory not found
/usr/sbin/rcrescached: Line 4: /etc/rc.d/functions: File or Directory not found
/usr/sbin/rcrescached: Line 13: stat_busy: Command not found.
/usr/sbin/rcrescached: Line 18: add_daemon: Command not found.
/usr/sbin/rcrescached: Line 19: stat_done: Command not found.

ms · 2012-12-07 20:34:38

> Could it be that rescached isn't compatible with up-to-date Arch Systems using systemd?

Yes, sorry for that. I don't have time to update the source yet.

For temporary solution you can run it manually by executing /sbin/rescached /etc/rescached/rescached.cfg.

olace · 2013-01-06 21:12:49

Seems to work nice without systemd, but I got this:

[vos::DNSQuery] extract_rr: (twitter.com) Record type '41' is not handle yet!

ms · 2014-12-18 20:06:27

So, four years has passed.

I just got spare time and fix the program. Many major improvements,

Fix memory leaks.
Support DNS record type 28 (AAAA or IPv6 for address).
Support reading and server hostname in /etc/hosts.

TODO:

blocking ads using hostname.

Any feedbacks will be really appreciate. Thank you!

ms · 2014-12-20 10:44:00

New feature: blocking ads using host name.

Ads list provided by http://pgl.yoyo.org.

ms · 2015-07-05 23:46:24

New features:

* Using TCP to connect to parent server. This is useful if your ISP block DNS connection (port 53) but forgot to block TCP connection (like my ISP).

If your ISP filtering your traffic (blocking porn, etc) by DNS, or not allowing public DNS, you can try to request using TCP to public DNS server using dig,

$ dig @8.8.8.8 +tcp www.reddit.com

If its works, use rescached as proxy between your computer and public DNS.

* Update hosts ads.
* Fixes on TCP request.
* Ads that being blocked now will be showed in log as "ads blocked" instead of "cached".

TODO:
* Statistics: show how many bytes that you save when using rescached.

Last edited by ms (2015-07-18 23:31:40)

ms · 2017-02-23 17:58:20

Hi Archer,

I would like announce the release of rescached v1.5.0 (2017-02-24).

The new tool `resolver` is coming to the town!

`resolver` is command line tool to query hostname to name server.
Its like `dig` (bind tools). Currently, it only provide function to query
nameserver in the following format,

$ resolver [@nameserver[:port]] [type] hostname

See resolver(1) manual for more information.

Enhancements:

* Rescached: refactoring client and resolver into threads

Rescached now have two threads: one is for processing client questions
(ClientWorker) and one is for reading answer from resolver.

* Update list of blocked hosts

* Minimize query to parent resolver by checking previous queue that has been
send to parent resolver.

* Split manual page into three parts: rescached(1), rescached.cfg(5), and
resolver(1).

Fixes:

* Fix timeout calculation and debug output on queue and name cache records

TODO:

I do not want to promise anything, but here is my plan for next release of rescached:

* Load host files in `/etc/rescached/hosts.d` directory

* Create CLI codename `resolver`

* Display overall cache statistic since started.

$ resolver CMD stat

* Display `n` number of host ranked from the top.

$ resolver CMD list [number]

* Search for name in cache and return their information.

$ resolver CMD search name

* Add new hostname directly without restarting the service,

$ resolver CMD add name ad.dr.es.ss

* Remove hostname from cache,

$ resolver CMD remove name

Thanks for playing!

ms · 2017-03-17 18:27:33

Rescached v1.6.0 (2017-03-18)

I would like to dedicated this release to clang's `scan-build` and `cppcheck` tools. Those are wonderful tools!

New Features:

* Load all host files from `/etc/rescached/hosts.d`

Breaking Changes:

* Blocked host file now will be installed inside `/etc/rescached/hosts.d`

This deprecated the config option for blocked host file
`file.hosts.block`, since it will automatically loaded.

Fixes:

* ResolverWorker: delete answer when fail at reading from server

Internal:

* Fix code as recommended by clang's `scan-build` and `cppcheck` tools

TODO

I do not want to promise anything, but here is my plan for next release of rescached:

* Display overall cache statistic since started.

$ resolver CMD stat

* Display `n` number of host ranked from the top.

$ resolver CMD list [number]

* Search for name in cache and return their information.

$ resolver CMD search name

* Add new hostname directly without restarting the service,

$ resolver CMD add name ad.dr.es.ss

* Remove hostname from cache,

$ resolver CMD remove name

Thanks for playing!

-- Sulhan

ms · 2018-09-29 00:27:40

Hi all,

We are back. We just release new rescached, v2.0.0-rc2, completely rewritten with Go.

= Introduction

rescached is a daemon that caching internet name and address on local memory
for speeding up DNS resolution.

rescached primary goal is only to caching DNS queries and answers, used by
personal or small group of users, to minimize unneeded traffic to outside
network.

This release is dedicated to Archer and github user `chadberg` and `skydrome`
who keeps reporting issues and may still use the original version.
Who ever you are, thank you!

= Features

List of current features,

- Enable to handle request from UDP and TCP connections
- Enable to forward request using UDP or TCP
- Load and serve addresses and hostnames in `/etc/hosts`
- Load and serve hosts formated files inside directory
`/etc/rescached/hosts.d/`
- Blocking ads and/or malicious websites through host list in
`/etc/rescached/hosts.d/hosts.block`
- Support loading and serving master (zone) file format from
`/etc/rescached/master.d`
- Integration with openresolv
- Support DNS over HTTPS (DoH) (draft 14)

For more information see https://github.com/shuLhan/rescached-go .

= Background

The original rescached software is written in C++ [1], written back since
2009. Over time, the more feature to add, the more painful to maintainance
and test it. You can't trust and remember your own code after being long
enough not meddling with it.

After having experienced with Go and confidence enough, I try to porting it.
The time its take to porting it is about one month.
The first step is to create DNS library [2] that support client and server
model.
Everything else in rescached is just a layer to manage caches from DNS
responses on top of the library.

Adding new features, like loading master file, integration with openresolv and
DoH is take about one months. Thanks to Go standard packages that provide
user all the required libraries for HTTP and TLS.

Overall I am very happy working with Go.

--
[1] https://github.com/shuLhan/rescached
[2] https://github.com/shuLhan/share/tree/master/lib/dns

ms · 2021-01-25 18:13:05

Hello fellow archers,

Long time no see. Two years has been passed (oh... home fast they flies) and the new rescached has been released. This new release bring web user interface which provide functionalities like monitoring active caches, managing caches, hosts, and zone files.

Some breaking changes on this release,

* The hosts.d directory, where custom hosts file stored, is now static to /etc/rescached/hosts.d
* The master.d directory, where zone file is located, is now static to /etc/rescached/zone.d

Below is some screenshots of the interface (please forgive me for the image size since I can not find how to render their size using img tag),

The front page .

The front page allow you monitor active caches and removing them directly.

The environment page .

This is the interface to configure the server directly on-the-fly with auto-restart.

The hosts blocks page .

The hosts blocks page allow enabling or disabling which hosts blocks sources you want to use.

The hosts.d page .

This hosts.d page allow you to create custom hosts file, add or remove the record in them.

The zone.d page .

The zone.d page allow you create zone file, adding or removing custom DNS records other than A record.

bialy39 · 2024-01-06 15:55:55

Does rescached supports diffrerent DNS server per domain?

Regards.

ms · 2024-01-06 16:04:50

> Does rescached supports diffrerent DNS server per domain?

Unfortunately, it does not support it right now. Personally, I have not found a use case where this feature useful to me. Feel free to open a new issue at https://github.com/shuLhan/rescached-go/issues though, if you have GitHub account.

Arch Linux

#1 2010-08-21 15:37:16

rescached - yet another DNS/resolver cache daemon

#2 2010-08-27 03:22:17

Re: rescached - yet another DNS/resolver cache daemon

#3 2010-08-28 10:38:07

Re: rescached - yet another DNS/resolver cache daemon

#4 2010-08-31 23:25:28

Re: rescached - yet another DNS/resolver cache daemon

#5 2010-09-01 00:39:27

Re: rescached - yet another DNS/resolver cache daemon

#6 2010-09-01 05:20:32

Re: rescached - yet another DNS/resolver cache daemon

#7 2010-09-01 10:59:19

Re: rescached - yet another DNS/resolver cache daemon

#8 2010-11-24 07:14:32

Re: rescached - yet another DNS/resolver cache daemon

#9 2011-05-22 11:55:38

Re: rescached - yet another DNS/resolver cache daemon

#10 2011-05-23 07:40:10

Re: rescached - yet another DNS/resolver cache daemon

#11 2011-05-23 15:02:23

Re: rescached - yet another DNS/resolver cache daemon

#12 2011-05-24 02:32:54

Re: rescached - yet another DNS/resolver cache daemon

#13 2012-12-07 12:35:39

Re: rescached - yet another DNS/resolver cache daemon

#14 2012-12-07 20:34:38

Re: rescached - yet another DNS/resolver cache daemon

#15 2013-01-06 21:12:49

Re: rescached - yet another DNS/resolver cache daemon

#16 2014-12-18 20:06:27

Re: rescached - yet another DNS/resolver cache daemon

#17 2014-12-20 10:44:00

Re: rescached - yet another DNS/resolver cache daemon

#18 2015-07-05 23:46:24

Re: rescached - yet another DNS/resolver cache daemon

#19 2017-02-23 17:58:20

Re: rescached - yet another DNS/resolver cache daemon

#20 2017-03-17 18:27:33

Re: rescached - yet another DNS/resolver cache daemon

#21 2018-09-29 00:27:40

Re: rescached - yet another DNS/resolver cache daemon

#22 2021-01-25 18:13:05

Re: rescached - yet another DNS/resolver cache daemon

#23 2024-01-06 15:55:55

Re: rescached - yet another DNS/resolver cache daemon

#24 2024-01-06 16:04:50

Re: rescached - yet another DNS/resolver cache daemon

Board footer