query to www.archlinux.org every 5 minutes

farmerdave · 2018-03-23 09:11:01

Hi,
I recently started paying more attention to my online privacy, so I installed Pi-Hole to a RaspberryPi and use it as my DNS server. I discovered that my desktop pc with an Arch installation queries www.archlinux.org 4 times (two on IPv4 and two on IPv6) precisely every five minutes.

I thought at first it might be the systemd-timesyncd.service, but mine is configured to use 0.arch.pool.ntp.org and there is a specific log of that query. I'm now struggling to think what else it could be, and how I can find it.

Can someone please enlighten me! What could this be and is it necessary?

Edit:
I'm not sure how much information I will need to provide, but I have plasma desktop running, and only a few services that I have enabled over and above the system services.

Last edited by farmerdave (2018-03-23 09:15:01)

phw · 2018-03-23 09:32:25

This is likely Network Manager, which regularly queries a URL to verify that an internet connection is setup. If this qeuest fails, your DE might indicate that you have a network connection, but no internet connection.

You can override it and configure it to use a different server, different interval or disable it completely by adding a new file to /etc/NetworkManager/conf.d and configure the [connectivity] section there. See https://developer.gnome.org/NetworkMana … .conf.html for a description of the available options.

I used to set this up to point to one of my own servers, but currently I just have it as the default.

EDIT: The default configuration is in /usr/lib/NetworkManager/conf.d/20-connectivity.conf and just sets the checl URL to http://www.archlinux.org/check_network_status.txt .

Last edited by phw (2018-03-23 09:35:31)

farmerdave · 2018-03-23 09:38:22

Thanks!
I guess I don't mind it happening providing it is generally considered ok by the community. I'll override it as you suggest and confirm this is in fact Network Manager (which I do use) and then most likely leave it at the default.

Trilby · 2018-03-23 12:28:30

farmerdave wrote:

I guess I don't mind it happening providing it is generally considered ok by the community.

That depends on your definition of ok. I'd say it is mostly harmless, and completely pointless.

If you are doing something that requires a network connection, you'll notice if/when it goes down long before the icon on the panel turns red (or whatever it would do). If you are not doing anything that requires a network connection, do you really need a red icon to interrupt what you are doing to tell you that you no longer have something you no longer need? Youd find out anyways as soon as it becomes relevant (i.e. the next time you tried to do anything requiring a connection).

An icon changing based on a check of a website at 5 minute intervals provides slightly less useful information than a weather stone.

Last edited by Trilby (2018-03-23 12:34:02)

NoSuck · 2018-03-23 15:38:04

I hadn't heard of Pi-Hole (and almost wrote it off as Pi-exclusive, given the name). Are you liking it so far, and do you know of any alternatives? Pi-Hole's list of features reads like a Swiss army knife, but if all the doodads are designed to work out of the box, I might be able to try it sooner rather than later.

Trilby wrote:

An icon changing based on a check of a website at 5 minute intervals provides slightly less useful information than a weather stone.

Conky ricers should set this as their desktop wallpapers instead of using so many weather widgets.

nomorewindows · 2018-03-23 18:08:41

Wouldn't necessarily need something that pings out to the internet. Mumble VOIP does a good job of letting me know if there is a network problem. Usually the sound I hear disappears before it tells me that the connection dropped. My cell phone doesn't bother to tell me that it dropped my connection while I'm still talking to somebody who isn't listening. Most of the time that's just across the local lan, but it could be used for the internet too...

eschwartz · 2018-03-23 18:13:19

Trilby wrote:

farmerdave wrote:
I guess I don't mind it happening providing it is generally considered ok by the community.
That depends on your definition of ok. I'd say it is mostly harmless, and completely pointless.
If you are doing something that requires a network connection, you'll notice if/when it goes down long before the icon on the panel turns red (or whatever it would do). If you are not doing anything that requires a network connection, do you really need a red icon to interrupt what you are doing to tell you that you no longer have something you no longer need? Youd find out anyways as soon as it becomes relevant (i.e. the next time you tried to do anything requiring a connection).
An icon changing based on a check of a website at 5 minute intervals provides slightly less useful information than a weather stone.

Well, that assumes that the network connectivity check is just a changed icon... but that is not really true. It's a networkmanager library feature, and can be queried via nmcli and various other means, and could probably also be used to block the networkmanager-wait-online target (not sure if this happens already).

That would actually be pretty useful IMHO.

Although admittedly the #1 reason captive portals annoy me for, is due to corrupting pacman's databases because pacman still doesn't check for db corruption *before* overwriting the sync databases with whatever it just downloaded. That's not fundamentally a network issue, it's fundamentally a pacman bug.

nomorewindows wrote:

Wouldn't necessarily need something that pings out to the internet. Mumble VOIP does a good job of letting me know if there is a network problem. Usually the sound I hear disappears before it tells me that the connection dropped. My cell phone doesn't bother to tell me that it dropped my connection while I'm still talking to somebody who isn't listening. Most of the time that's just across the local lan, but it could be used for the internet too...

Mumble can detect captive portals without pinging the internet???

It's hardly impressive if they can detect a missing network -- that much was always trivial. Captive portals exist on perfectly good, working networks.

Last edited by eschwartz (2018-03-23 18:15:08)

Trilby · 2018-03-23 19:19:20

Eschwartz wrote:

Well, that assumes that the network connectivity check is just a changed icon... but that is not really true. It's a networkmanager library feature, and can be queried via nmcli and various other means, and could probably also be used to block the networkmanager-wait-online target (not sure if this happens already).

Why query via nmcli when you can query with `ping` and get actual current results rather than an up-to-5-minute-old result? Or better yet, just check /proc/net/*

The wait-online.target may be a fair point - but on that you are not even sure if it works, and if it does, it is reached once at boot, so that is in no way an arguement for continued connections to a website at 5 minute intervals.

I didn't say that there was no utility in monitoring one's network connection (though it has much less than would be often assumed). But checking archlinux.org every 5 minutes is a braindead way of doing that.

nomorewindows · 2018-03-23 19:22:17

Eschwartz wrote:

Trilby wrote:
farmerdave wrote:
I guess I don't mind it happening providing it is generally considered ok by the community.
That depends on your definition of ok. I'd say it is mostly harmless, and completely pointless.
If you are doing something that requires a network connection, you'll notice if/when it goes down long before the icon on the panel turns red (or whatever it would do). If you are not doing anything that requires a network connection, do you really need a red icon to interrupt what you are doing to tell you that you no longer have something you no longer need? Youd find out anyways as soon as it becomes relevant (i.e. the next time you tried to do anything requiring a connection).
An icon changing based on a check of a website at 5 minute intervals provides slightly less useful information than a weather stone.
Well, that assumes that the network connectivity check is just a changed icon... but that is not really true. It's a networkmanager library feature, and can be queried via nmcli and various other means, and could probably also be used to block the networkmanager-wait-online target (not sure if this happens already).
That would actually be pretty useful IMHO.
Although admittedly the #1 reason captive portals annoy me for, is due to corrupting pacman's databases because pacman still doesn't check for db corruption *before* overwriting the sync databases with whatever it just downloaded. That's not fundamentally a network issue, it's fundamentally a pacman bug.
nomorewindows wrote:
Wouldn't necessarily need something that pings out to the internet. Mumble VOIP does a good job of letting me know if there is a network problem. Usually the sound I hear disappears before it tells me that the connection dropped. My cell phone doesn't bother to tell me that it dropped my connection while I'm still talking to somebody who isn't listening. Most of the time that's just across the local lan, but it could be used for the internet too...
Mumble can detect captive portals without pinging the internet???
It's hardly impressive if they can detect a missing network -- that much was always trivial. Captive portals exist on perfectly good, working networks.

Mumble only pings an internet mumble server, if you are using one. If you are using a private mumble server on your own lan, it pings that. Mumble doesn't cross over.
Teamspeak will talk to its' servers even if you are running your own private server. 'Upgrade to our latest version'.

Last edited by nomorewindows (2018-03-23 19:24:07)

phw · 2018-03-24 00:17:43

Trilby wrote:

But checking archlinux.org every 5 minutes is a braindead way of doing that.

How else would you check for a working internet connection other than querying some internet address?

Trilby · 2018-03-24 00:40:48

Oh, I don't know, perhaps exactly what I had already written in my previous post.

First, why do you need to check for an internet connection? Anything that requires an internet connection will tell you if/when it fails, then you know you don't have a connection. If nothing would otherwise tell you that you are lacking a connection, there is no purpose in NM doing so. But much more to the point: the problem is "every 5 minutes". NM will only give you stale data.

Imagine some process that needs to download something from some url. It could first check with NM whether there is a connection, but even if NM says there is, not may or may not still be correct. So the psuedo-code:

1) Check with NM if we have a connection
2) If NM reports we have a connection, go to 3, otherwise go to 5
3) Attempt download, if successful, exit, otherwise go to 4
4) Report an error downloading and exit
5) Attempt download anyways, if successful, exit, otherwise go to 4

Now, a non-NM alternative of the exact same program

1) Attempt download, if successful, exit, otherwise go to 2
2) Report error and exit.

The first version adds nothing on top of the second but complexity, bloat, and overall stupidity.

Docbroke · 2018-03-24 02:58:07

Trilby wrote:

Imagine some process that needs to download something from some url. It could first check with NM whether there is a connection, but even if NM says there is, not may or may not still be correct.

Imagine that user needs to download something from some url. He will look at the icon and if there is net connection he can go ahead. Otherwise in non-NM scenario he will have to keep trying till network becomes available and download is done successfully.

I am not the fan of NM or any utility to check net connection every few minutes using ping, but that doesn't imply no one should use that. Personally I only need to see that my computer is connected to my home wifi, that would be sufficient in most cases.

Trilby · 2018-03-24 03:15:20

Docbroke wrote:

Imagine that user needs to download something from some url. He will look at the icon and if there is net connection he can go ahead.

But this is my point, it adds absolutely no new information that said user would not already have. It may even occasionally give incorrect information (as it is up to 5 minutes old). So said user would look at the icon, if it shows the network is up, the download might still fail, if it shows the network is down, they'd restart NM or take some other action to get the network back up.

The attempt to download the desired resource is a direct test of whether the network is up. An additional indirect test adds absolutely no new information (only possibly some misinformation).

Last edited by Trilby (2018-03-24 03:16:50)

phw · 2018-03-24 08:30:10

Trilby wrote:

Oh, I don't know, perhaps exactly what I had already written in my previous post.

Sorry, I probably did not quote enough of your post in my previous comment. My question aimed at the point, that you said there are uses for checking for a working internet connection, but querying a URL every 5 minutes is braindead. Answering the question for an alternate solution with basically "don't do it" is kind of chicken out

Trilby wrote:

Anything that requires an internet connection will tell you if/when it fails, then you know you don't have a connection.

Actually no, you don't. You know that you have an issue connecting to this specific service. If you want to know more about why, you next would probably ping another reliable server to gather a second data point. Basically that's what the online check feature provides.

As I see it the feature has mainly two uses, both are more targeted at less experienced users:

- Allow to communicate to the user whether the network connection provides internet connectivity or not, basically providing the second data point I talked above.
- Allow detection of captive portals

The above, especially the first point, is mainly useful for the less experienced users. It might surprise you, but many people don't initially understand that network connection != Internet connection. With such a feature this can be communicated to them.

My main issue with the feature is actually not the delay of 5 minutes, it's more the fact that it can easily give false positives, since one specific URL is assumed to represent the internet. That's especially confusing for the target audience of less experienced users.

I was actually surprised when I discovered that Arch provides this functionality and does not simply disable it

I skip over your pseudo code example, as it is IMHO not describing one of the uses.

Trilby · 2018-03-24 11:53:36

phw wrote:

Sorry, I probably did not quote enough of your post in my previous comment. Answering the question for an alternate solution with basically "don't do it" is kind of chicken out

And what about admitting to not having read a post you are criticizing even after being told the response to your critique is right there? Sure, I'm a chicken, and you're an ass. I'm done here.

phw · 2018-03-25 12:04:19

Trilby wrote:

phw wrote:
Sorry, I probably did not quote enough of your post in my previous comment. Answering the question for an alternate solution with basically "don't do it" is kind of chicken out
And what about admitting to not having read a post you are criticizing even after being told the response to your critique is right there? Sure, I'm a chicken, and you're an ass. I'm done here.

Ok, sorry. I meant no offense. Bad wording on my part, apologies for that, I did not expect it to offend you that heavily

Maybe we are just talking past each other. Actually I was just talking about the periodic pinging, not about checking on demand, because the first is what this thread is about. Maybe let's take a step back for what use this functionality exists: applications can get informed about changes in internet connectivity

The main use for this is probably network connection applets or network management GUIs, which passively sit there and want to inform the user about current state. The polling is kind of a compromise of course, as information is not actually up-to-date this way. But for this kind of application it is maybe acceptable (or not, that's something we can disagree on). Another use case might be background tasks, for which it doesn't matter if they run now or 5 minutes later. Not sure if anybody uses it for this.

Trilby wrote:

I didn't say that there was no utility in monitoring one's network connection (though it has much less than would be often assumed). But checking archlinux.org every 5 minutes is a braindead way of doing that.

That's probably the point where I misunderstood you, not sure. What I thought was meant is, that there are uses for the use case I described above (continious monitoring), but pinging in in 5 minutes intervals is stupid. Allright, but I honestly would not know how to solve it better.

And I don't see a solution in your posts. You argue mainly if you need up-to-date information about the connection, the above does not help you should query the current status fresh (agreed*), and in cases where you do network requests you usually don't have to first check for an existing internet connection before trying the request (also agreed). But both do not address the use case above, which is passively getting the network status.

Your other answer to this is that it is "... completely pointless", which I kind of understand as a valid reasoning from the point of view of a specific user, but then you are actually not the target audience of this feature (see my previous post).

So my assumption that you agreed that there is some use for the use case above (which I might have misunderstood) together with the arguments that it is not useful for many use cases and even an icon sitting around that is only updated every 5 minutes also not useful led me to the statement that "Answering the question for an alternate solution with basically "don't do it" is kind of" evading the issue (better wording?)

That's it from my side, maybe my point of view is clear now, maybe not. If nothing else, please at least consider my apologies for having offended you. I did not expect my initial answer would escalate this quickly and the discussion would take so many words.

*) Btw, of course you can ask network manager for up to date status, in which case it will just query the check URL right away and update the internal status

eschwartz · 2018-03-25 12:19:55

Trilby wrote:

Imagine some process that needs to download something from some url. It could first check with NM whether there is a connection, but even if NM says there is, not may or may not still be correct. So the psuedo-code:
1) Check with NM if we have a connection
2) If NM reports we have a connection, go to 3, otherwise go to 5
3) Attempt download, if successful, exit, otherwise go to 4
4) Report an error downloading and exit
5) Attempt download anyways, if successful, exit, otherwise go to 4
Now, a non-NM alternative of the exact same program
1) Attempt download, if successful, exit, otherwise go to 2
2) Report error and exit.
The first version adds nothing on top of the second but complexity, bloat, and overall stupidity.

Alternative version of a different, badly behaved program:

1) attempt download, if successful, go to 3, otherwise go to 2
2) successfully download incorrect data like a captive portal that is marked with a 200 OK http status. go to 4
3) successfully download useful data
4) try to validate the data and use it for some purpose or another. Possibly throw a cryptic error when an http splash page isn't, say, a valid tarball-formatted flat file database.

As I mentioned above, pacman is a good example of a program that does just that. Worse, even if you later on fix your internet connection the last-modified date of the http captive portal is going to be newer than the remote databases, possibly for a while, which means you need to learn more than you probably wanted to know about the internal details of pacman's backend to debug the problem.

It is unwise to assume every program will error when it should, or provide a useful error when relevant... Arch users are *mostly* better at troubleshooting this anyway, but certainly having the feature is not a ridiculous notion.

Trilby · 2018-03-25 13:38:18

Eschwartz wrote:

As I mentioned above, pacman is a good example of a program that does just that.

Then pacman should be fixed.

But in either case, periodic checking of archlinux.org is not a sane remedy for this: does it even work? Does NM correctly identify captive portal responses? If NM says we have a good connection does that mean our programs should just trust it and not implement their own sanity checks on the data? Of course not.

Pacman should check the data it retrieves. Failing to do so is a bug to be fixed. If it is considered "not a bug" then pacman should list network manager as a dependency as - according to your logic - network manager's check is the only way pacman would know if it was getting good data. Clearly this is absurd.

nomorewindows · 2018-03-25 20:29:40

You can have an internet connection working and operational before any network indicator icon ever figures it out...no reason to wait on it...
Software usually has a problem correctly diagnosing it beyond a certain capacity written into the program...usually the PEBKC, can figure out what the error message is actually saying.

Last edited by nomorewindows (2018-03-25 20:32:10)

frostschutz · 2018-03-25 22:01:49

Now we know why archlinux.org is down all the time, it's getting hammered by an untold number of ArchLinux installations querying it non-stop.

nomorewindows · 2018-03-26 18:04:29

frostschutz wrote:

Now we know why archlinux.org is down all the time, it's getting hammered by an untold number of ArchLinux installations querying it non-stop.

I thought it was everyone from other distributions looking to Arch everytime they had a problem with their distribution...they couldn't find the answer anywhere else...
But does Archlinux.org go down everytime that phrak gets hacked?

Last edited by nomorewindows (2018-03-26 18:05:01)

eschwartz · 2018-03-26 18:07:58

Silly person, phrak is no longer allowed to host Archlinux.org from his hacked laptop. It's been years now... we even took away his signing key: https://www.archlinux.org/master-keys/

nomorewindows · 2018-03-26 18:19:16

Eschwartz wrote:

Silly person, phrak is no longer allowed to host Archlinux.org from his hacked laptop. It's been years now... we even took away his signing key: https://www.archlinux.org/master-keys/

They must think he's somebody important, they keep hacking at him anyway...
Are you sure the hacker didn't take his keys away?

Last edited by nomorewindows (2018-03-26 18:19:48)

Arch Linux

#1 2018-03-23 09:11:01

query to www.archlinux.org every 5 minutes

#2 2018-03-23 09:32:25

Re: query to www.archlinux.org every 5 minutes

#3 2018-03-23 09:38:22

Re: query to www.archlinux.org every 5 minutes

#4 2018-03-23 12:28:30

Re: query to www.archlinux.org every 5 minutes

#5 2018-03-23 15:38:04

Re: query to www.archlinux.org every 5 minutes

#6 2018-03-23 18:08:41

Re: query to www.archlinux.org every 5 minutes

#7 2018-03-23 18:13:19

Re: query to www.archlinux.org every 5 minutes

#8 2018-03-23 19:19:20

Re: query to www.archlinux.org every 5 minutes

#9 2018-03-23 19:22:17

Re: query to www.archlinux.org every 5 minutes

#10 2018-03-24 00:17:43

Re: query to www.archlinux.org every 5 minutes

#11 2018-03-24 00:40:48

Re: query to www.archlinux.org every 5 minutes

#12 2018-03-24 02:58:07

Re: query to www.archlinux.org every 5 minutes

#13 2018-03-24 03:15:20

Re: query to www.archlinux.org every 5 minutes

#14 2018-03-24 08:30:10

Re: query to www.archlinux.org every 5 minutes

#15 2018-03-24 11:53:36

Re: query to www.archlinux.org every 5 minutes

#16 2018-03-25 12:04:19

Re: query to www.archlinux.org every 5 minutes

#17 2018-03-25 12:19:55

Re: query to www.archlinux.org every 5 minutes

#18 2018-03-25 13:38:18

Re: query to www.archlinux.org every 5 minutes

#19 2018-03-25 20:29:40

Re: query to www.archlinux.org every 5 minutes

#20 2018-03-25 22:01:49

Re: query to www.archlinux.org every 5 minutes

#21 2018-03-26 18:04:29

Re: query to www.archlinux.org every 5 minutes

#22 2018-03-26 18:07:58

Re: query to www.archlinux.org every 5 minutes

#23 2018-03-26 18:19:16

Re: query to www.archlinux.org every 5 minutes

Board footer