You are not logged in.

#101 2020-08-20 01:07:44

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,965
Website

Re: Reflector reborn

Version 2020.8.19 adds systemd integration with a custom configuration file. See the man page, configuration file (/etc/xdg/reflector/reflector.conf) and wiki for details.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#102 2020-08-26 02:30:58

starquake
Member
Registered: 2013-05-03
Posts: 13

Re: Reflector reborn

Should the folder /etc/xdg/reflector and file /etc/xdg/reflector/reflector.conf already exist? Because on my system they don't. And the wiki make it sound like they are already there.

EDIT: Nevermind, it seems I had a mirror that didn't get updates for quite awhile and I had an old version of reflector.

Last edited by starquake (2020-08-26 02:49:28)

Offline

#103 2020-11-02 18:39:44

berturion
Member
Registered: 2018-06-19
Posts: 22

Re: Reflector reborn

Hello,
I have a problem with reflector.service. As root, when I run "systemctl start reflector.service", the service fails:

# systemctl start reflector.service
Job for reflector.service failed because the control process exited with error code.
See "systemctl status reflector.service" and "journalctl -xe" for details.

Here is the result of "systemctl status reflector.service" right after that:

# systemctl status reflector.service
● reflector.service - Refresh Pacman mirrorlist with Reflector.
     Loaded: loaded (/usr/lib/systemd/system/reflector.service; disabled; vendor preset: disabled)
     Active: failed (Result: exit-code) since Mon 2020-11-02 22:28:14 +04; 8min ago
       Docs: https://wiki.archlinux.org/index.php/Reflector
   Main PID: 451347 (code=exited, status=1/FAILURE)

Nov 02 22:28:14 sheldon systemd[1]: Starting Refresh Pacman mirrorlist with Reflector....
Nov 02 22:28:14 sheldon reflector[451375]: error: Permission denied
Nov 02 22:28:14 sheldon systemd[1]: reflector.service: Main process exited, code=exited, status=1/FAILURE
Nov 02 22:28:14 sheldon systemd[1]: reflector.service: Failed with result 'exit-code'.
Nov 02 22:28:14 sheldon systemd[1]: Failed to start Refresh Pacman mirrorlist with Reflector..

But, if I run (as root) :

reflector --verbose --latest 5 --sort rate --save /etc/pacman.d/mirrorlist

then, the mirrorlist file is written correctly.

I have another Arch linux machine and the service is working perfectly. I don't understand.

Can someone help me please (answer below)?

EDIT: I found! I didn't remember I played with the owner and permissions of /etc/pacman.d/mirrorlist. Permissions where set as 755 and the file was owned by my usual user. I changed the owner to root and modified the permissions to 644 on this file and it works now. It is strange though because the service is run by root so it should be able to overwrite the file anyway. Well, that's fine now.

Last edited by berturion (2020-11-02 18:46:48)

Offline

#104 2020-12-02 06:27:12

riazufila
Member
Registered: 2019-09-30
Posts: 16

Re: Reflector reborn

When I "systemctl start reflector", the mirrorlist gets updated successfully. However, when I "systemctl enable reflector" and restarts my host machine. Then runs "systemctl status reflector", an error is given and mirrorlist is not updated.

Screenshot-from-2020-12-02-14-16-28.png

Offline

#105 2020-12-02 11:54:50

schard
Forum Moderator
From: Hannover
Registered: 2016-05-06
Posts: 2,111
Website

Re: Reflector reborn

What does the journal say?
My suspicion is a problem with your internet connectivity.


Inofficial first vice president of the Rust Evangelism Strike Force

Offline

#106 2020-12-02 13:31:23

riazufila
Member
Registered: 2019-09-30
Posts: 16

Re: Reflector reborn

schard wrote:

What does the journal say?
My suspicion is a problem with your internet connectivity.

reflector[449]: error: failed to retrieve mirrorstatus data: URLError: <urlopen error [Errno -3] Temporary failure in name resolution>

That is what it says. Should I paste my whole journalctl?
But the network online target happened way before the reflector.service starts..

Last edited by riazufila (2020-12-02 13:43:36)

Offline

#107 2020-12-02 13:41:56

schard
Forum Moderator
From: Hannover
Registered: 2016-05-06
Posts: 2,111
Website

Re: Reflector reborn

I'm not sure whether here is the right place for that, or not.
If not, you can also post a new topic on this here: https://bbs.archlinux.org/viewforum.php?id=18
Maybe some mod can clear this up.
In any case, the message confirms my suspicion.
Your DNS is not ready at the time reflector.service is being started.


Inofficial first vice president of the Rust Evangelism Strike Force

Offline

#108 2020-12-02 13:45:19

riazufila
Member
Registered: 2019-09-30
Posts: 16

Re: Reflector reborn

schard wrote:

I'm not sure whether here is the right place for that, or not.
If not, you can also post a new topic on this here: https://bbs.archlinux.org/viewforum.php?id=18
Maybe some mod can clear this up.
In any case, the message confirms my suspicion.
Your DNS is not ready at the time reflector.service is being started.

Does this mean, network-online.target doesn't really help? Asking this cause reflector is set up with that and I can confirm that the network-online.target is satisfied before reflector.service is executed.

Offline

#109 2020-12-02 14:37:57

schard
Forum Moderator
From: Hannover
Registered: 2016-05-06
Posts: 2,111
Website

Re: Reflector reborn

Probably. What service(s) do you use to connect to the network / internet?
I could think of NetworkManager being the problem, especially if NetworkManager-wait-online.service is not enabled.
Also you can increase the NM_ONLINE_TIMEOUT by systemd editing the unit.
That of course is, iff you use NetworkManager.


Inofficial first vice president of the Rust Evangelism Strike Force

Offline

#110 2020-12-02 15:19:03

riazufila
Member
Registered: 2019-09-30
Posts: 16

Re: Reflector reborn

schard wrote:

Probably. What service(s) do you use to connect to the network / internet?
I could think of NetworkManager being the problem, especially if NetworkManager-wait-online.service is not enabled.
Also you can increase the NM_ONLINE_TIMEOUT by systemd editing the unit.
That of course is, iff you use NetworkManager.

Yes. I use NetworkManager. The wait service is turned on and enabled which I can confirm. And I've also increased the value to 60 from 30 but the problems still persists.

Last edited by riazufila (2020-12-02 15:19:20)

Offline

#111 2020-12-03 17:59:54

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,965
Website

Re: Reflector reborn

Following the discussion here, threading support has been removed from the mirror rating to avoid skewing results and the target file has been changed to the community database (5 MB). Rating mirrors by speed will now take a little longer but the results should be more reliable. As mentioned in that thread, I personally think ranking mirrors by speed is mostly a waste of time and bandwidth. If you want to increase download speeds, I strongly recommend powerpill.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#112 2020-12-06 10:16:05

Utini
Member
Registered: 2015-09-28
Posts: 476
Website

Re: Reflector reborn

I wonder if this is a sane way of configuration and usage of reflector?

Since I travel quite I lot I prefer to manually run reflector depending on the change of location/country. For that I made my own script in /home/user/bin/syu:

#!/bin/bash
reflector --verbose --connection-timeout 1 --protocol https --country 'Austria,Belgium,Canada,Czechia,Finland,France,Germany,Hungary,Iceland,Italy,Netherlands,Norway,Slovakia,Slovenia,Sweden,Switzerland,United Kingdom' --age 24 --latest 7 --sort rate
yay -Syyu

Basically what I wanted to achieve:
1. connection-tiimeout: For all coming server checks (https,age,rate,..) limit the check timeout to 1sec. Everything that takes longer is probably slow.
2. protocol: From all available servers, remove all that don't support https
3. country: From all available servers, use only the ones I prefer. Those are mostly that countries that provide me with the fastest speeds
4. age: Use only mirrors that are up to date
5. latest: From the selection that is now left, select the 7 latest updated servers
6. sort: now test them for their speed and use the fastest one

I thought this would be the most resource friendly, fastest and most sane way of using reflector for my case?

Thanks!

Last edited by Utini (2020-12-06 10:16:31)


Setup 1: Thinkpad T14s G3, 14" FHD - R7 6850U - 32GB RAM - 2TB Solidigm P44 Pro NVME
Setup 2: Thinkpad X1E G1, 15.6" FHD - i7-8850H - 32GB RAM - NVIDIA GTX 1050Ti - 2x 1TB Samsung 970 Pro NVME
Accessories: Filco Majestouch TKL MX-Brown Mini Otaku, Benq XL2420T (144Hz), Lo(w)gitech G400, Puretrak Talent, Sennheiser HD800S + Meier Daccord FF + Meier Classic FF

Offline

#113 2020-12-06 12:56:53

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 30,330
Website

Re: Reflector reborn

Utini, that reflector use seems reasonable - though if you are scripting it, I'd expect there'd be a good way to determine what country you are currently in, and just use mirrors in that country.  But much more importantly - and the reason I commented - DON'T use `-Syyu'.  The additional 'y' either does absolutely nothing useful for you, or it just wastes more time, but it does pointlessly waste mirror server bandwidth.  Admittedly this use of bandwidth is fairly trivial, but as there is absolutely no benefit to using the extra 'y', just don't: that is not the proper way to update a system.


"UNIX is simple and coherent" - Dennis Ritchie; "GNU's Not Unix" - Richard Stallman

Offline

#114 2020-12-06 14:13:37

archlinux20130905
Member
Registered: 2013-09-05
Posts: 45

Re: Reflector reborn

riazufila wrote:
schard wrote:

Probably. What service(s) do you use to connect to the network / internet?
I could think of NetworkManager being the problem, especially if NetworkManager-wait-online.service is not enabled.
Also you can increase the NM_ONLINE_TIMEOUT by systemd editing the unit.
That of course is, iff you use NetworkManager.

Yes. I use NetworkManager. The wait service is turned on and enabled which I can confirm. And I've also increased the value to 60 from 30 but the problems still persists.

I have exactly the same error message:

reflector[449]: error: failed to retrieve mirrorstatus data: URLError: <urlopen error [Errno -3] Temporary failure in name resolution>

However in my case this particular error only occurs if I try to run the reflector.service in a VirtualBox Guest Installation of ArchLinux. It doesn't occur if I run

systemctl start reflector

as root on the host of the VirtualBox Guest. I experimented a little bit with the file

/usr/lib/systemd/system/reflector.service

and found that removing

AF_UNIX

from

RestrictAddressFamilies=~AF_AX25 AF_IPX AF_APPLETALK AF_X25 AF_DECnet AF_KEY AF_NETLINK AF_PACKET AF_RDS AF_PPPOX AF_LLC AF_IB AF_MPLS AF_CAN AF_TIPC AF_BLUETOOTH AF_ALG AF_VSOCK AF_KCM AF_UNIX AF_XDP

makes the service work in the Archlinux Virtualbox Guest as well. Does anyone know what could be the reason for that? More importantly from a security standpoint what kind of attack becomes possible if one removes AF_UNIX from RestrictAddressFamilies deny list? Note that one has to call

systemctl daemon-reload

after making changes to reflector.service.

Offline

#115 2020-12-06 19:55:07

Gesh
Member
Registered: 2010-04-12
Posts: 5

Re: Reflector reborn

Thanks for the tool Xyne.
Attached below is a patch to restore the old download_timeout option -- I am sufficiently far from the highest-scoring mirrors that even sampling a small number of them for speed is unbearably slow without it.
The implementation is admittedly hacky, but it's better than nothing.

If this isn't the proper route to contribution, please say so.

--- Reflector.py	2020-12-03 00:00:00.000000000 +0000
+++ Reflector.py	2020-12-06 00:00:00.000000000 +0000
@@ -37,6 +37,7 @@
 import shlex
 import socket
 import subprocess
+import signal
 import sys
 import tempfile
 import time
@@ -59,6 +60,7 @@
 MIRRORLIST_ENTRY_FORMAT = "Server = " + MIRROR_URL_FORMAT + "\n"
 
 DEFAULT_CONNECTION_TIMEOUT = 5
+DEFAULT_DOWNLOAD_TIMEOUT = 5
 DEFAULT_CACHE_TIMEOUT = 300
 
 SORT_TYPES = {
@@ -131,6 +133,12 @@
 
 # ------------------------------ Miscellaneous ------------------------------- #
 
+class SlowConn(Exception):
+    pass
+
+def dieIfSlow(sig, frame):
+    raise SlowConn()
+
 def get_logger():
     '''
     Get the logger used by this module. Use this to be sure that the right logger
@@ -193,7 +201,8 @@
 
 # ---------------------------------- Rating ---------------------------------- #
 
-def rate_rsync(db_url, connection_timeout=DEFAULT_CONNECTION_TIMEOUT):
+def rate_rsync(db_url, connection_timeout=DEFAULT_CONNECTION_TIMEOUT,
+        download_timeout=DEFAULT_DOWNLOAD_TIMEOUT):
     '''
     Download a database via rsync and return the time and rate of the download.
     '''
@@ -203,6 +212,8 @@
         '--contimeout={}'.format(connection_timeout),
         db_url
     ]
+    signal.signal(signal.SIGALRM, dieIfSlow)
+    signal.alarm(download_timeout)
     try:
         with tempfile.TemporaryDirectory() as tmpdir:
             time_0 = time.time()
@@ -217,16 +228,20 @@
             )
             ratio = size / time_delta
             return time_delta, ratio
-    except (subprocess.CalledProcessError, subprocess.TimeoutExpired, FileNotFoundError):
+    except (subprocess.CalledProcessError, subprocess.TimeoutExpired,
+            FileNotFoundError, SlowConn):
         return 0, 0
 
 
-def rate_http(db_url, connection_timeout=DEFAULT_CONNECTION_TIMEOUT):
+def rate_http(db_url, connection_timeout=DEFAULT_CONNECTION_TIMEOUT,
+        download_timeout=DEFAULT_DOWNLOAD_TIMEOUT):
     '''
     Download a database via any protocol supported by urlopen and return the time
     and rate of the download.
     '''
     req = urllib.request.Request(url=db_url)
+    signal.signal(signal.SIGALRM, dieIfSlow)
+    signal.alarm(download_timeout)
     try:
         with urllib.request.urlopen(req, None, connection_timeout) as handle:
             time_0 = time.time()
@@ -234,11 +249,13 @@
             time_delta = time.time() - time_0
         ratio = size / time_delta
         return time_delta, ratio
-    except (OSError, urllib.error.HTTPError, http.client.HTTPException):
+    except (OSError, urllib.error.HTTPError, http.client.HTTPException,
+            SlowConn):
         return 0, 0
 
 
-def rate(mirrors, connection_timeout=DEFAULT_CONNECTION_TIMEOUT):
+def rate(mirrors, connection_timeout=DEFAULT_CONNECTION_TIMEOUT,
+        download_timeout=DEFAULT_DOWNLOAD_TIMEOUT):
     '''
     Rate mirrors by timing the download of the community repo's database from
     each one.
@@ -265,9 +282,11 @@
         scheme = urllib.parse.urlparse(url).scheme
 
         if scheme == 'rsync':
-            time_delta, ratio = rate_rsync(db_url, connection_timeout)
+            time_delta, ratio = rate_rsync(db_url, connection_timeout,
+                    download_timeout)
         else:
-            time_delta, ratio = rate_http(db_url, connection_timeout)
+            time_delta, ratio = rate_http(db_url, connection_timeout,
+                    download_timeout)
 
         kibps = ratio / 1024.0
         logger.info(fmt.format(url, kibps, time_delta))
@@ -468,11 +487,13 @@
     def __init__(
         self,
         connection_timeout=DEFAULT_CONNECTION_TIMEOUT,
+        download_timeout=DEFAULT_DOWNLOAD_TIMEOUT,
         cache_timeout=DEFAULT_CACHE_TIMEOUT,
         min_completion_pct=1.0,
         url=URL
     ):
         self.connection_timeout = connection_timeout
+        self.download_timeout = download_timeout
         self.cache_timeout = cache_timeout
         self.min_completion_pct = min_completion_pct
         self.url = url
@@ -601,10 +622,10 @@
         help='The number of seconds to wait before a connection times out. Default: %(default)s'
     )
 
-#   parser.add_argument(
-#     '--download-timeout', type=int, metavar='n',
-#     help='The number of seconds to wait before a download times out. The threshold is checked after each chunk is read, so the actual timeout may take longer.'
-#   )
+    parser.add_argument(
+        '--download-timeout', type=int, metavar='n', default=DEFAULT_DOWNLOAD_TIMEOUT,
+        help='The number of seconds to wait before a download times out. Default: %(default)s'
+    )
 
     parser.add_argument(
         '--list-countries', action=ListCountries, nargs=0,
@@ -763,7 +784,7 @@
     if not mirrorstatus:
         mirrorstatus = MirrorStatus(
             connection_timeout=options.connection_timeout,
-            #       download_timeout=options.download_timeout,
+            download_timeout=options.download_timeout,
             cache_timeout=options.cache_timeout,
             min_completion_pct=(options.completion_percent / 100.),
             url=options.url

Last edited by Gesh (2020-12-06 20:11:22)

Offline

#116 2020-12-07 03:56:38

kcrmson
Member
From: 20 min from the Cobra Kai dojo
Registered: 2018-06-01
Posts: 12

Re: Reflector reborn

after making changes to reflector.service.[/quote]

Might I suggest something that will survive an update (if this isn't fixed in the next update of reflector)? Copy /usr/lib/systemd/system/reflector.service to /etc/systemd/system/reflector.service and perform the modification on that, run a systemctl daemon-reload then restart reflector and you should be golden.

Offline

#117 2020-12-07 04:00:16

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: Reflector reborn

Oooor you could both look at "systemctl edit reflector.service" whcih will guide you in using drop-in overrides like you're supposed to. smile


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#118 2020-12-07 04:08:42

kcrmson
Member
From: 20 min from the Cobra Kai dojo
Registered: 2018-06-01
Posts: 12

Re: Reflector reborn

eschwartz wrote:

Oooor you could both look at "systemctl edit reflector.service" whcih will guide you in using drop-in overrides like you're supposed to. smile

Completely forgot about this, yes, definitely this way is best. Especially if you've set your preferred $EDITOR.

Offline

#119 2020-12-07 06:09:07

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,965
Website

Re: Reflector reborn

@Utini
If you are using the latest 7, using the age criterion before it is probably useless. The only way it would change anything is if there were fewer than 7 mirrors that had synced in the last 24 hours, but that is unlikely with a list that long.

@archlinux20130905
The current service file is the one submitted by the Arch iso devs. I admit that I didn't bother researching all of the options and the reason for their inclusion, so I can't answer your question about which vulnerabilities its removal would create. If there is a good argument for removing it (utility for users vs security vulnerabilities) then I will consider it, but given that it seems to be limited to a few use cases, I recommend using "systemctl edit reflector.service" as suggested by eschwartz.

@Gesh
Thanks for the patch. I've applied it with some changes. It revealed an existing bug in the way some options were passed through to the download functions (they weren't tongue). I've also created a context manager to time downloads and restore previous alarms when done. I'll upload it shortly.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#120 2020-12-07 14:56:22

contra
Member
Registered: 2020-12-07
Posts: 2

Re: Reflector reborn

Recently got tired of editing the mirrorlist every time I do an update, so thanks a lot for the script.

I changed the script a bit, I'm here to share my changes. The '--sort country' option isn't very useful right now, as it sorts countries by alphabetical order, which is never what you want. I added another option to move a selected country (or countries) to the top of the mirrorlist, when sorting.

I see other people have posted patches here, I hope that's the correct way to contribute to the tool.

diff --git a/Reflector.py b/Reflector.py
index 7b6b7ab..b21836f 100644
--- a/Reflector.py
+++ b/Reflector.py
@@ -225,13 +225,26 @@ class DownloadTimer():
 
 # --------------------------------- Sorting ---------------------------------- #
 
-def sort(mirrors, by=None, **kwargs):  # pylint: disable=invalid-name
+def sorting_key_gen(spec_items_priority, item):
+    """
+    Handles the generation of the key for sorting
+    """
+    if item not in spec_items_priority:
+        return len(spec_items_priority), item
+    else:
+        return spec_items_priority.index(item), item
+
+
+def sort(mirrors, by=None, spec_items_priority=None, **kwargs):  # pylint: disable=invalid-name
     '''
     Sort mirrors by different criteria.
     '''
     # Ensure that "mirrors" is a list that can be sorted.
     if not isinstance(mirrors, list):
         mirrors = list(mirrors)
+    # ensure spec_items_priority is around
+    if spec_items_priority is None:
+        spec_items_priority = []
 
     if by == 'age':
         mirrors.sort(key=lambda m: m['last_sync'], reverse=True)
@@ -242,7 +255,7 @@ def sort(mirrors, by=None, **kwargs):  # pylint: disable=invalid-name
 
     else:
         try:
-            mirrors.sort(key=lambda m: m[by])
+            mirrors.sort(key=lambda m: sorting_key_gen(spec_items_priority, m[by]))
         except KeyError as err:
             raise MirrorStatusError('attempted to sort mirrors by unrecognized criterion: "{}"'.format(by)) from err
 
@@ -603,7 +616,7 @@ class MirrorStatus():
         msf = MirrorStatusFilter(min_completion_pct=self.min_completion_pct, **kwargs)
         yield from msf.filter_mirrors(mirrors)
 
-    def sort(self, mirrors, **kwargs):
+    def sort(self, mirrors, countries_priority=None, **kwargs):
         '''
         Sort mirrors by various criteria.
         '''
@@ -611,7 +624,7 @@ class MirrorStatus():
             mirrors = self.get_mirrors()
         kwargs.setdefault('connection_timeout', self.connection_timeout)
         kwargs.setdefault('download_timeout', self.download_timeout)
-        yield from sort(mirrors, **kwargs)
+        yield from sort(mirrors, spec_items_priority=countries_priority, **kwargs)
 
     def rate(self, mirrors=None, **kwargs):
         '''
@@ -717,6 +730,16 @@ def add_arguments(parser):
         help='Sort the mirrorlist. {}.'.format(sort_help)
     )
 
+    parser.add_argument(
+        '--priority_country',
+        help='List of countries to put on top of the list when sorting by country.'
+             'Multiple countries may be selected using commas (e.g. "France,Germany") or by passing this option multiple times. '
+             'Use "--list-countries" to see which are available.',
+        action='append',
+        dest='priority_countries',
+        metavar='<priority_country>'
+    )
+
     parser.add_argument(
         '--verbose', action='store_true',
         help='Print extra information to STDERR. Only works with some options.'
@@ -832,7 +855,7 @@ def parse_args(args=None):
     )
     parser = add_arguments(parser)
     options = parser.parse_args(args)
-    for list_arg in ('countries', 'protocols'):
+    for list_arg in ('countries', 'protocols', 'priority_countries'):
         setattr(options, list_arg, list(split_list_args(getattr(options, list_arg))))
     return options
 
@@ -883,7 +906,10 @@ def process_options(options, mirrorstatus=None, mirrors=None):
         mirrors = itertools.islice(mirrors, options.fastest)
 
     if options.sort and not (options.sort == 'rate' and options.fastest):
-        mirrors = mirrorstatus.sort(mirrors, by=options.sort)
+        if options.sort == 'country':
+            mirrors = mirrorstatus.sort(mirrors, by=options.sort, countries_priority=options.priority_countries)
+        else:
+            mirrors = mirrorstatus.sort(mirrors, by=options.sort)
 
     if options.number:
         mirrors = list(mirrors)[:options.number]

(Sorry, if I've messed something up, a bit new to sharing my code with others.)

Last edited by contra (2020-12-07 18:41:51)

Offline

#121 2020-12-13 19:30:53

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,965
Website

Re: Reflector reborn

@contra
I like the idea and the patch is fine, although I'll likely tweak the implementation a bit to handle both country names and country codes synonymously. I wonder if it's necessary though to add a new option. If a user has country preferences then I expect that they would filter by country prior to sorting. In that case, I would just use the arguments passed with "--country" to set the sort order.

Thoughts?


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#122 2020-12-16 03:45:27

romstor
Member
Registered: 2018-08-18
Posts: 54

Re: Reflector reborn

Have the same "failed to retrieve mirrorstatus data" issue as @archlinux20130905 on a fresh Arch installation when running "systemctl start reflector". Same errors too. Runs ok from command line. Using iwd for wireless networking and it works well with all the other programs and services - doesn't look like connectivity issue.

Tried fixes suggested in this thread with no success.

Interestingly enough "systemctl start reflector" works well on Arch ISO when booted live. I tried diffing reflector.conf and reflector.service with versions that I have installed and found minor differences in .conf (applying those didn't help either) and no differences in .service file. ISO has some overrides applied to its configuration - applying the same archiso override didn't seem to help either.

I found fairly recent ticket on Arch bug tracker regarding this exact issue - FS#68557 : reflector - systemd service fails to start due to a name resolution failure], but it was closed by @Foxboron without any explanation stating only that it is not a bug. Looking a bit more on wiki pages and google didn't yield any results.

If this is a known "not a bug" issue -- can someone kindly point to explanation / resolution. I have trouble tracking further difference in configuration between Arch ISO and my install. This smells a lot like systemd permissions issue.

Offline

#123 2020-12-17 22:09:09

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,965
Website

Re: Reflector reborn

The linked "bug" (FS#68557) has a comment in the closing notes that the issue was resolved by the submitter after fixing an error in resolv.conf. If you're getting name resolution failures then I suspect that your system's network settings are misconfigured, especially if name resolution works on the live medium on the same hardware with the same network.

<edit>
Search the forum for help with name resolution errors. For example, I just saw this thread in the list of active topics.
</edit>

If you can reproduce the error in a terminal, post the exact command along with the downloaded mirrorstatus data ($XDG_CACHE_HOME/mirrorstatus.json, which defaults to ~/.cache/mirrorstatus.json) and I will examine it for possible bugs.

Last edited by Xyne (2020-12-17 22:13:01)


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#124 2020-12-17 23:29:29

romstor
Member
Registered: 2018-08-18
Posts: 54

Re: Reflector reborn

Thank you @Xyne. I had a closer look at my DNS configuration per your suggestion. I use iwd with systemd-resolved, but not systemd-networkd (iwd has an option to be a simple network manager in simpler setups). When I installed systemd-resolved, I left the original /etc/resolv.conf untouched. Like I mentioned above - DNS name resolution worked well for me everywhere. If you leave the original /etc/resolv.conf around, documentation says it makes DNS config management more backwards compatible. It worked in my case, so I didn't need to change it.

This time I went and replaced /etc/resolf.conf with a link as per documentation in wiki. After that systemctl start reflector began working with no issues. Need to dig deeper about what the difference is - still not sure why reflector.service wouldn't work in that backwards compatible mode. May be this is because I don't run systemd-networkd (which has some magic when working with systemd-resolved in tandem. Arch ISO has both enabled and may be that is why reflector.service doesn't have issues there).

Thank you for your help!

Last edited by romstor (2020-12-18 00:35:59)

Offline

#125 2020-12-18 09:10:42

nl6720
The Evil Wiki Admin
Registered: 2016-07-02
Posts: 646

Re: Reflector reborn

romstor wrote:

When I installed systemd-resolved, I left the original /etc/resolv.conf untouched. Like I mentioned above - DNS name resolution worked well for me everywhere. If you leave the original /etc/resolv.conf around, documentation says it makes DNS config management more backwards compatible.

It was bound to break something eventually. There are software that perform domain name resolution themselves using the name servers from /etc/resolv.conf instead of using glibc provided functions (which would go through nss-resolve because of /etc/nsswitch.conf).
https://wiki.archlinux.org/index.php/Sy … solved#DNS says that leaving /etc/resolv.conf as a file is "less disruptive" in the context that if something else is managing it. If no other software is managing it or you're not managing it manually, then you'll want to symlink it to /run/systemd/resolve/stub-resolv.conf (or /run/systemd/resolve/resolv.conf if the software that reads /etc/resolv.conf insists on the listed name servers being real DNS servers).

Offline

Board footer

Powered by FluxBB