[Solved] Symbols point to non-existent gcc headers on min Arch install

sn0ot · 2020-05-03 03:13:31

Preface:

I am a noob trying to learn c++ and learn debugging with gdb (staying away from IDEs that do some of the repetitive work for me for the sake of knowing how the magic happens). I will be a little verbose in explaining so I would appreciate if replys would point out any of my misunderstanding that lead to problem.

I am using Arch because everything here should be a result of me putting it there. Except I cannot figure out why my debugger is sourcing files (.h and .cc) that do not exist on my machine (files listed at end).

I am posting in the Arch forums because I believe the headers and source files are of the stdlibc++ provided by GNU Compiler Collection (which Arch distributes, along with compiled .so objects(?) in the core repository). At least I think the issue is somewhere there (but after a few days I cannot figure it out and need some help).

Minimum Code To Replicate the Issue:

I have a very basic install of Arch,:

 sudo pacman -S base linux linux-firmware base-devel linux-lts linux-lts-headers

Plus browser, vim, networking, etc...

I am debugging a basic program

#include <iostream>                                                                                                                                   
int main(){                                                                                                         
    std::cout << "Hello Word" << std::endl;                                                   
    return 0;                                                                             
};

I compile with the following flags:

 g++ -Wall -g hello_world.cc

Then when I step through in gdb I step into the public member function std::operator<< and get this garbage

 std::operator<< <std::char_traits<char> > (__out=..., __s=0x555555556005 "Hello World") at /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/ostream:565

/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.h: No such file or directory.

and this continues seemingly forever while it steps through standard lib code, until it actually calls fwrite()

0x00007ffff7b01050 in fwrite () from /usr/lib/libc.so.6

I decided I would like to skip over all steps into the standard library and only view my code.

My attempted solution:

I edited my ~/.gdbinit file to script and exclude all files that have absolute paths that begin with /usr/include/* and /build/gcc/*. I confirmed this worked for /usr/include/* by checking info skip from within gdb and all appropriate files /usr/include/* are skipped. I also verified by writing code that would step into those header (that is outside this post).

But this won't work for absolute paths starting with /build/gcc/* >:(

From inside GDB I type info sources and a whole bunch of "Source files for which symbols will be read in on demand:" are listed that don't exist and I cannot skip. (I will include a small sample of these files at the end of the post).

Conclusion

I believe this is something to do with the debug symbols on the libc .so objects but it seems I have the source code for them installed (?) and I can't quite piece it together. I used readelf -a to look at the symbols for libc objects and they do not appear to be stripped.

1. If it is symbol related, where can I get the /build/gcc/* headers so I can step through them
2. Directory /build/gcc/* does not exist on my machine, and these paths are not pulled out of thin air, so how did they get on my machine, and where are they specified? Was it compiled elsewhere and the paths to source files was left in the objs which I installed?

"Source files for which symbols will be read in on demand:"

/usr/include/c++/9.3.0/iostream, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/string_view, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.h, 
/build/gcc/src/gcc/libstdc++-v3/src/c++17/cow-string-inst.cc, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/stl_iterator_base_funcs.h, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/char_traits.h, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/ext/new_allocator.h, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/allocator.h, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.tcc, /build/gcc/src/gcc/libstdc++-v3/src/c++17/string-inst.cc, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/ostream, 
/build/gcc/src/gcc/libstdc++-v3/src/c++17/ostream-inst.cc, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/locale_conv.h, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/system_error, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/functional_hash.h, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/predefined_ops.h, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/stl_iterator.h, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/array, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/stl_pair.h, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/string_view.tcc, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/stl_algobase.h, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/stl_algo.h, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/stl_uninitialized.h, /build/gcc/src/gcc/libstdc++-v3/libsupc++/new, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/tuple, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/unique_ptr.h, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/stl_construct.h, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/ext/atomicity.h, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/alloc_traits.h, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/move.h, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/allocated_ptr.h, /build/gcc/src/gcc/libstdc++-v3/libsupc++/typeinfo, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/ext/aligned_buffer.h, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/fs_path.h, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/codecvt.h, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h, /build/gcc/src/gcc/libstdc++-v3/src/c++17/fs_path.cc, /build/gcc/src/gcc/libstdc++-v3/src/c++17/cow-fs_path.cc, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/stl_stack.h, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/deque.tcc, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/stl_deque.h, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/chrono, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/functional, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/fs_ops.h, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/fs_dir.h, /usr/include/sys/stat.h, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/fs_fwd.h, 
/build/gcc/src/gcc/libstdc++-v3/src/c++17/../filesystem/ops-common.h, /build/gcc/src/gcc/libstdc++-v3/src/c++17/fs_ops.cc, /build/gcc/src/gcc/libstdc++-v3/src/c++17/cow-fs_ops.cc, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/utility, /build/gcc/src/gcc/libstdc++-v3/src/c++17/../filesystem/dir-common.h, /build/gcc/src/gcc/libstdc++-v3/src/c++17/fs_dir.cc, 
/build/gcc/src/gcc/libstdc++-v3/src/c++17/cow-fs_dir.cc, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/vector.tcc, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/memory, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/atomic_base.h, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/atomic, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bit, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/stl_vector.h, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/shared_mutex, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/std_mutex.h, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/stl_function.h, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/memory_resource, /build/gcc/src/gcc/libstdc++-v3/src/c++17/memory_resource.cc, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/ext/alloc_traits.h, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_ios.h, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/postypes.h, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/ios_base.h, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/streambuf, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/fstream, 
/build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/ext/stdio_filebuf.h, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/locale_facets.h, /build/gcc/src/gcc/libstdc++-v3/src/c++11/wstring-io-inst.cc, 
/build/gcc/src/gcc/libstdc++-v3/libsupc++/initializer_list, /build/gcc/src/gcc/libstdc++-v3/src/c++11/wstring-inst.cc, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/locale_facets.tcc, 
/build/gcc/src/gcc/libstdc++-v3/src/c++11/locale-inst.cc, /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/c++locale.h, ...............................

Last edited by sn0ot (2020-05-07 15:11:13)

eschwartz · 2020-05-03 05:24:24

File paths in /build/gcc/* are for the build chroot that created the package, so they're not expected to exist. That being said, the arch tools do support creating packages with full debug support, we're just not set up to distribute them ATM.

For packages which do distribute split *-debug packages, you'd see 'info sources' pointing instead at /usr/src/debug/. I have a custom repo that distributes splitdebug, here: https://wiki.archlinux.org/index.php/Un … #eschwartz

So if you were to switch to my glibc-git package, and also pacman -S glibc-git-debug, you'd have sources. Assuming you wanted to debug stuff inside glibc itself.

sn0ot wrote:

I edited my ~/.gdbinit file to script and exclude all files that have absolute paths that begin with /usr/include/* and /build/gcc/*. I confirmed this worked for /usr/include/* by checking info skip from within gdb and all appropriate files /usr/include/* are skipped. I also verified by writing code that would step into those header (that is outside this post).
But this won't work for absolute paths starting with /build/gcc/* >:(
From inside GDB I type info sources and a whole bunch of "Source files for which symbols will be read in on demand:" are listed that don't exist and I cannot skip. (I will include a small sample of these files at the end of the post).

What's the exact contents of .gdbinit, and does 'info skip' report that files in /build/ will be skipped as well?

sn0ot · 2020-05-04 04:41:32

eschwartz wrote:

What's the exact contents of .gdbinit, and does 'info skip' report that files in /build/ will be skipped as well?

Oof, I managed to skip the /build/* directories while writing this post. Thank God. I will keep the content that I wrote because it points at a shortcoming extending gdb with python and command post hooks (or I did it incorrectly), and a noobs misunderstanding about file globbing if using the gdb's build-in functionality. Maybe somebody will find it helpful. Probably not though, noobs never search for old posts

I will mark this post solved once I ask a few clarifying questions pertaining to the original question in the thread. But I need to look into some more things before I can ask a knowledgeable question and I'm already too sleepy.

I am using gdb version 9.1. I tried three methods to skip files with absolute files prefixed with /build/*
The three methods are:

Simple skip file <path>
- This actually worked for doing one file at a time, but I needed to include all std files (not manually)
File globbing to specify the files to skip in the debugger (I believe introduced in GDB 7.12 if I read the articles correctly)
- Originally didn't work because I used it incorrectly
```
skip -gfi /build/*
```
- Instead I had needed to glob (?) as deep as the directories were so the following worked but will probably have to adjust if source files are higher in the directory tree. Just put this in the ~/.gdbinit
```
 skip -gfi /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/*/* 
```
Extending gdb with python and command post hooks
- I liked the python method the most because I could just define the prefix and it'll dynamically skip files (that exist). But it does not work on files that don't exist. I slapped the following in my ~/.gdbinit to execute with the "start" command post hook:
```
# skip all STL source files
define skipstl
python
# get all sources loadable by gdb
def GetSources():
    sources = []
    for line in gdb.execute('info sources',to_string=True).splitlines():
        if line.startswith("/"):
            sources += [source.strip() for source in line.split(",")]
    return sources

# skip files of which the (absolute) path begins with 'dir'
def SkipDir(dir):
    sources = GetSources()
    for source in sources:
        if source.startswith(dir):
            gdb.execute('skip file %s' % source, to_string=True)

# apply only for c++
if 'c++' in gdb.execute('show language', to_string=True):
    SkipDir("/usr")
    SkipDir("/build")
end
end

define hookpost-start
    skipstl
end
```
info skip only populates for /usr/* directories and not /build/* directories. If I remove SkipDir("/usr") line in ~/.gdbinit and only leave in the SkipDir("/build"), info skip from inside gdb returns "not skipping any files or functions".

If you run info sources, it appears that all the /build/* paths are categorized as "Source files for which symbols will be read in on demand". In case that was the culprit, I launched gdb with the --readnow flag to read each symbol file's entire symbol table immediately
```
 gdb --readnow -tui a.out 
```
Now info sources from inside gdb did not list any /build/* directories and I was at half-mast. But alas, when I stepped through inside the debugger I still entered those pesky /build/* headers. >:'(

Last edited by sn0ot (2020-05-04 04:43:03)

sn0ot · 2020-05-07 03:40:10

eschwartz wrote:

File paths in /build/gcc/* are for the build chroot that created the package, so they're not expected to exist.

- Looks like the I download pre-compiled binaries from the core repository? I mean, I didn't compile the package locally during initial install of OS?
- If I were to try your repository, how do I go about it without blowing away my install of libstdc++, just linking /usr/lib/stdlibc++.so to custom built lib instead of my current /usr/lib/stdibc++.so.6.0.28 ?

eschwartz wrote:

For packages which do distribute split *-debug packages, you'd see 'info sources' pointing instead at /usr/src/debug/.

Concept of split debugging is a brand new concept to me, thanks! From what I gather that is when you can put a program’s debugging information in a file separate from the executable itself? It actually appears the /usr/lib/libstdc++.so.6.0.28 already has all the debug information embedded in itself (symbol table and a bunch of DWARF sections). Here are just a few.

There are 41 section headers, starting at offset 0x1590d80:
Section Headers:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 4] .dynsym           DYNSYM           0000000000008ee0  00008ee0
       00000000000235e0  0000000000000018   A       5     1     8
  [ 5] .dynstr           STRTAB           000000000002c4c0  0002c4c0
       0000000000047beb  0000000000000000   A       0     0     1
  [31] .debug_aranges    PROGBITS         0000000000000000  001e42d4
       0000000000014de0  0000000000000000           0     0     1
  [32] .debug_info       PROGBITS         0000000000000000  001f90b4
       00000000007cbdd4  0000000000000000           0     0     1
  [33] .debug_abbrev     PROGBITS         0000000000000000  009c4e88
       00000000000558be  0000000000000000           0     0     1
  [34] .debug_line       PROGBITS         0000000000000000  00a1a746
       000000000017cdfa  0000000000000000           0     0     1
  [35] .debug_str        PROGBITS         0000000000000000  00b97540
       000000000012e393  0000000000000001  MS       0     0     1
  [36] .debug_loc        PROGBITS         0000000000000000  00cc58d3
       000000000069c994  0000000000000000           0     0     1
  [37] .debug_ranges     PROGBITS         0000000000000000  01362267
       00000000001a27a0  0000000000000000           0     0     1
  [38] .symtab           SYMTAB           0000000000000000  01504a08
       000000000002e218  0000000000000018          39   1838     8
  [39] .strtab           STRTAB           0000000000000000  01532c20
       000000000005dfb6  0000000000000000           0     0     1
  [40] .shstrtab         STRTAB           0000000000000000  01590bd6
       00000000000001a3  0000000000000000           0     0     1

What seems to be missing is the source code for each compilation unit. I hypothesize if the source code was distributed, organized in the same tree structure when it was compiled and placed in the same paths, then GDB would have no issues finding the source files.

Run this command on the executable and notice the DWARF ATtributes, DW_AT_name: and DW_AT_comp_dir:. Note those /build/gcc/* directories. Must be where GDB is getting those pesky paths from.

readelf --debug-dump=info /usr/lib/libstdc++.so.6.0.28 | head -n 12

Contents of the .debug_info section:

  Compilation Unit @ offset 0x0:
   Length:        0x96e5 (32-bit)
   Version:       4
   Abbrev Offset: 0x0
   Pointer Size:  8
 <0><b>: Abbrev Number: 133 (DW_TAG_compile_unit)
    <d>   DW_AT_producer    : (indirect string, offset: 0x5af): GNU C++98 9.3.0 -march=x86-64 -mtune=generic -mshstk -g -O2 -std=gnu++98 -fno-implicit-templates -ffunction-sections -fdata-sections -frandom-seed=compatibility.lo -fno-plt -fcf-protection=full -fPIC
    <11>   DW_AT_language    : 4	(C++)
    <12>   DW_AT_name        : (indirect string, offset: 0x1d8d): /build/gcc/src/gcc/libstdc++-v3/src/c++98/compatibility.cc
    <16>   DW_AT_comp_dir    : (indirect string, offset: 0x40d0): /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/src

If you could throw me a bone and point me to where I can get the source, in the same organization as it was compiled. I would appretiate that :')

Last edited by sn0ot (2020-05-07 03:41:41)

eschwartz · 2020-05-07 04:26:12

Certain core packages are not stripped because it breaks a lot of stuff for them to be missing and Arch doesn't have -debug packages by default.

This includes gcc-libs and glibc.

Split debug packages are entirely different, as you've gathered, the debug information is extracted and placed into files in /usr/lib/debug/.

For properly formatted splitdebug pacman packages, there's also a special gcc compiler option, -fdebug-prefix-map, which is used to modify DW_AT_comp_dir, DW_AT_name so that they ultimately point to a file in /usr/src/debug instead of random places in /build -- and these splitdebug packages therefore provide *both* the separated debug info *and* the source code for each compilation unit.

My custom repo demos this with various packages I run from git, including glibc and pacman. They can be installed instead of the main packages and generally serve as drop-in replacements.
I also provide stable copies of glib2 and gtk3 due to high demand. Those will be installed by simply enabling the repo and running pacman -Syu.

In each case, to get debug info you'd need to install the replacement package and then *also* install 'gtk3-debug' or whichever set of debuginfo you wanted.

I do NOT currently provide gcc builds, however.

...

You can create your own debug packages for *any* Arch package, using the process generally described at https://wiki.archlinux.org/index.php/De … ing_Traces

sn0ot · 2020-05-07 15:16:59

Thank you! makes much more sense, Ill experiment with glibc and pacman that you host.

Glad I poked around your server LOL

null1480 · 2021-05-01 06:18:05

Thanks @sn0ot for providing really useful information, I wanted to accomplish the same and I found your post which really helped me. The code block shown below contains what I've included in my ".gdbinit" file.

It's a shame that the Python snippet you wrote doesn't actually make "gdb" ignore those files. That would avoid having to have a triangle of asterisks in my configuration file.

skip -gfi /usr/include/c++/*

# I don't want "/build/*" files to be shown when skipping because I
# only want to debug those functions that I've written.
#
# I tried the following but it didn't work
#
# skip -gfi /build/*
# 
# I think this is because the glob pattern don't recurse into the
# directories. That is, it only matches the files in that specific
# depth.
#
# As a workaround, I did the following: Whenever an undesired file is
# shown when executing "step", I added to the list shown below
#
# skip -gfi /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/*/*
# skip -gfi /build/gcc/src/gcc/libstdc++-v3/libsupc++/*
# 
# However, I got tired of doing that and after trying multiple
# solutions, the following is the only solution that have made gdb
# ignore all the directories source files under /build/.

skip -gfi /build/gcc/src/*
skip -gfi /build/gcc/src/*/*
skip -gfi /build/gcc/src/*/*/*
skip -gfi /build/gcc/src/*/*/*/*
skip -gfi /build/gcc/src/*/*/*/*/*
skip -gfi /build/gcc/src/*/*/*/*/*/*
skip -gfi /build/gcc/src/*/*/*/*/*/*/*
skip -gfi /build/gcc/src/*/*/*/*/*/*/*/*
skip -gfi /build/gcc/src/*/*/*/*/*/*/*/*/*
skip -gfi /build/gcc/src/*/*/*/*/*/*/*/*/*/*
skip -gfi /build/gcc/src/*/*/*/*/*/*/*/*/*/*/*
skip -gfi /build/gcc/src/*/*/*/*/*/*/*/*/*/*/*/*
skip -gfi /build/gcc/src/*/*/*/*/*/*/*/*/*/*/*/*/*
skip -gfi /build/gcc/src/*/*/*/*/*/*/*/*/*/*/*/*/*/*
skip -gfi /build/gcc/src/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*
skip -gfi /build/gcc/src/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*
skip -gfi /build/gcc/src/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*
skip -gfi /build/gcc/src/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*
skip -gfi /build/gcc/src/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*
skip -gfi /build/gcc/src/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*/*

Last edited by null1480 (2021-05-01 06:24:55)

Arch Linux

#1 2020-05-03 03:13:31