You are not logged in.
Here is my bootstrap script.
repo=$1
if [[ -z "$repo" ]]; then
echo "Usage: ./bootstrap <repo>"
exit
fi
build() {
pushd $1
shift
sudo makechrootpkg -r /var/lib/archbuild/${repo}-x86_64/ -- -i -r $@
popd
}
${repo}-x86_64-build -c
build linux-api-headers
build glibc --nocheck
build binutils --nocheck
build gcc --nocheck
build glibc
build binutils
build gcc
Offline
This is probably not the whole list of everything, isn't it? If you upgrade glibc/gcc, don't you also need to recompile the linux kernel? As you memtion earlier, maybe a list of kernel modules also need to be recompiled?.
Read the rest of that reply. Other packages Arch rebuilds was covered under point #3.
Offline
Allan pointed me to a set of patches that provided a fully working toolchain in December: https://pkgbuild.com/~alex19ep/patches/
I've uploaded the toolchain patches with those patches applied here along with a modified version of Allan's bootstrap script. He advised me not to apply the gcc PGO patch but I forgot to omit it while going through the motions.
The current status is that glibc fails on the second build pass due to a single failed test:
FAIL: nss/tst-nss-files-hosts-long
My system journal also gets spammed with error messages about core dumps but those seem to be expected. I haven't looked into it yet. My plan was to reproduce the working configuration from December as a starting point for further updates, but it may be better to just skip directly to the latest releases.
Building the toolchain is new to me but I'm hoping that it can be a collaborative effort and learning experience. Feel free to fork the repo or request access. I'll post updates here if I make any progress but I have some other pressing tasks right now so I'm not sure when that will be.
My Arch Linux Stuff • Forum Etiquette • Community Ethos - Arch is not for everyone
Offline
Hrm... that is not a test I recall seeing fail before. Do you have an IPv6 address?
Offline
Hrm... that is not a test I recall seeing fail before. Do you have an IPv6 address?
Yep. I also have Unbound running. Looks like I'll have to jump to 2.35: https://sourceware.org/git/?p=glibc.git … 34a8767581
If I have time I'll play with it some more today.
My Arch Linux Stuff • Forum Etiquette • Community Ethos - Arch is not for everyone
Offline
Hey Xyne, thanks for your work.
I tried your script on my local VM. Here's my modification to your script:
diff --git a/scripts/bootstrap.sh b/scripts/bootstrap.sh
index 2cdd119..d7b7200 100755
--- a/scripts/bootstrap.sh
+++ b/scripts/bootstrap.sh
@@ -74,7 +74,7 @@ done
REPO=${1:-$DEFAULT_REPO}
ARCH=$(uname -m)
BUILD_CMD=("$REPO-$ARCH-build")
-MAKECHROOTPKG_CMD=(sudo makechrootpkg -r "/var/lib/archbuild/$REPO-$ARCH/" -- -ir)
+MAKECHROOTPKG_CMD=(sudo --preserve-env=MAKEFLAGS makechrootpkg -r "/var/lib/archbuild/$REPO-$ARCH/" -- -ir)mkdir -p "$BUILD_DIR"
rsync -rtvv "$PKGBUILD_DIR/" "$BUILD_DIR"
@@ -128,20 +128,20 @@ function resolve_libiberty_conflicts() {
log "Re-initializing $REPO-$ARCH"
"${BUILD_CMD[@]}" -c || [[ $? -eq 255 ]]-build linux-api-headers
+build linux-api-headers --skipchecksums --skipinteg --skippgpcheck-build glibc --nocheck
-resolve_libiberty_conflicts
-build binutils --nocheck
-build gcc --nocheck
+build glibc --nocheck --skipchecksums --skipinteg --skippgpcheck
+resolve_libiberty_conflicts --skipchecksums --skipinteg --skippgpcheck
+build binutils --nocheck --skipchecksums --skipinteg --skippgpcheck
+build gcc --nocheck --skipchecksums --skipinteg --skippgpcheck# Move the unchecked packages to a separate directory as a checkpoint.
move_pkgs unchecked glibc binutils gcc-build glibc
-build binutils
-build gcc
+build glibc --skipchecksums --skipinteg --skippgpcheck
+build binutils --skipchecksums --skipinteg --skippgpcheck
+build gcc --skipchecksums --skipinteg --skippgpcheck-build linux
-build libtools
-build valgrind
+build linux --skipchecksums --skipinteg --skippgpcheck
+build libtools --skipchecksums --skipinteg --skippgpcheck
+build valgrind --skipchecksums --skipinteg --skippgpcheck
and I got one error on glibc 2.34:
FAIL: malloc/tst-mallocfork3-mcheck
Did you know about this?
However, I don't see the tst-nss-files-hosts-long error you mentioned.
In the mean time, I might as well try 2.35 to see if the problem persisted.
Thanks again.
Last edited by johnyan (2022-02-06 17:11:51)
Offline
Preserving the environment for sudo makes sense to me by why did you disable the integrity checks? There's a script to import the required PGP keys to the build user's toolchain if necessary and the checksums should remain valid in the VM.
My Arch Linux Stuff • Forum Etiquette • Community Ethos - Arch is not for everyone
Offline
Preserving the environment for sudo makes sense to me by why did you disable the integrity checks? There's a script to import the required PGP keys to the build user's toolchain if necessary and the checksums should remain valid in the VM.
The skipping checks is not going to land on production repo. This is just to easy the process to find errors and fixes.
also, thanks for the tip.
Offline
updating linux-api-headers to 5.16.7 and glibc to 2.35 as follows:
diff --git a/pkgbuilds/glibc/PKGBUILD b/pkgbuilds/glibc/PKGBUILD
index 9f7eb0a..5558c94 100644
--- a/pkgbuilds/glibc/PKGBUILD
+++ b/pkgbuilds/glibc/PKGBUILD
@@ -6,7 +6,7 @@
pkgbase=glibc
pkgname=(glibc lib32-glibc)
-pkgver=2.34
+pkgver=2.35
pkgrel=1
arch=(x86_64)
url='https://www.gnu.org/software/libc'
diff --git a/pkgbuilds/linux-api-headers/PKGBUILD b/pkgbuilds/linux-api-headers/PKGBUILD
index 23ad306..0c7bad8 100644
--- a/pkgbuilds/linux-api-headers/PKGBUILD
+++ b/pkgbuilds/linux-api-headers/PKGBUILD
@@ -4,7 +4,7 @@
# toolchain build order: linux-api-headers->glibc->binutils->gcc->binutils->glibc
pkgname=linux-api-headers
-pkgver=5.16.5
+pkgver=5.16.7
pkgrel=1
pkgdesc='Kernel headers sanitized for use in userspace'
arch=('any')
Got the following result:
XPASS: conform/UNIX98/ndbm.h/linknamespace
XPASS: conform/XOPEN2K/ndbm.h/linknamespace
XPASS: conform/XOPEN2K8/ndbm.h/linknamespace
XPASS: conform/XPG42/ndbm.h/linknamespace
UNSUPPORTED: elf/tst-audit10
UNSUPPORTED: elf/tst-avx512
UNSUPPORTED: elf/tst-cet-legacy-8
UNSUPPORTED: elf/tst-cet-property-2
UNSUPPORTED: elf/tst-env-setuid
UNSUPPORTED: elf/tst-env-setuid-tunables
XPASS: elf/tst-ifunc-isa-1
XPASS: elf/tst-ifunc-isa-1-static
XPASS: elf/tst-ifunc-isa-2
XPASS: elf/tst-ifunc-isa-2-static
XPASS: elf/tst-protected1a
XPASS: elf/tst-protected1b
UNSUPPORTED: elf/tst-valgrind-smoke
FAIL: malloc/tst-mallocfork3-mcheck
UNSUPPORTED: math/test-double-libmvec-acos-avx512f
UNSUPPORTED: math/test-double-libmvec-acosh-avx512f
UNSUPPORTED: math/test-double-libmvec-asin-avx512f
UNSUPPORTED: math/test-double-libmvec-asinh-avx512f
UNSUPPORTED: math/test-double-libmvec-atan-avx512f
UNSUPPORTED: math/test-double-libmvec-atan2-avx512f
UNSUPPORTED: math/test-double-libmvec-atanh-avx512f
UNSUPPORTED: math/test-double-libmvec-cbrt-avx512f
UNSUPPORTED: math/test-double-libmvec-cos-avx512f
UNSUPPORTED: math/test-double-libmvec-cosh-avx512f
UNSUPPORTED: math/test-double-libmvec-erf-avx512f
UNSUPPORTED: math/test-double-libmvec-erfc-avx512f
UNSUPPORTED: math/test-double-libmvec-exp-avx512f
UNSUPPORTED: math/test-double-libmvec-exp10-avx512f
UNSUPPORTED: math/test-double-libmvec-exp2-avx512f
UNSUPPORTED: math/test-double-libmvec-expm1-avx512f
UNSUPPORTED: math/test-double-libmvec-hypot-avx512f
UNSUPPORTED: math/test-double-libmvec-log-avx512f
UNSUPPORTED: math/test-double-libmvec-log10-avx512f
UNSUPPORTED: math/test-double-libmvec-log1p-avx512f
UNSUPPORTED: math/test-double-libmvec-log2-avx512f
UNSUPPORTED: math/test-double-libmvec-pow-avx512f
UNSUPPORTED: math/test-double-libmvec-sin-avx512f
UNSUPPORTED: math/test-double-libmvec-sincos-avx512f
UNSUPPORTED: math/test-double-libmvec-sinh-avx512f
UNSUPPORTED: math/test-double-libmvec-tan-avx512f
UNSUPPORTED: math/test-double-libmvec-tanh-avx512f
UNSUPPORTED: math/test-float-libmvec-acosf-avx512f
UNSUPPORTED: math/test-float-libmvec-acoshf-avx512f
UNSUPPORTED: math/test-float-libmvec-asinf-avx512f
UNSUPPORTED: math/test-float-libmvec-asinhf-avx512f
UNSUPPORTED: math/test-float-libmvec-atan2f-avx512f
UNSUPPORTED: math/test-float-libmvec-atanf-avx512f
UNSUPPORTED: math/test-float-libmvec-atanhf-avx512f
UNSUPPORTED: math/test-float-libmvec-cbrtf-avx512f
UNSUPPORTED: math/test-float-libmvec-cosf-avx512f
UNSUPPORTED: math/test-float-libmvec-coshf-avx512f
UNSUPPORTED: math/test-float-libmvec-erfcf-avx512f
UNSUPPORTED: math/test-float-libmvec-erff-avx512f
UNSUPPORTED: math/test-float-libmvec-exp10f-avx512f
UNSUPPORTED: math/test-float-libmvec-exp2f-avx512f
UNSUPPORTED: math/test-float-libmvec-expf-avx512f
UNSUPPORTED: math/test-float-libmvec-expm1f-avx512f
UNSUPPORTED: math/test-float-libmvec-hypotf-avx512f
UNSUPPORTED: math/test-float-libmvec-log10f-avx512f
UNSUPPORTED: math/test-float-libmvec-log1pf-avx512f
UNSUPPORTED: math/test-float-libmvec-log2f-avx512f
UNSUPPORTED: math/test-float-libmvec-logf-avx512f
UNSUPPORTED: math/test-float-libmvec-powf-avx512f
UNSUPPORTED: math/test-float-libmvec-sincosf-avx512f
UNSUPPORTED: math/test-float-libmvec-sinf-avx512f
UNSUPPORTED: math/test-float-libmvec-sinhf-avx512f
UNSUPPORTED: math/test-float-libmvec-tanf-avx512f
UNSUPPORTED: math/test-float-libmvec-tanhf-avx512f
UNSUPPORTED: misc/tst-adjtimex
UNSUPPORTED: misc/tst-clock_adjtime
UNSUPPORTED: misc/tst-ntp_adjtime
UNSUPPORTED: nptl/test-cond-printers
UNSUPPORTED: nptl/test-condattr-printers
UNSUPPORTED: nptl/test-mutex-printers
UNSUPPORTED: nptl/test-mutexattr-printers
UNSUPPORTED: nptl/test-rwlock-printers
UNSUPPORTED: nptl/test-rwlockattr-printers
UNSUPPORTED: nptl/tst-pthread-gdb-attach
UNSUPPORTED: nptl/tst-pthread-gdb-attach-static
UNSUPPORTED: posix/tst-cet-vfork-1
FAIL: posix/tst-waitid
UNSUPPORTED: stdlib/tst-secure-getenv
UNSUPPORTED: string/tst-memchr-rtm
UNSUPPORTED: string/tst-memcmp-rtm
UNSUPPORTED: string/tst-memmove-rtm
UNSUPPORTED: string/tst-memrchr-rtm
UNSUPPORTED: string/tst-memset-rtm
UNSUPPORTED: string/tst-strchr-rtm
UNSUPPORTED: string/tst-strcpy-rtm
UNSUPPORTED: string/tst-strlen-rtm
UNSUPPORTED: string/tst-strncmp-rtm
UNSUPPORTED: string/tst-strrchr-rtm
UNSUPPORTED: time/tst-clock_settime
UNSUPPORTED: time/tst-settimeofday
Summary of test results:
2 FAIL
5030 PASS
86 UNSUPPORTED
12 XFAIL
10 XPASS
Last edited by johnyan (2022-02-06 21:19:44)
Offline
Please edit your post to use code tags, rather than quote tags, for terminal output.
Sakura:-
Mobo: MSI MAG X570S TORPEDO MAX // Processor: AMD Ryzen 9 5950X @4.9GHz // GFX: AMD Radeon RX 5700 XT // RAM: 32GB (4x 8GB) Corsair DDR4 (@ 3000MHz) // Storage: 1x 3TB HDD, 6x 1TB SSD, 2x 120GB SSD, 1x 275GB M2 SSD
Making lemonade from lemons since 2015.
Offline
Please edit your post to use code tags, rather than quote tags, for terminal output.
Thank you for the tip. It would be even nicer if we can specify what kind of code is used. maybe something like [ code=diff]?
Offline
It turns out both of these issues are TIMEOUT problems. I added the following patch to fix both.
diff --git a/pkgbuilds/glibc/PKGBUILD b/pkgbuilds/glibc/PKGBUILD
index 9f7eb0a..ccd4aad 100644
--- a/pkgbuilds/glibc/PKGBUILD
+++ b/pkgbuilds/glibc/PKGBUILD
@@ -6,7 +6,7 @@
pkgbase=glibc
pkgname=(glibc lib32-glibc)
-pkgver=2.34
+pkgver=2.35
pkgrel=1
arch=(x86_64)
url='https://www.gnu.org/software/libc'
@@ -146,6 +146,7 @@ check() {
_skip_test tst-adjtime time/Makefile
_skip_test tst-clock2 time/Makefile
+ export TIMEOUTFACTOR=16
make -O check
}
Now glibc 2.35 2nd pass is clean.
Last edited by johnyan (2022-02-06 23:11:42)
Offline
No test failure on binutils 2.37:
=== libctf Summary ===
# of expected passes 3
# of unsupported tests 2
=== binutils Summary ===
# of expected passes 296
# of unsupported tests 1
=== gas Summary ===
# of expected passes 1559
# of unsupported tests 1
=== ld Summary ===
# of expected passes 2749
# of expected failures 59
# of untested testcases 1
# of unsupported tests 29
Offline
It turns out both of these issues are TIMEOUT problems. I added the following patch to fix both.
I am amazed people are still seeing these... The timeouts were already extended a good amount upstream.
Don't expect gcc to be test suite failure free. I have patches on my branch that will help, but I'd expect go/ada/d failures. No-one cares about those...
Offline
I am currently stuck at at a weird issue.
As I memtion earlier, binutils completes without issue. But gcc won't run anymore due to missing symbol in libctf.so.0 (in binutils package). The missing symbol is "sha1_finish_ctx".
I checked the libctf.so.0 using "readelf -Ws". The libctf binary produced from the 1st pass do not have this symbol at all. It was introduced somehow during the 2nd pass.
The sha1_finish_ctx is a function defined in libiberty/sha1.c.
I compare the configure options between the old PKGCONFIG and the new one from Xyne. I saw "--enable-install-libiberty" was introduced in the new PKGCONFIG.
I tried to remove this options on the 2nd pass but didn't seem to fix the issue.
Any thoughts/ideas/suggestions?
I googled around, almost no search result regarding to missing sha1_finish_ctx.
Offline
libiberty moved from gcc to binutils with these PKGBUILDs. This sounds like you installed the binutils libiberty, then overwrote it with the old one from gcc. (I know the error is in libctf, but this is the cause).
Edit - or just build binutils against the current gcc without libiberty.
Offline
There's a function in the bootstrap script that removes the conflicting libiberty files just before the first installation of binutils. It also removes those files from gcc's filelist in the pacman local db. I wrote it quickly and used the list of conflicting files as the basis for what to remove. There may be mistakes in it.
@johnyan
I plan to create a new branch with your patches and test it when I have some time today. Do you have a fork online already that I could track?
I've also started a local WIP branch that uses Allan's latest packages as a starting point for re-introducing multilib support. I haven't had time to finish and test it yet but I'll upload it as soon as I do.
My Arch Linux Stuff • Forum Etiquette • Community Ethos - Arch is not for everyone
Offline
No - me returning to packaging just helps patch over the problems of the distro. I'd prefer them to be exposed.
https://www.phoronix.com/scan.php?page= … ing-Behind
I guess they are now exposed lol.
What surprises me about this issue is that the current maintainer or even the current Archlinux leader has not yet spoken out. Lack of transparency is not good.
Excuse my poor English.
Offline
I do love them quoting me talking about me in the third person...
Offline
Hey Allan and Xyne, thanks for the suggestion. Forgive me not knowing how to fork Xyne's repo on gitlab. I pushed my current WIP repo on github instead: https://github.com/john-yan/arch-linux-toolchain
Instead of fixing the libiberty issue, I tried to remove Xyne's fix (move --enable-install-libiberty back to gcc) and it seems NOT working. (run manually, don't use the bootstrap script)
Last edited by johnyan (2022-02-07 13:27:13)
Offline
https://lists.archlinux.org/pipermail/a … 30729.html
Hi All,
Most of you are aware that our toolchain is currently outdated. We always had very few people
working on it through the years and I took over once Barth left, because it was needed.
However, I have been having very little time to work on Arch related stuff lately and the
toolchain is the most noticeable victim, given it is one of the most time consuming.
In this meantime, a few things also happened that compounded to the issue, among them, we enabled LTO.
Right now I'm working on bringing new glibc 2.35 and also waiting on binutils release so we
can bring the toolchain up to date. I'm aware we also have a GCC release coming out soon, and
the toolchain will need a new rebuild then.
For the future, we are trying to bring more people to work with the whole toolchain, so it is
not too much of a bus factor. We should have at least two toolchain maintainers, not just one.
I hope this serves to assuage the concerns over the current status of our toolchain, both present
and going future.
Regards,
Giancarlo Razzolini
Excuse my poor English.
Offline
That's awesome news. Thanks for sharing with us.
Offline
My current issue is that the libiberty.a compiled in binutils does not have the symbols we need. Here's the readelf output of the new vs old libiberty.so
[root@john lib]# readelf -a ./libiberty.a | grep sha1
File: ./libiberty.a(sha1.o)
[root@john lib]# readelf -a /usr/lib/libiberty.a | grep sha1
File: /usr/lib/libiberty.a(sha1.o)
00000000150e 000600000004 R_X86_64_PLT32 0000000000000070 sha1_process_block - 4
0000000015da 000600000004 R_X86_64_PLT32 0000000000000070 sha1_process_block - 4
00000000163c 000600000004 R_X86_64_PLT32 0000000000000070 sha1_process_block - 4
00000000167f 000600000004 R_X86_64_PLT32 0000000000000070 sha1_process_block - 4
00000000174a 000600000004 R_X86_64_PLT32 0000000000000070 sha1_process_block - 4
000000001875 000400000004 R_X86_64_PLT32 0000000000000000 sha1_init_ctx - 4
000000001883 000a00000004 R_X86_64_PLT32 0000000000001540 sha1_process_bytes - 4
00000000188e 000900000004 R_X86_64_PLT32 00000000000013f0 sha1_finish_ctx - 4
000000001902 000400000004 R_X86_64_PLT32 0000000000000000 sha1_init_ctx - 4
000000001951 000600000004 R_X86_64_PLT32 0000000000000070 sha1_process_block - 4
00000000197e 000900000004 R_X86_64_PLT32 00000000000013f0 sha1_finish_ctx - 4
0000000019ba 000a00000004 R_X86_64_PLT32 0000000000001540 sha1_process_bytes - 4
000000001520 000500000004 R_X86_64_PLT32 0000000000000040 sha1_read_ctx - 4
4: 0000000000000000 49 FUNC GLOBAL DEFAULT 1 sha1_init_ctx
5: 0000000000000040 46 FUNC GLOBAL DEFAULT 1 sha1_read_ctx
6: 0000000000000070 4980 FUNC GLOBAL DEFAULT 1 sha1_process_block
9: 00000000000013f0 330 FUNC GLOBAL DEFAULT 1 sha1_finish_ctx
10: 0000000000001540 760 FUNC GLOBAL DEFAULT 1 sha1_process_bytes
13: 0000000000001840 121 FUNC GLOBAL DEFAULT 1 sha1_buffer
14: 00000000000018c0 261 FUNC GLOBAL DEFAULT 1 sha1_stream
using default options from @Xyne
Offline
But gcc won't run anymore due to missing symbol in libctf.so.0 (in binutils package). The missing symbol is "sha1_finish_ctx".
I'm not up-to-speed (yet) but your issue could be related to an earlier problem [1] we hit whereby libctf was linking against the wrong libiberty (but I could easily be wrong here and your issue might have a different cause). I'm pretty sure the fix didn't make it into binutils-2.37 but it should be in the soon-to-be-released 2.38.
Offline
just quick note: I did install manjaro on son laptop some weeks ago and of course, doing updates. there is also glibc 2.33-5. But I dont know more details, son is just using it ;-)
I have seen the update pass by. I went to see exactly what was changed in the commits. Nothing! It got recompiled only because a new python version has been released. AFAIK, a few python scripts are executed as part of the build process but the python version used to build will have no effect at all on the resulting binaries... If I am correct, updating a core package for no good reason represents a waste of bandwidth considering the number of downloads this will trigger.
A few posts ago, Allan did propose a few explanations about why the package is not updated. My take is simply that somehow a newbie got the job and he simply does not understand everything that he should know to occupy this role. I am all for giving a young dev a chance to demonstrate what he is capable of but if that is the case, he should receive actively some mentoring so that he can learn and be able to do the job...
Offline