You are not logged in.

#1 2017-08-09 10:33:20

Starfish
Member
From: Germany
Registered: 2015-10-21
Posts: 133

[SOLVED] c++ ASan - shadow memory interleaves with existing memory

Hello there,

this is a question for those who are familiar with google's address sanitizer (asan) and memory management of processes in general.
Since a week or so, when I compile my code with asan enabled and run it, the program usually crashes and gives me the following memory layout

$ LD_PRELOAD=/usr/lib/libasan.so ./ttn <input-file>
==26907==Shadow memory range interleaves with an existing memory mapping. ASan cannot proceed correctly. ABORTING.
==26907==ASan shadow was supposed to be located in the [0x00007fff7000-0x10007fff7fff] range.
==26907==Process memory map follows:
	0x00a80c7d7000-0x00a80c944000	/home/markus/ttn/ttn
	0x00a80cb43000-0x00a80cb44000	/home/markus/ttn/ttn
	0x00a80cb44000-0x00a80cb4b000	/home/markus/ttn/ttn
	0x00a80cb4b000-0x00a80cb4c000	
	0x7f8fd425f000-0x7f8fd45b1000	
	0x7f8fd45b1000-0x7f8fd45ef000	/usr/lib/libquadmath.so.0.0.0
	0x7f8fd45ef000-0x7f8fd47ef000	/usr/lib/libquadmath.so.0.0.0
	0x7f8fd47ef000-0x7f8fd47f0000	/usr/lib/libquadmath.so.0.0.0
	0x7f8fd47f0000-0x7f8fd47f1000	/usr/lib/libquadmath.so.0.0.0
	0x7f8fd47f1000-0x7f8fd49cb000	/usr/lib/libgfortran.so.4.0.0
	0x7f8fd49cb000-0x7f8fd4bcb000	/usr/lib/libgfortran.so.4.0.0
	0x7f8fd4bcb000-0x7f8fd4bcc000	/usr/lib/libgfortran.so.4.0.0
	0x7f8fd4bcc000-0x7f8fd4bcd000	/usr/lib/libgfortran.so.4.0.0
	0x7f8fd4bcd000-0x7f8fd4bce000	
	0x7f8fd4bce000-0x7f8fd4be7000	/usr/lib/libpthread-2.25.so
	0x7f8fd4be7000-0x7f8fd4de6000	/usr/lib/libpthread-2.25.so
	0x7f8fd4de6000-0x7f8fd4de7000	/usr/lib/libpthread-2.25.so
	0x7f8fd4de7000-0x7f8fd4de8000	/usr/lib/libpthread-2.25.so
	0x7f8fd4de8000-0x7f8fd4dec000	
	0x7f8fd4dec000-0x7f8fd4df3000	/usr/lib/librt-2.25.so
	0x7f8fd4df3000-0x7f8fd4ff2000	/usr/lib/librt-2.25.so
	0x7f8fd4ff2000-0x7f8fd4ff3000	/usr/lib/librt-2.25.so
	0x7f8fd4ff3000-0x7f8fd4ff4000	/usr/lib/librt-2.25.so
	0x7f8fd4ff4000-0x7f8fd4ff7000	/usr/lib/libdl-2.25.so
	0x7f8fd4ff7000-0x7f8fd51f6000	/usr/lib/libdl-2.25.so
	0x7f8fd51f6000-0x7f8fd51f7000	/usr/lib/libdl-2.25.so
	0x7f8fd51f7000-0x7f8fd51f8000	/usr/lib/libdl-2.25.so
	0x7f8fd51f8000-0x7f8fd5395000	/usr/lib/libc-2.25.so
	0x7f8fd5395000-0x7f8fd5594000	/usr/lib/libc-2.25.so
	0x7f8fd5594000-0x7f8fd5598000	/usr/lib/libc-2.25.so
	0x7f8fd5598000-0x7f8fd559a000	/usr/lib/libc-2.25.so
	0x7f8fd559a000-0x7f8fd559e000	
	0x7f8fd559e000-0x7f8fd55b4000	/usr/lib/libgcc_s.so.1
	0x7f8fd55b4000-0x7f8fd57b3000	/usr/lib/libgcc_s.so.1
	0x7f8fd57b3000-0x7f8fd57b4000	/usr/lib/libgcc_s.so.1
	0x7f8fd57b4000-0x7f8fd57b5000	/usr/lib/libgcc_s.so.1
	0x7f8fd57b5000-0x7f8fd57ea000	/usr/lib/libjsoncpp.so.1.8.1
	0x7f8fd57ea000-0x7f8fd59e9000	/usr/lib/libjsoncpp.so.1.8.1
	0x7f8fd59e9000-0x7f8fd59ea000	/usr/lib/libjsoncpp.so.1.8.1
	0x7f8fd59ea000-0x7f8fd59eb000	/usr/lib/libjsoncpp.so.1.8.1
	0x7f8fd59eb000-0x7f8fd5afc000	/usr/lib/libm-2.25.so
	0x7f8fd5afc000-0x7f8fd5cfb000	/usr/lib/libm-2.25.so
	0x7f8fd5cfb000-0x7f8fd5cfc000	/usr/lib/libm-2.25.so
	0x7f8fd5cfc000-0x7f8fd5cfd000	/usr/lib/libm-2.25.so
	0x7f8fd5cfd000-0x7f8fd6384000	/usr/lib/liblapack.so.3.7.1
	0x7f8fd6384000-0x7f8fd6583000	/usr/lib/liblapack.so.3.7.1
	0x7f8fd6583000-0x7f8fd6587000	/usr/lib/liblapack.so.3.7.1
	0x7f8fd6587000-0x7f8fd6588000	/usr/lib/liblapack.so.3.7.1
	0x7f8fd6588000-0x7f8fd65d2000	
	0x7f8fd65d2000-0x7f8fd6626000	/usr/lib/libblas.so.3.7.1
	0x7f8fd6626000-0x7f8fd6825000	/usr/lib/libblas.so.3.7.1
	0x7f8fd6825000-0x7f8fd6826000	/usr/lib/libblas.so.3.7.1
	0x7f8fd6826000-0x7f8fd6827000	/usr/lib/libblas.so.3.7.1
	0x7f8fd6827000-0x7f8fd69a1000	/usr/lib/libstdc++.so.6.0.24
	0x7f8fd69a1000-0x7f8fd6ba0000	/usr/lib/libstdc++.so.6.0.24
	0x7f8fd6ba0000-0x7f8fd6baa000	/usr/lib/libstdc++.so.6.0.24
	0x7f8fd6baa000-0x7f8fd6bac000	/usr/lib/libstdc++.so.6.0.24
	0x7f8fd6bac000-0x7f8fd6baf000	
	0x7f8fd6baf000-0x7f8fd6cfe000	/usr/lib/libasan.so.4.0.0
	0x7f8fd6cfe000-0x7f8fd6efd000	/usr/lib/libasan.so.4.0.0
	0x7f8fd6efd000-0x7f8fd6f00000	/usr/lib/libasan.so.4.0.0
	0x7f8fd6f00000-0x7f8fd6f03000	/usr/lib/libasan.so.4.0.0
	0x7f8fd6f03000-0x7f8fd7b68000	
	0x7f8fd7b68000-0x7f8fd7b8b000	/usr/lib/ld-2.25.so
	0x7f8fd7d4d000-0x7f8fd7d55000	
	0x7f8fd7d69000-0x7f8fd7d8b000	
	0x7f8fd7d8b000-0x7f8fd7d8c000	/usr/lib/ld-2.25.so
	0x7f8fd7d8c000-0x7f8fd7d8d000	/usr/lib/ld-2.25.so
	0x7f8fd7d8d000-0x7f8fd7d8e000	
	0x7fff280da000-0x7fff280fb000	[stack]
	0x7fff281ec000-0x7fff281ef000	[vvar]
	0x7fff281ef000-0x7fff281f1000	[vdso]
	0xffffffffff600000-0xffffffffff601000	[vsyscall]
==26907==End of process memory map.

The error message in the first line is self-explaining. You can see that asan's shadow memory interleaves with the binary image "ttn" and that I also tried preloading asan, which is mentioned here: https://github.com/google/sanitizers/wi … itizer#faq

First of all, I find two things puzzling:
(1) When I start the program multiple times, the binary is located at different positions. I did not compile it with fPIC, which means absolute addresses can be present, which means, in my understanding, it should be loaded into the exact same position every time I start it. Every fifth time or so, it is apparently within a memory range that does not interleave with the shadow memory and the program runs flawlessly.
(2) Why the hell is the image loaded so far into the virtual memory in the first place? The range 0x000000000000 - 0x00a80c7d7000 is wasted for what, security reasons?

Finally I would be happy if someone told me how to make asan work again without errors. Here's some further input:

$ pacman -Qlq gcc-libs | grep asan
/usr/lib/libasan.so
/usr/lib/libasan.so.4
/usr/lib/libasan.so.4.0.0
$ pacman -Qi gcc-libs | grep Install\ Date
Install Date    : Fri 14 Jul 2017 11:01:57 AM CEST

The crashes did not occur immediately after July 14th. Besides, I have no clue what changes in my code could have provoked this behaviour.

Any help is appreciated.

Last edited by Starfish (2017-08-21 08:25:43)


"Yesterday is history, tomorrow is a mystery, but today is a gift. That is why it is called the present." - Master Oogway

Offline

#2 2017-08-10 12:16:28

mattyclarkson
Member
Registered: 2014-08-29
Posts: 6

Re: [SOLVED] c++ ASan - shadow memory interleaves with existing memory

I've noticed exactly the same error since updating my system. Trying to figure out what has caused this.

Offline

#4 2017-08-10 12:32:13

mattyclarkson
Member
Registered: 2014-08-29
Posts: 6

Re: [SOLVED] c++ ASan - shadow memory interleaves with existing memory

Thanks for the information! To solve, either wait for this to be fixed in mainline or install the long term support kernel.

Offline

#5 2017-08-10 13:47:39

Starfish
Member
From: Germany
Registered: 2015-10-21
Posts: 133

Re: [SOLVED] c++ ASan - shadow memory interleaves with existing memory

@Uriel_Bernhard48: Thanks for the links!
Never heard of PIE before, which responds to my question why the binaries work in different position.
And they are indeed loaded into different positions each time for security reasons: https://en.wikipedia.org/wiki/Address_s … domization

Well, let's see who is going to adapt to whom, asan to linux or the other way around. I'm not sure if this qualifies as a violation of Torvalds' number one rule "You do not break userspace!", since asan makes a rather bold assumption about the internal memory layout of a process and not some standardized system call or sysfs file.


"Yesterday is history, tomorrow is a mystery, but today is a gift. That is why it is called the present." - Master Oogway

Offline

#6 2017-08-20 09:13:50

novist
Member
Registered: 2014-03-14
Posts: 47

Re: [SOLVED] c++ ASan - shadow memory interleaves with existing memory

mattyclarkson wrote:

Thanks for the information! To solve, either wait for this to be fixed in mainline or install the long term support kernel.

I did install LTS kernel but same error still occurs. Yes, i booted right kernel:

~ % cat /proc/version
Linux version 4.9.42-1-lts

Offline

#7 2017-08-20 17:11:19

Eschwartz
Bug Wrangler
Registered: 2014-08-08
Posts: 1,162

Re: [SOLVED] c++ ASan - shadow memory interleaves with existing memory

FS#55008

This is fixed in linux 4.12.8-2 which adds this patch: https://patchwork.kernel.org/patch/9886105/

Offline

#8 2017-08-21 08:25:29

Starfish
Member
From: Germany
Registered: 2015-10-21
Posts: 133

Re: [SOLVED] c++ ASan - shadow memory interleaves with existing memory

Affirmative. Just did an upgrade and the kernel and asan love each other again.


"Yesterday is history, tomorrow is a mystery, but today is a gift. That is why it is called the present." - Master Oogway

Offline

Board footer

Powered by FluxBB