Stack Smashing/signal 11/SEGV when trying to launch Steam game.

ACE7F22 · 2024-08-06 00:11:47

Hi, I've been searching all over the place for ways to fix this. Stack smashing seeming to be a program trying to access more RAM than it's given, and "smashing" nearby data stacks. But I have 32GB of RAM, and I'm only trying to launch a game. This sounds to me like Linux isn't taking advantage of the available RAM, and instead is smashing things together for no reason.

Aug 05 19:35:50 archlinux steam[7816]: Adding process 8817 for gameID 386180
Aug 05 19:35:56 archlinux steam[7816]: *** stack smashing detected ***: terminated
Aug 05 19:35:56 archlinux systemd-coredump[8843]: Process 8629 (Main Thread) of user 1000 terminated abnormally with signal 11/SEGV, processing...
Aug 05 19:35:56 archlinux systemd[1]: Started Process Core Dump (PID 8843/UID 0).

If anyone can explain this, it would be greatly appreciated.

A lot of search results mention Stack Smashing as an attack, but I'm sure that's not the case here, and I'm aware that there's a method for deactivating it's detection, but that's not recommended. I've read it's mostly a developer thing, but since this game works on windows without issue, I feel it's more an OS problem. Can't find answers as to how to just give the game more RAM.

BTW, I've had several games fail to launch, so I don't think it's just this game. Does my system have memory management issues?

MB = Asus Strix z790H (new)
CPU= i9-12900k (duel GPU enabled in bios) (new)
GPU= RTX 4060 (MSI) (new)
Driver= Nvidia
RAM= DDR5-6000 32GiB (Kingston Fury Renegade) (XMP2) (new)
OS=Arch Linux (Sata Sam. 870 1TB Evo) (installed with ArchInstall)
Alt OS= Windows 11 (T.G. NVME SSD 1TB)

Note - OSs booted separately from MB UEFI.
Each OS uses different file systems, and Linux doesn't mount NTFS, Windows I don't think can even read BTRFS? As far as I'm aware, the 2 are completely separate from each other on my system, only sharing USB devices, like my exFAT Steam library, but these problems happen even with that disconnected.

Game: Crossout (journal mentioned above)
Tried multiple proton versions. The ProtonDB used to say Gold I think, it recently broke, and changed to not working.

Last edited by ACE7F22 (2024-08-06 22:13:57)

V1del · 2024-08-06 01:20:14

More actual information maybe? Which game, which hardware, which drivers? Is the game supposed to work (check protondb) ?

If I check, according to the gameid, things look dire: https://www.protondb.com/app/386180 -- might as well be a game bug/interaction issue with wine. Pick/test a game that's supposed to work and rated platin on protondb.

Last edited by V1del (2024-08-06 01:20:46)

ACE7F22 · 2024-08-06 01:39:18

My apologies, I was focused on the memory issue.

I updated the post with the info, will add more if needed.

I have had Warframe and Distance run, but Geometry Wars 3 and some other games fail to start at all.

Warframe had lag issues, but I haven't even tried to optimize it yet.

Distance is Plat.

I can make a full list, but I'm really suspicious of how the OS is allocating memory, or seeing SEGV. I'm wondering if the game is even the problem.

cryptearth · 2024-08-06 03:18:39

I doubt that this is a software issue but to me it rather looks like bad hardware - please run memtest
also you seem to misunderstand what a stack smashing attack is - please read up on that
also also: you mention the game runs fine on windows - do you dual-boot arch with windows on the same system?
with that powerful hardware and your main intention to play games - why you use linux?

ACE7F22 · 2024-08-06 06:19:50

Sooooooo, I broke something, and now I can't log in, or rather, the desktop doesn't seem to load. Lol

Excuse me while I try to remedy this. Might just reinstall and start over.

To answer your question in the meantime....

I recently built this PC just 2 months ago, and I did a memory test with the built-in one on the motherboard. Passed flawlessly.
(edit - just did a test with the arch Linux installer, and it passed, no errors, took about 10 minutes.)

I understand that a stack smashing event can be an attack, but can also be the result of bad coding, or just a program going rouge. I've been reading up on it everywhere I can for over a week, maybe the past few weeks. "stack smashing is when a program exceeds, or spills over out of its own allocated memory space, potentially corrupting memory of other programs as it writes over their space, thus the term smashing, however, there's a built-in security that uses a buffer called a conary to detect stack smashing. When the conary is corrupted, indicating that the process is spilling over, systemd kills the offending processes. A stack smashing attack (fun note, listening to Attack by SOAD as I write this lol) is an intentional attempt to corrupt the data of running programs to disrupt a system, change data, or insert scripts forcefully, etcetera." As this is a video game in question, one that I've played for years, I'm not at all concerned it's an intentional attack, which leaves me to believe that the system is mishandling, maybe being stingy with allocating memory, not giving the game sufficient RAM space.

As for duel booting, I've been trying to move away from windows, and hope to never have to boot Windows at all. My intention is to set up a VM for Windows 11 within Linux, running Arch Linux as my primary OS, and W11 for compatibility purposes. Currently the W11 OS is on it's own NVMe SSD, and Arch Linux is on its own SATA SSD, but I'm planning to reinstall on another NVME. I reboot the PC, and manually choose a drive to boot from to run with Windows or Linux. Does that count as duel booting?

Thanks for the compliment, though I know my system isn't as "powerful" as it could be. Waiting to upgrade the GPU, the 4060 was just to get up and running. To answer your question though, I switched to Linux because I wanted control over my computer, and to be able to fully benefit from the hardware, instead of a chunk if it's power being used to spy on me, and do who knows what. Been wanting to do this for a long time, but Microsoft really made it clear it was time to switch. Also, I intend to get into developing games, working on music, assets, and maybe my own programs.

Last edited by ACE7F22 (2024-08-06 06:42:05)

cryptearth · 2024-08-06 06:37:27

if you duAl boot with windows make sure you have fast boot disabled in power settings

ACE7F22 · 2024-08-06 06:49:55

I prefer clean starts, even if they're slower, so yea, fast boot is off.

UEFI - Fast Boot = Disabled

Last edited by ACE7F22 (2024-08-06 06:50:22)

seth · 2024-08-06 09:01:21

3rd link below. Mandatory.
Disable it (it's NOT the BIOS setting!) and reboot windows and linux twice for voodo reasons.

it passed, no errors, took about 10 minutes

Meaningful RAM tests are measured in days, not minutes.

But wine "detects" smash stacking more often…
If you run into this again, post the full wine/proton/steam log for the invocation as well as the https://wiki.archlinux.org/title/Core_d … _core_dump

cryptearth · 2024-08-06 09:36:42

ACE7F22 wrote:

I prefer clean starts, even if they're slower, so yea, fast boot is off.
UEFI - Fast Boot = Disabled

as seth linked the fast start I talk about is not the setiing in the uefi but in windows energy settings

ACE7F22 · 2024-08-06 17:21:14

seth wrote:

3rd link below. Mandatory.
Disable it (it's NOT the BIOS setting!) and reboot windows and linux twice for voodo reasons.
it passed, no errors, took about 10 minutes
Meaningful RAM tests are measured in days, not minutes.
But wine "detects" smash stacking more often…
If you run into this again, post the full wine/proton/steam log for the invocation as well as the https://wiki.archlinux.org/title/Core_d … _core_dump

My understanding is the memory test runs in bios, and renders my PC completely unusable while it runs. Can you give me a minimum test time to be sufficient? At least 4 hours? 12? 24? Do I absolutely have to run it none stop for several days?

Also, does a number of passes matter? Like, I assume the test just runs laps, repeating itself, and the test says the RAM speed is aprox. 24000Mib/s, so if my system is faster, would a sufficient result take less time? In other words, could I get a reliable test in less real world time? In comparison to an average computer that is.

Also, I will check the Windows OS settings next time I get the chance, though I'm pretty sure I disabled that to. Although, Microsoft loves to force settings on people, so I wouldn't be surprised if it changed back. Thank God Linux exists, can't wait to ditch windows completely.
(so I checked, and fast startup is unchecked/off)
(as for hibernate, I use it manually when I want the computer to "sleep" for a faster start later, but otherwise, I use a regular shutdown, which my understanding with the settings in windows and the UEFI, it is configured to fully shut down and clear the running state.)
(I updated my original post with more details.)

Last edited by ACE7F22 (2024-08-06 18:15:20)

cryptearth · 2024-08-06 18:26:37

usually memtest does sequential tests first and then a number of random patterns
depending on the actual fault (if there is any) it can happen that it only shows during random patterns if the pattern hits the problematic pattern

as an example: on my old platform 1 of the 4 dimm is faulty - and just for fun I recently ran a memtest and for my particular error it showed up both during the sequential tests and the random ones - but the latter showed a different error pattern so it seems that it does depend on the pattern that hits the defective range
I can't tell from the top of my had but one run takes more than an hour for 32gb ddr3 1600 on a amd fx-8350 8-core @4ghz

and yes, memtest runs bare metal so it does render the system unuseable during the test - which is the point as only this way all memory can be tested
running on an os stuff like address space layout randomization and virtual memory mapping would interfere with the test as the physical location would differ from the logical addressed one

ACE7F22 · 2024-08-06 20:10:26

cryptearth wrote:

usually memtest does sequential tests first and then a number of random patterns
depending on the actual fault (if there is any) it can happen that it only shows during random patterns if the pattern hits the problematic pattern
as an example: on my old platform 1 of the 4 dimm is faulty - and just for fun I recently ran a memtest and for my particular error it showed up both during the sequential tests and the random ones - but the latter showed a different error pattern so it seems that it does depend on the pattern that hits the defective range
I can't tell from the top of my had but one run takes more than an hour for 32gb ddr3 1600 on a amd fx-8350 8-core @4ghz
and yes, memtest runs bare metal so it does render the system unuseable during the test - which is the point as only this way all memory can be tested
running on an os stuff like address space layout randomization and virtual memory mapping would interfere with the test as the physical location would differ from the logical addressed one

Understood.

I get that it runs "bare metal", my point is that I don't want to go days without being able to use my only computer if it's not necessary.

My understanding (I could be wrong), the memtest does all this random sequence and stuff, so one full test is sufficient, isn't it? A longer repeated test is just testing stability, not if it's working correctly, right?

If I need to test RAM for troubleshooting this "stack smashing" issue, I need to know the extent of which to test, if one full pass isn't sufficient. Otherwise, I see "TEST PASSED", and I'm satisfied.

Thank you everyone for the responses and all, it's very much appreciated.

seth · 2024-08-06 20:15:42

I get that it runs "bare metal", my point is that I don't want to go days without being able to use my only computer if it's not necessary.

Not the BIOS POST, run https://wiki.archlinux.org/title/Stress … MemTest86+ overnight.

I use a regular shutdown, which my understanding with the settings in windows and the UEFI, it is configured to fully shut down and clear the running state.

The default behavior (and yes, it frequently gets reset w/ windows updates) is to actually just hibernate the system and call that "fast start"

And first and foremost, get use the logs and data around that "smashing" because, as hinted, wine can hallucinate that.

ACE7F22 · 2024-08-06 20:34:16

seth wrote:

I get that it runs "bare metal", my point is that I don't want to go days without being able to use my only computer if it's not necessary.
Not the BIOS POST, run https://wiki.archlinux.org/title/Stress … MemTest86+ overnight.
I use a regular shutdown, which my understanding with the settings in windows and the UEFI, it is configured to fully shut down and clear the running state.
The default behavior (and yes, it frequently gets reset w/ windows updates) is to actually just hibernate the system and call that "fast start"
And first and foremost, get use the logs and data around that "smashing" because, as hinted, wine can hallucinate that.

Ok, I'll run that test tonight, and give an update tomorrow. I assume I can turn off the monitor while it runs.

When I referred to the "settings in Windows", I meant specifically how I have it configured, however, of course it's possible Microsoft might have changed it. "Fast Start" is disabled in both UEFI and Windows power settings. Hibernate is only enabled for the sleep command, as far as I'm able to tell.

I might give updates slow on the "Stack Smashing" issue, as Linux has been refusing to start the Plasma Desktop after putting in password. I might be rebuilding the OS on a new drive, but I'm aware I can do the memtest in the meantime.

Thanks again.

seth · 2024-08-06 20:55:39

I might give updates slow on the "Stack Smashing" issue, as Linux has been refusing to start the Plasma Desktop after putting in password.

Try to login, switch to a different TTY (ctrl+alt+F3) and post your complete system journal for the boot:

sudo journalctl -b | curl -F 'file=@-' 0x0.st

ACE7F22 · 2024-08-06 21:00:48

seth wrote:

I might give updates slow on the "Stack Smashing" issue, as Linux has been refusing to start the Plasma Desktop after putting in password.
Try to login, switch to a different TTY (ctrl+alt+F3) and post your complete system journal for the boot:
sudo journalctl -b | curl -F 'file=@-' 0x0.st

I can't open Firefox to paste it on the computer though?

I can only think that if I fetch the journal, then save the log as a file I can access on Windows, then I can copy and paste it that way?

I was planning on rebuilding the OS regardless, just to build experience, and flush out any unnecessary things I did, or mistakes I've made. I also intended to give an update to my posted issues based on a clean install. Still, I'll provide the journal if I'm able.

Oh, by the way, I tried changing the disk mounting point for the USB Drive to Home, and that's the only significant thing I can think of, that preceded this desktop loading issues. The external drive only had the steam library, and some windows backups, nothing vital to Linux, but I haven't been able to mount it lately. I'm working on figuring it out. Just thought I'd mention the potential cause of not being able to get to desktop environment. I wasn't looking for help yet on that, but I do appreciate the advice.

Last edited by ACE7F22 (2024-08-06 21:07:03)

seth · 2024-08-06 21:11:27

The command will upload the journal to 0x0.st from any interactive shell - no firefox required.

Oh, by the way, I tried changing the disk mounting point for the USB Drive to Home,

If your $HOME is now most likely stale, the fat DEs cannot handle that so you'll have to fix (only that) and re-installation won't do anything about that.
Add

(lsblk -f; cat /etc/fstab) | curl -F 'file=@-' 0x0.st

ACE7F22 · 2024-08-06 21:32:52

seth wrote:

The command will upload the journal to 0x0.st from any interactive shell - no firefox required.
Oh, by the way, I tried changing the disk mounting point for the USB Drive to Home,
If your $HOME is now most likely stale, the fat DEs cannot handle that so you'll have to fix (only that) and re-installation won't do anything about that.
Add
(lsblk -f; cat /etc/fstab) | curl -F 'file=@-' 0x0.st

By "reinstall" I mean a complete rebuild. This is my first go at Arch Linux, and I want to build more familiarity with configuring the operating system. Ultimately, I'd like to be competent enough to install and configure without needed to visit the Wiki, and do it efficiently without adding unnecessary packages, or modifying parameters I don't need to touch. I'm mot calling it a "fix", more a "start from the beginning.

I assumed that setting the mount location for a USB Drive would just show it as a folding in that directory?

seth · 2024-08-06 21:37:21

When you mount a device into a specific path the filesystem of the device "replaces" (shadows) everything in that path.
If you mount some random usb key into your $HOME, you don't have a usable $HOME anymore, if you mount it into /home you don't have a $HOME directory at all.

That's all I can tell you w/o details on the condition (the journal and output of the commands in #17)

ACE7F22 · 2024-08-06 21:49:26

seth wrote:

When you mount a device into a specific path the filesystem of the device "replaces" (shadows) everything in that path.
If you mount some random usb key into your $HOME, you don't have a usable $HOME anymore, if you mount it into /home you don't have a $HOME directory at all.
That's all I can tell you w/o details on the condition (the journal and output of the commands in #17)

Ah, I may have messed up then. lol

Thanks for educating me. I'll do more research on this. My assumption was I'd just have /home/usb.

By the way, I logged into tty3, and ran "systemctl status sddm".

sddm-helper[1007]: pam_systemd(sddm:session): Failed to create session: Job 1293 for unit 'session-2.scope' failed with 'dependency'

Had to type this manually. Hope it makes sense. However, I don't want to get side tracked. I very much appreciate the help, but I don't think I'm supposed to drift from the topic issue.

seth · 2024-08-07 07:30:55

Had to type this manually.

No you don't, you can redirect the files and outputs into 0x0.st and get a short url to share in return. That'S what the commands I posted do. Please take advantage of that.

ACE7F22 · 2024-08-07 17:45:59

seth wrote:

Had to type this manually.
No you don't, you can redirect the files and outputs into 0x0.st and get a short url to share in return. That'S what the commands I posted do. Please take advantage of that.

Ok, let me rephrase to "I chose to type this".

I'll do more research into how this works, but I'm currently in the middle of just reinstalling. Trying to rule out any miss-configuration.
(by the way, I installed on a new drive, so I should still be able to boot up the old one if you're curious about the boot journal for the desktop issue.)

I did the momtest86+ last night, did 9 passes, no errors. I also did research on memory testing, and most sources say one pass is enough, and very few errors would need extended tests. Extended tests are more for over clocking stability test, which I'm not concerned about, I haven't overclocked yet. I'm suspicious about the memory manager and allocation. Really feels like it's just not giving the program the memory it needs.

Also, I recognize it's been said a few times that Wine is trigger happy about calling stack smashing, so maybe that's where I need to focus my attention. Steam uses Proton, which is a modified version of Wine, isn't it?

Thanks again for all the info and help by the way. I hope this thread can become a great resource for others trying to troubleshoot.(thumbs up emoji)

seth wrote:

I might give updates slow on the "Stack Smashing" issue, as Linux has been refusing to start the Plasma Desktop after putting in password.
Try to login, switch to a different TTY (ctrl+alt+F3) and post your complete system journal for the boot:
sudo journalctl -b | curl -F 'file=@-' 0x0.st

I tried this, and got...

curl: (26) Failed to open/read local data from file/application

My fresh install seems to be working though, so I'm not concerned about this. Just wanted to follow up and say I tried your suggestion.

I'm going to try rebuilding for specifically running a few games, and try to minimize variables. Considering trying GE Proton. I'll give an update once I've tested.

Last edited by ACE7F22 (2024-08-08 05:00:47)

Arch Linux

#1 2024-08-06 00:11:47

Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#2 2024-08-06 01:20:14

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#3 2024-08-06 01:39:18

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#4 2024-08-06 03:18:39

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#5 2024-08-06 06:19:50

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#6 2024-08-06 06:37:27

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#7 2024-08-06 06:49:55

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#8 2024-08-06 09:01:21

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#9 2024-08-06 09:36:42

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#10 2024-08-06 17:21:14

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#11 2024-08-06 18:26:37

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#12 2024-08-06 20:10:26

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#13 2024-08-06 20:15:42

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#14 2024-08-06 20:34:16

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#15 2024-08-06 20:55:39

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#16 2024-08-06 21:00:48

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#17 2024-08-06 21:11:27

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#18 2024-08-06 21:32:52

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#19 2024-08-06 21:37:21

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#20 2024-08-06 21:49:26

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#21 2024-08-07 07:30:55

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

#22 2024-08-07 17:45:59

Re: Stack Smashing/signal 11/SEGV when trying to launch Steam game.

Board footer