[SOLVED] SAMBA segfaults since updating OS yesterday

audiomuze · 2016-01-02 09:06:26

Ran an OS update last night

pacman -Syu --noconfirm

on my Arch server (which I noticed included a kernel update) and shut it down after having had it up for the better part of the week. This morning my music server hosted on a Windows machine couldn't connect to the Arch server. Investigating on the Arch server shows:

From dmesg:

[   26.179116] nmbd[622]: segfault at 7f098c9090f8 ip 00007f096c6deff7 sp 00007ffc95bd9718 error 4 in libhcrypto-samba4.so.5.0.1[7f096c6d7000+32000]
[   32.845932] smbd[655]: segfault at 7fabf418d0f8 ip 00007fabd3f62ff7 sp 00007ffde27e5428 error 4 in libhcrypto-samba4.so.5.0.1[7fabd3f5b000+32000]

from testparm:

testparm
Segmentation fault (core dumped)

Nothing at all has been changed on SAMBA config, so I'm guessing this is a bug in the SAMBA update. I'm guessing a kernel rollback is required to get things working again?

Last edited by audiomuze (2016-01-04 12:19:17)

audiomuze · 2016-01-03 18:45:34

There something wrong with the way I asked the question or am I the only person that's picked up SAMBA issues since the last kernel update?

ewaller · 2016-01-03 21:48:09

Nothing wrong with how you phrased the question.

Generic solutions:
Does the running kernel ( uname -a) version match that of the installed modules ( pacman -Qi linux ) ?
Are your microcode updates installed and configured.

audiomuze · 2016-01-04 05:35:11

Thanks. It seems the running kernel and installed modules do match.

Linux alibbtrfs 4.3.3-2-ARCH #1 SMP PREEMPT Wed Dec 23 20:09:18 CET 2015 x86_64 GNU/Linux
Name : linux
Version : 4.3.3-2
Description : The Linux kernel and modules
Architecture : x86_64
URL : http://www.kernel.org/
Licenses : GPL2
Groups : base
Provides : None
Depends On : coreutils linux-firmware kmod mkinitcpio>=0.7
Optional Deps : crda: to set the correct wireless channels of your country
Required By : None
Optional For : None
Conflicts With : None
Replaces : None
Installed Size : 74.11 MiB
Packager : Tobias Powalowski <tpowa@archlinux.org>
Build Date : Wed 23 Dec 2015 21:10:36 SAST
Install Date : Fri 01 Jan 2016 16:00:00 SAST
Install Reason : Explicitly installed
Install Script : Yes
Validated By : Signature

There don't seem to have been any microcode updates:

[    0.312673] microcode: CPU0 sig=0x306c3, pf=0x2, revision=0x12
[    0.312676] microcode: CPU1 sig=0x306c3, pf=0x2, revision=0x12
[    0.312680] microcode: CPU2 sig=0x306c3, pf=0x2, revision=0x12
[    0.312685] microcode: CPU3 sig=0x306c3, pf=0x2, revision=0x12
[    0.312714] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba

tom.ty89 · 2016-01-04 05:43:29

What's your CPU? I got update to revision 0x1c for the CPUs with same sig and pf here

[tom@localhost ~]$ dmesg | grep microcode
[    0.000000] microcode: CPU0 microcode updated early to revision 0x1c, date = 2014-07-03
[    0.098721] microcode: CPU1 microcode updated early to revision 0x1c, date = 2014-07-03
[    0.123608] microcode: CPU2 microcode updated early to revision 0x1c, date = 2014-07-03
[    0.148510] microcode: CPU3 microcode updated early to revision 0x1c, date = 2014-07-03
[    0.323653] microcode: CPU0 sig=0x306c3, pf=0x2, revision=0x1c
[    0.323658] microcode: CPU1 sig=0x306c3, pf=0x2, revision=0x1c
[    0.323662] microcode: CPU2 sig=0x306c3, pf=0x2, revision=0x1c
[    0.323667] microcode: CPU3 sig=0x306c3, pf=0x2, revision=0x1c
[    0.323696] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba

It's Intel(R) Core(TM) i5-4570S CPU @ 2.90GHz (Haswell). Any chance that it's just the ucode is not loaded somehow?

audiomuze · 2016-01-04 08:25:20

Will have to check my EFI config. Just ran dmidecode on two identical machines to check if BIOS' were the same (both machines run Arch, only one has no GUI installed). They're not, and the machine with the later BIOS isn't having SAMBA issues. Bizarre, but I'll flash the BIOS and then check if that resolves the issue. If not I'll check the EFI config to ensure the microcode is loaded. Thx.

Desktop machine (works):

# dmidecode 3.0
Getting SMBIOS data from sysfs.
SMBIOS 2.7 present.
70 structures occupying 2726 bytes.
Table at 0xCD685018.
Handle 0x0000, DMI type 0, 24 bytes
BIOS Information
Vendor: American Megatrends Inc.
Version: 0907
Release Date: 05/29/2014
Address: 0xF0000
Runtime Size: 64 kB
ROM Size: 8192 kB
Characteristics:
PCI is supported
APM is supported
BIOS is upgradeable
BIOS shadowing is allowed
Boot from CD is supported
Selectable boot is supported
BIOS ROM is socketed
EDD is supported
5.25"/1.2 MB floppy services are supported (int 13h)
3.5"/720 kB floppy services are supported (int 13h)
3.5"/2.88 MB floppy services are supported (int 13h)
Print screen service is supported (int 5h)
8042 keyboard services are supported (int 9h)
Serial services are supported (int 14h)
Printer services are supported (int 17h)
ACPI is supported
USB legacy is supported
BIOS boot specification is supported
Targeted content distribution is supported
UEFI is supported
BIOS Revision: 4.6
Handle 0x0001, DMI type 1, 27 bytes
System Information
Manufacturer: ASUS
Product Name: All Series
Version: System Version
Serial Number: System Serial Number
UUID: 4BC45180-D7DA-11DD-A63C-D850E64A8903
Wake-up Type: Power Switch
SKU Number: All
Family: ASUS MB

Server machine (segfaults)

# dmidecode 3.0
Getting SMBIOS data from sysfs.
SMBIOS 2.7 present.
70 structures occupying 2726 bytes.
Table at 0x000EC1F0.
Handle 0x0000, DMI type 0, 24 bytes
BIOS Information
Vendor: American Megatrends Inc.
Version: 0903
Release Date: 01/07/2014
Address: 0xF0000
Runtime Size: 64 kB
ROM Size: 8192 kB
Characteristics:
PCI is supported
APM is supported
BIOS is upgradeable
BIOS shadowing is allowed
Boot from CD is supported
Selectable boot is supported
BIOS ROM is socketed
EDD is supported
5.25"/1.2 MB floppy services are supported (int 13h)
3.5"/720 kB floppy services are supported (int 13h)
3.5"/2.88 MB floppy services are supported (int 13h)
Print screen service is supported (int 5h)
8042 keyboard services are supported (int 9h)
Serial services are supported (int 14h)
Printer services are supported (int 17h)
ACPI is supported
USB legacy is supported
BIOS boot specification is supported
Targeted content distribution is supported
UEFI is supported
BIOS Revision: 4.6
Handle 0x0001, DMI type 1, 27 bytes
System Information
Manufacturer: ASUS
Product Name: All Series
Version: System Version
Serial Number: System Serial Number
UUID: 0641D800-D88C-11DD-8CA0-E03F49ACB9B0
Wake-up Type: Power Switch
SKU Number: All
Family: ASUS MB
Handle 0x0002, DMI type 2, 15 bytes
Base Board Information
Manufacturer: ASUSTeK COMPUTER INC.
Product Name: Z87I-PRO
Version: Rev X.0x
Serial Number: 140121580900054
Asset Tag: To be filled by O.E.M.
Features:
Board is a hosting board
Board is replaceable
Location In Chassis: To be filled by O.E.M.
Chassis Handle: 0x0003
Type: Motherboard
Contained Object Handles: 0

Last edited by audiomuze (2016-01-04 11:39:18)

audiomuze · 2016-01-04 09:47:56

Updates BIOS to latest on both machines, identical BIOS config. SAMBA works on the desktop machine, not on the headless one. Reverted to out of box smb.conf, still segfaults. Time to look into the microcode updates. Doesn't make any sense through that it'd work on one machine and not the other.

audiomuze · 2016-01-04 10:04:14

Disabled smbd.service and nmbd.service, rebooted and tried to start smdb manually:

Job for smbd.service failed because a fatal signal was delivered causing the control process to dump core. See "systemctl status smbd.service" and "journalctl -xe" for details.

# systemctl -l status smbd.service
● smbd.service - Samba SMB/CIFS server
   Loaded: loaded (/usr/lib/systemd/system/smbd.service; disabled; vendor preset: disabled)
   Active: failed (Result: core-dump) since Mon 2016-01-04 11:55:44 SAST; 1min 40s ago
  Process: 653 ExecStart=/usr/bin/smbd -D (code=dumped, signal=SEGV)

Jan 04 11:55:43 alibbtrfs systemd[1]: Starting Samba SMB/CIFS server...
Jan 04 11:55:44 alibbtrfs systemd[1]: smbd.service: Control process exited, code=dumped status=11
Jan 04 11:55:44 alibbtrfs systemd[1]: Failed to start Samba SMB/CIFS server.
Jan 04 11:55:44 alibbtrfs systemd[1]: smbd.service: Unit entered failed state.
Jan 04 11:55:44 alibbtrfs systemd[1]: smbd.service: Failed with result 'core-dump'.
Jan 04 11:55:44 alibbtrfs systemd-coredump[654]: Process 653 (smbd) of user 0 dumped core.
                                                 
                                                 Stack trace of thread 653:
                                                 #0  0x00007f035b026ff7 n/a (libhcrypto-samba4.so.5)

# journalctl -xe


<snip>

-- 
-- The start-up result is done.
Jan 04 11:55:43 alibbtrfs systemd[1]: Starting Samba SMB/CIFS server...
-- Subject: Unit smbd.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit smbd.service has begun starting up.
Jan 04 11:55:44 alibbtrfs kernel: smbd[653]: segfault at 7f037b2510f8 ip 00007f035b026ff7 sp 00007ffe2ec15f88 error 4 in libhcrypto-samba4.so.5.0.1[7f035b01f000+32000]
Jan 04 11:55:44 alibbtrfs systemd[1]: smbd.service: Control process exited, code=dumped status=11
Jan 04 11:55:44 alibbtrfs systemd[1]: Failed to start Samba SMB/CIFS server.
-- Subject: Unit smbd.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit smbd.service has failed.
-- 
-- The result is failed.
Jan 04 11:55:44 alibbtrfs systemd[1]: smbd.service: Unit entered failed state.
Jan 04 11:55:44 alibbtrfs systemd[1]: smbd.service: Failed with result 'core-dump'.
Jan 04 11:55:44 alibbtrfs systemd-coredump[654]: Process 653 (smbd) of user 0 dumped core.
                                                 
                                                 Stack trace of thread 653:
                                                 #0  0x00007f035b026ff7 n/a (libhcrypto-samba4.so.5)
-- Subject: Process 653 (smbd) dumped core
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- Documentation: man:core(5)
-- 
-- Process 653 (smbd) crashed and dumped core.
-- 
-- This usually indicates a programming error in the crashing program and
-- should be reported to its vendor as a bug.

audiomuze · 2016-01-04 10:22:13

Ok, made sure all microcode updates are loaded:

dmesg | grep microcode
[    0.000000] microcode: CPU0 microcode updated early to revision 0x1c, date = 2014-07-03
[    0.096693] microcode: CPU1 microcode updated early to revision 0x1c, date = 2014-07-03
[    0.312419] microcode: CPU0 sig=0x306c3, pf=0x2, revision=0x1c
[    0.312424] microcode: CPU1 sig=0x306c3, pf=0x2, revision=0x1c
[    0.312428] microcode: CPU2 sig=0x306c3, pf=0x2, revision=0x1c
[    0.312433] microcode: CPU3 sig=0x306c3, pf=0x2, revision=0x1c
[    0.312463] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba

and...it makes no difference...SAMBA still segfaults.

# systemctl start smbd
Job for smbd.service failed because a fatal signal was delivered causing the control process to dump core. See "systemctl status smbd.service" and "journalctl -xe" for details.

Last edited by audiomuze (2016-01-04 10:24:20)

lucke · 2016-01-04 10:39:43

Try reinstalling smbclient (pacman -S smbclient).

audiomuze · 2016-01-04 12:17:41

lucke wrote:

Try reinstalling smbclient (pacman -S smbclient).

Client or server? The client is Roon running on the Win box.

Anyhow, reinstalled both and it's working again. Thank you for your assistance. I learned something new today... microcode, and Arch can be buggy.

lucke · 2016-01-04 12:22:49

smbclient apparently didn't get installed properly (libhcrypto-samba4.so.5.0.1 belongs to the smbclient package). Why? Maybe your drive is faulty or cosmic rays are shifting bits - can happen in any distro.

Last edited by lucke (2016-01-04 12:23:03)

audiomuze · 2016-01-04 13:18:48

The install is on a flash drive, perhaps data rot crept into the equation. Just glad it's up and running again.

Last edited by audiomuze (2016-01-04 13:29:31)

Arch Linux

#1 2016-01-02 09:06:26

[SOLVED] SAMBA segfaults since updating OS yesterday

#2 2016-01-03 18:45:34

Re: [SOLVED] SAMBA segfaults since updating OS yesterday

#3 2016-01-03 21:48:09

Re: [SOLVED] SAMBA segfaults since updating OS yesterday

#4 2016-01-04 05:35:11

Re: [SOLVED] SAMBA segfaults since updating OS yesterday

#5 2016-01-04 05:43:29

Re: [SOLVED] SAMBA segfaults since updating OS yesterday

#6 2016-01-04 08:25:20

Re: [SOLVED] SAMBA segfaults since updating OS yesterday

#7 2016-01-04 09:47:56

Re: [SOLVED] SAMBA segfaults since updating OS yesterday

#8 2016-01-04 10:04:14

Re: [SOLVED] SAMBA segfaults since updating OS yesterday

#9 2016-01-04 10:22:13

Re: [SOLVED] SAMBA segfaults since updating OS yesterday

#10 2016-01-04 10:39:43

Re: [SOLVED] SAMBA segfaults since updating OS yesterday

#11 2016-01-04 12:17:41

Re: [SOLVED] SAMBA segfaults since updating OS yesterday

#12 2016-01-04 12:22:49

Re: [SOLVED] SAMBA segfaults since updating OS yesterday

#13 2016-01-04 13:18:48

Re: [SOLVED] SAMBA segfaults since updating OS yesterday

Board footer