UEFI PXE reason it doesn't work

nomorewindows · 2024-02-08 18:42:44

 DHCP-Message (53), length 1: Discover
	    MSZ (57), length 2: 1464
	    Parameter-Request (55), length 35: 
	      Subnet-Mask (1), Time-Zone (2), Default-Gateway (3), Time-Server (4)
	      IEN-Name-Server (5), Domain-Name-Server (6), Hostname (12), BS (13)
	      Domain-Name (15), RP (17), EP (18), RSZ (22)
	      TTL (23), BR (28), YD (40), YS (41)
	      NTP (42), Vendor-Option (43), Requested-IP (50), Lease-Time (51)
	      Server-ID (54), RN (58), RB (59), Vendor-Class (60)
	      TFTP (66), BF (67), GUID (97), Unknown (128)
	      Unknown (129), Unknown (130), Unknown (131), Unknown (132)
	      Unknown (133), Unknown (134), Unknown (135)

tcpdump of port 67 request from UEFI PXE client before it'll complete the PXE transaction. If it doesn't get it, it doesn't PXE boot, even if secure boot is disabled. Like it expects a windows server in order to make it work. Especially the unknown options. What are they?

Have to use legacy PXE to boot linux. One machine won't do legacy, UEFI PXE only.

Last edited by nomorewindows (2024-02-08 18:44:44)

cryptearth · 2024-02-14 20:38:37

To me it sounds like you have at least one additional dhcp server on your network and the pxe one isn't correctly working in proxy mode.
So: What other dhcp servers do you have running on your network? What happens if you disable them? What you use for your pxe server?

nomorewindows · 2024-02-25 06:44:32

cryptearth wrote:

To me it sounds like you have at least one additional dhcp server on your network and the pxe one isn't correctly working in proxy mode.
So: What other dhcp servers do you have running on your network? What happens if you disable them? What you use for your pxe server?

Well it's strange, because I have openwrt routers and the dhcp server on it doesn't seem to work, but before I put the openwrt router in, I was doing pxe and dhcp entries for pxe from arch server. When arch server goes down, openwrt won't take over as dhcp server. So it shouldn't be conflicted. Didn't realize openwrt dhcp wasn't working right until I subnetted another openwrt to join the main and it won't dhcp either.

Might be something to check out, but don't think it will be that though. I think it's something in the PXE specification for UEFI. Can't remember that I have really ever gotten a boot from UEFI PXE. Always have to use legacy boot mode. Going to be annoying if I can't UEFI PXE going on a unit that has a eMMC as the main drive and no legacy mode. I even tried iPXE and it won't find the network device.

There was another unit I had that had PXE stack, but it was on another port besides BOOTP. It was definitely a Micro$oft friendly PXE UEFI specification. It was on port 4000 something I think.

Last edited by nomorewindows (2024-02-25 06:46:03)

cryptearth · 2024-02-25 10:52:44

Hm, to me still sounds like issues with more than one dhcp conflicting eachother.
Anyway, all I can give you as input is this:

On my network I have two DHCP working in parallel - the one in my ISPs router (can't disable it, and can't change its settings - it's fixed) and a pxe-proxy on my "server" (one of those single-board-computers - but x86_64 based instead of arm).
The dhcp on my router serves out a specific class c network in the range 192.168.178.100 to 192.168.178.200 with only client-ip and itself as both the dns and gateway.
On my server I use a ISC dhcpd and gave it this config:

option domain-name-servers 192.168.178.1;
option routers 192.168.178.1;
ddns-update-style none;
default-lease-time 180;
allow booting;
class "pxe-bios" {
  match if substring (option vendor-class-identifier, 0, 20) = "PXEClient:Arch:00000";
}
class "pxe-uefi" {
  match if substring (option vendor-class-identifier, 0, 20) = "PXEClient:Arch:00007";
}
subnet 192.168.178.0 netmask 255.255.255.0 {
  option broadcast-address 192.168.178.255;
  pool {
    allow members of "pxe-bios";
    filename "ipxe.pxe";
    next-server 192.168.178.137;
    range 192.168.178.201 192.168.178.220;
  }
  pool {
    allow members of "pxe-uefi";
    filename "ipxe.efi";
    next-server 192.168.178.137;
    range 192.168.178.221 192.168.178.240;
  }
}

How does it work?
First I filter to only server PXE clients - these identify themselfs by a specific vendor-class string.
Care must be taken to select between old bios / uefi csm clients and modern uefi clients.
I then assign both types a range, for bios the range 201 - 220 and for uefi the range 221 - 240 - and serve the specific bootp file name which then gets loaded via TFTPd.
In my case I use the very default iPXE but only with a one-line script to break the infinite loop of the standard built but to fetch a config file also provided via tftp which contains entries for chaining like arch, opensuse, windows pe, memtest and a couple others.

For me this works as the dhcp in my router seems to ignore pxe clients alltogether. And this is what I suspect goes wrong at your setup: To my knowledge OpenWRT already comes along with PXE-capable dhcp and tftp server - so it should be able to provide all these features without require you to have some additional stuff set up. But as you do it seems some stuff does conflict and hence already the initial DHCP DISCOVER fails as it gets two replies: one without a bootp filename - and a second one with a bootp filename. So now you have a race-condition: What happens depends on which DHCP OFFER is received first by the client.

I would recommend to check out your openWRT settings whats going on there - as you state that its dhcp doesn't seem to work correctly. I would start there.
Otherwise try to disable the dhcp n the openwrt router completely so you only have your pxe dhcp server running - then it should work reliable.
Another option: Just use ISC DHCP, iPXE and copy my config - that should do the trick as well.

nomorewindows · 2024-02-26 06:46:23

I get pxe on legacy mode just fine, it's the uefi that's the strange point. I tried it on a machine that allows me to being up the bios boot menu. I flipped back and forth between legacy pxe and uefi pxe. Uefi pxe no dice. Legacy pxe right on.

I do know about the pxe function on openwrt, I think I'm not using it, may have to check. My arch server is using dnsmasq although in the past I ran tftpd, dhcpd, dns caching separate.

Ok, so port 67 appears closed from outside the openwrt router. Even though odhcpd is supposedly running on openwrt, it isn't open from nmap scan outside router. Explains why openwrt doesn't take over dhcp when arch server goes down.

Last edited by nomorewindows (2024-02-26 07:19:19)

cryptearth · 2024-02-27 18:13:33

Well, I have a bit of an odd question, maybe it could be solved by showing your config: Do you know, you have to provide two different NBP (network bootstrap program) depending on if the client is either legacy or uefi?
To me it sounds like you try to provide ONE image to both types - which of course doesn't work as a legacy NBP is not a valid EFI image.
How does your config chose which image to serve?

As you can see in my config I do this based on the client identifier.

nomorewindows · 2024-02-28 19:12:15

cryptearth wrote:

Well, I have a bit of an odd question, maybe it could be solved by showing your config: Do you know, you have to provide two different NBP (network bootstrap program) depending on if the client is either legacy or uefi?
To me it sounds like you try to provide ONE image to both types - which of course doesn't work as a legacy NBP is not a valid EFI image.
How does your config chose which image to serve?
As you can see in my config I do this based on the client identifier.

I used a slightly different dialect in my dhcpd.conf

if option architecture = 00:07 {
filename = "/syslinux.efi";
}
else
{
filename "/lpxelinux.0";
}

On dnsmasq.conf it uses

pxe-service=x86PC, "Install Linux",lpxelinux

And

pxe-service=X86-64PC_EFI,"PXELINUX (UEFI)",syslinux.efi

Last edited by nomorewindows (2024-02-28 19:19:15)

cryptearth · 2024-02-28 21:23:12

Well, ok, the actual syntax might be different - but the effect is the same.
Are the different file-names just a typo?

Although syslinux can be build as efi I would recommend ipxe - just because it's newer. Depending on the age of the clients it could be that syslinux is just too old and doesn't have the required drivers.

Just to check if it's not some NBP issue: Have you tried these images: https://archlinux.org/releng/netboot/ ? They should work just fine with most hardware and could give an indication if someting's wrong with the binaries you provide.

Tarqi · 2024-02-28 23:01:54

Maybe add 9 as additional architecture.

dhcpd.conf

option vendor-class-identifier code 60 = string;
option architecture-type code 93 = unsigned integer 16;

class "pxe" {
  match if substring (option vendor-class-identifier, 0, 9) = "PXEClient";
  if    option architecture-type = 00:00 { filename "bios/pxelinux.0"; }
  elsif option architecture-type = 00:07 { filename "efi64/syslinux.efi"; }
  elsif option architecture-type = 00:09 { filename "efi64/syslinux.efi"; }
  else { filename "bios/pxelinux.0"; }
}

dnsmasq.conf

dhcp-match=set:bios,option:client-arch,0
dhcp-match=set:efi-x86_64,option:client-arch,7
dhcp-match=set:efi-x86_64,option:client-arch,9
dhcp-boot=tag:bios,bios/lpxelinux.0
dhcp-boot=tag:efi-x86_64,efi64/syslinux.efi

This works for me with the machines I have access to; you also might add 6 for 32-bit.

Edit: Grammar / added missing dhcpd options

Last edited by Tarqi (2024-02-28 23:10:20)

cryptearth · 2024-02-29 04:09:34

Good point about the additional platform IDs. For reference I found this: https://serverfault.com/questions/34969 … m-rfc-4578
There seems to be some conflict about IDs 7 and 9.

nomorewindows · 2024-02-29 16:54:51

cryptearth wrote:

Well, ok, the actual syntax might be different - but the effect is the same.
Are the different file-names just a typo?
Although syslinux can be build as efi I would recommend ipxe - just because it's newer. Depending on the age of the clients it could be that syslinux is just too old and doesn't have the required drivers.
Just to check if it's not some NBP issue: Have you tried these images: https://archlinux.org/releng/netboot/ ? They should work just fine with most hardware and could give an indication if someting's wrong with the binaries you provide.

I tried ipxe from hard drive. It has another distribution pre-installed, so I added ipxe to grub. And it wouldn't find the network interface. Can't remember if I tried putting arch ipxe on it, but I was looking at netboot unsure if it was going to find network interface any different than the one I tried.

Tarqi · 2024-02-29 18:10:26

Just stay with syslinux, this works flawlessly, see my last post. Also, the problem is that you're just not getting the file, regardless of its type.

Edit: Grammar

Last edited by Tarqi (2024-02-29 19:56:55)

cryptearth · 2024-03-01 09:48:28

Tarqi wrote:

Just stay with syslinux, this works flawlessly

I highly recommend using iPXE over syslinux
1) last syslinux update was years ago - ipxe is in active developement
1a) syslinux heavily relies on UNDI on modern systems as it lacks a lot of own drivers for modern nics
2) syslinux is quite limited in what it can do - ipxe offers a much wider featureset
3) as linked ipxe is the base for current arch netboot images - and they're designed as drop-in replacement
in short: there's no reason for using syslinux over ipxe whrn dealing with modern uefi hardware - and even in mixed environments it's better to just use ipxe for both instead of struggle around between syslinux for legacy hardware vs ipxe for uefi: way too error prone

Tarqi · 2024-03-01 10:07:59

I don't care; as written, it works for me perfectly and the features of iPXE are not needed (for me). Has iPXE a boot menu like syslinux? However, I just wanted to help the OP and have written my point(s), so I'm out here now.

cryptearth · 2024-03-02 12:34:45

Well, it obviously depends on the config you provide - and as for menu stuff iPXE is more sophisticated: you can use, set and change variables (have a look at the arch-ipxe menu: you can select a mirror, can modify kernel parameters - something syslinux can't do).
Also: as uefi was mentioned we're speaking about post-2010 - according to wikipedia syslinux got it's last official release 6.03 in 2014 while ipxe 1.21.1 was released end of 2020.
I also don't argue about which is better - I just presented some facts why syslinux should not be recommended in favour instead of ipxe for modern uefi hardware. If it comes down to "what tool to use" you can even make grub2 network bootable and use it instead of either syslinux or ipxe.

Until now it's still unclear if OP has some network issue like if the uefi client doesn't even get the pxe-dhcp-offer or if there's an issue with the provided efi binary - hence I posted my config and recommended known working binaries meant as a drop-in replacement.
Unfortunately I can't just grab my laptop and a patch cable - walk over next door to OP - and figure it out myself as I have quite some experience in trouble shooting PXE. But I'm unable to figure out possible issues from what OP posted so far.

If syslinux works for you - ok then - but it's more likely to fail sooner as ipxe will just due to last ipxe latest release is several years newer - and it's overall a somewhat successor to syslinux as it's specifically targeted to pxe - hence its name. pxelinux is just one of the artifacts of syslinux enable it to load config and data over tftp/http instead of from the filesystem it booted from.

jeez ... I still don't get why so many linux users are so stubborn and toxic ... it's not rocket science - it's a simple question-and-answer as for windows - although the end of the story often is a fix rather than a clean reinstall of the entire OS
yes, network stuff is complicated and so far OP hasn't provided much more than "uefi doesn't work" and hasn't replied to "have you tried this or that?" - but why the fricking heck someone has to come in - yell at everyone else "you're wrong - use this!" and tries to get this as a somewhat accepted answer like over on the stackexchange platform - instead of simple troubleshooting?
if you think you have to feel so much over the top above anyone else - go sign up for nasa and bring the next murican back on the moon - and get yourself a one-way ticket along

Tarqi · 2024-03-02 13:57:34

Yes, I said I'm out, but that last paragraph amused me. Maybe you should take a look at my signature and reread the thread, and then you will possibly find out, who in this thread is stubborn, toxic and yelling. And now I'm really out; this is getting dumb.

nomorewindows · 2024-03-02 16:18:11

Somehow after messing around with grub and adding another kernel to tftp and adding new entry to dnsmasq I got it to work. Just booted up to a pxemenu from dnsmasq. I had pxeprompt on dnsmasq before but maybe I forgot to restart dnsmasq after making changes to conf file before. Not exactly sure what I did but it worked.

But trying it on another machine still doesn't boot efi pxe. So still a mystery.

Last edited by nomorewindows (2024-03-02 16:56:16)

nomorewindows · 2024-03-02 18:15:43

Trying on machine that already has Arch, still won't boot uefi pxe from firmware, so put ipxe.efi in ESP partition, and it was able to at least open network interface but still refuses pxe menu items.

Gets an IP address I can see that from flashing by on firmware uefi boot. And it gets an IP on ipxe also. So it at least gets that far.

The netboot files are apparently in the ipxe package.

Last edited by nomorewindows (2024-03-02 18:19:13)

cryptearth · 2024-03-03 17:35:33

1) Does the UEFI client get the pxe-dhcp-offer?
2) Do you see the UEFI client try to download the efi nbp in the logs?
3) Have you tried the arch ipxe binaries?

Just for the lolz: Have you tried without the network but only connect two systems directly together and try to boot one from the other to rule out the multiple dhcp servers?

nomorewindows · 2024-03-04 18:38:41

cryptearth wrote:

1) Does the UEFI client get the pxe-dhcp-offer?
2) Do you see the UEFI client try to download the efi nbp in the logs?
3) Have you tried the arch ipxe binaries?
Just for the lolz: Have you tried without the network but only connect two systems directly together and try to boot one from the other to rule out the multiple dhcp servers?

Where is the nbp logs?
I did try the ipxe binaries. Loads network interface on one, but still doesn't load nbp, the other still doesn't even bring up network interface.

Last edited by nomorewindows (2024-03-04 18:39:34)

cryptearth · 2024-03-04 19:13:59

Check your logs of both your dhcp server and your tftp server.
You want to see a log entry in the dhcp log that a new pxe client has requested an offer and got a lease - and then in the tftp log that the pxe client try to download the file you given in the file name.
If you don't see a log in the dhcp server logs then try to just use two systems and a cable to directly connect them with eachother to get the rest of your network out of the question. One has to run a dhcp and tftp server while the other has to pxe boot.
Log the network traffic on the system running the server to see if it even receives the DHCP DISCOVER package.

nomorewindows · 2024-03-04 19:23:22

cryptearth wrote:

Check your logs of both your dhcp server and your tftp server.
You want to see a log entry in the dhcp log that a new pxe client has requested an offer and got a lease - and then in the tftp log that the pxe client try to download the file you given in the file name.
If you don't see a log in the dhcp server logs then try to just use two systems and a cable to directly connect them with eachother to get the rest of your network out of the question. One has to run a dhcp and tftp server while the other has to pxe boot.
Log the network traffic on the system running the server to see if it even receives the DHCP DISCOVER package.

I'm currently using dnsmasq, so that might not be where I think it is.

Looking in /etc/dnsmasq.conf, looks like I have to uncomment log-dhcp and log-queries to get anything like that. Not a default option apparently. So most likely, nothing to see atm. Turned on options, restarted dnsmasq. Shows dns queries, but nothing on dhcp.

Last edited by nomorewindows (2024-03-04 19:34:08)

Arch Linux

#1 2024-02-08 18:42:44

UEFI PXE reason it doesn't work

#2 2024-02-14 20:38:37

Re: UEFI PXE reason it doesn't work

#3 2024-02-25 06:44:32

Re: UEFI PXE reason it doesn't work

#4 2024-02-25 10:52:44

Re: UEFI PXE reason it doesn't work

#5 2024-02-26 06:46:23

Re: UEFI PXE reason it doesn't work

#6 2024-02-27 18:13:33

Re: UEFI PXE reason it doesn't work

#7 2024-02-28 19:12:15

Re: UEFI PXE reason it doesn't work

#8 2024-02-28 21:23:12

Re: UEFI PXE reason it doesn't work

#9 2024-02-28 23:01:54

Re: UEFI PXE reason it doesn't work

#10 2024-02-29 04:09:34

Re: UEFI PXE reason it doesn't work

#11 2024-02-29 16:54:51

Re: UEFI PXE reason it doesn't work

#12 2024-02-29 18:10:26

Re: UEFI PXE reason it doesn't work

#13 2024-03-01 09:48:28

Re: UEFI PXE reason it doesn't work

#14 2024-03-01 10:07:59

Re: UEFI PXE reason it doesn't work

#15 2024-03-02 12:34:45

Re: UEFI PXE reason it doesn't work

#16 2024-03-02 13:57:34

Re: UEFI PXE reason it doesn't work

#17 2024-03-02 16:18:11

Re: UEFI PXE reason it doesn't work

#18 2024-03-02 18:15:43

Re: UEFI PXE reason it doesn't work

#19 2024-03-03 17:35:33

Re: UEFI PXE reason it doesn't work

#20 2024-03-04 18:38:41

Re: UEFI PXE reason it doesn't work

#21 2024-03-04 19:13:59

Re: UEFI PXE reason it doesn't work

#22 2024-03-04 19:23:22

Re: UEFI PXE reason it doesn't work

Board footer