[SOLVED] "Invalid URL" on a perfectly valid URL because of underscore

daksh_s · 2026-01-13 00:41:54

okay so apparently because of the underscore, this is an invalid URL:-

https://daksh_s.codeberg.page

when in-fact, it is perfectly correct? (and no it's not because of the missing w3 prefix because somehow codeberg pages doesn't work with the w3 prefix lmao, atleast mine shows an SSL error)

I understand that this may be a security feature or system limitation on the server side but I hope it can be fixed.

Last edited by daksh_s (2026-01-15 22:36:31)

mpan · 2026-01-13 02:53:51

It’s neither a security feature nor system limitation.

An underscore is not among characters that may be in a hostname. These are ASCII letters, digits, and a hyphen.

If that address resolves for you, it’s by pure coincidence. DNS servers do not waste resources on actively validating RRs. Since labels may in general contain any octet, the invalid A and AAAA records slip through; to a DNS server they’re just another opaque data passed around for some label.

It’s not even a bug on Codeberg’s end, beause it’s not a record intentionally disseminated by them. The DNS has a wildcard that resolves any name to their host. It will accept these two too:

a-totally-non-existent-username-dxywgy3m.codeberg.page
even*clearly^invalid%names.codeberg.page

Last edited by mpan (2026-01-13 03:33:44)

cryptearth · 2026-01-13 08:10:14

daksh_s wrote:

missing w3 prefix because somehow codeberg pages doesn't work with the w3 prefix

because the usual www prefix mostly doesn't designate a hostname of a FQDN but rather the service of a website
have a look at this very site: it also doesn't start with www but with bbs because it doesn't serve the regular main page of archlinux.org but rather the specific service of this forum
another common prefix for this is forum
the prefix mail, however, can serve two ideas: either a webmail service or it can be the specific hostname of the MX for a domain - or even both
so - just because everyone used www in the 90s doesn't mean it's required - and never was - it's just one of many service designators - in the case of www with original meaning of "i request the website of some service"

daksh_s · 2026-01-13 16:59:20

so uhh, how exactly do i link my website to my profile now?

i cant change the url because its based on username

cryptearth · 2026-01-13 18:40:42

this is a service issue - contact codeberg support about they allow usernames which result in invalid url

seth · 2026-01-13 19:46:28

so uhh, how exactly do i link my website to my profile now?

https://duckduckgo.com/?q=tiny+url

mpan · 2026-01-14 08:31:24

daksh_s wrote:

so uhh, how exactly do i link my website to my profile now?

By first having any link. I want to underline this: the site loads in (some?) browsers due to a coincidence. A combination of Codeberg’s wildcard setup and DNS software blindly passing data around.

Even if — hypothetically — this forum makes an exception for you,⁽¹⁾ it solves nothing. That URL would remain invalid and the trouble is just pushed one step lower: to users and services trying to access it. It’s not even helping you. Sure, short-term it’s out of your mind. But it comes back tenfold worse later: when the address becomes associated with you, you wish people to access your content, and they can’t.

As a temporary solution, you may put it in your signature. It’s under Profile → Personality, “Compose your signature”.

daksh_s wrote:

i cant change the url because its based on username

Preferably by opening a bug report in Codeberg/pages-server. A solution would be to provide a mechanism separate from username-based hostname. For example an URL like:

https://userpages.codeberg.page/USERNAME

While reporting, make sure your tone is right. There isn’t much fault at Codeberg’s end. It’s not an error in their code. A simple oversight; missing that some users have names that can’t be accessed the way Codeberg offers.

⁂

seth wrote:

https://duckduckgo.com/?q=tiny+url

Please do not. Aside from consequences of using link obscuring services, it is not solving the core problem.

____
⁽¹⁾ It can be updated directly in the database. I believe the value is later treated as any other text. I do not suggest doing so, since it introduces maintenance burden, but hypothetically it’s possible.

Last edited by mpan (2026-01-14 08:47:59)

seth · 2026-01-14 10:32:22

Fwwi,

mpan wrote:

That URL would remain invalid and … wish people to access your content, and they can’t … site loads in (some?) browsers

https://en.wikipedia.org/wiki/Hostname#Syntax

Chrome, Firefox, Internet Explorer, Edge, and Safari allow underscores in hostnames

and also curl and wget accept it - while formally illegal, underscores are today de-facto universally tolerated.

daksh_s · 2026-01-14 19:51:22

okay. thanks guys.

daksh_s · 2026-01-15 22:53:58

https://codeberg.org/Codeberg/pages-server/issues/540

for future reference

cryptearth · 2026-01-15 23:52:29

well - I dug a bit deeper through various RFCs and archives of official mailing lists - and it seems we collectively fell for an important difference:
the LDH rule is specifically for hostnames - while DNS uses labels
none of the relevant RFCs actively denies the use of the underscore - rather the LDH rule reference should be understand as what has to be minimal supported
later RFCs, such as 2782 for the SRV RR define the underscore as designator for service specific labels
in fact both 1738 (URL) as well as 3986 (URI) define quite a lot of additional chars - both including the underscore
given this it could be argued that the url/uri validator here is too strict but not allowing the otherwise legal and widely used underscore (and maybe others) - because, as mentioned: an url/uri is not a FQDN nor a hostname and hence is not bound to enforcing the LDH rule

(note: I not used AI for the initial digging through the rfcs but only compared the outout of several about my findings - and all of them agree upon: "while the underscore is not part of the explicit characters for a hostname this doesn't result in it being illegal in DNS or url/uri and hence is fine to use and actually widely adopted")

mpan · 2026-01-16 00:30:49

cryptearth: as I mentioned earlier, “labels may in general contain any octet.” But hostnames are not consisting of arbitrary labels. Their labels may only contain letters a-z (either case), digits 0-9, and hyphens. This is originally defined by RFC 952, which is later referenced (directly and not) by other RFCs, including (assumed or not) the update that allows hyphens as the first character.

This can’t be reversed anymore, because many other standards depend on hostname never containing an underscore. This is why an underscore is chosen to be present in labels acting as keywords. An example is the RFC 2782 you mentioned, another is ACME (using `_acme-challenge`). Their existence is possible only because it’s certain that no host will ever bear a name with an underscore.

cryptearth · 2026-01-16 06:23:29

@mpan
agreed - no hostname should ever have any other char than a-z 0-9 and "-"
but hostnames - something handled local by the kernel - are independent of DNS labels
an entry in a dns zone does not have to match the hostname set on the machine it points to - and aside from protocols which include this "service designator" as part of thier layer 4 data structure, like http or tls (sni), the underlying IP layer doesn't even know about it

it doesn't matter what the bbs in bbs.archlinux.org would be called - as long as it resolves to the ip which points to the machine which serves it - and it be able to parse it correctly - which would make _bbs perfectly fine - and i doubt that the system which serves this forum has its hostname set to "bbs"

again - i question the fixing on hostnames - because they are NOT part of DNS and hence the LDH rule doesn't apply to DNS labels
if the url/uri parser uses the ldh rule for hostnames then it is at fault here because it tries to validat one thing, a url/uri, against a different ruleset, a hostname - which not just doesn't make sense but is just wrong IMO

seth · 2026-01-16 09:24:07

No. This is also touched upon in the previously linked wiki paragraph.

The key is the scheme - the moment you put http(s):// in front of anything, the following has to … errr… follow a specific pattern.

http://user:password@host:port/path?query

("user:password@" actually now being deprecated)
It doesn't matter that underscores are generally allowed as aspect of the URI or DNS labels.
As a matter of fact and as mpan exampled, using underscores in a hostname is especially troublesome since it can run you into conflicts w/ active keywords.

The reason why DNS servers will (now) resolve _foo.bar.com is https://www.rfc-editor.org/rfc/rfc7816 which got into wider use somewhen in 2019 when bind defaulted to it and is now a proposed standard since 2022, https://www.rfc-editor.org/rfc/rfc9156
That doesn't make _foo a legal hostname and therefore also not a legal host component in a http URI.

FluxBB discontinued in 2018/2019 (don't start, there're enough threads about that), otherwise one would probably file a bug to be more lenient about this because of the de-facto situation (illegal hostnames being accepted by all major http clients and slipping through DNS queries) but https://_foo.bar.com/ is NOT a valid url.

Arch Linux

#1 2026-01-13 00:41:54

[SOLVED] "Invalid URL" on a perfectly valid URL because of underscore

#2 2026-01-13 02:53:51

Re: [SOLVED] "Invalid URL" on a perfectly valid URL because of underscore

#3 2026-01-13 08:10:14

Re: [SOLVED] "Invalid URL" on a perfectly valid URL because of underscore

#4 2026-01-13 16:59:20

Re: [SOLVED] "Invalid URL" on a perfectly valid URL because of underscore

#5 2026-01-13 18:40:42

Re: [SOLVED] "Invalid URL" on a perfectly valid URL because of underscore

#6 2026-01-13 19:46:28

Re: [SOLVED] "Invalid URL" on a perfectly valid URL because of underscore

#7 2026-01-14 08:31:24

Re: [SOLVED] "Invalid URL" on a perfectly valid URL because of underscore

#8 2026-01-14 10:32:22

Re: [SOLVED] "Invalid URL" on a perfectly valid URL because of underscore

#9 2026-01-14 19:51:22

Re: [SOLVED] "Invalid URL" on a perfectly valid URL because of underscore

#10 2026-01-15 22:53:58

Re: [SOLVED] "Invalid URL" on a perfectly valid URL because of underscore

#11 2026-01-15 23:52:29

Re: [SOLVED] "Invalid URL" on a perfectly valid URL because of underscore

#12 2026-01-16 00:30:49

Re: [SOLVED] "Invalid URL" on a perfectly valid URL because of underscore

#13 2026-01-16 06:23:29

Re: [SOLVED] "Invalid URL" on a perfectly valid URL because of underscore

#14 2026-01-16 09:24:07

Re: [SOLVED] "Invalid URL" on a perfectly valid URL because of underscore

Board footer