You are not logged in.

#1 2020-12-24 05:13:38

zenix
Member
From: Earth - Save it!
Registered: 2007-08-05
Posts: 104
Website

alpm-html - Build a static website to browse an Arch repo

So I've been noodling on this all year, spending a few minutes here and there until it's finally in a moderately reasonable shape. And here it is!

A pretty simple python script that takes in a directory full of pacman packages and a couple of Jinja templates and spits out a bunch of HTML that you can use to browse the list of packages in your browser. And since it's purpose built to be used for Arch repositories, the results are styled similarly to Arch's website.

Why have a static website at all for your repo? It's not like users need to see it and pacman doesn't care. Well, because I wanted something prettier than the automatic index pages generated by conventional webservers. And the resulting page can be used as a kind of homepage for it, rather than a mention somewhere else in a list. But mostly, I just wanted to. big_smile

See the source code at my Gitlab instance.
Grab the package from the AUR.
See an example of it with my own personal repo.


I made an AUR helper once.
I also go by evaryont and nogweii elsewhere on the internet.
Check out my projects and packages.

Offline

#2 2020-12-24 05:52:37

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 6,581
Website

Re: alpm-html - Build a static website to browse an Arch repo

Nice! I like the clean layout with an Arch theme. This should be perfect for dropping in repos on static sites.

Can you add the packager to the package info page for repos with multiple packagers? I also noticed some some warnings and errors in the HTML validator.

Sorry for being to lazy right now to check the code but does this handle multiple repos in the same directory (e.g. two repos for different architectures in the same directory that share the common "any" packages and have duplicates of others for each architecture)?


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#3 2020-12-24 06:45:53

zenix
Member
From: Earth - Save it!
Registered: 2007-08-05
Posts: 104
Website

Re: alpm-html - Build a static website to browse an Arch repo

Xyne wrote:

Nice! I like the clean layout with an Arch theme. This should be perfect for dropping in repos on static sites.

Thanks for the kind words! I was extremely nervous, since it's been quite some time since I published/advertised a personal project.

Xyne wrote:

Can you add the packager to the package info page for repos with multiple packagers?

You can customize the template used, either by editing the file directly (installed at /usr/share/alpm-html/package.html.j2) or by passing a different directory to tell alpm-html to look for that template by passing --resources DIR as a CLI argument. The information is already exposed to the template, so no need to edit the python code.

Since the repo is maintained solely by myself, I felt it was kind of redundant to include the packager in the details page. I was thinking earlier about whether or not to include it, and decided then against it. I intend to shape the default template as somewhat generic, but tailored more to my needs. I feel like the options allow you to customize/override that choice easily enough.

Xyne wrote:

I also noticed some some warnings and errors in the HTML validator.

Whoops! That will be fixed very soon.

Edit: This is now fixed and my repo example above has been regenerated using it.

Xyne wrote:

Sorry for being to lazy right now to check the code but does this handle multiple repos in the same directory (e.g. two repos for different architectures in the same directory that share the common "any" packages and have duplicates of others for each architecture)?

I think so, unintentionally. The code is really naive, looking for tarballs in the directory specified (or current directory) and passing them to pyalpm/libalpm to pull metadata from. I was thinking about doing something similar to what you describe as I've been looking at cross-compiling some packages for Archlinux ARM. It definitely doesn't do a good job if those repos were spread across multiple directories.

Last edited by zenix (2020-12-24 21:31:13)


I made an AUR helper once.
I also go by evaryont and nogweii elsewhere on the internet.
Check out my projects and packages.

Offline

#4 2020-12-24 22:41:33

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 6,581
Website

Re: alpm-html - Build a static website to browse an Arch repo

Thanks for the reply. I should have examined the code to check if the displayed information was configurable. Exposing all of the data to a customizable template is a perfect solution. I also agree that omitting the packager by default is best for single-packager repos.

Would it be easy to also generate a static JSON file with all of the package metadata alongside the generated HTML? The JSON could be used to create different utilities such as a cross-site package search engine. If enough people were to begin using alpm-html to host their repos online, it could even create a unified package ecosystem. It would then be trivial to write a tool to list available repos with relevant metdata (maintainer, signing keys) and auto-generate an include file for pacman (e.g. an ncurses-based command-line utility).

I realize that this is probably well beyond the scope of the project so I'm not actually requesting that you implement this. I'm just exploring the idea.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#5 2020-12-27 23:41:04

eschwartz
Trusted User/Bug Wrangler
Registered: 2014-08-08
Posts: 3,870

Re: alpm-html - Build a static website to browse an Arch repo

Very cool!

Have you thought about making the Dependencies section link to either https://archlinux.org/packages/$repo/$arch/$name/ if available in the current syncdbs via pyalpm, or a relative link to the static site if it is another package in the same repo?


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#6 2020-12-28 00:00:17

zenix
Member
From: Earth - Save it!
Registered: 2007-08-05
Posts: 104
Website

Re: alpm-html - Build a static website to browse an Arch repo

eschwartz wrote:

Very cool!

Have you thought about making the Dependencies section link to either https://archlinux.org/packages/$repo/$arch/$name/ if available in the current syncdbs via pyalpm, or a relative link to the static site if it is another package in the same repo?

I was thinking about this, but punted on it for the moment since a) I wanted to just get something out there (avoiding perfection, etc) and b) I wasn't sure how to distinguish those 2 visually, which I want to do. Might not be truly necessary to do that.

Xyne wrote:

Would it be easy to also generate a static JSON file with all of the package metadata alongside the generated HTML?

Easy to generate? Yup, since python has a native JSON library. But what is included in that JSON? And how do you discover it?

However, the idea of a tool that can be pointed at a repo URL and automatically generate the pacman configuration would be awesome. Especially making the discovery of signing keys a little more built-in feeling.


I made an AUR helper once.
I also go by evaryont and nogweii elsewhere on the internet.
Check out my projects and packages.

Offline

#7 2020-12-30 05:00:36

zenix
Member
From: Earth - Save it!
Registered: 2007-08-05
Posts: 104
Website

Re: alpm-html - Build a static website to browse an Arch repo

I've just released 0.10.0 which includes some of the feedback!

Added

  • Dependencies now link to the arch website or sibling packages in this repo when found

  • Optional support for minifying HTML using the htmlmin library

Changed

  • Package detail pages link to the index page

  • External links have an indicator, derived from Wikipedia

  • Verbose and debug logs can now be optionally colored using the daiquri library

  • The "License" and "Group" labels will now be pluralized only if necessary, using the inflect library

Xyne wrote:

Would it be easy to also generate a static JSON file with all of the package metadata alongside the generated HTML?

Something I didn't note from before is that the package pages include https://schema.org/SoftwareApplication HTML microdata embedded inline. Doesn't address the repo-wide data you were talking about, though.


I made an AUR helper once.
I also go by evaryont and nogweii elsewhere on the internet.
Check out my projects and packages.

Offline

Board footer

Powered by FluxBB