[SOLVED] Commit message for updated packages

Salkay · 2015-12-28 09:59:15

When packages from the official repositories are updates, I often like to check the commit message, which often acts as a mini-changelog. For example, mesa was updated recently. I'd navigate to the mesa package page, click View Changes, then click on the latest commit, i.e. upgpkg: mesa 11.1.0-2. I would then read the commit message.

upgpkg: mesa 11.1.0-2
Add libgcrypt dependency (FS#47540)

git-svn-id: file:///srv/repos/svn-packages/svn@257307 eb2447ed-0c53-47e4-bac8-5bc4a241df78

Is there a way to access this message via the command line? Presumably pacman cannot do this, but perhaps a short script?

Last edited by Salkay (2016-01-02 11:36:43)

x33a · 2015-12-28 13:29:07

You either have to clone the git repo and search for the commit messages inside that or you'll have to write a web scraper. Another option is to use the atom feed for the packages.

dolik.rce · 2015-12-29 06:02:46

You can get the latest commit messages from the web in command line. It can be done quite easily with the w3m browser (from w3m package) and using a simple sed filter:

# for Arch packages:
w3m -dump 'https://projects.archlinux.org/svntogit/packages.git/log/trunk?h=packages/mesa&showmsg=1' | sed -n '/Commit message/,/^\s*$/p'

# for packages from AUR:
w3m -dump 'https://aur.archlinux.org/cgit/aur.git/log/?h=debootstrap&showmsg=1' | sed -n '/Commit message/,/^\s*$/p'

It is probably possible to do this also using links, lynx, eLinks or any other console based web browser.

Salkay · 2015-12-29 09:48:05

x33a wrote:

You either have to clone the git repo and search for the commit messages inside that or you'll have to write a web scraper. Another option is to use the atom feed for the packages.

Thank you for the ideas. I could clone the svn repo, and I get the logs from that.

$ svn checkout --depth=empty svn://svn.archlinux.org/packages
Checked out revision 257362.
$ cd packages
$ svn update mesa
...
$ svn log mesa -l 2
------------------------------------------------------------------------
r257308 | lcarlier | 2015-12-28 08:27:21 +1100 (Mon, 28 Dec 2015) | 1 line

archrelease: copy trunk to extra-i686, extra-x86_64
------------------------------------------------------------------------
r257307 | lcarlier | 2015-12-28 08:26:19 +1100 (Mon, 28 Dec 2015) | 3 lines

upgpkg: mesa 11.1.0-2

Add libgcrypt dependency (FS#47540)
------------------------------------------------------------------------

This seems workable, and I could wrap it in a script, but the svn logs seem slightly less "clean" than the git logs. In the latter, the latest entry is "Add libgcrypt dependency", and there is no entry for the extraneous "copy trunk to…". However, I couldn't quite work out how to clone the git repos. I couldn't find a reference to that in the wiki. Probably getting remote git logs would be more efficient than the svn route. I'll also see how the web scraper version compares.

dolik.rce wrote:

You can get the latest commit messages from the web in command line.

Thank you for the code. This works well. I'll check out the other options, but this might be best.

dolik.rce · 2015-12-29 11:31:28

Salkay wrote:

I could clone the svn repo, and I get the logs from that.

Actually, you don't need to clone it at all. With subversion, you can get the history directly from the remote server:

svn checkout --depth=empty svn://svn.archlinux.org/packages

Salkay wrote:

However, I couldn't quite work out how to clone the git repos. I couldn't find a reference to that in the wiki. Probably getting remote git logs would be more efficient than the svn route. I'll also see how the web scraper version compares.

There is no way to get remote git log in the way you can do it with subversion. The closest you could do is similar to your previous approach with svn:

git clone git://projects.archlinux.org/svntogit/packages.git packages
cd pacakges
git log -- mesa

But the repository is quite huge and there is no simple way to clone only single package. So you'd always have to download the entire thing, which is definitely not what you want.

progandy · 2015-12-29 11:35:28

dolik.rce wrote:

But the repository is quite huge and there is no simple way to clone only single package. So you'd always have to download the entire thing, which is definitely not what you want.

It is possible to clone single packages. Each package has its own branch and you only have to fetch the branche you are interested in, not the whole repository. There is even a tool that helps with it.
https://aur.archlinux.org/packages/asp-git/

Last edited by progandy (2015-12-29 11:36:51)

dolik.rce · 2015-12-29 11:57:09

progandy wrote:

dolik.rce wrote:
But the repository is quite huge and there is no simple way to clone only single package. So you'd always have to download the entire thing, which is definitely not what you want.
It is possible to clone single packages. Each package has its own branch and you only have to fetch the branche you are interested in, not the whole repository. There is even a tool that helps with it.
https://aur.archlinux.org/packages/asp-git/

Oh, I didn't know the repository was organized this way. So the script could be

git clone --single-branch  -b packages/mesa git://projects.archlinux.org/svntogit/packages.git
cd packages
git log -- trunk

But it still doesn't feel right to download it locally. I'd still prefer the w3m way I mentioned in my first post.

Salkay · 2015-12-31 22:37:24

Thanks for the hints everyone. They all seem to work well for mesa. The w3m way seems to be fastest by far.

However, it seems that one problem with all of these methods is that you need to know which repository they are from. For example, I tried the commands on notmuch's recent update. However, because it's in the [community] repo, I had to change the path/url a little. Would there be a way to not have to look up the repository as well?

dolik.rce · 2016-01-01 20:56:37

Salkay wrote:

Would there be a way to not have to look up the repository as well?

Sure, just ask pacman:

pacman -Ss '^mesa$' | sed 's|/.*||;q'

It returns repository name (core, extra, community, ...). If it returns nothing, then the package doesn't exist in repositories, which means it either doesn't exist at all or it is from AUR. You can write a simple case statement in your script to figure out the correct url based on this.

Salkay · 2016-01-02 11:36:27

Thanks for all your help! I've written a small script to output the last commit messages for a package. I've also included a flag (-l) to only show the last x commits.

#!/usr/bin/env bash
# Output the comments for the last commits of a package (specified as the script's argument)
# Optionally, only display the first NUM commits, with -l NUM.

# Define usage.
usage() {
  cat >&2 <<EOF
Usage: $0 [-l NUM] PACKAGE_NAME | -h
  -l NUM      optionally, only display NUM latest entries
  -h          display this help text and exit
EOF
  exit 1
}

# Parse options.
while getopts 'l:h' opt; do
  case "${opt}" in
    l) length=$OPTARG ;;
    *) usage ;;
  esac
done

# Check -l argument. If $length is empty, then equivalent to flag never being specified. Zero is okay (NOP), negative is not.
if ! [[ $length =~ ^[0-9]*$ ]]; then
  echo 'ERROR: wrong argument supplied to -l'
  exit 1
fi

shift $((OPTIND-1))

# Check primary argument(s)
if [[ $# != 1 ]]; then
  echo 'ERROR: specify a single package as argument'
  exit 1
fi

# Determine which repository package is from, and build url
repo=$(pacman -Ss ^${1}$ | sed 's|/.*||;q')

case $repo in
core|extra|testing)
	pre_url='https://projects.archlinux.org/svntogit/packages.git/log/trunk?h=packages/'
	;;
community|multilib)
	pre_url='https://projects.archlinux.org/svntogit/community.git/log/trunk?h=packages/'
	;;
'') # AUR or package not found
	pre_url='https://aur.archlinux.org/cgit/aur.git/log/?h='
	;;
*)
	echo "ERROR: Unknown repository returned (\"${repo}\")" >&2
	exit 1
	;;
esac

post_url='&showmsg=1'

# Parse commit log
output=$(w3m -dump "${pre_url}${1}${post_url}" | sed -n '/Commit message/,/^\s*$/p')

# Check for presence of log
if [ -z "$output" ]; then
	echo "ERROR: no log found for ${1}"
	exit 1
fi

# Truncate if necessary
if [ -z $length ]; then
  echo "$output"
else
  <<<"$output" gawk -v l=$length 'BEGIN {nth=0}; /^\s/ {print; next} {if (nth == l){exit} else {print}; nth++}'
fi

dolik.rce · 2016-01-02 13:18:33

Salkay wrote:

Thanks for all your help! I've written a small script to output the last commit messages for a package. I've also included a flag (-l) to only show the last x commits.

That looks good, I might even use it sometimes too :-) Good work!

I found one little improvement/simplification I overlooked before. To find the correct repo, -Sp can be used with --print-format to give the repo name directly (instead of -Ss and sed):

# Determine which repository package is from, and build url
repo=$(pacman -Sp --print-format %r ${1} 2>/dev/null)

Salkay · 2016-01-09 12:21:19

dolik.rce wrote:

That looks good, I might even use it sometimes too :-) Good work!

Thank you! And thanks for the help. I've uploaded it to AUR, in case it's useful to anyone else.

dolik.rce wrote:

I found one little improvement/simplification I overlooked before.

Nice! I've modified the script appropriately.

Salkay · 2016-01-16 23:35:19

I just realised that my script doesn't work for packages with a different base-package name, for example, qt5-base. For the script, I'd use the string `qt5-base` to show the package is in the extra repository (with `pacman -Sp...` as normal). However, I'd have to use the base-package name, `qt5` to construct the url for the commit log.

In order to determine the base-package name, the only way I know is to parse the qt5-base package webpage. This is possible, but requires an extra download step for every single package, which slows the process. Is there an alternative way that won't require an extra download step? I couldn't seem to find this information in `pacman -S` anywhere.

eschwartz · 2016-01-17 01:24:05

I don't think so, it isn't in the repo database and the main repos have no RPC interface either... although it is in the *.pkg.tar.xz:.PKGINFO

Salkay · 2016-01-17 02:50:30

Okay, thank you. For the purposes of this script, I don't think I'll rely on the presence of the original tarred package, so I guess I'll have to code an extra download step.

x33a · 2016-01-17 11:16:12

I feel that this thread has a place in "Programming & Scripting" instead of Newbie Corner.

Moving.

Arch Linux

#1 2015-12-28 09:59:15

[SOLVED] Commit message for updated packages

#2 2015-12-28 13:29:07

Re: [SOLVED] Commit message for updated packages

#3 2015-12-29 06:02:46

Re: [SOLVED] Commit message for updated packages

#4 2015-12-29 09:48:05

Re: [SOLVED] Commit message for updated packages

#5 2015-12-29 11:31:28

Re: [SOLVED] Commit message for updated packages

#6 2015-12-29 11:35:28

Re: [SOLVED] Commit message for updated packages

#7 2015-12-29 11:57:09

Re: [SOLVED] Commit message for updated packages

#8 2015-12-31 22:37:24

Re: [SOLVED] Commit message for updated packages

#9 2016-01-01 20:56:37

Re: [SOLVED] Commit message for updated packages

#10 2016-01-02 11:36:27

Re: [SOLVED] Commit message for updated packages

#11 2016-01-02 13:18:33

Re: [SOLVED] Commit message for updated packages

#12 2016-01-09 12:21:19

Re: [SOLVED] Commit message for updated packages

#13 2016-01-16 23:35:19

Re: [SOLVED] Commit message for updated packages

#14 2016-01-17 01:24:05

Re: [SOLVED] Commit message for updated packages

#15 2016-01-17 02:50:30

Re: [SOLVED] Commit message for updated packages

#16 2016-01-17 11:16:12

Re: [SOLVED] Commit message for updated packages

Board footer