Clyde - A better libalpm/makepkg wrapper

quarkup · 2011-02-03 11:39:21

Found issue (posted on github)

SPLIT-PKG support (ie: http://aur.archlinux.org/packages.php?ID=32908 )

juster · 2011-02-04 22:52:54

Hund wrote:

:: vala-git conflicts with vala. Remove vala? [Y/n]
error: failed to prepare transaction (conflicting dependencies)
Im getting this error everytime it wants to replace some deps.

That should be fixed now. The same with SpeederCat's corrupt package bug.

juster · 2011-02-04 23:01:24

I thought I'd reply some more.

Jimi wrote:

Sorry if this has probably been asked before, but is there a way to make the package compression multi-threaded?

Do you mean "package building" instead? That would make more sense, since package compression only happens when building packages. I imagine the compression is the least time-consuming part in building packages. Building package would be nice to do in parallel, but no it is not implemented.

hellnest wrote:

Is there any config to "move" the default "tmp" folder to another directory?

rekado on github sent me a pull request that allows you to override the build directory. I have it working along with other code refactoring but I need to test it before merging it to the master branch. I think I will push it as the test branch in the meantime.

johanbcn · 2011-02-05 21:38:13

bratmaxe wrote:

I have two Bugs to report:

1. Replacing conflicting packages with each other does not work:

Example:

resolving dependencies...
looking for inter-conflicts...
:: libxft conflicts with libxft-lcd. Remove libxft-lcd? [Y/n] y
error: failed to prepare transaction (conflicting dependencies)
:: libxft: conflicts with libxft-lcd (libxft)

This is still happening, specially with AUR packages...

Last edited by johanbcn (2011-02-05 21:39:57)

juster · 2011-02-06 00:44:08

It works for me...

==> Finished making: libxft-lcd 2.2.0-1 (Sat Feb  5 16:40:37 PST 2011)
loading package data...
resolving dependencies...
looking for inter-conflicts...
:: libxft-lcd conflicts with libxft. Remove libxft? [Y/n] y
checking package integrity...
(1/1) checking for file conflicts                  [#######################] 100%
(1/1) removing libxft                              [#######################] 100%
(1/1) installing libxft-lcd                        [#######################] 100%
[juster@artemis clyde]$

Make sure you are reinstalling clyde from the AUR. Anyone else having this problem still?

edit: and in case someone wants to be a smarty pants, yes the opposite (removing libxft-lcd when in conflict) also works for me.

Last edited by juster (2011-02-06 00:46:58)

Jimi · 2011-02-06 09:32:22

juster wrote:

Jimi wrote:
Sorry if this has probably been asked before, but is there a way to make the package compression multi-threaded?
Do you mean "package building" instead? That would make more sense, since package compression only happens when building packages. I imagine the compression is the least time-consuming part in building packages. Building package would be nice to do in parallel, but no it is not implemented.

Yeah I do mean package building. To my knowledge only p7zip the only utility currently capable of compressing to .xz using multiple threads.

Last edited by Jimi (2011-02-06 09:34:42)

juster · 2011-02-06 13:35:54

Well Jimi, to be honest... when building packages, all clyde does is use the makepkg script that comes with pacman. This is why I had not considered multi-threaded compression because it would have to be implemented by makepkg. This could be as easy as replacing the command line that compresses the package directory. If you know bash programming just open /usr/bin/makepkg in an editor and check it out! Personally, I would rather have multiple makepkg processes running in parallel. I think this would be even faster.

Jimi · 2011-02-07 11:28:44

Never knew makepkg was simply bash. I took a look at it and it's pretty simple to switch out the xz command with the proper 7z command. Shame is for some reason 7z won't xz a pkg.tar, which is strange. Looks like we'll have to wait for xz-utils to support multithreaded compression.

Looks like making them run in parallel is the best choice out there currently.

Last edited by Jimi (2011-02-07 11:30:23)

juster · 2011-02-11 15:27:48

I updated clyde last night. I tried to change the WARNING message to be less obnoxious. Let me know if you don't like the way it looks and I'll change it back. Rekado's patch from github was incorporated. This adds a --builddir option to syncing and adds a BuildDir configure file field. Thanks rekado! Here is the Changelog entry:

* Version 0.3.10 [2011-02-10 Thu]

** AUR Customize Messages

The great big warning has been changed to yellow and now only the arrow
next to it blinks. And there was much rejoicing. A new "Customizing..."
status message was added just to be complete...

** AUR Build Directory is Overridable                                   :NEW:

You can override the build directory now. The build directory is where
source packages are extracted to. Alot of the AUR code was re-written
and moved to the aur.lua file. Keen!

The build directory must exist, clyde will not create it for you.
Thanks to "rekado" on github for the patch that started it all.

** Change Log                                                           :NEW:

I started this change log because I tend to forget what I change. I
wanted a higher-level view of changes than git commit logs give
me. This text file uses org-mode in Emacs.

ffjia · 2011-02-12 01:39:28

Thanks, "--builddir" switch is very convenient to me.

vkumar · 2011-02-12 02:23:54

Hi. I just built clyde from the AUR a couple seconds ago. Here's the error I'm getting;

[vk@vk ~]$ sudo clyde -Rsuc
Password: 
You do not yet have a clyde.conf, would you like to create one? [Y/n] y
Please enter your default text editor: 
You entered '' as your default editor, is this correct?  [Y/n] n
Editor: geany
You entered 'geany' as your default editor, is this correct?  [Y/n] 
lua: /usr/bin/clyde:672: attempt to call field 'getbuilduser' (a nil value)
stack traceback:
    /usr/bin/clyde:672: in function 'config_init'
    /usr/bin/clyde:983: in function 'main'
    /usr/bin/clyde:1112: in main chunk
    [C]: ?

Sorry if this has been posted already. Thanks for all the work.

akurei · 2011-02-12 21:28:39

vkumar wrote:

Hi. I just built clyde from the AUR a couple seconds ago. Here's the error I'm getting;

[vk@vk ~]$ sudo clyde -Rsuc
Password: 
You do not yet have a clyde.conf, would you like to create one? [Y/n] y
Please enter your default text editor: 
You entered '' as your default editor, is this correct?  [Y/n] n
Editor: geany
You entered 'geany' as your default editor, is this correct?  [Y/n] 
lua: /usr/bin/clyde:672: attempt to call field 'getbuilduser' (a nil value)
stack traceback:
    /usr/bin/clyde:672: in function 'config_init'
    /usr/bin/clyde:983: in function 'main'
    /usr/bin/clyde:1112: in main chunk
    [C]: ?

Sorry if this has been posted already. Thanks for all the work.

Same here.

rotave · 2011-02-12 22:48:28

akurei wrote:

vkumar wrote:

Hi. I just built clyde from the AUR a couple seconds ago. Here's the error I'm getting;

[vk@vk ~]$ sudo clyde -Rsuc
Password: 
You do not yet have a clyde.conf, would you like to create one? [Y/n] y
Please enter your default text editor: 
You entered '' as your default editor, is this correct?  [Y/n] n
Editor: geany
You entered 'geany' as your default editor, is this correct?  [Y/n] 
lua: /usr/bin/clyde:672: attempt to call field 'getbuilduser' (a nil value)
stack traceback:
    /usr/bin/clyde:672: in function 'config_init'
    /usr/bin/clyde:983: in function 'main'
    /usr/bin/clyde:1112: in main chunk
    [C]: ?

Sorry if this has been posted already. Thanks for all the work.

Same here.

I'm getting this error too. I've just started using Arch and thought it was something I might have done! Any help on fixing this would be appreciated.

thecdn · 2011-02-14 02:09:26

I've been using clyde for awhile and just noticed this error in a new install. Glad to see it's not just me.

aloyr · 2011-02-14 04:56:05

just started with arch and got this error with clyde... had no idea it wasn't something i did until i hit this post...

anonymous_user · 2011-02-14 06:06:43

Does Clyde output messages when a package cannot be found or a mirror is down? iirc, Clyde silently just tries the next mirror and doesn't tell the user. This behavior should be changed.

juster · 2011-02-14 15:31:32

vkumar wrote:

Hi. I just built clyde from the AUR a couple seconds ago. Here's the error I'm getting;

[vk@vk ~]$ sudo clyde -Rsuc
Password: 
You do not yet have a clyde.conf, would you like to create one? [Y/n] y
Please enter your default text editor: 
You entered '' as your default editor, is this correct?  [Y/n] n
Editor: geany
You entered 'geany' as your default editor, is this correct?  [Y/n] 
lua: /usr/bin/clyde:672: attempt to call field 'getbuilduser' (a nil value)
stack traceback:
    /usr/bin/clyde:672: in function 'config_init'
    /usr/bin/clyde:983: in function 'main'
    /usr/bin/clyde:1112: in main chunk
    [C]: ?

Sorry if this has been posted already. Thanks for all the work.

Sorry I didn't test creating a new config file. I missed one getbuilduser function call. I moved getbuilduser to another file and renamed it, causing this problem. This is fixed now in version 0.03.11. The forum also did not mail me a notification for the topic... I got an email from github thanks to thechitownscubs reporting it there. Thanks for posting it here too.

juster · 2011-02-14 15:39:31

anonymous_user wrote:

Does Clyde output messages when a package cannot be found or a mirror is down? iirc, Clyde silently just tries the next mirror and doesn't tell the user. This behavior should be changed.

Are you talking about searching (-Ss) or syncing (-S without -s)? Searching matching pacman's behavior of not printing anything if a matching package is not found. Syncing has similar behavior to pacman's. The message should probably be changed from "<package> not found in sync db", though.

[juster@artemis clyde]$ sudo ./clyde -S lkasjdf
:: lkasjdf package not found, searching for group...
:: lkasjdf group not found, searching AUR...
error: 'lkasjdf': not found in sync db
[juster@artemis clyde]$ sudo pacman -S lkasjdf
error: 'lkasjdf': could not find or read package
[juster@artemis clyde]$

I find the :: messages a little spammy and unnecessary myself. Adding more messages like :: "lkasjdf" not found in [core] and :: "lkasjdf" not found in [extra] seems spammy and unnecessary. I'm not sure if this is what you mean, though. Better error reporting for downed mirrors could be nice but does it really make much different which mirror of a repository is used? Simple examples of input and output would help me to understand what you mean, specifically.

Ramses de Norre · 2011-02-14 15:54:45

I'm having trouble updating clyde:

clyde -S clyde-git
<snip>
==> Leaving fakeroot environment.
==> Finished making: clyde-git 20110214-1 (Mon Feb 14 16:53:31 CET 2011)
lua: ./clydelib/aur.lua:299: attempt to index global 'alpm' (a nil value)
stack traceback:
        ./clydelib/aur.lua:299: in function <./clydelib/aur.lua:298>
        [C]: in function 'sort'
        ./clydelib/aur.lua:302: in function 'installpkg'
        ./clydelib/sync.lua:1140: in function <./clydelib/sync.lua:1070>
        (tail call): ?
        ./clydelib/sync.lua:1547: in function 'clyde_sync'
        ./clydelib/sync.lua:1553: in function <./clydelib/sync.lua:1552>
        (tail call): ?
        /usr/bin/clyde:1054: in function 'main'
        /usr/bin/clyde:1112: in main chunk
        [C]: ?

Currently running

    .--.          .---.        Clyde v0.03.10 - lualpm v0.02
   '-._ \ .-.  .-|O O  | ~~    Copyright (C) 2010 Robert Djubek
   .-'  / '-'  '-|~~~  |  ~~~  This program may be freely redistributed under
    '--'         |.-.-.| ~~    the terms of the GNU General Public License.

EDIT: see pull request at github.

Last edited by Ramses de Norre (2011-02-14 16:10:46)

anonymous_user · 2011-02-14 17:27:51

@juster - I'm talking about a scenario where you are syncing using an outdated mirror (and you get file not found due to version mismatches) like this:

http://img834.imageshack.us/img834/9333 … ecuzio.png

or the mirror itself is down or nonexistant:

~ $ sudo pacman -S vim
resolving dependencies...
looking for inter-conflicts...

Targets (2): vim-runtime-7.3.102-1 [4.20 MB]  vim-7.3.102-1 [0.84 MB]

Total Download Size:    5.03 MB
Total Installed Size:   26.82 MB

Proceed with installation? [Y/n] y
:: Retrieving packages from extra...
error: failed retrieving file 'vim-runtime-7.3.102-1-i686.pkg.tar.xz' from archlinux.unixheads.org : File unavailable (e.g., file not found, no access)
error: failed retrieving file 'vim-7.3.102-1-i686.pkg.tar.xz' from archlinux.unixheads.org : File unavailable (e.g., file not found, no access)
warning: failed to retrieve some files from extra
error: failed to commit transaction (File unavailable (e.g., file not found, no access))
Errors occurred, no packages were upgraded.
~ $ sudo clyde -S vim
resolving dependencies...
looking for inter-conflicts...

Targets (2): vim-runtime-7.3.102-1 [4.20 MB]  vim-7.3.102-1 [0.84 MB]  

Total Download Size:    5.03 MB
Total Installed Size:   26.82 MB

==> Proceed with installation? [Y/n] y
:: Retrieving packages from extra...
~ $

To me, it doesn't matter what mirror is used, but I'd at least like to know if a mirror is down so I can change my mirrorlist.

juster · 2011-02-14 22:41:39

RamsesDeNorre: Your patch has been merged (current version is now 0.03.12). I wish I knew why I can't reproduce that bug. Thanks for the patch!

anonymous_user: That sounds reasonable I will add it to my TODO list. I will see how pacman accomplishes that and maybe copy it.

Berseker · 2011-02-19 08:38:44

juster have you taken a look also at this behaviour of clyde?
https://bbs.archlinux.org/viewtopic.php … 33#p864333
maybe you could fix that in one of next releases!

FrozenFox · 2011-02-19 08:51:45

It seems every package I try to install from AUR that drags in other AUR dependencies fails to work with clyde now. For this example, sudo clyde -S indicator-pastie will do. It, along with others I've tried, returns this same message:

error: /usr/share/lua/5.1/clydelib/aur.lua:246: attempt to concatenate upvalue 'mkpkgopts' (a nil value)

Anyone else have such trouble?

EDIT: Appears to be resolved by today's git update. Thanks

Last edited by FrozenFox (2011-02-20 04:02:50)

Ramses de Norre · 2011-02-19 09:19:00

Berseker wrote:

juster have you taken a look also at this behaviour of clyde?
https://bbs.archlinux.org/viewtopic.php … 33#p864333
maybe you could fix that in one of next releases!

Berseker wrote:

Two questions, I don't know if it's a lack on Clyde functionality or if I missed the informations on wiki\man page
see this:
berseker@mysystem /tmp/clyde-berseker $ ls
clyde-git
berseker@mysystem /tmp/clyde-berseker $ cd clyde-git/
berseker@mysystem /tmp/clyde-berseker/clyde-git $ ls
clyde-git clyde-git.tar.gz
berseker@mysystem /tmp/clyde-berseker/clyde-git $ cd clyde-git/
berseker@mysystem /tmp/clyde-berseker/clyde-git/clyde-git $ ls
clyde-git-20101211-1-i686.pkg.tar.gz pkg PKGBUILD src
berseker@mysystem /tmp/clyde-berseker/clyde-git/clyde-git $
Can Clyde be modified so that the downloaded source file is erased, and files directly extracted in the current directory instead of creating that (IMHO) ugly directory/directory thing?
Also, can the directory where clyde downloads and packages all the AUR\ABS stuff been modified in some way in the clyde.conf or I have to use symlinks?

1) I think it's a feature that the source is kept, I often do a few rebuilds of packages in which case it is very convenient not to have to download the source over and over again.

2) clyde -S clyde-git and have a look at the --builddir option.

Ramses de Norre · 2011-02-19 09:58:45

juster wrote:

RamsesDeNorre: Your patch has been merged (current version is now 0.03.12). I wish I knew why I can't reproduce that bug. Thanks for the patch!

It happens only when an already built package with a different version than the currently being built package is present in the builddir (like a left over from a previous build). Only then the code containing the alpm call (and also the pkg_ver code I fixed some days ago) is called through the tables.sort() call.

Arch Linux

#551 2011-02-03 11:39:21

Re: Clyde - A better libalpm/makepkg wrapper

#552 2011-02-04 22:52:54

Re: Clyde - A better libalpm/makepkg wrapper

#553 2011-02-04 23:01:24

Re: Clyde - A better libalpm/makepkg wrapper

#554 2011-02-05 21:38:13

Re: Clyde - A better libalpm/makepkg wrapper

#555 2011-02-06 00:44:08

Re: Clyde - A better libalpm/makepkg wrapper

#556 2011-02-06 09:32:22

Re: Clyde - A better libalpm/makepkg wrapper

#557 2011-02-06 13:35:54

Re: Clyde - A better libalpm/makepkg wrapper

#558 2011-02-07 11:28:44

Re: Clyde - A better libalpm/makepkg wrapper

#559 2011-02-11 15:27:48

Re: Clyde - A better libalpm/makepkg wrapper

#560 2011-02-12 01:39:28

Re: Clyde - A better libalpm/makepkg wrapper

#561 2011-02-12 02:23:54

Re: Clyde - A better libalpm/makepkg wrapper

#562 2011-02-12 21:28:39

Re: Clyde - A better libalpm/makepkg wrapper

#563 2011-02-12 22:48:28

Re: Clyde - A better libalpm/makepkg wrapper

#564 2011-02-14 02:09:26

Re: Clyde - A better libalpm/makepkg wrapper

#565 2011-02-14 04:56:05

Re: Clyde - A better libalpm/makepkg wrapper

#566 2011-02-14 06:06:43

Re: Clyde - A better libalpm/makepkg wrapper

#567 2011-02-14 15:31:32

Re: Clyde - A better libalpm/makepkg wrapper

#568 2011-02-14 15:39:31

Re: Clyde - A better libalpm/makepkg wrapper

#569 2011-02-14 15:54:45

Re: Clyde - A better libalpm/makepkg wrapper

#570 2011-02-14 17:27:51

Re: Clyde - A better libalpm/makepkg wrapper

#571 2011-02-14 22:41:39

Re: Clyde - A better libalpm/makepkg wrapper

#572 2011-02-19 08:38:44

Re: Clyde - A better libalpm/makepkg wrapper

#573 2011-02-19 08:51:45

Re: Clyde - A better libalpm/makepkg wrapper

#574 2011-02-19 09:19:00

Re: Clyde - A better libalpm/makepkg wrapper

#575 2011-02-19 09:58:45

Re: Clyde - A better libalpm/makepkg wrapper

Board footer