You are not logged in.

#1 2018-08-19 18:06:09

ganileni
Member
Registered: 2017-07-11
Posts: 4

speeding up node-gyp aur package builds

I need to have a lot of AUR packages installed on my computer that require a node-gyp build (node-gyp being a node.js utility that builds software). Every time I update my machine the process takes a lot of time.

Now, I know that node-gyp can do builds faster by using multiple threads. The way to do that is to set an environment variable like this:

$ export JOBS=max

before starting node-gyp. So one way to activate multicore build by default would be to add `export JOBS=max` to my `~/.bashrc`, but I don't like it because it's not very clean, since the variable would be set everytime I use bash.

Is there a way to have makepkg automatically set the env variable when it's launched, and delete it after? maybe touching `/etc/makepkg.conf`?

(I thought that one could write a bash script that sets and deletes the env variable when calling makepkg, but it would require changing the file in `/usr/bin` and it looks brittle - the file could be overwritten anytime.)

Offline

#2 2018-08-19 18:32:57

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: speeding up node-gyp aur package builds

makepkg.conf is a bash-compatible configuration file which is sourced as bash, so you could just set it there...

But you never know when some Makefile for an unrelated program will use JOBS as an internal variable and your exporting it will break everything. See e.g. https://lists.archlinux.org/pipermail/a … 31314.html

You never know what will set something off, and therefore the best way to export variables is to do so *carefully*, using either standards or namespaces.

...

Pity that there's no standard for setting this. Well, there is MAKEFLAGS=-j$(nproc) and most cheap make imitations support -j, but unfortunately that doesn't help when they don't read $MAKEFLAGS, also $MAKEFLAGS can have any other make options as well (even though it very rarely does)...


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#3 2018-08-20 11:03:06

ganileni
Member
Registered: 2017-07-11
Posts: 4

Re: speeding up node-gyp aur package builds

Thanks for the answer.

We'll have to do a feature request on makepkg then. Or, if we have time, write our own patch and then a pull request smile

For the moment, I settled for a script that does all the updates chores [i.e. it runs pacman -Syu, then my AUR helper with -Syu, then cleans cache with -Scc and removes useless dependencies with -Rsn $(pacman -Qdtq)]. This script sets JOBS=max when run, and then puts it back to whatever original value it had on exit (on my bash it is empty by default). If some makefile or other script tries to change JOBS during update, it should in theory still update fine, but it will prevent multicore building on node-gyp. Let's see which bugs come up while I test this partial solution. [Note: as Eschwartz said before, this is NOT guaranteed to work safely]

p.s.: this is not exactly a bug, and we didn't exactly solve it. Should I update the thread's title to [SOLVED]? (sorry for being a newbie)

Offline

Board footer

Powered by FluxBB