Cmake section for makepkg.conf

quequotion · 2016-01-08 17:29:43

I think makepkg.conf could use a Cmake section.

Cmake scripts are complex works of build-system obfuscation that are often broken and unintentionally discard CFLAGS, etc (ie, not to override them with developer's selected flags; just neglecting to pass the variable from one script to another). Can't we specify cmake parameters as bash variables?

Here's a start:

#-- CMAKE flags
CMAKE_C_COMPILER="command gcc $CFLAGS"
CMAKE_CXX_COMPILER="command g++ $CXXFLAGS"
CMAKE_LINKER="command ld $LDFLAGS"

This should make sure it uses the proper executable (if there's a better way, let me know) and honor makepkg's cflags.

Trilby · 2016-01-08 17:46:22

Those variables can be defined in the build function of any PKGBUILD building a broken package.

quequotion · 2016-01-09 11:02:44

Trilby wrote:

Those variables can be defined in the build function of any PKGBUILD building a broken package.

Yes of course.

Just considering doing it en-masse for the frequent occurence of broken packages.

Is makepkg considered designed for gnu make only? Can't think of another reason why makepkg shouldn't set up for cmake.

Don't get me wrong, I despise cmake; this just might fix one of its several nightmares.

Trilby · 2016-01-09 13:39:35

quequotion wrote:

Is makepkg considered designed for gnu make only? Can't think of another reason why makepkg shouldn't set up for cmake.

Sorry, I don't know cmake well enough to know what should be defined, but to answer this part, I'd say makepkg should work with any properly used major build system. The CFLAGS variable is not read by make, nor by gcc*: but it is explicitly used by any properly written Makefile. If I used gnu make incorrectly by only using the variable MY_SILLY_CFLAGS in my Makefile and ignoring CFLAGS, then there would be no reason for makepkg to try to work around my silliness: it would be up to the packager to set MY_SILLY_CFLAGS=$CFLAGS in the PKGBUILD.

Now what I don't know about cmake is whether the proper use of cmake will use the variables defined in Makepkg.conf. If it will, then report problems to upstream package developers or to the AUR package maintainers just like one would report MY_SILLY_CFLAGS. If something is missing for the support of the proper use of cmake, then I'd support a bug-report or feature-request against makepkg.conf.

*EDIT: These variables are used by implicit build rules in gnu make. But for anything else, the Makefile author must include these variables themselves on the compiler line, or they will not be included.

EDIT 2: It seems cmake (when properly used, at least) does use CXXFLAGS. So report the lack of use as bugs to the source of those bugs: package developers and/or packagers who misuse cmake. Makepkg can't possibly compensate for every misuse of a tool: there are infinitely many ways to use the tool incorrectly.

Scimmia · 2016-01-09 15:43:04

quequotion wrote:

Is makepkg considered designed for gnu make only? Can't think of another reason why makepkg shouldn't set up for cmake.

You seem to have a serious misunderstanding of what cmake is. It is not an alternative to GNU Make in any way.

quequotion · 2016-01-09 16:10:49

Scimmia wrote:

quequotion wrote:
Is makepkg considered designed for gnu make only? Can't think of another reason why makepkg shouldn't set up for cmake.
You seem to have a serious misunderstanding of what cmake is. It is not an alternative to GNU Make in any way.

No, more like a front-end. I don't understand it well, but I know what it basically does. It's a further level of automation on top of other build scripts--one I find particularly inconvenient to work with.

Scimmia · 2016-01-09 16:11:46

No, it's not a front-end, either. It's a way to generate Makefiles.

quequotion · 2016-01-09 16:24:19

I don't recall which was the first package I noticed this really common problem with, Posibly something in mmug-qq.
At some point I figured out that a lot of cmake pacakges were not respecting makepkg configuration and building default packages (which generally works, but ignores both arch's defaults and user-selected configuration).

I've made several local versions of packages with included scripts to call /usr/bin/{g{cc,++},ld} $MAKEPKGSFLAGS, and specified those as the compilers and linker for cmake in the PKGBUILD.

To test a package, I try adding profile guided optimization to makepkg.conf. If the configuration isn't being respected, the standard build won't respond to changes in $CFLAGS and won't output any .gcda files. If, after hooking up the appproriate variables (usually on the cmake command line with -DMAKEPKGSFLAGS), changes to makepkg configuration are repected--then you are looking at a cmake package that has failed to set those variables appropriately on its own.

quequotion · 2016-01-09 16:27:13

Scimmia wrote:

No, it's not a front-end, either. It's a way to generate Makefiles.

It generates makefiles and build scripts and also executes them; sounds rather front-endish to me, but yeah it does twice the work of your average front-end. I often wonder just what cmake is really intended to do, other than make it extremely difficult to debug faulty build scripts.

Trilby · 2016-01-09 16:55:01

quequotion wrote:

then you are looking at a cmake package that has failed to set those variables appropriately on its own.

OK. Then report it as a packaging bug to that package's upstream source, or the AUR packager.

quequotion · 2016-01-09 17:00:11

Trilby wrote:

OK. Then report it as a packaging but to that packages upstream source, or the AUR packager.

Another valid proposal.
I'm curious, how did we settle on the variables specified on makepkg.conf as they are?
There must be some line between a flag that fixes lots of packages but can't be expected to be included by packagers and a flag so obvious not including it should be reported as a bug.

Trilby · 2016-01-09 17:03:25

Trilby wrote:

I'd say makepkg should work with any properly used major build system.

The variables included are those necessary to make build systems work the way their authors intended them to work. Variables are not included to compensate for the misuse of build systems. I'm getting the feeling you are not reading any of my replies, so I don't think I'll post any more.

Scimmia · 2016-01-09 17:08:47

quequotion wrote:

Scimmia wrote:
No, it's not a front-end, either. It's a way to generate Makefiles.
It generates makefiles and build scripts and also executes them....

Again, no, it doesn't.

quequotion · 2016-01-09 17:31:15

Trilby wrote:

you are not reading any of my replies

This is what I said about your reply, after I read it:

quequotion wrote:

Another valid proposal.

Scimmia wrote:

Again, no, it doesn't.

It doesn't execute make but it does everything autogen and/or configure would do, only much worse. I guess I mean to say its a frontend / replacement for automake.

Last edited by quequotion (2016-01-09 17:36:12)

Scimmia · 2016-01-09 17:32:42

quequotion wrote:

It doesn't execute make but it does everything autogen and/or configure would do...

IE, generate Makefile. Like I said.

CFLAGS, CXXFLAGS, and LDFLAGS are "make" variables, so they are pretty universal. The ones you're proposing are not.

Last edited by Scimmia (2016-01-09 17:36:01)

quequotion · 2016-01-09 17:44:28

Scimmia wrote:

The ones you're proposing are not.

I know this.

I brought it up because I don't think upstream packagers can be relied on to make the changes to fix so many broken packages.
It seems like common practice to ignore the particulars of how cmake scripts get packages built, or perhaps cmake scripts are so complex that it is common to be unaware that they are broken (in this way, since it often does not break compile).

I think it would be simpler to work around it in makepkg.conf, if that kind of thing were possible.

It's not that important to me if it gets implemented.

I'm comfortable having less problems all by myself.

Last edited by quequotion (2016-01-09 17:48:20)

branch · 2016-02-02 20:33:02

While CMake does use CFLAGS, LDFLAGS, and CXXFLAGS from the environment, it uses them differently from autotools. In cmake these environment variables are sampled the first time cmake is run in a given build directory and the values are stored as the cmake cache variables CMAKE_C_FLAGS and CMAKE_CXX_FLAGS. The environment variables are not checked on subsequent runs of cmake in the same build directory, nor by the resultant Makefiles.
For example:

# Works
$ CFLAGS="-D MYVAR" cmake /path/to/src/dir
$ make

# Does not work
$ cmake /path/to/src/dir
$ CFLAGS="-D MYVAR" cmake /path/to/src/dir
$ make

# Does not work
$ cmake /path/to/src/dir
$ CFLAGS="-D MYVAR" make

This is to prevent broken partial rebuilds when compiler flags that affect the ABI are changed.

So in a clean build directory these environment variables will be respected. However, if you build the package, change the flags in makepkg.conf and rebuild the package then the changes to makepkg.conf will not be reflected in the new package. Deleting CMakeCache.txt from the build directory will cause the environment variables to be re-sampled and a full rebuild to be performed.

edit: removed erroneous cmake linker flag variable name

Last edited by branch (2016-02-02 22:44:02)

eschwartz · 2016-02-02 21:59:36

Modifying CMAKE_C_COMPILER wouldn't help, surely... it would just make the cache have a different set of stale values, wouldn't it?

On the other hand, this will always pass the current $CFLAGS into cmake, and force a full rebuild if necessary.

cmake -DCMAKE_C_FLAGS:STRING="${CFLAGS}" -DCMAKE_CXX_FLAGS:STRING="${CXXFLAGS}" /path/to/src

repeated invocations of makepkg should always rerun cmake...

...

Now, perhaps it could be argued that cmake should do this automatically, rather than sticking with the old value. But clearly you can get cmake to respect $CFLAGS just fine.

So what we really need is better education for packagers of cmake projects, or (:lol:) maybe do as some other distros do and provide our own wrappers for configure, make, cmake, etc.

branch · 2016-02-02 22:41:46

Eschwartz wrote:

On the other hand, this will always pass the current $CFLAGS into cmake, and force a full rebuild if necessary.
cmake -DCMAKE_C_FLAGS:STRING="${CFLAGS}" -DCMAKE_CXX_FLAGS:STRING="${CXXFLAGS}" /path/to/src
repeated invocations of makepkg should always rerun cmake...

Agreed, if the PKGBUILD is written as you suggest the behavior will be correct. Also, it seems LDFLAGS should be propagated to CMAKE_EXE_LINKER_FLAGS and CMAKE_SHARED_LINKER_FLAGS.

Perhaps such guidance should be added to the wiki?

Trilby · 2016-02-02 22:46:14

Can you clarify what problem this would solve? If there is a PKGBUILD that calls cmake twice, the much easier fix would be to remove the redundant (and apparently problematic) call to cmake.

If any guidance should be added to the wiki, it should be information on the proper use of cmake - not how to get away with using it improperly.

eschwartz · 2016-02-03 00:00:06

Trilby wrote:

Can you clarify what problem this would solve? If there is a PKGBUILD that calls cmake twice, the much easier fix would be to remove the redundant (and apparently problematic) call to cmake.
If any guidance should be added to the wiki, it should be information on the proper use of cmake - not how to get away with using it improperly.

How about... any time you rerun makepkg?

I don't usually bother to use `makepkg -C`, especially not when dealing with a *-git package... I like my timesaving incremental builds.
In fact, the only package I currently build from the AUR that uses cmake, is sigil-git.

So presumably, if it didn't already include these, I might find that changing the makepkg.conf flags had no effect. And not even realize it until much later (if ever).

Last edited by eschwartz (2016-02-03 00:00:20)

branch · 2016-02-03 00:05:04

The issue I see (and I don't know if it's the problem the OP actually had) is when CFLAGS, CXXFLAGS, or LDFLAGS are changed in makepkg.conf and a package is rebuilt with the old build directory still in place. In this case, the new package is built with the old flags. I am not sure if anyone actually considers this a problem, but people may want to be aware of the behavior.

A related issue is that cmake does not in any way use CPPFLAGS, so for a package to use this variable from makepkg.conf the PKGBUILD must explicitly add CPPFLAGS to both CMAKE_C_FLAGS and CMAKE_CXX_FLAGS. So currently the default value of '-D_FORTIFY_SOURCE=2' in makepkg.conf is being ignored by any package using cmake that does not explicitly do this.

Trilby · 2016-02-03 00:10:18

You can't keep the timesaving of incrememental builds and apply changes from makepkg.conf. For those changes to apply, you need to rebuild.

In fact, if you change your makepkg.conf build flags, then do an incrememental build with the new flags, you could very likely cause significant breakage as different modules would be build with different build flags. In such cases, the options of rebuild the whole package, or keep the original buildflags are the only sane options.

eschwartz · 2016-02-03 00:23:13

Trilby wrote:

You can't keep the timesaving of incrememental builds and apply changes from makepkg.conf. For those changes to apply, you need to rebuild.
In fact, if you change your makepkg.conf build flags, then do an incrememental build with the new flags, you could very likely cause significant breakage as different modules would be build with different build flags. In such cases, the options of rebuild the whole package, or keep the original buildflags are the only sane options.

Which is why it is nice that cmake does recreate everything that is affected by those flags.

Things that don't get rebuilt: automoc, translations, rcc...
So still a timesaver.

...

But if it didn't know to rebuild everything that depended on those flags, I expect I would realize when things failed to compile.
I know I didn't remember to do a cleanbuild for at least one package when the C++ ABI thing happened, and the large error messages during the build made it clear enough.

Last edited by eschwartz (2016-02-03 00:24:28)

branch · 2016-02-03 03:21:15

I would suggest cmake guidelines for C/C++ as the following:

1) Set the following definitions in adition to setting CMAKE_INSTALL_PREFIX:

cmake -DCMAKE_C_FLAGS:STRING="${CFLAGS} ${CPPFLAGS}" \
  -DCMAKE_CXX_FLAGS:STRING="${CXXFLAGS} ${CPPFLAGS}" \
  -DCMAKE_EXE_LINKER_FLAGS:STRING="${LDFLAGS}" \
  -DCMAKE_SHARED_LINKER_FLAGS:STRING="${LDFLAGS}"

2) Do not set CMAKE_BUILD_TYPE, as it may add options that overide those in makepkg.conf.

3) If the PKGBUILD performs an out of source build, which is highly recommended by the cmake authors, the build directory should be a subdirectory of the source directory so that "makepkg -C" still works as intended.

Arch Linux

#1 2016-01-08 17:29:43

Cmake section for makepkg.conf

#2 2016-01-08 17:46:22

Re: Cmake section for makepkg.conf

#3 2016-01-09 11:02:44

Re: Cmake section for makepkg.conf

#4 2016-01-09 13:39:35

Re: Cmake section for makepkg.conf

#5 2016-01-09 15:43:04

Re: Cmake section for makepkg.conf

#6 2016-01-09 16:10:49

Re: Cmake section for makepkg.conf

#7 2016-01-09 16:11:46

Re: Cmake section for makepkg.conf

#8 2016-01-09 16:24:19

Re: Cmake section for makepkg.conf

#9 2016-01-09 16:27:13

Re: Cmake section for makepkg.conf

#10 2016-01-09 16:55:01

Re: Cmake section for makepkg.conf

#11 2016-01-09 17:00:11

Re: Cmake section for makepkg.conf

#12 2016-01-09 17:03:25

Re: Cmake section for makepkg.conf

#13 2016-01-09 17:08:47

Re: Cmake section for makepkg.conf

#14 2016-01-09 17:31:15

Re: Cmake section for makepkg.conf

#15 2016-01-09 17:32:42

Re: Cmake section for makepkg.conf

#16 2016-01-09 17:44:28

Re: Cmake section for makepkg.conf

#17 2016-02-02 20:33:02

Re: Cmake section for makepkg.conf

#18 2016-02-02 21:59:36

Re: Cmake section for makepkg.conf

#19 2016-02-02 22:41:46

Re: Cmake section for makepkg.conf

#20 2016-02-02 22:46:14

Re: Cmake section for makepkg.conf

#21 2016-02-03 00:00:06

Re: Cmake section for makepkg.conf

#22 2016-02-03 00:05:04

Re: Cmake section for makepkg.conf

#23 2016-02-03 00:10:18

Re: Cmake section for makepkg.conf

#24 2016-02-03 00:23:13

Re: Cmake section for makepkg.conf

#25 2016-02-03 03:21:15

Re: Cmake section for makepkg.conf

Board footer