Official firefox-pgo thread

gav616 · 2010-02-14 19:47:48

harryNID wrote:

Wee it's not just me.
Let me ask you a question. Did you compile GTK2 yourself? If so then there's your culprit. Load the stock version and everything works, build your own and Organize Bookmarks go wonky and crashes firefox. Every time I try to compile GTK2 myself everything works but I can't use any of the Bookmark features (Organize Bookmarks, Bookmarks Sidebar etc.). I have no clue as to why unless it has something to do with how GTK2 is compiled. I have no other issues or errors with my compiled version of GTK2 yet firefox apparently doesn't like it.

I'm compiling with the stock gtk2, firefox in core works fine though.

its not gtk2 then....

Last edited by gav616 (2010-02-14 19:48:32)

davidm · 2010-02-14 20:53:30

Marcel- wrote:

davidm wrote:

What are you using for CFLAGS and what processor have you?

I was using the default ones:

# Changing the user's optimization flags is justified, because this is
# a package specifically for an optimized software build, and because of
# the official branding, binaries can't be redistributed anyways.
# These flags just set guidelines for the build, they are overridden in
# most compile job pieces by Firefox's better judgement.
export CFLAGS="-march=native -O2 -pipe"
export CXXFLAGS="-march=native -O2 -pipe"

on an Intel(R) Atom(TM) CPU N270 @ 1.60GHz stepping 02.

See this post within this thread: http://bbs.archlinux.org/viewtopic.php? … 75#p685875

You probably need something like:

export CFLAGS="-march=i686 -mtune=atom -O2 -pipe -fomit-frame-pointer"
export CXXFLAGS="-march=i686 -mtune=atom -O2 -pipe -fomit-frame-pointer"

Last edited by davidm (2010-02-14 20:59:51)

harryNID · 2010-02-15 19:59:26

gav616
I'm compiling with the stock gtk2, firefox in core works fine though.
sad its not gtk2 then....

Hmm! That is weird. Mine seems to work fine as long as I don't mess with GTK. Have you compiled any of these (or are you using a git/svn version of any of these)?

:
* atk>=1.28.0
* cairo>=1.8.8-2
* gnutls>=2.8.3
* heimdal>=1.3.1
* libcups>=1.4.2-3
* libtiff>=3.9.2-2
* libxcomposite
* libxcursor
* libxdamage
* libxi>=1.2.1
* libxinerama
* libxrandr>=1.3.0
* pango>=1.26.1
* shared-mime-info

or

:
* alsa-lib
* cairo dbus-glib>=0.82
* desktop-file-utils
* gcc-libs>=4.4
* gtk2>=2.18.0
* hunspell>=1.2.8
* libidl2>=0.8.13
* libnotify>=0.4
* libpng>=1.4.0
* libxt mime-types
* mozilla-common
* nss>=3.12.4
* startup-notification>=0.10

If not then maybe try to recompile it. Other than that I don't know what could be going on. I have a theory as to my issue with GTK but I want to test it first before I comment on it.

Good Luck!! Let us know if you find a fix or if recompiling worked.

Oh, if you are using a git/svn version(s) of one of the above then try the stock version(s). I just fixed a big "FUBAR" yesterday because I recompiled nss with volatile CFLAGS. Man, I won't do that again!!!!!!!! I had a hell of a time trying to figure out why firefox (stock and pgo) wouldn't work. Reloading the stock version of nss was the ticket.

Marcel- · 2010-02-15 21:27:49

davidm wrote:

You probably need something like:

export CFLAGS="-march=i686 -mtune=atom -O2 -pipe -fomit-frame-pointer"
export CXXFLAGS="-march=i686 -mtune=atom -O2 -pipe -fomit-frame-pointer"

I tried this (after a complete reinstall of Arch, because 'Security Tool' (not! ) and some virusscanner completely b0rked Bill's system, which wouldn't recover with another primary partition on my HDD), but GCC complained that it couldn't compile with these settings. A little search showed me that only GCC 4.5 supports -mtune=atom (the one Arch installs is 4.4). The same page on Stack Overflow led me to Gentoo's Wiki which recommended:

CFLAGS="-O2 -march=core2 -mtune=generic -mssse3 -mfpmath=sse -fomit-frame-pointer -pipe"
CXXFLAGS="${CFLAGS}"

Tried that, the first compilation run halted because it couldn't find a "screen", apparently X is needed to compile. Started X, now I get another error:

OBJDIR=/tmp/yaourt-tmp-mk/aur-firefox-pgo/firefox-pgo/src/mozilla-1.9.2/ff-pgo python /tmp/yaourt-tmp-mk/aur-firefox-pgo/fiefox-pgo/src/mozilla-1.9.2/ff-pgo/_profile/pgo/profileserver.py
INFO | automation.py | Application pid: 16476
Xlib:  extension "RANDR" missing on display ":99.0".
localhost.localdomain - - [15/Feb/2010 19:29:17] "GET /index.html HTTP/1.1" 200 -
TEST-UNEXPECTED-FAIL | automation.py | Exited with code -11 during test run
INFO | automation.py | Application ran for: 0:00:06.580611
make: *** [profiledbuild] Error 245
/usr/bin/makepkg: line 756:  4973 Killed                  LD_PRELOAD="" /usr/bin/Xvfb -nolisten tcp -extension GLX :99
==> ERROR: Build Failed.
    Aborting...
Error: Makepkg was unable to build firefox-pgo package.

Tried again, similar error. Now I will try building without jemalloc PGO.

EDIT: oh, I should have read better.

Xlib:  extension "RANDR" missing on display ":99.0".

Hmm, xrandr is present, I'll do a search again... and found this thread. Added '-extension RANDR' to the Xvfb line in PKGBUILD, now compiling again.

EDIT 2: no, that didn't do the trick. I searched a bit again and tested:

# grep -ni randr /var/log/Xorg.0.log
173:(II) intel(0): RandR 1.2 enabled, ignore the following RandR disabled message.
178:(--) RandR disabled
192:(II) Initializing built-in extension RANDR

Strange, those messages. Although I should ignore the message in line 178, could it still be related to this problem? For the record, I use xf86-video-intel 2.10.0-1 (working with the required KMS), one monitor connected through a DVI-cable, without Xinerama.

Last edited by Marcel- (2010-02-15 22:28:39)

davidm · 2010-02-15 22:13:46

Marcel

It looks like you are using yaoart? You may want to try it just using makepkg manually (it's actually very easy) so as to eliminate another source of possible problems.. Also when you are using yaoart you aren't trying it as root (su'ed from a terminal), are you? With a stock config I got that error when trying it as root as well when I first tried it.... Attempting it again as a normal user with a valid X session running and properly configured solved the issue. Hopefully it really is that simple for you.

Marcel- · 2010-02-15 22:31:38

@Davidm: Yes, I'm using Yaourt (I reported about the RANDR error messages again above). I will try it using makepkg.

(And erm..., yes, using sudo.)

davidm · 2010-02-15 23:44:06

Marcel- wrote:

@Davidm: Yes, I'm using Yaourt (I reported about the RANDR error messages again above). I will try it using makepkg.
(And erm..., yes, using sudo.)

Don't use su or sudo when doing makepkg. run it with the PGO as a normal user with an X session active. You only need to use it when invoking pacman to actually install the new tarball generated. I think this might be your issue.

add: Of course I set up my system so that sudo can be invoked automatically without a password being needed by makepkg. I thought I had better mention that.

Last edited by davidm (2010-02-16 00:59:23)

Marcel- · 2010-02-16 11:38:35

Thanks for the tip, makepkg is very easy indeed (and doesn't throw files away automatically, and yes, I also set up the sudo trick for makepkg -s).

Unfortunately, I ran into the same error message again (Xlib: extension "RANDR" missing on display ":99.0").

EDIT: adding '-extension XRANDR' to the Xvfb command resulted in the same error message, replacing 'GLX :99' with 'XRANDR' results in:

OBJDIR=/home/mk/firefox-pgo/src/mozilla-1.9.2/ff-pgo python /home/mk/firefox-pgo/src/mozilla-1.9.2/ff-pgo/_profile/pgo/profileserver.py
INFO | automation.py | Application pid: 30574
Error: cannot open display: :99
TEST-UNEXPECTED-FAIL | automation.py | Exited with code 1 during test run
INFO | automation.py | Application ran for: 0:00:00.196024
make: *** [profiledbuild] Error 1
/home/mk/firefox-pgo/PKGBUILD: line 142: kill: (19077) - No such process

Which leads me to the question: is "display: :99" correct? And I can't figure out what the GLX extension in Xvfb does, but don't I have to add something in xorg.conf for that? Just guessing...

EDIT2: no, GLX is activated, according to Xorg.0.log.

Last edited by Marcel- (2010-02-16 13:40:27)

Ranguvar · 2010-02-16 16:43:14

The RANDR missing message can be safely ignored.
The problem is not with Xvfb, the problem is that the PGO tests are failing after the first page:

localhost.localdomain - - [15/Feb/2010 19:29:17] "GET /index.html HTTP/1.1" 200 -
TEST-UNEXPECTED-FAIL | automation.py | Exited with code -11 during test run

gav616 · 2010-02-16 21:19:15

I 'think' my bookmark crash has to do with the mozconfig 'optimize' setting using '-03'..
because unsetting the cflags it doesn't crash.

will test....

Others with the crash could help out too....

by, trying diffrent flags to compile and see if it crashes

i.e.

disabling 'optimize' (03) but added 02

Last edited by gav616 (2010-02-16 21:23:23)

harryNID · 2010-02-16 23:26:15

I 'think' my bookmark crash has to do with the mozconfig 'optimize' setting using '-03'..
because unsetting the cflags it doesn't crash.
will test....
Others with the crash could help out too....
by, trying diffrent flags to compile and see if it crashes
i.e.
disabling 'optimize' (03) but added 02

Interesting hypothesis. I will look into that myself and see what I come up with.

harryNID · 2010-02-17 00:34:12

Where exactly are you getting "-O3" from?

Inside mozconfig you can add these parameters:

Optimization
ac_add_options --enable-optimize
Enables the default compiler optimization options
ac_add_options --enable-optimize=-O2
Choose particular compiler optimization options. In most cases, this will not give the desired results, unless you know the mozilla codebase very well.
ac_add_options --enable-debug
Enables verbose debugging macros and other debug-only code. This can significantly slow a build, but it is invaluable when writing patches.
ac_add_options --disable-optimize
Disables compiler optimization. This makes it much easier to step through code in a debugger.

If my understanding is right mozilla sets its own optimizations for most things automagically unless you set ac_add_options --enable-optimize=-O#. In which case it uses that as the default CFLAG optimization for everything thereby bypassing mozilla's recommended CFLAGS. I seem to remember and as stated in the warning above that this is not really good Ju-Ju. What I believe Ranguvar says he is doing (according to the PKGBUILD) is that he is setting a guideline for firefox to use if their script doesn't have anything specifically specified (hence he only has ac_add_options --enable-optimize and CFLAGS specified in the PKGBUILD).

With that being said I agree that something about the CFLAG export setting might be interfering with mozilla's scripted settings (notice I said might I don't know). It is one of the hypothesis I've been contemplating as to why march=native is causing seg faults for some of us but not others. It could also be likely (and more plausible) that a piece of code was introduced that is causing the conflict (yet again I don't know but would love to find out).

If somebody out there with more knowledge of mozilla/mozconfig scripting could weigh in here it would be appreciated. Could the CFLAGS specified in the PKGBUILD be causing the headaches? If so why and or how?

Marcel- · 2010-02-17 03:26:18

Building with potential-pgo-fix.patch led to the same PGO-test error, but disabling jemalloc-enable-pgo.patch resulted in a correct package. After installing it I got the same error I initially had, now using TWM (yes, the archaic window manager ).

I put the complete output of strace on Pastebin, if that suits someone's needs.

I'll now try to build a version without PGO at all.

gav616 · 2010-02-17 12:14:18

harryNID wrote:

Where exactly are you getting "-O3" from?
Inside mozconfig you can add these parameters:
Optimization
ac_add_options --enable-optimize
Enables the default compiler optimization options
ac_add_options --enable-optimize=-O2
Choose particular compiler optimization options. In most cases, this will not give the desired results, unless you know the mozilla codebase very well.
ac_add_options --enable-debug
Enables verbose debugging macros and other debug-only code. This can significantly slow a build, but it is invaluable when writing patches.
ac_add_options --disable-optimize
Disables compiler optimization. This makes it much easier to step through code in a debugger.
If my understanding is right mozilla sets its own optimizations for most things automagically unless you set ac_add_options --enable-optimize=-O#. In which case it uses that as the default CFLAG optimization for everything thereby bypassing mozilla's recommended CFLAGS. I seem to remember and as stated in the warning above that this is not really good Ju-Ju. What I believe Ranguvar says he is doing (according to the PKGBUILD) is that he is setting a guideline for firefox to use if their script doesn't have anything specifically specified (hence he only has ac_add_options --enable-optimize and CFLAGS specified in the PKGBUILD).
With that being said I agree that something about the CFLAG export setting might be interfering with mozilla's scripted settings (notice I said might I don't know). It is one of the hypothesis I've been contemplating as to why march=native is causing seg faults for some of us but not others. It could also be likely (and more plausible) that a piece of code was introduced that is causing the conflict (yet again I don't know but would love to find out).
If somebody out there with more knowledge of mozilla/mozconfig scripting could weigh in here it would be appreciated. Could the CFLAGS specified in the PKGBUILD be causing the headaches? If so why and or how?

Hi, I've built and using without crashing.

I added
mozconfig

ac_add_options --enable-optimize="-O2 -fno-omit-frame-pointer"

PKGBUILD

    unset CFLAGS
    unset CXXFLAGS
    export CFLAGS="-march=athlon-xp"
    export CXXFLAGS="-march=athlon-xp"
#    export CFLAGS="-march=native -O2 -pipe"
#    export CXXFLAGS="-march=native -O2 -pipe"

I read this:

Need -fno-omit-frame-pointer
With gcc 4.1, some optimization levels (e.g., -O, -Os, -O2) have started to imply -fomit-frame-pointer. (Verify this? Which levels?) If you want to use jprof or any of the other stack walking tools, you probably need to use
ac_add_options --enable-optimize="-O2 -fno-omit-frame-pointer"
when enabling optimization. (Perhaps retest this once patches from bug 331436 are landed.)

I realised that frame pointer could be the problem, which is in my makepkg.conf as default flags:

CFLAGS="-march=athlon-xp -O2 -pipe -fomit-frame-pointer"
CXXFLAGS="${CFLAGS}"

...which is why i think unsetting the flags makes it not crash.

sorry, TBH, I don't have a vast amount of linux knowledge, all this is pure guess work...

Last edited by gav616 (2010-02-17 12:20:57)

Marcel- · 2010-02-17 13:41:52

Even when commenting out all PGO-related patches, firefox segfaults.

For the record, extra/firefox and AUR/swiftfox-prescott run fine (at least for 15 minutes).

davidm · 2010-02-20 01:35:03

toxygen wrote:

wow, thanks. the ipc.plugins setting fixed it on the nightly release for 3.7a2. i had to set it to false on mine (i'm using last night's build though)

Hello just wanted to let you know that I tried todays nightly using the firefox-pgo-minefield (http://aur.archlinux.org/packages.php?ID=32030) package modified to get the current tree and the old bug which you ran into -- at least for me -- has been solved and I am now able to watch youtube videos again with ipc.plugins set to true. It looks like one of the fixes between then and now took care of the problem. You might give it a shot if you are interested in using OOPP.

Last edited by davidm (2010-02-20 01:35:23)

toxygen · 2010-02-20 06:42:15

Thanks for letting us know. I just rebuilt pgo-minefield, and youtube works once again with ipc.plugins set at true (default) and false. even youtube hd fullscreen works, when i turn compositing off in kde. the usual plugins and extensions seem to be stable as well. 980ms score on sunspider test (the relative average i get nowadays on 3.7a2 nightlies, compared to average of around 1050ms for 3.6). There is a slight hiccup when switching videos on youtube, but no crashes, and the hiccup is only for a half a second or so.

Last edited by toxygen (2010-02-20 06:42:46)

Arm-the-Homeless · 2010-02-20 18:05:15

What CFLAGS should I use with:

processor : 1
vendor_id : AuthenticAMD
cpu family : 15
model : 107
model name : AMD Athlon(tm) 64 X2 Dual Core Processor 4000+

Marcel- · 2010-02-22 14:08:29

I followed the advice in the AUR comments and built firefox-pgo with CFLAGS="-march=i686 -mtune=generic -O2 -pipe -fomit-frame-pointer" and now everything is fine.

Thanks Ranguvar and others for helping me (and hopefully other users) out.

harryNID · 2010-02-22 21:24:02

OK gav616, I gave your method a whirl (albeit with some changes from gav616's post) and I got it to build. Not only did I get it to build I got a slightly faster build (according to Sunspider average score).

Here is what I did:

PKGBUILD:
    unset CFLAGS
    unset CXXFLAGS
    export CFLAGS="-march=pentium4"
    export CXXFLAGS="-march=pentium4"

mozconfig:
## GNU/Linux stuff
ac_add_options --disable-installer --disable-updater
ac_add_options --enable-official-branding
ac_add_options --enable-startup-notification
ac_add_options --disable-gnomeui
ac_add_options --disable-gnomevfs
ac_add_options --disable-necko-wifi
ac_add_options --disable-xprint

## Optimize
ac_add_options --disable-pedantic --enable-jemalloc --enable-xterm-updates
### Reverse these lines (enable vs disable) for a debug build
ac_add_options --enable-optimize="-O2 -fomit-frame-pointer" --disable-debug --disable-tests
ac_add_options --enable-profile-guided-optimization
ac_add_options --enable-strip --enable-install-strip
###

Notice I didn't have to use -march=i686 this time around. It's weird! Anybody care to guess why this is working? Could makepkg be causing a hidden conflict between it's CFLAGS and the exported CFLAGS in the PKGBUILD? Could unsetting the CFLAGS first be the real key here? Anybody else having an issue with this want to give it a try and see how they fair?

Marcel- · 2010-02-24 00:00:14

As I was curious what I would get when doing this (using march=atom), I built gcc-snapshot 4.5.20100218-1 from AUR (as the Atom optimization was only realized in GCC 4.5). After that, I edited mozconfig like harryNID did (but I also disabled Xinerama support) and, as said, put CFLAGS="-march=atom" in PKGBUILD. My firefox-pgo also built succesfully and runs, still without problems (but I didn't test extensively).

I think the difference is in --enable-optimize="-O2 -fomit-frame-pointer". I remember from a previous build that something like "g++ [...] -O2 -fomit-frame-pointer [...] -O3 [...]" came across sometimes and I think the occasional -O3 overrided -O2, which could cause these problems/segfaults combined with anything else than -march=i686 (at least on my EEE Box). I watched some time (no, I didn't look three hours at make/gcc) and as far as I could see I didn't observe this flag again.

The difference between -march=i686 and -march=atom is rather minimal (although the built package is significantly larger): Sunspider resulted in 3947ms with the former versus 3875ms with the latter.

Gcccpuopt recommends "-march=atom -mfpmath=sse" as CFLAGS using GCC 4.5. Perhaps I'll try building firefox-pgo using "-mfpmath=sse" again and see if that makes a difference. But perhaps not too soon.

bangkok_manouel · 2010-02-24 01:31:51

you may want to use -fexcess-precision=fast with gcc-4.5
see this thread for reference http://gcc.gnu.org/ml/gcc-bugs/2009-12/msg01441.html

Marcel- · 2010-02-24 03:00:07

Thanks bangkok_manouel, I will add that, tomorrow or Thursday.

(Do you follow GCC development or something? How did you know this? Oh, it's also mentioned here, under caveats.)

Last edited by Marcel- (2010-02-24 03:12:42)

bangkok_manouel · 2010-02-24 04:13:44

[offtopic]

Marcel- wrote:

(Do you follow GCC development or something? How did you know this? Oh, it's also mentioned here, under caveats.)

no, I don't follow the gcc development but I have been testing gcc-4.5 (for fun) since few weeks and it's the first hit when you google gcc 4.5 performance (performance, the holy grail of all of you nerds - including myself - in this thread )
[/offtopic]

Marcel- · 2010-02-24 23:51:48

When building with CFLAGS="-march=atom -mfpmath=sse -fexcess-precision=fast", Firefox segfaults again on startup. Only using "-march=atom -fexcess-precision=fast" results in an executable that runs stable for several hours now. So one of the causes of my problem has something to do with -mfpmath=sse.

The Sunspider test now results in 3774 ms, so this time 100 ms of my life are saved.

One thing I noticed is that when searching for some hashtag in Twitter and expanding the results using the 'more' button, Firefox halts the animation of the loader indicator after half a second and few seconds later the famous alert pops up:

A script on this page may be busy, or it may have stopped responding. You can stop the script now, or you can continue to see if the script will complete.
Script: http://ajax.googleapis.com/ajax/libs/jq … .min.js:19

I didn't test this on other builds (like -march=i686), but does someone else have this problem?

I also proved that unsetting CFLAGS and CXXFLAGS has nothing to do with this bug (at least not in my case, I commented those two lines), which would be very strange anyway, because these variables are immediately set after unsetting.

Arch Linux

#526 2010-02-14 19:47:48

Re: Official firefox-pgo thread

#527 2010-02-14 20:53:30

Re: Official firefox-pgo thread

#528 2010-02-15 19:59:26

Re: Official firefox-pgo thread

#529 2010-02-15 21:27:49

Re: Official firefox-pgo thread

#530 2010-02-15 22:13:46

Re: Official firefox-pgo thread

#531 2010-02-15 22:31:38

Re: Official firefox-pgo thread

#532 2010-02-15 23:44:06

Re: Official firefox-pgo thread

#533 2010-02-16 11:38:35

Re: Official firefox-pgo thread

#534 2010-02-16 16:43:14

Re: Official firefox-pgo thread

#535 2010-02-16 21:19:15

Re: Official firefox-pgo thread

#536 2010-02-16 23:26:15

Re: Official firefox-pgo thread

#537 2010-02-17 00:34:12

Re: Official firefox-pgo thread

#538 2010-02-17 03:26:18

Re: Official firefox-pgo thread

#539 2010-02-17 12:14:18

Re: Official firefox-pgo thread

#540 2010-02-17 13:41:52

Re: Official firefox-pgo thread

#541 2010-02-20 01:35:03

Re: Official firefox-pgo thread

#542 2010-02-20 06:42:15

Re: Official firefox-pgo thread

#543 2010-02-20 18:05:15

Re: Official firefox-pgo thread

#544 2010-02-22 14:08:29

Re: Official firefox-pgo thread

#545 2010-02-22 21:24:02

Re: Official firefox-pgo thread

#546 2010-02-24 00:00:14

Re: Official firefox-pgo thread

#547 2010-02-24 01:31:51

Re: Official firefox-pgo thread

#548 2010-02-24 03:00:07

Re: Official firefox-pgo thread

#549 2010-02-24 04:13:44

Re: Official firefox-pgo thread

#550 2010-02-24 23:51:48

Re: Official firefox-pgo thread

Board footer