Compiling Firefox with ICC?

uncholowapo · 2010-02-02 20:10:40

Has anyone done it? I'm really interested on how it preforms against gcc. I will do it eventually when school work is not procrastinated on

Fackamato · 2010-03-02 11:15:44

uncholowapo wrote:

Has anyone done it? I'm really interested on how it preforms against gcc. I will do it eventually when school work is not procrastinated on

Yes: http://software.intel.com/en-us/article … -compiler/

Marcel- · 2010-03-30 20:55:30

Today I managed to compile FF 3.6.2 using ICC 11.1.069 and the article mentioned by Fackamato (with a tiny extra change).

I just downloaded the source tarball and unpacked it. I put the following files in {$srcdir}/mozilla-1.9.2:

.mozconfig (modified from Firefox-PGO):

ac_add_options --enable-application=browser
mk_add_options MOZ_CO_PROJECT=browser
ac_add_options --prefix=/usr --libdir=/usr/lib

## System libs are preferred
ac_add_options --with-system-nss --with-system-jpeg --with-pthreads --with-system-zlib
ac_add_options --with-system-bz2 --with-system-png --enable-system-cairo --with-system-hunspell
ac_add_options --with-system-sqlite --with-system-nspr

## GNU/Linux stuff
ac_add_options --disable-installer --disable-updater
ac_add_options --enable-official-branding
ac_add_options --enable-startup-notification

## disable GNOME stuff
ac_add_options --disable-gnomeui --disable-gnomevfs
ac_add_options --disable-necko-wifi
ac_add_options --disable-xprint
## disable Xinerama
ac_add_options --disable-xinerama

## Optimize
ac_add_options --disable-pedantic --enable-jemalloc --enable-xterm-updates
### Reverse these lines (enable vs disable) for a debug build
ac_add_options --enable-optimize="-O2 -fomit-frame-pointer" --disable-debug --disable-tests
#ac_add_options --enable-profile-guided-optimization
ac_add_options --enable-strip --enable-install-strip
###

## Edit these as you see fit
ac_add_options --disable-tests
ac_add_options --enable-cpp-rtti

ac_add_options --disable-crashreporter --disable-parental-controls 
ac_add_options --enable-printing #--enable-xinerama
ac_add_options --enable-default-toolkit=cairo-gtk2  # cairo-gtk2-dfb, cairo-gtk2, or cairo-qt
ac_add_options --enable-places --enable-svg --enable-pango --enable-canvas
ac_add_options --enable-smil --disable-java-xpcom
# Enabling this breaks PGO with 3.6, on x86_64.  Please test enabled with later versions.
ac_add_options --disable-canvas3d
# This is disabled in order to speed Firefox up, it prevents checking Google
# for every single URL Firefox processes.  If you're using this package, it's likely you already
# know enough to be safe online, or are paranoid (perhaps with reason) about
# Google getting your URLs.  Also, this is GNU/Linux, we have somewhat less to fear ;-)
ac_add_options --disable-safe-browsing

mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/ff-opt
#mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) $(MOZ_OBJDIR)/_profile/pgo/profileserver.py'
export BUILD_OFFICIAL=1
export MOZILLA_OFFICIAL=1
mk_add_options BUILD_OFFICIAL=1
mk_add_options MOZILLA_OFFICIAL=1

# Want to change the compiler?
export CC='icc'
export CXX='icpc'

icc-11.1.069.patch:

diff -Nur ./js/ctypes/libffi/src/x86/ffi64.c ./js/ctypes/libffi/src/x86/ffi64.c
--- ./js/ctypes/libffi/src/x86/ffi64.c  2010-03-16 10:55:18.000000000 +0100
+++ ./js/ctypes/libffi/src/x86/ffi64.c  2010-03-30 16:15:43.000000000 +0200
@@ -36,6 +36,8 @@
 #define MAX_GPR_REGS 6
 #define MAX_SSE_REGS 8

+typedef struct { int64_t m[2]; } __int128_t;
+
 struct register_args
 {
   /* Registers for argument passing.  */
@@ -415,10 +417,12 @@
                  break;
                case X86_64_SSE_CLASS:
                case X86_64_SSEDF_CLASS:
-                 reg_args->sse[ssecount++] = *(UINT64 *) a;
+                 reg_args->sse[ssecount++].m[0] = *(UINT64 *) a;
+                 reg_args->sse[ssecount++].m[1] = 0;
                  break;
                case X86_64_SSESF_CLASS:
-                 reg_args->sse[ssecount++] = *(UINT32 *) a;
+                 reg_args->sse[ssecount++].m[0] = *(UINT32 *) a;
+                 reg_args->sse[ssecount++].m[1] = 0;
                  break;
                default:
                  abort();
diff -Nur ./xpcom/ds/nsMathUtils.h ./xpcom/ds/nsMathUtils.h
--- ./xpcom/ds/nsMathUtils.h    2010-03-16 10:57:23.000000000 +0100
+++ ./xpcom/ds/nsMathUtils.h    2010-03-30 16:17:38.000000000 +0200
@@ -155,7 +155,7 @@
  */
 inline NS_HIDDEN_(double) NS_hypot(double x, double y)
 {
-#if __GNUC__ >= 4
+#if (__GNUC__ >= 4) && !__INTEL_COMPILER
     return __builtin_hypot(x, y);
 #elif defined _WIN32
     return _hypot(x, y);
diff -Nur ./modules/libpr0n/encoders/png/nsPNGEncoder.cpp ./modules/libpr0n/encoders/png/nsPNGEncoder.cpp
--- ./modules/libpr0n/encoders/png/nsPNGEncoder.cpp     2010-03-16 10:56:46.000000000 +0100
+++ ./modules/libpr0n/encoders/png/nsPNGEncoder.cpp     2010-03-30 18:35:43.000000000 +0200
@@ -134,8 +134,9 @@
   }

   // initialize
+  // changed png_voidp_NULL -> 0, due to https://bugs.webkit.org/show_bug.cgi?id=33287
   mPNG = png_create_write_struct(PNG_LIBPNG_VER_STRING,
-                                 png_voidp_NULL,
+                                 0,
                                  ErrorCallback,
                                  ErrorCallback);
   if (! mPNG)

The last chunk solves a missing symbol error when compiling against libpng 1.4 (note I didn't change other files containing missing symbols, like those in ./modules/libimg/png, as they were not compiled, probably due to using system libs. Neither was step 6 in the Intel article needed, for the same reason).

I then did:

$ patch -Np0 -i icc-11.1.069.patch
$ make -f client.mk configure
$ make -f client.mk build

Now a lot of ugly messages are spit out by the compiler, but after building I got a working Firefox version.

$ cd ff-opt/dist/bin
$ export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH
$ ./firefox

Well, here it is, using it right now.

Unfortunately, it's not faster than an optimized Firefox-PGO for my system (using GCC 4.5 and -march=atom), Sunspider test results a total time of 4152 ms, against 3774 ms using Firefox-PGO built with GCC 4.5.

Another time I'll try using more aggressive optimizations, mentioned in the second part of the Intel article.

Last edited by Marcel- (2010-03-31 12:46:24)

toxygen · 2010-03-31 15:49:34

that was an excellent write-up Marcel. thanks a lot for posting it. are you considering doing a PGO build with ICC? i'd be curious about how that performs vs gcc PGO build

broch · 2010-04-02 01:57:36

it is faster (ICC with optimization flags) than PGO

Marcel- · 2010-04-02 12:35:18

@toxygen: as Broch said ICC with IPO will probably result in a more performant binary than when using PGO, so I won't do that.

@broch: which flags did you use when building FF described in this post in the FF-PGO topic?

Last days I tried to build the thing using IPO (and more aggressive optimization settings), described in the second part of the Intel article. Apart from the patch mentioned in my previous post I changed ./security/coreconf/UNIX.mk and ./security/coreconf/rules.mk according to the article. I used the following .mozconfig:

ac_add_options --enable-application=browser
mk_add_options MOZ_CO_PROJECT=browser
ac_add_options --prefix=/usr --libdir=/usr/lib

## System libs are preferred
ac_add_options --with-system-nss --with-system-jpeg --with-pthreads --with-system-zlib
ac_add_options --with-system-bz2 --with-system-png --enable-system-cairo --with-system-hunspell
ac_add_options --with-system-sqlite --with-system-nspr

## GNU/Linux stuff
ac_add_options --disable-installer --disable-updater
ac_add_options --enable-official-branding
ac_add_options --enable-startup-notification

## disable GNOME stuff
ac_add_options --disable-gnomeui --disable-gnomevfs
ac_add_options --disable-necko-wifi
ac_add_options --disable-xprint
## disable Xinerama
ac_add_options --disable-xinerama

## Optimize
ac_add_options --disable-pedantic --enable-jemalloc --enable-xterm-updates
### Reverse these lines (enable vs disable) for a debug build
ac_add_options --enable-optimize="-O3 -fomit-frame-pointer -fp-model fast=2 -ipo1 -xSSE3_ATOM" --disable-debug --disable-tests
ac_add_options --enable-strip --enable-install-strip
###

## Edit these as you see fit
ac_add_options --disable-tests
ac_add_options --enable-cpp-rtti

ac_add_options --disable-crashreporter --disable-parental-controls.
ac_add_options --enable-printing
ac_add_options --enable-default-toolkit=cairo-gtk2  # cairo-gtk2-dfb, cairo-gtk2, or cairo-qt
ac_add_options --enable-places --enable-svg --enable-pango --enable-canvas
ac_add_options --enable-smil --disable-java-xpcom
# Enabling this breaks PGO with 3.6, on x86_64.  Please test enabled with later versions.
ac_add_options --disable-canvas3d
# This is disabled in order to speed Firefox up, it prevents checking Google
# for every single URL Firefox processes.  If you're using this package, it's likely you already
# know enough to be safe online, or are paranoid (perhaps with reason) about
# Google getting your URLs.  Also, this is GNU/Linux, we have somewhat less to fear ;-)
ac_add_options --disable-safe-browsing

mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/ff-opt
export BUILD_OFFICIAL=1
export MOZILLA_OFFICIAL=1
mk_add_options BUILD_OFFICIAL=1
mk_add_options MOZILLA_OFFICIAL=1

# Want to change the compiler?
export CC='icc'
export CXX='icpc'

Then I built FF using (as you can see, I put my optimization flags in .mozconfig):

$ make -f client.mk clean
$ make -f client.mk configure AR=xiar
$ make -f client.mk build AR=xiar

During build I got a strange error message about mcpcom having been "terminated by kill signal", which was caused by a lack of memory. So I added some extra swap space* and rerun 'make build'. Note that you don't want to try this on an Intel Atom processor: even with BFS and running X with SCHED_ISO policy, my system became rather unresponsive due to heavy disk usage.

One night of sleep later the process stopped at another out of memory error. Even when adding more swap space (resulting in a total memory of 11 GB), I got exactly the same error message, so perhaps it's due to lack of RAM (which is 1 GB in my case) or perhaps something totally different:

ipo: remark #11000: performing multi-file optimizations
ipo: remark #11005: generating object file /tmp/ipo_icpcRQxH1C.o
/home/mk/aur/firefox-pgo-3.6.2/manual/mozilla-1.9.2/media/libtheora/lib/x86/mmxidct.c(562) (col. 1): warning #963: no EMMS instruction before return

/home/mk/aur/firefox-pgo-3.6.2/manual/mozilla-1.9.2/media/libtheora/lib/x86/mmxfrag.c(288) (col. 1): warning #963: no EMMS instruction before return

/home/mk/aur/firefox-pgo-3.6.2/manual/mozilla-1.9.2/media/libtheora/lib/x86/mmxfrag.c(201) (col. 1): warning #963: no EMMS instruction before return

/home/mk/aur/firefox-pgo-3.6.2/manual/mozilla-1.9.2/media/libtheora/lib/x86/mmxfrag.c(149) (col. 1): warning #963: no EMMS instruction before return

/home/mk/aur/firefox-pgo-3.6.2/manual/mozilla-1.9.2/media/libtheora/lib/x86/mmxfrag.c(34) (col. 1): warning #963: no EMMS instruction before return

/home/mk/aur/firefox-pgo-3.6.2/manual/mozilla-1.9.2/media/libtheora/lib/x86/mmxstate.c(186) (col. 1): warning #963: no EMMS instruction before return

/home/mk/aur/firefox-pgo-3.6.2/manual/mozilla-1.9.2/media/libtheora/lib/x86/mmxstate.c(125) (col. 1): warning #963: no EMMS instruction before return

/home/mk/aur/firefox-pgo-3.6.2/manual/mozilla-1.9.2/media/libtheora/lib/x86/mmxstate.c(88) (col. 8): warning #964: no EMMS instruction before call

/home/mk/aur/firefox-pgo-3.6.2/manual/mozilla-1.9.2/media/libtheora/lib/x86/mmxstate.c(95) (col. 1): warning #963: no EMMS instruction before return

Fatal compilation error: Out of memory asking for 16781320.
icpc: error #10014: problem during multi-file optimization compilation (code 1)
make[4]: *** [libxul.so] Error 1
make[4]: Leaving directory `/home/mk/aur/firefox-pgo-3.6.2/manual/mozilla-1.9.2/ff-opt/toolkit/library'
make[3]: *** [libs_tier_toolkit] Error 2
make[3]: Leaving directory `/home/mk/aur/firefox-pgo-3.6.2/manual/mozilla-1.9.2/ff-opt'
make[2]: *** [tier_toolkit] Error 2
make[2]: Leaving directory `/home/mk/aur/firefox-pgo-3.6.2/manual/mozilla-1.9.2/ff-opt'
make[1]: *** [default] Error 2
make[1]: Leaving directory `/home/mk/aur/firefox-pgo-3.6.2/manual/mozilla-1.9.2/ff-opt'
make: *** [build] Error 2

BTW, someone on the Intel forums already tried several optimization flags and concluded that ICC didn't result in a faster Firefox than FF compiled using GCC.

Due to personal circumstances I'll stop experimenting with this for a while. But please let us know about your experiences with this, I'm still curious to know.

* In case someone forgot how to do this (I had to search a bit and ran into a little problem):

# dd if=/dev/zero of=/large_fs/swapfile bs=4K count=1M  # will create 4 GB file
# mkswap /large_fs/swapfile
# swapon /large_fs/swapfile

broch · 2010-04-02 14:10:46

@Marcel,
your problem has less to do with amount of physical RAM, but a lot with 32-bit

however this is best example why setting vm.swappiness less than 60 is stupid and has nothing to do with performance.

Anyway, on 32-bit system set:
-mozconfig:
ac_add_options --enable-optimize="-O2 -fomit-frame-pointer -fp-model fast=2 -ip -xSSSE3"

restrictions: -ipo will exhaust memory on 32-bit systems (reason I said it compiles but segfaults on tests = no final binary)
so do not use -ipo even though this would (in theory) generate best code
instead use -ip

-ipo stands for multi-file optimization
-ip stands for single-file optimization

-O3 is slower (on my machine) than -O2 in the case of firefox, but you can safely test this (successful compilation with tests)
so you can try
ac_add_options --enable-optimize="-O3 -fomit-frame-pointer -fp-model fast=2 -ip -xSSE3_ATOM"

the rest of the mozconfig options has no effect on compilation success so whatever you choose, it will compile.

set vm.swappiness = 100
this setting will help and in fact speed up compilation process if you have less than 4GB RAM. If you have more, this will have no effect whatsoever (... and still may help)

good luck

ps regarding results:
as i mentioned, in my case it went from 1408 (default Arch firefox) to 1257 (icc optimized). This is statistically meaningful, but user experience is unaffected (I don't "feel" speed).
In contrast on old winxp (E2200/2GB RAM) with ff 3.6.3 without any modifications, firefox gets ~800 ms in SunSpider test.
There is more than compiler that makes windows desktop faster than linux desktop.

Last edited by broch (2010-04-02 14:18:03)

Marcel- · 2010-04-08 01:12:23

Thanks Broch! I finally got enough motivation to do some FF compiling again and you were right. Indeed, it seems to be impossible to compile the beast with -ipo1 on a 32-bit machine. With

ac_add_options --enable-optimize="-O2 -fomit-frame-pointer -fp-model fast=2 -ip -xSSE3_ATOM"

it compiled fine and I got a working binary. It feels slightly faster/more responsive than when compiling FF-PGO with GCC 4.5 and using -march=atom, though SunSpider returns almost exactly the same total time (± 3870 ms).

broch wrote:

set vm.swappiness = 100
this setting will help and in fact speed up compilation process if you have less than 4GB RAM. If you have more, this will have no effect whatsoever (... and still may help)

I didn't know this setting, so I searched for it, but there seem to be many different opinions about the 'ideal' setting (like this old discussion; that said, I understand that the 'ideal' setting depends on whether I want a responsive desktop, regardless of whether the system is doing a lot of work in the background or not, or a system where tasks are performed with the least disc access as possible). But I also read your post which gives a different meaning to the setting nowadays. I'll do some further searches later.

broch wrote:

There is more than compiler that makes windows desktop faster than linux desktop.

Yeah, I read that even running FF/Win under Wine gives a better result than running native Linux Firefox. Well, I'm just fine with it.

Last edited by Marcel- (2010-04-08 01:13:38)

cecilyen · 2010-05-26 03:15:25

Anyone can make a PKGBUILD script for icc pgo build?

dunkles · 2010-07-20 16:44:07

cecilyen wrote:

Anyone can make a PKGBUILD script for icc pgo build?

please

Labello · 2010-07-21 15:30:34

i would like to see some benchmarks first >.<

Arch Linux

#1 2010-02-02 20:10:40

Compiling Firefox with ICC?

#2 2010-03-02 11:15:44

Re: Compiling Firefox with ICC?

#3 2010-03-30 20:55:30

Re: Compiling Firefox with ICC?

#4 2010-03-31 15:49:34

Re: Compiling Firefox with ICC?

#5 2010-04-02 01:57:36

Re: Compiling Firefox with ICC?

#6 2010-04-02 12:35:18

Re: Compiling Firefox with ICC?

#7 2010-04-02 14:10:46

Re: Compiling Firefox with ICC?

#8 2010-04-08 01:12:23

Re: Compiling Firefox with ICC?

#9 2010-05-26 03:15:25

Re: Compiling Firefox with ICC?

#10 2010-07-20 16:44:07

Re: Compiling Firefox with ICC?

#11 2010-07-21 15:30:34

Re: Compiling Firefox with ICC?

Board footer