[SOLVED] Large log files. Safe to delete?

BasioMeusPuga · 2012-03-31 13:26:42

Hey,

My /var/log directory has somehow reached 2031 MB on my relatively recent Arch install. The laptop seems to be running fine otherwise. The following is the size of each of the files:

0k      btmp
4k      cups
4k      old
4k      syslog.log
4k      syslog.log.1
4k      syslog.log.2
4k      syslog.log.3
8k      syslog.log.4
12k     faillog
24k     crond.log
24k     crond.log.1
24k     crond.log.2
24k     crond.log.4
28k     crond.log.3
28k     Xorg.0.log
28k     Xorg.0.log.old
32k     lastlog
32k     user.log
32k     user.log.2
36k     user.log.3
40k     user.log.1
48k     errors.log.3
56k     errors.log.4
60k     user.log.4
64k     dmesg.log
124k    auth.log.1
140k    auth.log.3
148k    auth.log
156k    auth.log.2
164k    ConsoleKit
180k    auth.log.4
208k    pacman.log
240k    wicd
252k    boot
276k    kdm.log
288k    daemon.log.3
316k    daemon.log
332k    daemon.log.1
380k    wtmp.1
468k    daemon.log.4
548k    pm-powersave.log
568k    daemon.log.2
664k    pm-suspend.log
1M      wtmp
1M      iptables.log.1
1M      iptables.log
1M      iptables.log.2
1M      messages.log.3
2M      messages.log
2M      messages.log.1
2M      messages.log.4
2M      messages.log.2
2M      kernel.log.3
2M      kernel.log.4
4M      iptables.log.4
7M      everything.log.4
10M     iptables.log.3
13M     everything.log.3
42M     errors.log.2
46M     kernel.log.2
48M     everything.log.2
88M     errors.log
90M     kernel.log
92M     everything.log
516M    errors.log.1
519M    kernel.log.1
521M    everything.log.1

Is there a) some reason for the logs to reach this size and b) can I just delete the larger logs?

Last edited by BasioMeusPuga (2012-04-02 06:49:47)

karol · 2012-03-31 13:51:44

First check what is in these logs, what messages are flooding them. Maybe you should take care of some errors.
My 3-year-old installation has 17MB /var/log.

ewaller · 2012-03-31 17:02:58

Is there a) some reason for the logs to reach this size and b) can I just delete the larger logs?

You are using logrotate, so you already are Your system rotates logs and keeps the last couple versions. Those are the numeric extensions on the filenames.

That said, I agree with Karol, go read your logs and see what is going on; the seem a bit large. I read my logs about once a month; more often if I am looking for something specific. Things I really want to know about are automatically mailed to me from my system.

Xyne · 2012-03-31 17:41:48

BasioMeusPuga wrote:

88M     errors.log
90M     kernel.log
92M     everything.log
516M    errors.log.1
519M    kernel.log.1

You should probably read those. Your system is desperately trying to tell you something.

BasioMeusPuga · 2012-03-31 19:51:03

Thanks for all the input.
Since I had no idea where I'd even start with a 500 MB log, I just deleted everything and crossed my fingers it won't show up on the next reboot. However, I did get 3 freshly made files of 39MB each (errors.log, kernel.log, everything.log).

In each of the three files, there are some 441,200 occurrences (I'm not making this up) of

Apr  1 00:50:37 Polaris kernel: [22812.050415] iwlwifi 0000:02:00.0: GF was set with SGI:SISO

I'm running the 3.2.13 kernel on an HP dv7 laptop. It has an Intel 5100agn card. I'm using wicd to connect to an n router with WPA2 encryption. The only problems I have is
1) The occasional loss of connection that rectifies itself (doesn't happen if I set aside a console session to ping 192.168.1.1 continuously) and;
2) Youtube and Twitter refuse to load when everything else is working fine (sometimes).

I've taken the liberty of posting the rest of the errors.log here.
I'll keep Googling but yes, I'm kinda out of my depth here.

P.S. I've also modified my /etc/logrotate.conf to keep log sizes down to 20 MB and compress them. Shows how much better you can get at sweeping things under the carpet with a little practice.

Last edited by BasioMeusPuga (2012-03-31 19:58:10)

Gcool · 2012-03-31 21:38:51

Not really sure what's causing those messages exactly, but you can suppress them by editing the relevant lines in your /etc/syslog-ng/syslog-ng.conf to something among the lines of:

filter f_kernel { facility(kernel) and not match("GF was set with SGI:SISO" value("MESSAGE")); };
filter f_err { level(err) and not match("GF was set with SGI:SISO" value("MESSAGE")); };

That should stop the spam in your logs.

a7 · 2012-04-01 11:14:14

Are you using a custom firmware on your wireless router by chance?

From intel's commit to the kernel we can see that the driver believes you should be aware that your wireless connectivity was being dropped to single mode (this being 802.11n) and using "extended" non-standard ways to communicate with the source. (GF=greenfield, I gather)

                 if (use_green) {
                         rate_n_flags |= RATE_MCS_GF_MSK;
                         if (is_siso(tbl->lq_type) && tbl->is_SGI) {
                                 rate_n_flags &= ~RATE_MCS_SGI_MSK;
                                 IWL_ERR(priv, "GF was set with SGI:SISO\n");
                         }

I'm going to stab in the dark here and assume that your computer wants to talk one way and your router disagrees.

Check you router settings for such related things like greenfield or whatnot and perhaps audit your placement as well. 802.11n is a lot busier then 802.11g was.

BasioMeusPuga · 2012-04-01 16:20:38

@Gcool

Thanks. There's still all kinds of nonsense showing up in the dmesg, but the log files are clean now.

@a7
I have a garden variety DLink 2750U router that gives me 1/10th of the advertised n speeds. All I've done is play around with the settings a little bit, but I stopped when the current config worked for all the devices at my place.
There is no specific mention of GF anywhere, but I will try fiddling with the knobs. And inform the neighbor he should really stop using WEP.

Again, thank you for taking the time.

a7 · 2012-04-01 18:21:49

BasioMeusPuga wrote:

garden variety DLink 2750U router that gives me 1/10th of the advertised n speeds

Yes, this another clue. SISO mode on wireless n is like having 8 pistons but driving only on two. Meanwhile, your router wants to use the fast lane but your dashboard light is on and the car is in safe mode.

There is no specific mention of GF anywhere.

Perhaps 'preamble' or something of the like?

If you have wireless-g (don't forget about smartphones here) or b items in the area using the router then the router won't be using 802.11n MIMO (firing on all pistons). If I read the documentation right and, again, I'm stabbing in the dark here but maybe your router is 'adapting' to connections coming on wireless-g and toggling for compatibility. 802.11n allows for legacy connections but you cannot have your cake and eat it too.

If this continues to elude you then maybe edit the post title to include some wireless type keywords and attract the attention of someone who has seen this themselves.

edit: spelling and post-coffee reread

Last edited by a7 (2012-04-01 18:25:14)

BasioMeusPuga · 2012-04-02 04:52:26

Yup, the good old g/n compatibility issues (You're supposed to add, not replace).
In any case, I've taken all the non n devices off the network and restarted the router. So far the dmesg is clean of the repeated "iwlwifi 0000:02:00.0: GF was set with SGI:SISO". So I guess that takes care of that.
Thanks.

One last thing, the iwconfig for wlan0 reads as follows:

wlan0 IEEE 802.11abgn  ESSID:"DLink"  
          Mode:Managed  Frequency:2.437 GHz  Access Point: 84:C9:B2:E8:11:69   
          Bit Rate=135 Mb/s   Tx-Power=15 dBm   
          Retry  long limit:7   RTS thr:off   Fragment thr:off
          Power Management:off
          Link Quality=60/70  Signal level=-50 dBm  
          Rx invalid nwid:0  Rx invalid crypt:0  Rx invalid frag:0
          Tx excessive retries:85114  Invalid misc:135   Missed beacon:0

As you can see, the "Tx excessive retries" counter is rather high and jumping by about 40-100 per second. The man page says this relates to packet loss. Would this be a fault of the router too or should I take a look at my Wifi settings?

Edit: The problem described in this thread is similar: https://bbs.archlinux.org/viewtopic.php?id=132079
The packet loss disappears if I disable wireless n by reloading the iwlwifi module using

modprobe iwlwifi 11n_disable=1 swcrypto=1

So should I just forget this is happening/sacrifice a goat over a copper bowl/submit a bug report/keep googling or march on DLink headquarters with a lynch mob suitably armed with torches and pitchforks?

Last edited by BasioMeusPuga (2012-04-02 05:36:03)

a7 · 2012-04-02 05:36:07

Do you recall if you had such packet loss before the recent changes, and not to patronize, but are your syslog filters still in place? Years of troubleshooting had me double check and never assume. Your router and client are playing 52 card pickup. If your filters are off and you are not getting SISO errors from the kernel then it is more likely that there is a complication in the wireless path between the client and router; otherwise your router needs a setting tweak so I linked a PDF of the product manual for you.

A few things to try are :

scan the area with the router to see what channels are busy
if your router does not support scanning then select a non-default channel
if your router supports auto-sensing for best channels use it
keep microwaves, large water sources and powerful electronics from resting between the clients and router
keep the client and router from being way too close

Really before you get busy. Do the youtube test a few times and again at each 'confident' stage when tweaking. Good luck and ask anything you need.

resource: DSL-2750U Product Manual [pdf]

Last edited by a7 (2012-04-02 05:36:39)

a7 · 2012-04-02 05:45:17

Edit: The problem described in this thread is similar: https://bbs.archlinux.org/viewtopic.php?id=132079
The packet loss disappears if I disable wireless n by reloading the iwlwifi module using
modprobe iwlwifi 11n_disable=1 swcrypto=1
So should I just forget this is happening/sacrifice a goat over a copper bowl/submit a bug report/keep googling or march on DLink headquarters with a lynch mob suitably armed with torches and pitchforks?

Just caught your edit. Disabling `n` support 'fixes' the issue so your router wants to be in legacy mode. In my opinion not using a fast rate when both client and router support it is unacceptable. The product manual I linked before is useless on page 77 where the specific tweaks would help here so tech support may be need to earn their pay this day explaining what the options will help.

BasioMeusPuga · 2012-04-02 06:13:52

I had actually read through that manual before posting. I especially liked the part where the all important Page 77 is a blurry screenshot. In fact, that's where I'd closed it (and where Okular reopened it). I'm not buying DLink again, that's for sure.

Also, I had removed syslog-ng filters too; dmesg is still clean. I'll recheck (and redo the filters) as and when a g device needs to access the router. I guess then it's time for Russian Roulette with the settings.

Once again man, thanks.

Last edited by BasioMeusPuga (2012-04-02 06:14:24)

a7 · 2012-04-02 06:21:15

It was my pleasure but one request. Can you verify that with the module setting below that your bitrate is above 54?

With

modprobe iwlwifi 11n_disable=1 swcrypto=1

do you still get?

          Bit Rate=135 Mb/s   Tx-Power=15 dBm

BasioMeusPuga · 2012-04-02 06:26:50

Nope. I checked using a 30 second "watch -n1 -d iwconfig wlan0" . It's stable at:

Bit Rate=54 Mb/s   Tx-Power=15 dBm

a7 · 2012-04-02 06:28:04

BasioMeusPuga wrote:

Nope. I checked using a 30 second "watch -n1 -d iwconfig wlan0" . It's stable at:
Bit Rate=54 Mb/s   Tx-Power=15 dBm

Yea, that is what I thought. Just so you know, your wireless N network is really just a fancy G network now.

BasioMeusPuga · 2012-04-02 06:48:42

Nothing a little self deception can't fix.
With the gentle yet unyielding hope that nobody buys an Intel/DLink combo in the near future; I'm marking the thread [Solved] at least for the initial problem.

Xyne · 2012-04-03 16:35:15

BasioMeusPuga wrote:

Nothing a little self deception can't fix.

For those that truly master it, self-deception is an amazing generic solution. We should add an article about it to the wiki.

Arch Linux

#1 2012-03-31 13:26:42

[SOLVED] Large log files. Safe to delete?

#2 2012-03-31 13:51:44

Re: [SOLVED] Large log files. Safe to delete?

#3 2012-03-31 17:02:58

Re: [SOLVED] Large log files. Safe to delete?

#4 2012-03-31 17:41:48

Re: [SOLVED] Large log files. Safe to delete?

#5 2012-03-31 19:51:03

Re: [SOLVED] Large log files. Safe to delete?

#6 2012-03-31 21:38:51

Re: [SOLVED] Large log files. Safe to delete?

#7 2012-04-01 11:14:14

Re: [SOLVED] Large log files. Safe to delete?

#8 2012-04-01 16:20:38

Re: [SOLVED] Large log files. Safe to delete?

#9 2012-04-01 18:21:49

Re: [SOLVED] Large log files. Safe to delete?

#10 2012-04-02 04:52:26

Re: [SOLVED] Large log files. Safe to delete?

#11 2012-04-02 05:36:07

Re: [SOLVED] Large log files. Safe to delete?

#12 2012-04-02 05:45:17

Re: [SOLVED] Large log files. Safe to delete?

#13 2012-04-02 06:13:52

Re: [SOLVED] Large log files. Safe to delete?

#14 2012-04-02 06:21:15

Re: [SOLVED] Large log files. Safe to delete?

#15 2012-04-02 06:26:50

Re: [SOLVED] Large log files. Safe to delete?

#16 2012-04-02 06:28:04

Re: [SOLVED] Large log files. Safe to delete?

#17 2012-04-02 06:48:42

Re: [SOLVED] Large log files. Safe to delete?

#18 2012-04-03 16:35:15

Re: [SOLVED] Large log files. Safe to delete?

Board footer