Server box just went down, log shows Bing/MSN crashing it?

colbert · 2011-03-17 05:51:31

I don't quite know what this means but randomly I was watching a Myth recording tonight and then xbmc froze, torrentflux died and I went downstairs to check the server. My keyboard was flashing lights as thought it had rebooted and couldn't detect a kb, I hit restart and waited, now via SSH I am seeing this output from /var/log/httpd/access.log:

192.168.1.101 - bobby [17/Mar/2011:00:04:53 -0400] "GET /mythweb/tv/upcoming HTTP/1.1" 200 10468 "http://192.168.1.100:xxxx/mythweb/tv/upcoming" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.15) Gecko/20110304 Firefox/3.6.15"
65.52.108.59 - - [17/Mar/2011:00:06:09 -0400] "GET /robots.txt HTTP/1.1" 403 - "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
65.52.108.59 - - [17/Mar/2011:00:06:51 -0400] "GET /?C=M;O=A HTTP/1.1" 200 2686 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
65.52.108.59 - - [17/Mar/2011:00:06:51 -0400] "GET /?C=N;O=D HTTP/1.1" 200 2686 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
65.52.108.59 - - [17/Mar/2011:00:06:52 -0400] "GET /?C=D;O=A HTTP/1.1" 200 2686 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
65.52.108.59 - - [17/Mar/2011:00:06:52 -0400] "GET /?C=S;O=A HTTP/1.1" 200 2686 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
65.52.108.59 - - [17/Mar/2011:00:06:52 -0400] "GET /cgi/ HTTP/1.1" 200 922 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
65.52.108.59 - - [17/Mar/2011:00:06:52 -0400] "GET /drakfire/ HTTP/1.1" 200 1006 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
65.52.108.59 - - [17/Mar/2011:00:06:53 -0400] "GET /img/ HTTP/1.1" 200 702 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
65.52.108.59 - - [17/Mar/2011:00:06:53 -0400] "GET /index.html.backup HTTP/1.1" 200 11580 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
65.52.108.59 - - [17/Mar/2011:00:06:53 -0400] "GET /index_files/ HTTP/1.1" 200 2247 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
65.52.108.59 - - [17/Mar/2011:00:06:53 -0400] "GET /owncloud/ HTTP/1.1" 200 170 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
65.52.108.59 - - [17/Mar/2011:00:06:53 -0400] "GET /test.php HTTP/1.1" 200 10848 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
65.52.108.59 - - [17/Mar/2011:00:06:53 -0400] "GET /torrentflux-b4rt-backup/ HTTP/1.1" 302 26 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
65.52.108.59 - - [17/Mar/2011:00:06:54 -0400] "GET /torrentflux-b4rt/ HTTP/1.1" 302 26 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

The first line is me locally as normal, then a foreign IP and it is at the exact moment my machine went down.

Here is relevant output from /var/log/httpd/errors.log:

[Thu Mar 17 00:04:33 2011] [error] [client 192.168.1.101] client denied by server configuration: /home/httpd, referer: http://192.168.1.100:1758/mythweb/tv/upcoming
[Thu Mar 17 00:06:09 2011] [error] [client 65.52.108.59] File does not exist: /home/bobby2/public_html/robots.txt
[Thu Mar 17 00:06:09 2011] [error] [client 65.52.108.59] client denied by server configuration: /home/httpd
[Thu Mar 17 00:06:51 2011] [error] [client 65.52.108.59] Negotiation: discovered file(s) matching request: /home/bobby2/public_html/index.html (None could be negotiated).
[Thu Mar 17 00:06:51 2011] [error] [client 65.52.108.59] Negotiation: discovered file(s) matching request: /home/bobby2/public_html/index.html (None could be negotiated).
[Thu Mar 17 00:06:51 2011] [error] [client 65.52.108.59] Negotiation: discovered file(s) matching request: /home/bobby2/public_html/index.html (None could be negotiated).
[Thu Mar 17 00:06:51 2011] [error] [client 65.52.108.59] Negotiation: discovered file(s) matching request: /home/bobby2/public_html/index.html (None could be negotiated).
[Thu Mar 17 00:06:51 2011] [error] [client 65.52.108.59] Negotiation: discovered file(s) matching request: /home/bobby2/public_html/index.html (None could be negotiated).
[Thu Mar 17 00:06:51 2011] [error] [client 65.52.108.59] Negotiation: discovered file(s) matching request: /home/bobby2/public_html/index.html (None could be negotiated).
[Thu Mar 17 00:06:52 2011] [error] [client 65.52.108.59] Negotiation: discovered file(s) matching request: /home/bobby2/public_html/index.html (None could be negotiated).
[Thu Mar 17 00:06:52 2011] [error] [client 65.52.108.59] Negotiation: discovered file(s) matching request: /home/bobby2/public_html/index.html (None could be negotiated).
[Thu Mar 17 00:06:52 2011] [error] [client 65.52.108.59] Negotiation: discovered file(s) matching request: /home/bobby2/public_html/index.html (None could be negotiated).
[Thu Mar 17 00:06:52 2011] [error] [client 65.52.108.59] Negotiation: discovered file(s) matching request: /home/bobby2/public_html/index.html (None could be negotiated).
[Thu Mar 17 00:06:52 2011] [error] [client 65.52.108.59] Negotiation: discovered file(s) matching request: /home/bobby2/public_html/index.html (None could be negotiated).
[Thu Mar 17 00:06:52 2011] [error] [client 65.52.108.59] Negotiation: discovered file(s) matching request: /home/bobby2/public_html/index.html (None could be negotiated).
[Thu Mar 17 01:32:10 2011] [warn] Init: Session Cache is not configured [hint: SSLSessionCache]

Again first line is me, then the same IP from access.log at same time and last line is me rebooting the machine just moments ago.

Admittedly I'm furthest from an expert from networking/security so what does this reveal (besides, potentially, my stupidity)?

Stebalien · 2011-03-17 17:20:18

Bing tried to index you because you don't have a robots.txt.
You really shouldn't be running a world accessible http server on port 80 unless you REALLY need it. In addition, something you are running is buggy and crashed when it got an unexpected request.

colbert · 2011-03-21 18:09:22

Hmm, what do you mean world accessible on port 80? I have my server on a specific port opened in my router, not 80, unless I misunderstand you. I would like to make sure this don't happen again though!

arch_gala · 2011-03-23 20:53:55

So if you put the ip address of your router/torrent flux server in a web browser and click go, you don't get any page/text displayed ? I say that your web server is not configured correctly.

ehlo · 2011-03-24 15:14:54

If you read the first access log entry clearly you can see that the server is not necessarily running on port 80. Search bots will index content on any port if it finds a link to it from another source.

Stebalien · 2011-03-24 19:16:57

colbert wrote:

Hmm, what do you mean world accessible on port 80? I have my server on a specific port opened in my router, not 80, unless I misunderstand you. I would like to make sure this don't happen again though!

Sorry, I obviously made several incorrect assumptions about your server and bing. Putting a robots.txt file in your server's root directory will prevent search engines from indexing you but your server still should not have crashed.

Arch Linux

#1 2011-03-17 05:51:31

Server box just went down, log shows Bing/MSN crashing it?

#2 2011-03-17 17:20:18

Re: Server box just went down, log shows Bing/MSN crashing it?

#3 2011-03-21 18:09:22

Re: Server box just went down, log shows Bing/MSN crashing it?

#4 2011-03-23 20:53:55

Re: Server box just went down, log shows Bing/MSN crashing it?

#5 2011-03-24 15:14:54

Re: Server box just went down, log shows Bing/MSN crashing it?

#6 2011-03-24 19:16:57

Re: Server box just went down, log shows Bing/MSN crashing it?

Board footer