You are not logged in.

#1 2012-11-10 16:42:43

impact
Member
Registered: 2012-10-28
Posts: 55

How to trace http links called within programs?

Is there some way to discover a source of an ad banner displayed within a program?
For example something to show me to what http links my computer is connecting at a given moment?

Offline

#2 2012-11-10 18:01:18

mzneverdies
Member
Registered: 2012-02-04
Posts: 147

Re: How to trace http links called within programs?

give wireshark a try, it's in the extra repo

extra/wireshark-cli 1.8.3-2
    A free network protocol analyzer for Unix/Linux and Windows - CLI version
extra/wireshark-gtk 1.8.3-2
    A free network protocol analyzer for Unix/Linux and Windows - GTK frontend

Wireshark allows the user to put network interface controllers that support promiscuous mode into that mode, in order to see all traffic visible on that interface, not just traffic addressed to one of the interface's configured addresses and broadcast/multicast traffic.

http://en.wikipedia.org/wiki/Wireshark

Offline

#3 2012-11-11 14:12:01

impact
Member
Registered: 2012-10-28
Posts: 55

Re: How to trace http links called within programs?

Thanks. Wireshark turned out to be quit tricky in tracing those thing, but I got TCPview and run it on Windows, which kind-of did what I wanted.

Offline

#4 2012-11-11 22:45:57

Strike0
Member
From: Germany
Registered: 2011-09-05
Posts: 1,277

Re: How to trace http links called within programs?

Have a look at iftop, easy to use use for that: https://www.archlinux.org/packages/?sort=&q=iftop
Start it like

sudo iftop -i <interface>

Offline

#5 2012-11-11 22:56:36

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,793
Website

Re: How to trace http links called within programs?

There's also netstat.

Offline

#6 2012-11-12 12:11:50

impact
Member
Registered: 2012-10-28
Posts: 55

Re: How to trace http links called within programs?

Thanks but logs returned by those commands are too massive - they show every protocol and every program. I only need to see those requests made by one specific program.

Offline

#7 2012-11-12 15:24:24

x33a
Forum Moderator
Registered: 2009-08-15
Posts: 3,532
Website

Re: How to trace http links called within programs?

you can use filters in wireshark, for e.g. filter to show only port 80 destination side.

Also make sure not to do any network activity while logging to keep useless hits to a minimum.

Offline

#8 2012-11-12 16:23:15

impact
Member
Registered: 2012-10-28
Posts: 55

Re: How to trace http links called within programs?

I know about filters in wireshack, but strangely, it did not show those url records that TCPview showed on windows.

Offline

#9 2012-11-14 11:09:49

hunterthomson
Member
Registered: 2008-06-22
Posts: 794
Website

Re: How to trace http links called within programs?

In BASH

read -p 'what is the name of the program you want to watch? ' pname && netstat -p TCP -veWc |grep $pname

Really though, it is just grep'ing for the string you enter. So, you could just type http or whatever. The -c flag will update every 1 second.

This may not be exacticaly what you are looking for. However, you can now see that by stringing commands together you will be able to get what you want. You may have better luck with tshark instead of netstat (pacman -S wireshark-cli)

Here is some samle output..

tcp        0      0 archpc:52605            a23-0-12-164.deploy.akamaitechnologies.com:www-http TIME_WAIT   root       0          -                   
tcp        0      0 archpc:37206            165.254.47.120:www-http TIME_WAIT   root       0          -                   
tcp        0      0 archpc:37205            165.254.47.120:www-http TIME_WAIT   root       0          -                   
tcp        0      0 archpc:56170            alderaan.archlinux.org:https ESTABLISHED user     1403647    1904/firefox        
tcp        0      0 archpc:37208            165.254.47.120:www-http TIME_WAIT   root       0          -                   
tcp        0      0 archpc:52728            98.124.248.70:https     ESTABLISHED user     1401832    1904/firefox        
tcp        0      0 archpc:56163            alderaan.archlinux.org:https ESTABLISHED user     1403639    1904/firefox        
tcp        0      0 archpc:56171            alderaan.archlinux.org:https ESTABLISHED user     1406176    1904/firefox        
tcp        0      0 archpc:56169            alderaan.archlinux.org:https ESTABLISHED user     1406175    1904/firefox        
tcp        0      0 archpc:56168            alderaan.archlinux.org:https ESTABLISHED user     1403644    1904/firefox        
tcp        0      0 archpc:37207            165.254.47.120:www-http TIME_WAIT   root       0          -                   
tcp        0      0 archpc:56159            alderaan.archlinux.org:https ESTABLISHED user     1403634    1904/firefox        
tcp        0      0 archpc:56164            alderaan.archlinux.org:https ESTABLISHED user     1403641    1904/firefox        
netstat: no support for `AF INET (sctp)' on this system.
netstat: no support for `AF INET (sctp)' on this system.
netstat: no support for `AF IPX' on this system.
netstat: no support for `AF AX25' on this system.
netstat: no support for `AF X25' on this system.
netstat: no support for `AF NETROM' on this system.

Last edited by hunterthomson (2012-11-14 11:32:56)


OpenBSD-current Thinkpad X230, i7-3520M, 16GB CL9 Kingston, Samsung 830 256GB
Contributor: linux-grsec

Offline

#10 2012-11-14 14:14:32

impact
Member
Registered: 2012-10-28
Posts: 55

Re: How to trace http links called within programs?

hunterthomson, thanks for your suggestions, but this command doesn't work for me:

$ read -p spotify pname && netstat -p TCP -veWc |grep $pname
spotify
Usage: grep [OPTION]... PATTERN [FILE]...
Try 'grep --help' for more information.
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
$

Offline

#11 2012-11-14 14:19:09

kokoko3k
Member
Registered: 2008-11-14
Posts: 1,546

Re: How to trace http links called within programs?

If the program support proxies, you can run it via a local proxy like tinyproxy with logging enabled.

Offline

#12 2012-11-14 15:04:05

impact
Member
Registered: 2012-10-28
Posts: 55

Re: How to trace http links called within programs?

Thanks kokoko, that worked but it turned out the program is not pulling ads in a way that would be visible in the proxy logs.

Offline

#13 2012-11-17 11:54:16

hunterthomson
Member
Registered: 2008-06-22
Posts: 794
Website

Re: How to trace http links called within programs?

Well ya,

read -p spotify pname && netstat -p TCP -veWc |grep $pname

Is not going to work. That is why I put...

read -p 'what is the name of the program you want to watch? ' pname && netstat -p TCP -veWc |grep $pname

The `read` command is printing...
what is the name of the program you want to watch?

And saving the value of what ever you type to the variable $pname until you hit the enter key

Then runs this command only if the read command compleated without errors

netstat -p TCP -veWc

Then pipes the output into grep, which will parse the output for the value of $pname

Last edited by hunterthomson (2012-11-17 12:01:55)


OpenBSD-current Thinkpad X230, i7-3520M, 16GB CL9 Kingston, Samsung 830 256GB
Contributor: linux-grsec

Offline

#14 2012-11-17 11:57:48

impact
Member
Registered: 2012-10-28
Posts: 55

Re: How to trace http links called within programs?

How is it different from what I did?

Offline

#15 2012-11-17 11:59:03

hunterthomson
Member
Registered: 2008-06-22
Posts: 794
Website

Re: How to trace http links called within programs?

Cut & Past this one line program into a BASH shell.

read -p 'what is the name of the program you want to watch? ' pname && netstat -p TCP -veWc |grep $pname

OpenBSD-current Thinkpad X230, i7-3520M, 16GB CL9 Kingston, Samsung 830 256GB
Contributor: linux-grsec

Offline

#16 2012-11-17 12:03:06

impact
Member
Registered: 2012-10-28
Posts: 55

Re: How to trace http links called within programs?

Ah, I see what you did there. But still no traffic is being shown.

Offline

#17 2012-11-17 12:09:17

hunterthomson
Member
Registered: 2008-06-22
Posts: 794
Website

Re: How to trace http links called within programs?

Ya, there may not be. It will refress every second.

Just start it and leave it running in a shell. Then start spotify and watch for connections being made.

Maybe I made it all to complicated.

Just run this in a shell and watch everything.

netstat -p TCP -veWc

OpenBSD-current Thinkpad X230, i7-3520M, 16GB CL9 Kingston, Samsung 830 256GB
Contributor: linux-grsec

Offline

#18 2012-11-17 14:43:10

kokoko3k
Member
Registered: 2008-11-14
Posts: 1,546

Re: How to trace http links called within programs?

@impact: are you sure the program you're monitoring didn't cached that banners?

Offline

#19 2012-11-18 09:51:48

impact
Member
Registered: 2012-10-28
Posts: 55

Re: How to trace http links called within programs?

hunterthomson wrote:

Ya, there may not be. It will refress every second.

Just start it and leave it running in a shell. Then start spotify and watch for connections being made.

It works! Thanks!

But I blocked all the urls I can see accessed, and yet some banners still show.

Last edited by impact (2012-11-18 09:54:50)

Offline

#20 2012-11-19 09:10:43

hunterthomson
Member
Registered: 2008-06-22
Posts: 794
Website

Re: How to trace http links called within programs?

Awe, you know... what you could do is this...

Install privoxy and configure it to use "Fanboy's Adblock List" Adblock Plus list. You can convert these to work with privoxy.

Install proxychains or a program like it, that will send all connections a program makes through the proxy.

Then like start the spotify like

proxychains spotify

OpenBSD-current Thinkpad X230, i7-3520M, 16GB CL9 Kingston, Samsung 830 256GB
Contributor: linux-grsec

Offline

#21 2012-11-19 10:59:30

impact
Member
Registered: 2012-10-28
Posts: 55

Re: How to trace http links called within programs?

You mean to use adblock? The thing is I already have one running (through hosts file) and I am just adding new addresses when I detect them. The database is huge and I am pretty sure it includes those from Fanboy's Adblock List.

Offline

Board footer

Powered by FluxBB