Arch Linux

Nick_Name

Member

Registered: 2022-11-07

Posts: 18

How to identify a proccess which trying to establish connection?

Hello linux users! Can you help me with one strange question?...

Is there any not very complicated approach to identify a process, which trying to establish connection on local loop interface in the case when connection can not be established?

For instance: polybar has a connection with mopidy on lo on 6600 port. If I kill mopidy, polybar will atempt to establish connection via tcp protocol with mopidy, requesting 6600 port every few seconds with no success.

If connection is not established I can not see information about this in utils like ss, nethogs, lsof, iftop, netstat, etc.

But in wireshark - yes, and in tcpdump yes, and in iptables log yes. But from these sources I could not identify process, which trying to establish connection with killed mopidy.

So, is it possible to identify such process?

Thank you!

Last edited by Nick_Name (2024-01-30 14:07:44)

Strike0

Member

From: Germany

Registered: 2011-09-05

Posts: 1,429

Re: How to identify a proccess which trying to establish connection?

Interesting question. Looking into it, I've come across https://github.com/mvitale1989/ptrap but did not get around to trying it.
I relies on syslog, but is just a bash script - perhaps it helps.

seth

Member

Registered: 2012-09-03

Posts: 51,348

Re: How to identify a proccess which trying to establish connection?

https://wiki.archlinux.org/title/Audit :
https://unix.stackexchange.com/question … hey-happen

Otherwise if you poll ss *very* tightly you might be able to catch the process in the act - not overly likely, though.

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

Nick_Name

Member

Registered: 2022-11-07

Posts: 18

Re: How to identify a proccess which trying to establish connection?

Interesting question. Looking into it, I've come across https://github.com/mvitale1989/ptrap but did not get around to trying it.
I relies on syslog, but is just a bash script - perhaps it helps.

https://wiki.archlinux.org/title/Audit :
https://unix.stackexchange.com/question … hey-happen

Otherwise if you poll ss *very* tightly you might be able to catch the process in the act - not overly likely, though.

Thank you for your answers!

NewArchUser0001

Member

Registered: 2024-03-29

Posts: 15

Re: How to identify a proccess which trying to establish connection?

Hello linux users! Can you help me with one strange question?...

Is there any not very complicated approach to identify a process, which trying to establish connection on local loop interface in the case when connection can not be established?

For instance: polybar has a connection with mopidy on lo on 6600 port. If I kill mopidy, polybar will atempt to establish connection via tcp protocol with mopidy, requesting 6600 port every few seconds with no success.

If connection is not established I can not see information about this in utils like ss, nethogs, lsof, iftop, netstat, etc.

But in wireshark - yes, and in tcpdump yes, and in iptables log yes. But from these sources I could not identify process, which trying to establish connection with killed mopidy.

So, is it possible to identify such process?

Thank you!

This is something i have been looking into for years on and off. I was also trying to find a way to see what process was doing what connections out to the internet. Needless to say like you i couldn't find the right solution. In my case i wanted to see the DNS names that matched the IP and this was something that ended up turning into two different processes to get the answer.

As I'm sure you're aware SS will give IP addresses and process name however it will not give you proper DNS names that made the call to that IP address even when using resolve DNS entries on some programs they couldn't resolve the DNS for the IP or didn't come back with the DNS i wanted to see.

So an example of this is using SS i can see Firefox but Firefox has a load of IP's show up if you have a few pages open, so then you need something else recording all the IP traffic coming and going to the computer and to do that i use TCPDUMP. I have a special command line that uses grep to only grab the A? and A records from the TCPDUMP and these are linked with a ID so you can match them up and then with that i can see what IP and process belongs to a certain DNS name that a program used to talk out to the internet.

The problem was i wanted something in real time-ish that would get the process name and the IP and SS does that but like mentioned in this thread some process can execute so fast
that you miss stuff and so with that in mind i put together a python scrip that would run the SS command and do so fairly fast about every 200 ms and do this while inside a loop that runs like 50 passes and each pass it appends the text output to a buffer with the results of the SS command, after the 50 passes i do a sort on that and de duplicate on the buffer and then print out the results and with this i can have some idea in realtime whats going on but again i still have to do that second stage of skimming the tcpdump text file to match up any IP's and DNS names from SS IP i haven't seen before.

This is not an ideal thing I'm doing but the only way i can see to get the info i want I'm really surprised there isn't a tool that can link all this stuff up and give you a complete picture
in real time.

Note that my python script is made so its on a 10 second delay so it can dump the data to screen and that 10 second delay allows me to glance at whats there and while delay is
active the buffer is refilling in the background and getting ready to display the next page of data after it clears the screen.

When chat-gpt came out i was messing around with that to see what all it could do in python and i did get it to spit out some code that could display very fasts aka short lived processes and so i may re visit that at some point and see if can get SS to run quick enough to not miss much of anything. Right now SS and my python script seem to have issues catching things like the ping command when you run that it makes a UDP call out to 192.168.1.1:53 and it can take quite a while to catch that and i run ping every minute to keep tabs on a server so it can be tricky to catch.

The reason i picked the 250 ms loop was that didn't bog the cpu down at all if you start going now nuts and say your loop is at 50 ms or less you cpu usage can climb quite a bit. Also note I'm not a programmer so I'm sure someone with some skills could do a much better job with something like this.

Brocellous

Member

Registered: 2017-11-27

Posts: 146

Re: How to identify a proccess which trying to establish connection?

Use bpf. There are (many) example bpf scripts, programs, and tools that do exactly this. For example the following script shipped in the examples with bpftrace will do what you want:

$ sudo /usr/share/bpftrace/tools/tcpconnect.bt
Tracing tcp connections. Hit Ctrl-C to end.
TIME     PID      COMM             SADDR                                   SPORT  DADDR                                   DPORT  
21:27:23 57516    nc               127.0.0.1                               35998  127.0.0.1                               1234  

NewArchUser0001

Member

Registered: 2024-03-29

Posts: 15

Re: How to identify a proccess which trying to establish connection?

Use bpf. There are (many) example bpf scripts, programs, and tools that do exactly this. For example the following script shipped in the examples with bpftrace will do what you want:

$ sudo /usr/share/bpftrace/tools/tcpconnect.bt
Tracing tcp connections. Hit Ctrl-C to end.
TIME     PID      COMM             SADDR                                   SPORT  DADDR                                   DPORT  
21:27:23 57516    nc               127.0.0.1                               35998  127.0.0.1                               1234  

Can bpftrace also get UDP connection information as well as DNS information along with TCP information all at the same time?

Brocellous

Member

Registered: 2017-11-27

Posts: 146

Re: How to identify a proccess which trying to establish connection?

To be clear, bpf sure can do those things, but it's a lower level tool and not always the easiest solution.

You can monitor udp connections, but most udp sockets are connectionless. You can monitor the fib lookups with bpf I guess, but you'll have to filter that down to the set you're interested in. And you already have a good solution for monitoring dns resolutions via tcpdump, I don't think bpftrace is more ergonomic there. You could also use tools like `resolvectl monitor`, if you don't anticipate that your process of interest will circumvent the local resolver — that will give you insight into DoT connections made by sd-resolved on behalf of your clients. If you need to search backward, and if you're using sd-resolved, `resolvectl show-cache --json` will dump the resolver's caches, and you can search the json document for the names that map to a given address. This is usually more insightful than PTR info, especially when the target server is a cloud tenant.