lsof is a great tool for debugging applications running on linux. lsof is short for “list open files”. As the name suggests, the tool lists and searches for open files on a linux host. Because everything is a file, lsof can not only be used see who is touching what file, but can also be used to get a quick view of what the network is doing.

Here are some useful snippets I find myself using frequently are some useful snippets I find myself using frequently.

This is not a comprehensive guide to lsof.

What applications are listening on a port?

You can find out what applications are listening on a port with -i. Running this on my laptop, I can find out that nginx is listening on port 80.

$ sudo lsof -i :80 -s TCP:LISTEN
nginx   1305 nginx    6u  IPv4 1613873      0t0  TCP *:http (LISTEN)
nginx   1305 nginx    7u  IPv6 1613874      0t0  TCP *:http (LISTEN)
nginx   1306 nginx    6u  IPv4 1613873      0t0  TCP *:http (LISTEN)
nginx   1306 nginx    7u  IPv6 1613874      0t0  TCP *:http (LISTEN)
...

what ports are being listened to by an application?

You can also go the other way. Say you know a host is running some application, but you’ve forgotten which port it’s listening on. First get the pid of the application, then use lsof -p.

$ sudo lsof -p 1305 | grep LISTEN
nginx   1305 nginx    6u     IPv4            1613873      0t0     TCP *:http (LISTEN)
nginx   1305 nginx    7u     IPv6            1613874      0t0     TCP *:http (LISTEN)

What logs are my applications writing to?

Sometimes you need to debug an application, but you don’t know where the logs are.

You can list all files open by a process. Often grepping for log will reveal where a process is logging to:

nginx   1305 nginx    2w      REG              253,0      606 2490627 /var/log/nginx/error.log
nginx   1305 nginx    4w      REG              253,0      606 2490627 /var/log/nginx/error.log
nginx   1305 nginx    5w      REG              253,0      833 2491029 /var/log/nginx/access.log

Note that this only works when the application is writing directly to log files itself. It’s more common for applications to be configured to log to stdout/stderr and delegate the log file management elsewhere. In those cases, these commands won’t work.

What network connections are my applications making?

This is helpful to verify that an application is making the right connections to other hosts. This is often helpful with services that talking to eachother, but I decided to see what my local Google Chrome was connecting to. This is only a subset of what I found, but you can see connections over IPv4 to Github and AWS.

$ sudo lsof  -p 3129  | grep TCP
chrome  3129 leeavital   58u     IPv4            2168005       0t0     TCP localhost.localdomain:42752->lb-192-30-253-124-iad.github.com:https (ESTABLISHED)
chrome  3129 leeavital   59u     IPv4            2165690       0t0     TCP localhost.localdomain:42020->ec2-52-55-142-125.compute-1.amazonaws.com:https (ESTABLISHED)
...

Note here that lsof is doing some rewriting of hostnames and ports to make the output easier to read. For example, lb-192-30-253-124-iad.github.com instead of the IP address, and https instead of port 443. Sometimes this helpful rewriting is slow or wrong. It can be disabled with -n and -P respectively.

$ sudo lsof  -p 3129  -nP | grep TCP
chrome  3129 leeavital   58u     IPv4            2168005       0t0     TCP 192.168.0.165:42752->192.30.253.124:443 (ESTABLISHED)
chrome  3129 leeavital   59u     IPv4            2165690       0t0     TCP 192.168.0.165:42020->52.55.142.125:443 (ESTABLISHED)