# Debugging a mystery Unix server

I was given an Unix server to administer with no runbook. The immediate task was to find the application listening on port 8888, make some modifications, and restart.

### Step 1: Find process listening on port 8888

The [netstat](https://man7.org/linux/man-pages/man8/netstat.8.html) command shows the network statistics on a Unix server, including which process is listening to what port

`netstat -plan | grep 8888`  
  
The output is

> tcp     0   0 :::8888   :::\*    LISTEN      2700/java 

This shows that a java program with Process ID (PID) 2700 is listening on port 8888

### Step 2: Find process info of PID 2700

The [ps](https://man7.org/linux/man-pages/man1/ps.1.html) command shows the current process snapshot.

`ps -fp 2700`  
  
The output is

> oracle  2700   1  0 18:55 ?     0:00:02   
> java -Dapex.port=8888 -jar /home/oracle/listener/apex.war

PID 2700 is a Java program executing apex.war as the Unix user oracle. A quick Google search reveals that Oracle publishes a product called [Oracle Apex](https://www.oracle.com/apex/).

### Step 3: Find the configuration and log files

The [lsof](https://man7.org/linux/man-pages/man8/lsof.8.html) command lists open files by PID. This can help us find configuration files and log directories.

`lsof -p 2700`

The output is

> java    2700 oracle    2w   REG 3,1 1700 1796967 /tmp/apex\_listener.log  
> java    2700 oracle    3r   REG. 3,65. 9828505  134585 /home/oracle/listener/apex.war  
> java    2700 oracle    4r   REG 3,1 51796975  361047 /usr/java/jdk1.6.0\_20/jre/lib/rt.jar  
> java    2700 oracle    5r   REG. 3,65  186902  132940 /home/oracle/config/apex.xml

With a bit of deductive reasoning, we know the configuration file is at /home/oracle/config/apex.xml and the log file is located at /tmp/apex\_listener.log

### Step 4: Stop and start the server

After editing the apex.xml configuration file, most java programs require a restart to apply the new settings. If you scroll back up to Step 2 and look at the `ps` output, the number after PID 2700 in the third column is the parent PID 1.

The command

`ps -fp 1`

gives the output

> root    1     0  0 18:54 ?    00:00:00 init \[5\]

The [init](https://linux.die.net/man/8/init) command is the process that executes when Unix first boots up.  The startup scripts are in directory ***/etc/init.d***

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">Note from 2026: This article was first written in 2013, hence the XML config file and the init command. In 2026, it would be YML config files and systemd at PID 1.</div>
</div>

The [find](https://man7.org/linux/man-pages/man1/find.1.html) command to find which script stops and starts Oracle Apex

`find /etc/init.d -type f -exec grep -Hi apex {} \;`

The output is

> /etc/init.d/oracle-apex:   su - oracle -c "/home/oracle/apex/listener.sh start"
> 
> /etc/init.d/oracle-apex:   su - oracle -c "/home/oracle/apex/listener.sh restart"
> 
> /etc/init.d/oracle-apex:   su - oracle -c "/home/oracle/apex/listener.sh stop"

Reading the `oracle-apex` Bash script, the command to restart Oracle Apex is

`/etc/init.d/oracle-apex restart`