Re: SV: [cobalt-security] bindshell'... INFECTED (PORTS: 1524 31337)

[Michael Stauber <cobalt@xxxxxxxxxxxxxx>]

Hi Kai,

> I will have a go for this. I must admit that I am a newbie and would be
> glad if somone could tell me a little spesefict how to do this:

Ok this is how you get LSOF:

Login to your machine by Telnet or SSH as admin and "su" to root. Then issue 
the following command to fetch and install LSOF:

rpm -hUv 
ftp://fr2.rpmfind.net/linux/redhat/6.2/en/os/i386/RedHat/RPMS/lsof-4.47-2.i386.rpm

The above is actually one long line.

Then issue the following command:

/usr/sbin/lsof -nl |grep LISTEN

That will fire up LSOF. The "-n" will suppress hostname lookups, "-l" will 
show UID numbers and we also pipe the output to "grep" to see on which ports 
the server is listening for connections.

Here is a sample line of that output:

squid     16030        0   11u  IPv4   27799531                 TCP 
206.131.240.157:8383 (LISTEN)

The first entry "squid" shows the name of the programm. As you can see it has 
opened a TCP connection on 206.131.240.157 on port 8383 and it's listening 
for connections. You'll never see that on a regular Cobalt, so this would be 
worth investigation.

In your case you'll take special care in the program names which listen on 
ports 1524 and 31337. LSOF will help you to determine what the programm name 
is. In the above case it was "squid", which is something we can then hunt for 
with the "find" command.

However, notice the "16030" in the above output. That is the PID (Process ID) 
of the process which opened the connection. So we now know which PID we can 
kill with the "kill" command.

But let's take a closer look with "ps", as we hunt for the process ID:

ps -auxw|grep 16030

Which in my case returns the following:

named    16030  0.3  1.9  8984 7452 ?        S    22:19   0:14 (squid)
root     22821  0.0  0.1  1196  476 pts/1    S    23:25   0:00 grep 16030

The first line is what we're looking for. The second line is just the process 
of our search as you can easily determine from the end of the line.

So process ID 16030 runs as user "named", runs since 22:19 MET (server time). 

We can now kill it with "klll -9 16030". However, you must keep two things in 
mind:

a) The "ps" command is often replaced by hackers with a compromised binary 
which will not show any hacker processes. So if LSOF (which we know to be 
unmodified) tells you PID 16030 is fishy and "ps" claims that there is no 
such process, then you know your "ps" has been tampered with.

b) If you kill the process, then you have not closed the backdoor. After the 
next reboot or most likely sooner it might be back, because one script or 
another just launches the bindshell again.

So you really need to find out how the hacker got in in first place (and you 
need to close that hole) and you need to find out what modifications he did 
to your system (and you need to fix 'em all back to normal).

If LSOF and/or "ps" tell you that "inetd" launched the process, then you know 
where to look: /etc/inetd.conf

Such a backdoor in inetd.conf could look like this:

2525 stream tcp nowait root /bin/sh

That line does bind a shell to port 2525 TCP. Upon connection by telnet to 
that port the attacker is directly in as user "root" without being prompted 
for a password.

If you find such a line, then remove it. Also check your /etc/passwd and 
/etc/shadow for user names and accounts you do not know. If you are in doubt 
if a username is that of a valid customer or maybe from a hacker, then assign 
this account the shell /bin/badsh which will make it impossible for them to 
login by SSH or Telnet. 

The shell is assigned at the end of each line in /etc/passwd

Also make sure that there are no cronjobs which you do not know about, 
especially root owned ones.

Run "crontab -l" on your machine as root. Cobalts do normally not have any 
cronjobs assigned this way. If you find any using "crontab -l" which you (or 
I for that matter <g>) put there, then remove 'em through "crontab -r".

Then go to /etc and sift through all the cron* directories in there. Check 
out every file you find in the different directories and make sure they are 
legit and untampered.

Also a nice way to check RPMs for their validity is to run "rpm -Va", which 
will cross-check the RPM database and installed binaries and will then return 
a (somewhat cryptic) list which can help to tack down binaries (and 
configuration files) which have been changed since they have been extracted 
from their originating RPMs. However, you will need to look at the manpages 
to look the codes up (see: man rpm) and thanks to the dirty PKG way 
Sun/Cobalt distributes patches this can turn up false hits even on legitimate 
files.

Once you have cleaned house and are sufficently certain that you have 
effectively removed all backdoors, then remain very, very vigilant. Install 
Logwatch, install Portsentry, get IPchains, get Tripwire or Fcheck to check 
the consistency and untampered state of your filesystem and if you feel 
adventurous, then even get Snort and Demarc up and running.

And check the output of those tools frequently to see if the hackers come 
back and can still worm their way into your system.

-- 

With best regards,

Michael Stauber
mstauber@xxxxxxxxxxxxxx
Unix/Linux Support Engineer