[cobalt-security] Re: Cobalt Raq 4 Hacked

[Cory Hollingsworth <Cory.Hollingsworth@xxxxxxxx>]

> Subject: [cobalt-security] Cobalt Raq 4 Hacked
> 
> Hi all.
> Today at 2:00 in the morning our RaQ4 was hacked.
> 
> The hack replaced all files named index.* with own
> hacker content.
> 
> We have had this issue a couple of weeks ago. That
> time the hack afected all index.* files under
> /home/sites.
> 
> Now the hack affected all index.* files under / so the
> Control Panel is affected too.
> 
> I must note a week ago the server wass rebuilded formm
> scratch and ALL upgrades were applied.
> 
> I have chkrootkit and portsentry installed. None of
> those reported strange activity.
> 
> I have checked open ports in the server and theres' no
> strange ports opened.
> 
> I want to know if somebody has experienced the same
> issue, and any help will be apreciated..
> 
> TIA 
> Pablo 

I've had a similar problem with a Raq 4 last week.  Instead my issue was someone using the server as a DOS attack machine.  Their attack removed /var/log and turned off all network services into the machine.  Also unbenounced to me on an earlier date Apache stopped logging in the /home/sites/XXX/log sub directories.

Even though the machine was fully patched, the Apache logging had halted before I applied the latest patches.  It may have been compromised months before it was used as a DOS attack machine.

Their DOS attack I believe unintentionally wiped out the root filesystem so the machine could no longer boot.  It took a couple of days of hacking at the serial console to get the machine working well enough to ssh the accounts and data off the machine.

I've since moved everything onto a Raq550 I had hanging around as a spare.  The only things lost were the users passwords as I couldn't easily load them from the old shadow file as the Raq550 stores passwords in a BTREE in /var/db/shadow.db.

As of yet I haven't figured out how the machine was exploited.  But we had 600+ users on the machine with CGI, so it is not infeasible the exploit could have been in user CGI rather than the core services.

Any one have any luck with tripwire on Raqs?  That is the only thing I can think of trying at this time.