[TriLUG] Server DEAD!

[auto668 at hush.com]

Sorry, had a round of meetings this afternoon that have really set me
back, I'm booting to disk 1 now.. gonna see what happens with Jeremys'
directions below...

Thanks for all the awesome follow ups..

I'll keep you all posted as I go..

L-

On Thu, 28 Aug 2003 09:17:57 -0700 Jeremy Portzer <jeremyp at pobox.com>
wrote:
>
>Unfortunately it sounds as you either have serious disk corruption,
> or
>your server has been cracked into and root kitted.
>
>I would try booting in the rescue environment (use the 1st installation
>CD and type "linux rescue") and check some things.  For example,
> you
>might want to see if packages like fileutils and procps have the
>correct
>timestaps:
>	rpm --root /mnt/sysimage -V fileutils
>	rpm --root /mnt/sysimage -V procps
>
>Etc.
>
>If commands like those fail; ie the MD5 sums don't match on a large
>number of files, you need to do a backup the data, and then do a
>complete format and reinstall.  Be careful backing up the data,
>as you
>don't want to get any "pieces" of the root kit with it; it would
>be best
>to restore from a known clean backup source.
>
>I have a little too much experience with computers that have been
>rootkitted.  It's not a lot of fun.
>
>You may also wish to run the "chkrootkit" program (see Google) though
>I
>don't know how well that will run in a rescue environment.
>
>--Jeremy
>
>On Thu, 2003-08-28 at 12:05, auto668 at hush.com wrote:
>> Serious issue here, I've had a server running for a couple weeks
>doing
>> some production virtual hosting.  All has been running great,
>everything
>> was configured and running fine I haven't done ANYTHING other
>than run
>> uup2date periodically.  Well, today I'm about to do a test on
>the box
>> after installing the Real Media server and here's what happens...
>> 
>> [root at www Helix]# /etc/rc.d/init.d/iptables stop
>> /etc/rc.d/init.d/iptables: line 41: 14950 Done               
>    /sbin/lsmod
>> 2>/dev/null
>>      14951 Segmentation fault      | grep -q ipchains
>> 
>> [root at www Helix]# /etc/rc.d/init.d/iptables restart
>> /etc/rc.d/init.d/iptables: line 41: 14966 Done               
>    /sbin/lsmod
>> 2>/dev/null
>>      14967 Segmentation fault      | grep -q ipchains
>> 
>> ****SO I DECIDE, I'M LOST, LET'S just try rebooting for the sake
>of reboting**
>> 
>> Now it won't even come back up, I can't copy/paste but here is
>some of
>> what I'm getting
>> 
>> 45 Segmentation Fault
>>      LC_ALL=C grep -q "Red Hat" /etc/redhat-release  RedHat Linux
>> 
>> Mounting proc filesystem                               [FAILED]
>>     /etc/rc.d/rc.sysinit :  Line 98:   Segmentation Fault   LC_ALL=C
>> grep -q 
>> 
>> Coninues this for about 3/4 more lines and totally quits after
>setting
>> hostname.
>> 
>> I literally, haven't done anything other than load the updates
>using
>> up2date form the command line.  Only had ssh/apache running.
>> 
>> Any ideas would be greatly appreciate as I said this is a production
>> box and one customer has already called since this happened1
>> 
>> WHY ME!
>> 
>> laura
>> 
>> 
>> 
>> 
>> Concerned about your privacy? Follow this link to get
>> FREE encrypted email: https://www.hushmail.com/?l=2
>> 
>> Free, ultra-private instant messaging with Hush Messenger
>> https://www.hushmail.com/services.php?subloc=messenger&l=434
>> 
>> Promote security and make money with the Hushmail Affiliate Program:
>>
>> https://www.hushmail.com/about.php?subloc=affiliate&l=427
>-- 
>/---------------------------------------------------------------
>------\
>| Jeremy Portzer       jeremyp at pobox.com       trilug.org/~jeremy
>    |
>| GPG Fingerprint: 712D 77C7 AB2D 2130 989F  E135 6F9F F7BC CC1A
>7B92 |
>\---------------------------------------------------------------
>------/
>



Concerned about your privacy? Follow this link to get
FREE encrypted email: https://www.hushmail.com/?l=2

Free, ultra-private instant messaging with Hush Messenger
https://www.hushmail.com/services.php?subloc=messenger&l=434

Promote security and make money with the Hushmail Affiliate Program: 
https://www.hushmail.com/about.php?subloc=affiliate&l=427