[TriLUG] help with fuser/ssh reporting lots of processes
Ken MacKenzie via TriLUG
trilug at trilug.org
Mon Jul 18 09:47:04 EDT 2016
I get lots of those, I hope it goes without saying you should use fail2ban
to catch repeat brute force offenders.
I have 2 jail setups, a quick ban and a repeat offender ban. On occasion I
check the logs and I add problem IP addresses to the blrules file (I use
shorewall for my firewall setup).
/etc/shorewall/blrules is the black list file for shorewall. Once in there
it is a permanent ban on the offending IP address. I eventually need to
automate that process but for now the occasional continued trouble maker is
not a big deal to handle manually.
Ken
On Mon, Jul 18, 2016 at 8:55 AM, Tim Jowers via TriLUG <trilug at trilug.org>
wrote:
> Thank you William and Roy,
>
> I see lots of things in /var/log/secure now! Tons of filed logins from
> China as I changed password and removed root from ssh login access. Duh! I
> know, slacker for not doing that before. Now my fuser ssh/tcp only shows
> one process and no others are showing up. Using an ssh key for ssh from now
> on and realize I can't be sloppy as crackers are out there. (I wonder if
> there is some lawsuit which could be filed against this outfit as I am sure
> they have US customers.)
> E.g. /var/log/secure has:
>
> Jul 18 10:02:12 test1 sshd[29252]: Failed password for root from
> 221.229.172.99 port 31660 ssh2
>
> Jul 18 10:02:14 test1 sshd[29254]: Failed password for root from
> 112.85.42.99 port 22571 ssh2
>
> Jul 18 10:02:14 test1 sshd[29252]: Failed password for root from
> 221.229.172.99 port 31660 ssh2
>
> Have a great week everyone,
>
> Tim
>
>
>
>
> On Mon, Jul 18, 2016 at 8:36 AM, Roy Vestal <rvestal at trilug.org> wrote:
>
> > Hi Tim,
> >
> > I did a quick whois on the 221.229.172.99 and found that is is a Chinese
> > IP:
> >
> > inetnum: 221.224.0.0 - 221.231.255.255
> > netname: CHINANET-JS
> > descr: CHINANET jiangsu province network
> > descr: China Telecom
> > descr: A12,Xin-Jie-Kou-Wai Street
> > descr: Beijing 100088
> > country: CN
> > admin-c: CH93-AP
> > tech-c: CJ186-AP
> > mnt-by: APNIC-HM
> > mnt-lower: MAINT-CHINANET-JS
> > mnt-routes: MAINT-CHINANET-JS
> > remarks: This object can only modify by APNIC hostmaster
> > remarks: If you wish to modify this object details please
> > remarks: send email to hostmaster at apnic.net with your
> > remarks: organisation account name in the subject line.
> > status: ALLOCATED PORTABLE
> > source: APNIC
> > mnt-irt: IRT-CHINANET-CN
> > changed: hm-changed at apnic.net 20030626
> >
> > irt: IRT-CHINANET-CN
> > address: No.31 ,jingrong street,beijing
> > address: 100032
> > e-mail: anti-spam at ns.chinanet.cn.net
> > abuse-mailbox: anti-spam at ns.chinanet.cn.net
> > admin-c: CH93-AP
> > tech-c: CH93-AP
> > auth: # Filtered
> > mnt-by: MAINT-CHINANET
> > changed: anti-spam at ns.chinanet.cn.net 20101115
> > source: APNIC
> >
> > role: CHINANET JIANGSU
> > address: 260 Zhongyang Road,Nanjing 210037
> > country: CN
> > phone: +86-25-86588231
> > phone: +86-25-86588745
> > fax-no: +86-25-86588104
> > e-mail: ip at jsinfo.net
> > remarks: send anti-spam reports to spam at jsinfo.net
> > remarks: send abuse reports to abuse at jsinfo.net
> > remarks: times in GMT+8
> > admin-c: CH360-AP
> > tech-c: CS306-AP
> > tech-c: CN142-AP
> > nic-hdl: CJ186-AP
> > remarks: www.jsinfo.net
> > notify: ip at jsinfo.net
> > mnt-by: MAINT-CHINANET-JS
> > changed: dns at jsinfo.net 20090831
> > changed: ip at jsinfo.net 20090831
> > changed: hm-changed at apnic.net 20090901
> > source: APNIC
> > changed: hm-changed at apnic.net 20111114
> >
> > person: Chinanet Hostmaster
> > nic-hdl: CH93-AP
> > e-mail: anti-spam at ns.chinanet.cn.net
> > address: No.31 ,jingrong street,beijing
> > address: 100032
> > phone: +86-10-58501724
> > fax-no: +86-10-58501724
> > country: CN
> > changed: dingsy at cndata.com 20070416
> > changed: zhengzm at gsta.com 20140227
> > mnt-by: MAINT-CHINANET
> > source: APNIC
> >
> > % Information related to '221.228.0.0/14AS23650'
> >
> > route: 221.228.0.0/14
> > descr: CHINANET jiangsu province network
> > country: CN
> > origin: AS23650
> > mnt-by: MAINT-CHINANET-JS
> > changed: ip at jsinfo.net 20030630
> > source: APNIC
> >
> >
> > Try an lsof of each port and see what is using the tcp connections:
> >
> > $> lsof -i :48079
> >
> > You should see the command, pid, and user that is using that port. From
> > there you could use the lsof command again to see what spawned the
> session:
> >
> > lsof -p PID (replace PID with the actual PID in the response)
> >
> >
> > From there you might be able to determine what is creating the ssh
> > connection.
> >
> > HTH,
> >
> > -Roy
> >
> >
> >
> > On 7/18/16 8:19 AM, Tim Jowers via TriLUG wrote:
> >
> >> Hi,
> >>
> >> I run these two less than a second apart:
> >>
> >> [root at test1 log]# fuser ssh/tcp
> >>
> >> ssh/tcp: 685 5066 5283 5284 5289 5290 5291 5292
> 5293
> >> 5294
> >>
> >> [root at test1 log]# fuser ssh/tcp
> >>
> >> ssh/tcp: 685 5066 5289 5290 5293 5294 5296 5297
> 5298
> >> 5299
> >>
> >>
> >> Any ideas how to troubleshoot? I think I have some Chinese search
> bot
> >> malware based on this:
> >>
> >> [root at test1 log]# lsof -i
> >>
> >> COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
> >>
> >> sshd 685 root 3u IPv6 350221175 0t0 TCP *:ssh (LISTEN)
> >>
> >> sshd 685 root 4u IPv4 350221177 0t0 TCP *:ssh (LISTEN)
> >>
> >> mysqld 811 mysql 10u IPv4 350221673 0t0 TCP *:mysql
> (LISTEN)
> >>
> >> sshd 5066 root 3r IPv4 4054471422 0t0 TCP
> >> 198-20-184-56-host.colocrossing.com:ssh->
> >> cpe-45-37-198-154.nc.res.rr.com:59006 (ESTABLISHED)
> >>
> >> sshd 5361 root 3r IPv4 4054875967 0t0 TCP
> >> 198-20-184-57-host.colocrossing.com:ssh->112.85.42.99:41796
> (ESTABLISHED)
> >>
> >> sshd 5362 sshd 3u IPv4 4054875967 0t0 TCP
> >> 198-20-184-57-host.colocrossing.com:ssh->112.85.42.99:41796
> (ESTABLISHED)
> >>
> >> sshd 5365 root 3r IPv4 4054877149 0t0 TCP
> >> 198-20-184-56-host.colocrossing.com:ssh->112.85.42.99:openmailpxy
> >> (ESTABLISHED)
> >>
> >> sshd 5366 sshd 3u IPv4 4054877149 0t0 TCP
> >> 198-20-184-56-host.colocrossing.com:ssh->112.85.42.99:openmailpxy
> >> (ESTABLISHED)
> >>
> >> sshd 5369 root 3r IPv4 4054886185 0t0 TCP
> >> 198-20-184-56-host.colocrossing.com:ssh->221.229.172.99:36122
> >> (ESTABLISHED)
> >>
> >> sshd 5370 sshd 3u IPv4 4054886185 0t0 TCP
> >> 198-20-184-56-host.colocrossing.com:ssh->221.229.172.99:36122
> >> (ESTABLISHED)
> >>
> >> sshd 5371 root 3r IPv4 4054886747 0t0 TCP
> >> 198-20-184-57-host.colocrossing.com:ssh->221.229.172.99:15096
> >> (ESTABLISHED)
> >>
> >> sshd 5372 sshd 3u IPv4 4054886747 0t0 TCP
> >> 198-20-184-57-host.colocrossing.com:ssh->221.229.172.99:15096
> >> (ESTABLISHED)
> >>
> >> java 18216 root 43u IPv6 3405192816 0t0 TCP *:webcache
> >> (LISTEN)
> >>
> >> java 18216 root 48u IPv6 3405192820 0t0 TCP *:8009
> (LISTEN)
> >>
> >> java 18216 root 72u IPv6 3405192937 0t0 TCP
> >> localhost.localdomain:8005 (LISTEN)
> >>
> >> httpd 26003 apache 3u IPv6 3253453758 0t0 TCP *:http
> (LISTEN)
> >>
> >> httpd 26361 apache 3u IPv6 3253453758 0t0 TCP *:http
> (LISTEN)
> >>
> >> httpd 27165 apache 3u IPv6 3253453758 0t0 TCP *:http
> (LISTEN)
> >>
> >> httpd 27818 root 3u IPv6 3253453758 0t0 TCP *:http
> (LISTEN)
> >>
> >> and
> >>
> >> [root at test1 log]# netstat -a
> >>
> >> Active Internet connections (servers and established)
> >>
> >> Proto Recv-Q Send-Q Local Address Foreign Address
> >> State
> >>
> >> tcp 0 0 *:ssh *:*
> >> LISTEN
> >>
> >> tcp 0 0 *:mysql *:*
> >> LISTEN
> >>
> >> tcp 0 0 198-20-184-57-host.colo:ssh 112.85.42.99:15265
> >> ESTABLISHED
> >>
> >> tcp 0 0 198-20-184-56-host.colo:ssh 221.229.172.99:48079
> >> TIME_WAIT
> >>
> >> tcp 0 0 198-20-184-56-host.colo:ssh 221.229.172.99:33195
> >> ESTABLISHED
> >>
> >> tcp 0 0 198-20-184-57-host.colo:ssh 221.229.172.99:44556
> >> ESTABLISHED
> >>
> >> tcp 0 0 198-20-184-57-host.colo:ssh 221.229.172.99:15096
> >> TIME_WAIT
> >>
> >> tcp 0 608 198-20-184-56-host.colo:ssh
> >> cpe-45-37-198-154.nc.:59006
> >> ESTABLISHED
> >>
> >> tcp 0 0 198-20-184-56-host.colo:ssh 112.85.42.99:42180
> >> ESTABLISHED
> >>
> >> tcp 0 0 *:webcache *:*
> >> LISTEN
> >>
> >> tcp 0 0 *:http *:*
> >> LISTEN
> >>
> >> tcp 0 0 *:ssh *:*
> >> LISTEN
> >>
> >> tcp 0 0 localhost.localdomain:8005 *:*
> >> LISTEN
> >>
> >> tcp 0 0 *:8009 *:*
> >> LISTEN
> >>
> >> tcp 0 0 198-20-184-56-host.col:http
> >> ns336619.ip-37-187-16:18286
> >> TIME_WAIT
> >>
> >> tcp 0 0 198-20-184-56-host.col:http
> >> hydrogen081.a.ahrefs.:30831
> >> TIME_WAIT
> >>
> >> and some StackOverflow article where someone posted that
> *221.229.172.99*
> >> is a Chinese search botnet.
> >>
> >> last and lastlog don't show anything. There is no /var/log/auth.log
> >> present. Not sure if there should be. Just tried things based on
> Internet
> >> searching.
> >>
> >> I guess there is no easy way to kill this? Sounds like I should just
> ask
> >> for a new server instance (ChicagoVPS)? I use SVN to back up my files
> >> there.
> >>
> >>
> >> Thanks for any ideas.
> >>
> >> Tim
> >>
> >
> >
> --
> This message was sent to: Ken M. <ken at mack-z.com>
> To unsubscribe, send a blank message to trilug-leave at trilug.org from that
> address.
> TriLUG mailing list : http://www.trilug.org/mailman/listinfo/trilug
> Unsubscribe or edit options on the web :
> http://www.trilug.org/mailman/options/trilug/ken%40mack-z.com
> Welcome to TriLUG: http://trilug.org/welcome
>
More information about the TriLUG
mailing list