[TriLUG] Cautionary Tale: Intel/McAfee VSEL & Python Web Apps

Fri Dec 16 14:16:45 EST 2016

Jack Sparrow <http://www.imdb.com/name/nm0000136/?ref_=tt_trv_qu>: The only
rules that really matter are these: what a man can do and what a man can't
do.
.......
Barbossa <http://www.imdb.com/name/nm0001691/?ref_=tt_trv_qu>: .... the
code is more what you'd call "guidelines" than actual rules.

jcj

On Fri, Dec 16, 2016 at 1:58 PM, Scott Chilcote via TriLUG <
trilug at trilug.org> wrote:

> Mr. Rogers raises an interesting point.
>
> McAfee VSEL, for all of its faults, also has a web console that can be
> quite useful in debugging issues like the one we had.  For example, one
> of its displays shows what the active scanner in doing, in something
> that approximates real-time.
>
> However, if your $CUSTOMER is obligated under contractual agreements to
> follow government cyber-security guidelines for the configuration of
> their servers and VSEL in particular, then $CUSTOMER's VSEL team may not
> realize that it exists.  That's because the Security Technical
> Implementation Guide recommends that the web UI in VSEL must be disabled.
>
> This is a routine policy intended to prevent it from being accessed by
> bad people via the internet.
> [Details here, if you're interested:
> https://www.stigviewer.com/stig/mcafee_vsel_1.92.0_
> managed_client/2016-04-06/finding/V-63051]
>
> I see this as a new-ish problem that the next generation of computer
> security people are going to need to address.
>
> Implementation guidelines are *not* commandments handed down from on
> high!  Just because it's a best practice to keep the UI disabled on a
> production server does NOT mean 1) that you can't ever turn it on, like
> when it might be useful to fix a problem, and 2) that you don't need to
> learn about it, experiment with it, and gain some experience with it so
> that you don't have to start from zero when that problem arrives.
>
> If certain persons responsible for VSEL in this situation had not taken
> the guidelines for stone carvings, we would have gotten through this
> investigation significantly faster.
>
>    Scott C.
>
>
> On 12/16/2016 11:00 AM, Rogers, Matthew wrote:
> > If you want to talk IDS/Antimalware, checkout Cisco AMP.  Has
> Linux/Mac/Win support and runs off Talos.  Huge fan of Cisco lately in this
> area.  You manage it all via web console.
> >
> > http://www.cisco.com/c/en/us/products/security/fireamp-
> endpoints/index.html
> >
> > Matthew Rogers
> > Senior Security Manager
> > O: +1.919.329.1130    M: +1.919.691.4636
> > allscripts.com | matthew.rogers2 at allscripts
> >
> > Allscripts: Building open, connected communities of health
> > _______________________________________________
> >
> > -----Original Message-----
> > From: TriLUG [mailto:trilug-bounces+matthew.rogers2=allscripts.
> com at trilug.org] On Behalf Of Scott Chilcote via TriLUG
> > Sent: Friday, December 16, 2016 10:55 AM
> > To: Triangle Linux Users Group discussion list <trilug at trilug.org>
> > Subject: [TriLUG] Cautionary Tale: Intel/McAfee VSEL & Python Web Apps
> >
> > Hi LUGgers,
> >
> > VSEL is VirusScan Enterprise for Linux.  My employer provides some web
> applications based on a JavaScript/Python framework for our $CUSTOMER.
> > $CUSTOMER mandates the use of Intel/McAfee VSEL for its RHEL webservers,
> which is not a bad idea in itself.  However, read on.
> >
> > VSEL does active signature-based scanning of all scripts launched via
> Apache, among much else.  We discovered in early testing that the VSEL
> scanner processes had a noticeable impact on our script performance.
> > This led to substantial troubleshooting.
> >
> > We ran "top" and found that the VSEL scanner tasks were all at the upper
> end, pushing python way down.  Scanner tasks were getting around 60% of
> core utilization, python around 3%.
> >
> > Some of our scripts are CPU intensive for brief periods.  They read zip
> files in a repository and build hashes based on their contents.  So we
> spoke with $CUSTOMER's people who handle VSEL.  They decided to try adding
> "exclusions" for our web app's python scripts, which is how you tell VSEL
> to trust installed scripts.
> >
> > The performance immediately improved, so we moved on.
> >
> > Some weeks further into the code cleanup and system tests, we find that
> VSEL is clobbering our scripts again.  It seems to have stopped honoring
> its exclusions.  The VSEL team runs a test using a "fake virus"
> > sentinel, and proves it.  They uninstall/reinstall VSEL, and we're good
> again.  Back to work.
> >
> > A few weeks later with final tests approaching, VSEL interference is
> worse than ever.  Even low demand script operations, like reading directory
> contents are staggering.
> >
> > We go back to work with the VSEL people.  They uninstall/reinstall VSEL
> again.  Nope.  They add all of the data directories as exclusions ("just in
> case").  No help.  They add wildcard exclusions, removing swaths of
> directories and file types we barely touch.  No help.  They call Intel
> Security (who bought McAfee recently).  After they finally get through,
> Intel's support people start working down a "we think your dumb"
> > checklist of steps that we ruled out weeks ago. S-l-o-w-l-y.
> Frustration!
> >
> > We go through more weeks of trying stuff and head scratching.  Deadlines
> impend.
> >
> > Then we have that zen moment.  "Do we have *all* of the python scripts
> that the web applications use excluded?", they ask.  "Well yeah, all of the
> ones we wrote.  The entire web application directories are wild-carded."
> >
> > "Are there others?"
> >
> > "Well, sure.  The python API libraries.  Those are part of the system.
> > Redhat installs those...!   Aren't they excluded by default??"
> >
> > We aren't VSEL guys.  We aren't HBSS or active IDS people.  Why would
> VSEL waste time scanning the system libraries?  Sure, doing a signature
> check on them following a system update, that makes sense.  But would VSEL
> be designed to signature scan the system libs *every* *time* they were
> accessed by a running application?  Seriously?
> >
> > As it turns out, McAfee thinks so.
> >
> > We added "/usr/lib64/python[2,3].[0-9]/.*/.*.py" to the exclusion list,
> which covers all of the system Python libraries.  The slowdown vanished.
> >
> > We bought our own license for VSEL to help this problem (which costs
> beer money).  I'm not saying everyone should, working with Intel Security
> brings its own challenges.  But for us it was time to learn up.  My idea of
> a decent IDS was tripwire, until recently.  With the Linux webserver hacks
> that are widespread in the last 4-5 years, active intrusion detection on
> internet facing services will become much more common.  If it isn't already.
> >
> > I haven't needed to understand how it works until this huge time-waster
> fell into our laps.  I suspect we'll have even more learning to do as this
> project proceeds.
> >
> > Scott C.
> >
> > --
> > Scott Chilcote
> > scottchilcote at ncrrbiz.com
> > Cary, NC USA
> >
> > --
> > This message was sent to: Matthew <matthew.rogers2 at allscripts.com> To
> unsubscribe, send a blank message to trilug-leave at trilug.org from that
> address.
> > TriLUG mailing list : http://www.trilug.org/mailman/listinfo/trilug
> > Unsubscribe or edit options on the web        :
> http://www.trilug.org/mailman/options/trilug/matthew.
> rogers2%40allscripts.com
> > Welcome to TriLUG: http://trilug.org/welcome
>
>
> --
> Scott Chilcote
> scottchilcote at ncrrbiz.com
> Cary, NC USA
>
> --
> This message was sent to: jc jones <jc.jones at tuftux.com>
> To unsubscribe, send a blank message to trilug-leave at trilug.org from that
> address.
> TriLUG mailing list : http://www.trilug.org/mailman/listinfo/trilug
> Unsubscribe or edit options on the web  : http://www.trilug.org/mailman/
> options/trilug/jc.jones%40tuftux.com
> Welcome to TriLUG: http://trilug.org/welcome
>

-- 
Jc Jones
Blogs -
http://www.wendellgeek.com/weblog/
http://www.wendellgeek.com/kixtech/