[TriLUG] seeking advice for a bullet-proof mail server

[Ryan Leathers]

Thanks all for the various ideas and warnings.

-----Original Message-----
From: Matt Pusateri [mailto:mpusateri at wickedtrails.com]
Sent: Friday, November 05, 2004 2:15 PM
To: Triangle Linux Users Group discussion list
Subject: Re: [TriLUG] seeking advice for a bullet-proof mail server


On Fri, November 5, 2004 1:37 pm, Aaron S. Joyner said:
> Ryan Leathers wrote:
>
>>Thanks for the follow-up question Matt.
>>
>>Certain systems will send mail to customers, and their responses to
>>predetermined mailboxes will be parsed for further action or read by
>> humans
>>in some cases.
>>
>>Another system which can only speak SMTP will send messages with
>> distinct
>>subject fields.  I will need to hand off incoming mail which matches
>> an
>>array of subject fields to a home-brew application which parses,
>> talks to a
>>database, fires off some jobs, and finally responds with another
>> email.
>>
>>I know its ugly.  Its not my choice.  If I could do JMS I would.
>> SMTP is
>>what I'm stuck with.
>>This is obviously asynchronous, but I can only afford minutes - not
>> hours of
>>delay, which is why MX weighting is not a practical solution for me.
>>Perhaps Aaron J's suggestion of putting all my eggs in one REALLY
>> good
>>basket is the right move, but I'm hoping for something better.  I
>> know I
>>could get the access behavior I want out of LVS, but I'm not sure how
>> to get
>>the data shared by both MTAs without a headache should hardware fail.
>>  That
>>is, unless I add a separate storage subsystem, which brings me back
>> to my
>>original post.
>>
>>
> Does it matter what machine processes your request?  Is it possible to
> have different boxes responsible for fulfilling any particular
> request?
> Originally I wasn't reading closely enough to realize that you were
> automating a task other than the recieval of email - my bad there.  If
> your secondary server is configured to accept all mail locally for the
> automation domain as well, and forward any accounts in your regular
> domain to the normal mail server, then you can have that second host
> deliver mail properly *and* have it still do your "fancy task".  Note
> that if you're going to try to direct all mail based on subject,
> things
> are going to be really confusing.  I'd *strongly* suggest doing it
> based
> on destination address, if you have any control over it and a desire
> to
> keep your sanity.
>
> If on the other hand your task can only be completed by one machine,
> for
> concurrency / storage / whatever reasons, or what-have-you, then
> you're
> back to my previous suggestion.
>
> Aaron S. Joyner
>
> PS - Also note that it's entirely possible to do not only fail over,
> but
> load balancing with MX trickery.  You can have more than one MX with
> the
> same priority, and traffic will be equally divided between them.  If
> your application needs to scale, and can be processed by multiple
> machines, that's the way to go about it.
> --
> TriLUG mailing list        :
> http://www.trilug.org/mailman/listinfo/trilug
> TriLUG Organizational FAQ  : http://trilug.org/faq/
> TriLUG Member Services FAQ : http://members.trilug.org/services_faq/
> TriLUG PGP Keyring         : http://trilug.org/~chrish/trilug.asc
>

Ok, I sure there are some holes in this, but I am just suggesting it
not  implementing it :)  Also there may be ways of doig this with
postfix, qmail etc but I don't use them so I don't know.

1. Have minimum two email gateway boxes recieving mail via round-robin
DNS.
2. Have two backend boxes that can do the automation part, a primary
box that runs the automation part and a secondary box that sits idle
and ready to go.  Have the secondary box run a script that monitors
the primary box and when the automation process is down on the primary
box have it start up the automation process to handle inbound mail.

3. Use sendmail and either Synonym milter or Mimedefang(Mimedefang
rocks, if you can script it in perl mimedefang can do it.) to
automatically copy the email message to the second rendundant
automation server.  This way a copy of every message will go to the
primary and secondary automation servers.  The secondary automation
server will normally only run the automation process when primary
server is down.  You could run the same synonym/mimedefang setup on
the primary and secondary automation servers to send copies of
inbound/outbound emails to a third server thus archiving all
inbound/outbound mail from all automation servers to a central
location or have the email gateways send to all three servers.

This does a couple of things;
One, you don't care which automation server handles the request.
Two, you only duplicate emails for messages that need automated. Three
if both servers archive to a third then you have one email store you
can check and you really don't care whether you different email stores
on each automation box.  Now granted this all requires SMTP.

I would figure you would want a montioring box such as Nagios to
monitor the machines and notify someone who could baby sit the
problem.  You would have to make sure when the primary box came back
up it didn't duplicate work that was done by the backup automation
server.

You don't have to figure out how to synchronize mail stores.  Which is
one of your biggest single points of failure if all automation boxes
have to look one place for mail.  I guess you could write all mail to
a DB and then replicate, but I don't know what you buy from that as
you still have to write to one db and then sync.  If the primary db is
down you still have the same problem.

MimeDefang integrates nicely with ClamAV and Spamassassin.


Sorry for the rambling.

Matt

-- 
TriLUG mailing list        : http://www.trilug.org/mailman/listinfo/trilug
TriLUG Organizational FAQ  : http://trilug.org/faq/
TriLUG Member Services FAQ : http://members.trilug.org/services_faq/
TriLUG PGP Keyring         : http://trilug.org/~chrish/trilug.asc