Portland Technology Consultant Scott Hendison has an interesting post about advertising your new (or old, I suppose) weblog; to comment on other peoples’ weblogs and websites, and maybe you would get some back-traffic.
First, he recommends, as I have evangelized in the past, “have decent content.” Second, submit it (your blog) to blog directories (see his article for his list of 11, 10 of which I hadn’t met before). And third, blog about it.
I was making a case yesterday that sendmail/mimedefang/clamd/spamassassin weren’t causing a heavy load on our systems. I had to back up my claims with some cold facts, so here’s what I put together.
First, I ran “sar” to get the system activity report; it’s some statistics that are taken every 10 minutes throughout a day on our Fedora Core 4 server. Then I grepped for today’s mail information in the maillog file (grep ^Feb\ 16 maillog) and saved it to another file. Then I ran this script to match total processed mail messages against the system activity report; I ran the script and saved the output in a file, then used “paste sarreport mailreport > foo” to get the whole report.
Sometimes my tools are just too much fun.
open( FH, "maillog.stripped");
# set default values
my $remote=$relay=$file=$local=$total=0;
my $avgtotal=$avgremote=$avglocal=$avgrelay=$avgfile=0;
my $totals=$remotes=$locals=$relays=$files=0;
my $stattime="00:00";
my $checktime="00:00";
# print these spaces to match the loadinfo sheet
print "\n";
print "\tMessages/min (10min avg)\n";
while(>fh< >){
# only need the mailer= lines
if (m/mailer=/) {
$line=$_;
# we know where the date stamp lives
# whichever mail item this is, increment that mailer count
# this was originally a minute-by-minute script, then broke it u
p for 10 min avg.
$checktime=substr( $line, 7, 5);
if ( $checktime eq $stattime ) {
if ( m/mailer=local/ ) { $local++; }
elsif ( m/mailer=esmtp/ ) { $remote++; }
elsif ( m/mailer=relay/ ) { $relay++; }
else { $file++; };
}
else {
# calculate totals etc
$total=$remote+$local+$relay+$file;
my( $hour, $minute ) = split /:/, $stattime;
# if we're at the 10 minute marke, print out the averages
if ( $minute % 10 == 0 ) {
$avgtotal=$totals/10 ;
$avgremote=$remotes/10;
$avglocal=$locals/10;
$avgrelay=$relays/10;
$avgfile=$files/10;
# print "$stattime|\t$avgtotal\t$avgremote\t$avglo
cal\t$avgrelay\t$avgfile\n";
print "\t$avgtotal\t$stattime\n";
$totals=$remotes=$locals=$relays=$files=0;
}
else {
$totals += $total;
$remotes += $remote;
$locals += $local;
$relays += $relay;
$files += $file;
}
$remote=$relay=$file=$local=$total=0;
$stattime=$checktime;
}
}
}
close( FH);
(END)
The server’s upward spiral for system load continued; we’re currently examining the qpopper process. What it does is copy the mailbox file over to a temporary place, parse the copy, and then if the user leaves mail on the server copies it back. Some of the users have over 20MB of saved mail. People, we’re not gmail here. So I’ve been writing a perl script to weed out some older files; I’m letting them keep 14 days of stuff, and only checking files over a certain size.
Here’s the completed script; obviously I’m not a perl master.
#!/usr/local/bin/perl
# need this to run through the mailboxes.
use Mail::Box::Manager;
# this size is the maximum we allow them to grow to
# let's start with 5 MB
my $maximumsize=5*1024*1024;
# this is the maximum age of a file in a lage mailbox;
# start with 28 days, measured in seconds
my $maximumage=14*24*3600;
my $now=time();
# get the list of all the mailboxes on the system.
my @mailboxes=;
my $mgr=Mail::Box::Manager->new;
my $worklog="";
foreach $mailbox (@mailboxes) {
# if the message ends in 'old' then skip it
next if /\.old$/;
if ( -s $mailbox > $maximumsize ) {
cleanout ($mailbox );
}
}
print $worklog;
sub cleanout {
my ( $checkme ) = @_;
`cp -f $checkme $checkme.old`;
# if we don't change the file we don't need to rewrite it
my $writeme = false;
my ( $imfrom, $imto, $imon );
$workdir .= "Now opening $checkme\n";
my $folder = $mgr->open(folder=>$checkme, access=>"rw");
foreach $msg ($folder->messages) {
my $howlongago = $now-$msg->timestamp;
if ( $howlongago > $maximumage ) {
my Mail::Message::Head $head = $msg->head;
$imfrom=$head->get('from');
$imto=$head->get('to');
$imon=$head->get('Date');
$workdir .= "Removing message from $imfrom to $imto on $imon\n";
$msg->delete;
$writeme=true;
}
}
if ( $writeme ) {
$folder->write;
$workdir .= "writing folder\n";
}
}
When you move to bring your archaic Named configuration files into a database, the mysql bind sdb patch looks like a terrific idea.
It’s pretty slick, but there are a couple of caveats; First, no wildcard entries are allowed; you can’t specify a wildcard server name and have mysql-bind track down the right server. As well, there’s no documentation as far as indexing the database files for better results. So, as the documentation specifies, having each domain in its own database table is probably the best (it offered me, on testing, a tenfold improvement over one main table).
Sendmail keeps its email in one big file, and as a mail-user reads their mail and elects to ‘leave messages on server,’ these files can get pretty large. I’d like to limit their size, but if I use quotas, it’ll bounce new messages. That won’t work on the mailboxes that are the worst offenders; one in particular is the destination of some automated scripts. So I need a way to remove the oldest messages.
Here’s the beginning of such a script
#!/usr/local/bin/perl
use Mail::Box::Manager;
my $checkme='/home/mail/sysmon';
my $mgr=Mail::Box::Manager->new;
my $folder = $mgr->open(folder=>$checkme);
my $msg= $folder->message(0);
my Mail::Message::Head $head = $msg->head;
my $then= $msg->timestamp;
my $now=time();
my $howlongago=($now-$then)/3600;
print $howlongago;
print "\n";
Our qmail client called up and asked me, since they had a huge backlog of mail and it all looked like spam to non-existent users, how they could just automatically round file it; sending it to /dev/null.
I recalled that you could specify /dev/null in the .qmail-default file, but that just putting in one line, commented out, was faster and just as good (It convinces qmaiil that all of the delivery options have been performed). So I instructed them to do that. A few hours later, the customer called back; they weren’t receiving any email for some aliased accounts, but others were working ok.
As it turned out, I needed to specify ‘| /home/vpopmail/bin/vdelivermail ” /dev/null’ rather than just “#”; because of the combination of qmail and vpopmail. So remember, if you’re using qmail+vpopmail, everything changes, including default delivery to /dev/null.
