ArghWebWorks

Here’s a quick example of how to pipe a message to a PHP script from Evolution (the gnome-tastic email client).

First, write a script, using very clear paths. And use php://stdin to capture the input stream:


#!/usr/bin/php -q
< ?php
include_once( dirname(__FILE__).'/includeme.php' );
$input='';
$fh=fopen( 'php://stdin', 'r' );
while( ! feof( $fh ) ) {
$input .= fread( $fh, 4096);
}
fclose( $fh );


Do whatever you need to with the $input variable. I had subscribed to a newsletter, and was using PHP to parse out the interesting bits of the newsletter (everything between the h1, h2 and h3 tags, so I knew if I needed to read it), and remail that to a more-frequently checked email address.

You can test your file with a shell command of "cat (testfile) | $somescript.php "

When it's perfect, then go to your evolution program and use the "message filters" to "pipe a message to a program." And pipe it to this one (including of course the explicit paths).

The things to take away here are the php://stdin slurp, and the fact that you have to use explicit paths (and you'll have a dickens of a time tracking down any errors).

Hope that helps ya!

The ominously named “git” is a rather dull knife in my toolbox. I’m used to subversion (svn). I use svn routinely for my own work, as a backup. So I modify some files, do a svn commit and then push my files to where ever they need to go.

Git’s another beast. Because it’s an distributed system, doing a “git commit” doesn’t send my files to the backup. I have to do a “git commit;git push” for that. I’m liking the .gitignore file (compared to svn propedit syntaxes). And I’m liking the easiness of branching. I’ve got a couple of projects that are pretty much duplicates of one another with different filling. Like making a pie, but this one is apple, that one is cherry. With subversion, it feels a lot more like those projects are full of stuff. But git feels leaner. Which is good, right?

Every morning, I had a client calling me and asking “Is our server abnormally unresponsive?” They were browsing their website and it seemed to them that the pages were taking longer than usual to render. Once I figured out why they were asking and what they were measuring with, I realized I could graph this amount of time, the important thing was figuring out how to get this information.

Here’s what I came up with: I was already running mrtg-rrd to do much of their graphing, so I wanted to create a MRTG Target line to correspond to “how long does our page take to render.” I wrote a script to run “/path/towget –delete-after -pq (page) -O /tmp/garbage” and display the amount of time it took. I wrote it in php because it was easiest.


$wget="/usr/bin/wget -pq --delete-after -O /tmp/garbage";
$toget = "$site";
$start=microtime(true);
exec( "$wget $toget", $out, $e );
$end=microtime(true);
$dur=round( $end-$start, 3 );
print "$dur\n";


The problem with this is that I was running it on their gateway; their gateway is also the monitor and grapher. It was taking less than a second to render the page ( because of local network speeds ). Instead, I wanted it to more closely correspond to what the client would be seeing. Their development server is at their office; so I put the script there and wrote the MRTG target to point at a local ( on the gateway ) script that got the value from the development server, printed it twice, printed the date and the name of the site. It looks like this:

$user='foo';
$password='bar';
$sitename='example.com';
exec("/usr/bin/lynx -auth=$user:$password --dump http://developmentserver.com/scriptname.php ", $out, $e );
$time=0;
while ( ($time==0) && (count( $out ) > 0 ) ) {
$time=(float) array_shift( $out );
}
print "$time\n";
print "$time\n";
print date("r\n");
print "$sitename\n";


A couple of important caveats. The day after I started running this ( it averages around 4.6 seconds, btw, much too long imho ), the displayed time doubled. It could have been a change to the page we’re looking for, but it turned out to be a DNS issue on the development server. So DNS is important. This doesn’t take into account the amount of time a browser takes to render a page; it’s just the time to download all the pieces. So processor intensive stuff, like super heavy javascript or super heavy tables, won’t show up here. However, if you keep in mind what the value is- how long to resolve and download all the pieces of a web page, this is a useful metric.

Oy. Permissions. Permissions in a Linux / Unix environment can be a headache; and who knows what all those little numbers and letters mean?

I do, and you will too.

Of course, every book in the world will tell you that there are three sets of permissions; user, group and world. So something that’s rw-r–r– is writable only by the user but readable by everyone. There’s read, write and executable settings for each one, and they’re marked by “bits”. You know that computers think in binary, and in binary there are only zeroes and ones. Here’s what we’re interested in for the next few minutes.

Decimal 0 = Binary 000
Decimal 1 = Binary 001
Decimal 2 = Binary 010
Decimal 4 = Binary 100

If you look those over, you’ll see there’s only one way to get each of the numbers from 0 to 7 using only one of a number ( ie, you can’t replace a 4 with 2 2s ). A seven has to be a four, a two and a one. No other options. So when you’re saying “change the ownership of somefile to 755″ what you mean is you want the ownership to be read:ON, write:ON, execute:ON for the owner, and read:ON write:OFF execute:ON for everyone else.

If you’re ssh’d into your server and you run an “ls -la somefile” you’ll see something that looks like

-rw-r--r--

which translates to

011000100

or, taking off the first one ( because it’s special and I won’t talk about it here),

110 100 100

If you treat those each as binary numbers and translate them to decimal, you’ll have 644.

What’s this mean?
Basically, 4 is read permissions only. 2 is write permissions only but you can add them together and get 6 which is read and write permissions. There’s only one way to get a six; a four and a two ( you can’t have two threes or three twos ). When you add “executable” to the mix, you’ll see that you can add executable to any of these numbers and get 5 ( read and execute, don’t write ), 3 ( write and execute, don’t read ( weird ) ), 7 ( read, write and execute ).

Note that it doesn’t really make sense to “execute” a directory. The “executable bit” for directories translates to “traversable.” Or, in english, a user can go through the directory without needing read permissions. For instance, to use a contrived example, you could have a /var/logs/user/ directory and just give the users permission to see their own directory; the logs directory could not be readable or writable but be traversable; they could go through it. But if they did a “ls -la /var/logs”, they’d get back an error.

You may be looking for a domain with a specific keyword. It used to be that you needed a thesaurus handy in order to come up with as many domain name options as possible, then input them into a bulk lookup form. Now, there’s a second breed of these lookups available; any google search for “Domain Name Spinner” should result in multiple solutions to that problem.

However, I have to say that DomainsBot is the best of the ones I’ve tried; I especially like being able to uncheck the options for .us and .biz (which scream “spam” to me) and uncheck the “for sale” and “expiring” domains. A great tool.

I did something rash.
I created a /etc/procmailrc file:

:0
* ^Subject:.*(Viagra|Cialis)
/tmp/viagra.spam

and then a /etc/logrotate.d/spamstorage file:

/tmp/viagra.spam {
daily
rotate 7
compress
notifempty
missingok
size=100k
}


If you have a linux server (or a workstation, but servers are more prone to OS stagnation, I’ve found), there might come a day when you insert a new hard drive. Perhaps you’re migrating the /home/ directory from a full disk to its own disk. If you do this, your newer hard drive might spin faster and have more cache than the old one.

If you run ‘hdparm -i (device)’, you might get an output like this:

/dev/hda:
Model=WDC WD400BB-00DEA0, FwRev=05.03E05, SerialNo=WD-WMAD15201933
Config={ HardSect NotMFM HdSw>15uSec SpinMotCtl Fixed DTR>5Mbs FmtGapReq }
RawCHS=16383/16/63, TrkSize=57600, SectSize=600, ECCbytes=40
BuffType=DualPortCache, BuffSize=2048kB, MaxMultSect=16, MultSect=16
CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=78165360
IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: mdma0 mdma1 mdma2
UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5
AdvancedPM=no WriteCache=enabled
Drive conforms to: device does not report version:

Compare this against:

/dev/hdb:
Model=ST3250823A, FwRev=3.06, SerialNo=5ND0B7XN
Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs RotSpdTol>.5% }
RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4
BuffType=unknown, BuffSize=8192kB, MaxMultSect=16, MultSect=16
CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=268435455
IORDY=on/off, tPIO={min:240,w/IORDY:120}, tDMA={min:120,rec:120}
PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: mdma0 mdma1 mdma2
UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5
AdvancedPM=no WriteCache=enabled

Access to this hard drive will be quicker, in general, because of this cache. You might decide to move your cache over to this drive. Here’s how to do it. First, fire up fdisk; “fdisk /dev/hdb” (or whatever your new hard drive is set up as). Create a new small partition (I set up a 4GB one, which is overkill). When you choose what type to set it up as, instead of choosing type 83 (Linux), choose type 82 (swap). Set up any other partitions you need, and (w)rite the partition map out.

Just like a new partition needs the mkfs command run on it, you need to ‘mkswap’ your new swap partition (mkswap /dev/hdb1). This’ll take a little while, just like the mkfs command does. Once this is complete run a quick ‘top’ and notice in the top of the, err, top, how much swap you have. Now run ’swapon’ and your partition (swapon /dev/hdb1). This is a quick command, like a ‘mount’ might be. Once this is complete, you can run top again and make sure you have buckets more swap.

Before you reboot, you might want to update your fstab ( or your swap space won’t be remounted when you reboot). Back up the file (cp /etc/fstab /etc/fstab.old) and edit the file. You can change the line with ’swap’ in it to have your partition mounted (/dev/hda2 swap swap defaults 0 0). When you reboot, you should have the new amount of swap space available.

As part of two positions, I often have to reboot a remote system, on a remote power server. I mean ‘power cycle,’ not just reboot. What this entails is telnetting into a remote server, and walking through a predictible set of menus and commands. I finally gave up and scripted it; I used ‘expect’ because it was, frankly, the easiest.

‘Expect’ is used to mimic a series of commands. It’s a scripting language like PHP or Perl, but older, and it has a different syntax. It’s also related to Tcl (tickle) programming… Here’s the example. Use it as you’d like; I’m sure there’s more ways to make it work (it’d be nice to add more options, so I can use the same script to reboot several servers).


set timeout 60
spawn telnet ip address
expect "User Name*"
send "username\r"
expect "Password*"
send "password\r"
expect ">"
send "1\r"
expect ">"
send "1\r"
expect ">"
send "1\r"
expect ">"
send "3\r"
expect "cancel*"
send "YES\r"


The FUSE modules, bringing file systems into user-space, are pretty fascinating. I was tempted by the encryption modules, the ntfs modules, but the most seductive one has been sshfs. SSHFS is a mechanism for “mounting” a remote system’s filesystem as a directory on your system over the ssh protocol.

How did I do it?

Well, first, the system is running Debian, so I had to install the fuse module source (I used Synaptic and just searched for FUSE and likely candidates). Then I ran module assistant (which I had previously installed for nvidia support) with this command as root.

m-a a-i fuse

That built the module, then I had to install the module. “insmod fuse” didn’t work, so I had to walk down the /lib/modules/2.6.15/ path until I got close enough to track it down;

# insmod /lib/modules/2.6.15/kernel/fs/fuse.ko

Once that was done, I had to, as root, mount the filesystem somewhere. I created a mount point and then issued this command (as root)

#sshfs (user)@(server):(directory) (mountpoint) -o uid=(my uid),allow_other


It worked appreciably well- as my normal user I can traverse the directory and use bluefish and whatnot on the files within, just as if it were a local drive.

Let’s hear it for SSHFS!

I’m working on a couple of projects; I’m keeping the web development projects I’m doing (like this site, for instance) in subversion repository. I’m trying to get better at this, but some of the instructions out there assume that you actually know what you’re doing.

So, here’s how to include another repository in your working copy; I’m building a ‘common library’ of code, that I update, but I plan to maintain as its own entity; that way I won’t have multiple copies of various things running around.

First, checkout your working copy (assumes you have a repository already set up. Happily, there are many tutorials on that).
$>svn checkout file:///var/subversion/repositoryname
Then go into this working copy and use ’svn propedit’ to edit the properties of the root directory:
$>svn propedit svn:externals .
This’ll bring up an editor window for you. Place into it a tab delimition of directory and repository:
(in the file)
phplibs file:///var/subversion/libraryrepository
(out of the file)
now exit, and run svn update to import the externals.
Voila- you’re done! Pat yourself on the back.