ArghWebWorks

When Des Traynor says “Quality is Fractal he means that by examining one piece of a whole, one can pass a judgement on the whole. In his example of a steak, it doesn’t matter if the steak is terrific, if the gravy is served cold or the waiter is ugly and smells of cigarettes.

I’m not convinced. And this could be because I’m working on my overall consistency. I think this idea requires a consistency that may not be present in the end product. For instance, I might write a glorious function for part of a website but put it together with the rest of the framework and my implementation is less than glorious. You could examine the function and make a judgment that the whole is terrific — but the whole is flawed and weak.

I won’t argue that if one piece is great and another is not so great, that the lower “average” between the two drags the overall quality down. That’s true too; I think that inconsistency is a flaw in a product. ( That’s why I’m working on it myself. ) That’s not what I’m saying; what I’m saying is that it’s hard to judge the whole effectively when there’s different levels of greatness.

With chatter today about website speed, here’s a quick test to see how long a webpage takes to download.

Keep in mind that this should be run at some interval with an average taken. Server load and speed should be taken into effect.

$ time wget –quiet –delete-after –page-requisites http://www.gmail.com

An example output is:
real 0m0.421s
user 0m0.000s
sys 0m0.012s

“time” gives you how much time something took; the command after the time command. You want the “real” number.
“wget” is a great tool for downloading stuff from the web. The –page-requisites flag gets the CSS(link) (Cascading Style Sheets), JS, images and other things necessary for the page to be displayed. The other options (quiet, delete after) are just there to clean up the output.

This also doesn’t take into account how much time a browser might take arranging the pieces into a pie. A bunch of nested tables will cause the client to slow down. Not specifying image height and width will cause browsers to slow down. A slow computer … well, you get the idea. Run this and take an average of the REAL value and you should have a pretty good idea of how slow your site is.

Note
If you’re plugging this into a php script, keep in mind that “time” is outputting that into the STDERR stream.

The ominously named “git” is a rather dull knife in my toolbox. I’m used to subversion (svn). I use svn routinely for my own work, as a backup. So I modify some files, do a svn commit and then push my files to where ever they need to go.

Git’s another beast. Because it’s an distributed system, doing a “git commit” doesn’t send my files to the backup. I have to do a “git commit;git push” for that. I’m liking the .gitignore file (compared to svn propedit syntaxes). And I’m liking the easiness of branching. I’ve got a couple of projects that are pretty much duplicates of one another with different filling. Like making a pie, but this one is apple, that one is cherry. With subversion, it feels a lot more like those projects are full of stuff. But git feels leaner. Which is good, right?

One of my clients has an extremely busy website. And it’s a popular place for people to show his images and offer a cracked version of his product for sale. We can see them in the apache logs- the referrer shows their domain name. One thing I like to do in order to keep an eye on this is to tail his apache logs and pull out the refer data. I’m using this:

[code]
tail -f access.log | cut -d\ -f9,11 | grep -v (domain name)
[/code]

That’s two spaces after the -d\; the backslash is escaping the space used as the delimiter. You may need different field numbers (9 and 11 are the ones for his combined logs that show the HTTP status code and the referrer). And of course I need to grep out the domain name of his site because it’s all over those lines.

Hope that helps ya.

Seth Godin could not have posted this at a more appropriate time for me.

Not that I was doing 4% less, but it did totally suck. Here’s the situation. I have a site I’ve built for a client. I do regular maintenance on the site and I keep it in a subversion repository. I also do development for this site. I have a development site set up for it where only I can see the results.

Early last month, I was asked to develop a feature for it. It was actually similar to many other parts of the site but to implement it right, I needed to rewrite a fairly major portion of the navigation pages. Then I had to duplicate it for a different category of navigation, and since it’s either working one way through a standard link (product.php?id=4) and through a javascript update of elements, I had to do a lot of testing.

I did a lot, a lot of testing of it and was really proud of how it worked. I developed it in Linux / Firefox and tested it with Windows Firefox 3 and IE 7. I don’t really have to support IE6 at the moment. I tested all of the static link possibilities and then I tested all of the javascript possibilities. It was set, and I committed it to the repository. The client wasn’t ready for it yet, and since we’ve had problems where I’ve made something live where either he or I weren’t around to make sure it was right (thus the emphasis on testing ), I was waiting for him to say to make it live.

Before he did, however, he had some smaller things to change. So I changed them and made those changes live. I had typos, however, on the copies I committed to the repository. I changed his code live for these smaller things, so these weren’t tested. This was a problem- I wasn’t ready to roll out the bigger changes to “live” but I did need to roll out some later newer changes.

So when he was ready for these bigger changes, I rolled them out. And then this happened over IM:

client: dont you test this shit before you go live
client: just forget it
client: i'll do it differently
client: less chance of you destroying my site that way
client: revert back to before you broke it
me: i'll brb. off to shoot myself.
client: how about before you do that you get my frikking page to load in IE?
client: you're making us look bad
... ( client asks if I can handle the other way without taking down his site) ...
me: If I can't handle it, I'll subcontract some eight year old in india to do it for me.
client: or a 3 year old in china
client: do you practice being inept in front of a mirror or is it just a natural state
client: does it just come naturally?
client: a gift?

I had to take over 200 dollars off the invoice from last month ( for the changes that didn’t work ) and my entire day was put into a blender and frappiccinnoed. Nice, huh? And all because I made his smaller, newer changes live and didn’t properly test them with the revisions that I had tested

Here’s a couple of things I should have done. First, made the changes to my development area and tested them before I did them to the live site. Second, made use of the “branch” and “tagging” features and branched the code when he needed the smaller changes made, then merged it afterwards (and tested). Really, it’s the testing that’s the problem. Even in my “full time gig,” my testing could use some work.

Every morning, I had a client calling me and asking “Is our server abnormally unresponsive?” They were browsing their website and it seemed to them that the pages were taking longer than usual to render. Once I figured out why they were asking and what they were measuring with, I realized I could graph this amount of time, the important thing was figuring out how to get this information.

Here’s what I came up with: I was already running mrtg-rrd to do much of their graphing, so I wanted to create a MRTG Target line to correspond to “how long does our page take to render.” I wrote a script to run “/path/towget –delete-after -pq (page) -O /tmp/garbage” and display the amount of time it took. I wrote it in php because it was easiest.


$wget="/usr/bin/wget -pq --delete-after -O /tmp/garbage";
$toget = "$site";
$start=microtime(true);
exec( "$wget $toget", $out, $e );
$end=microtime(true);
$dur=round( $end-$start, 3 );
print "$dur\n";


The problem with this is that I was running it on their gateway; their gateway is also the monitor and grapher. It was taking less than a second to render the page ( because of local network speeds ). Instead, I wanted it to more closely correspond to what the client would be seeing. Their development server is at their office; so I put the script there and wrote the MRTG target to point at a local ( on the gateway ) script that got the value from the development server, printed it twice, printed the date and the name of the site. It looks like this:

$user='foo';
$password='bar';
$sitename='example.com';
exec("/usr/bin/lynx -auth=$user:$password --dump http://developmentserver.com/scriptname.php ", $out, $e );
$time=0;
while ( ($time==0) && (count( $out ) > 0 ) ) {
$time=(float) array_shift( $out );
}
print "$time\n";
print "$time\n";
print date("r\n");
print "$sitename\n";


A couple of important caveats. The day after I started running this ( it averages around 4.6 seconds, btw, much too long imho ), the displayed time doubled. It could have been a change to the page we’re looking for, but it turned out to be a DNS issue on the development server. So DNS is important. This doesn’t take into account the amount of time a browser takes to render a page; it’s just the time to download all the pieces. So processor intensive stuff, like super heavy javascript or super heavy tables, won’t show up here. However, if you keep in mind what the value is- how long to resolve and download all the pieces of a web page, this is a useful metric.

Not really. Because Profiteering is like pirating. Sort of.

But I am trying to make some internet profits happen. Over at A Buck A Night I’m exploring internet passive income streams. I’ll let you know how it goes; so far it’s kind of depressing.

I made this theraputic purchase, and followed it up with a fair bit of work. When I asked, after I bought the domain, about some of the more “automagic” portions of the site, ones that I would expect; the seller laughed.

Anyway, I bought a Pagerank Checker service, and have put together some other changes. I have a few more things to put together, but for now, it’s working as I want it to.

I want to like IMified. I think it’s a terrific idea; a way to combine AJAX (Asynchronous Javascript And XML) and IMs; that’s what it is, really, a structured way to proces REST requests through your IM client.

Unfortunately, even with the plugins, it’s not very helpful to me. The most helpful ones would either be the network tools (ping, traceroute, etc) or the Basecamp tool- but unfortunately both of those require that I “push” the data to them. And honestly, I’d just as soon open a terminal window and use traceroute there. Or just go log into my basecamp account.

What would really help would be a rss feed tool, or some way for a service to send me an alert; I can totally see a place for Nagios to alert me via IM when a machine is down, and if it’s easier than the nagios plugin to configure, that’d be awesome.

IMified; keep an eye on it.

Basil Vandegriend has written about five crucial practices when you’re developing software. Let’s take a look at what he has to say.

1. Automated Tests
2. Version Control
3. Refactoring
4. Repeatable Build and Deploy
5. Communication

Obviously he’s coming from an Agile mindset. Agile methodologies are essentially to rewrite the specs in terms of “tests” and write those tests as scripts, break the code up into small pieces to match the tests, code quickly, and code in order to successfully pass those tests. And then go back over the code and refactor, tighten and clean up. Pretty straightforward.

That’s why he has the two items “Automated Tests” and “Refactoring;” they’re pretty much straight from the Agile handbook. And if you’re coding quickly, it’s important to run your tests frequently. And if you’re running your tests frequently, you might as well automate them. That way you can just happily burble along, coding up your test-hurdle-jumping scripts and just watch your email for the broken alerts. In fact, I think “Repeatable Build and Deploy” is right there along the automated tests- building and deploying should be as straightforward as typing “make” or “svn export (url);” the fewer different buttons you have to push, the more likely you are to push those buttons.

Don’t get me wrong; I totally agree with him about “automated tests,” “refactoring,” and “repeatable build and deploy.” However, his background and mindset pushes these items into his top five list. I agree for “automated tests,” and for “build and deploy” but I’m not so sure about “refactoring.” Because I tend to plan more in advance and try to figure out the functions and classes I will be needing, I don’t think refactoring is as high on my list.

“Version control” is an excellent suggestion; I really like subversion. I tend to forget stuff, so if I can figure out from the logs what I checked in on a particular day and what the log was (as well as any file changes), then I can figure out where I’m going a lot easier. Especially if you, like me, tend to work on projects and then drop them off, and then work on them after a while. And it makes “rolling back” a change very simple, in those rare (heh) cases where the client changes their mind.

Basil closes with his strongest topic; communication. I think this is hands down the most important essential practice when developing; whether it’s communication with the client or communication with other team members. If you don’t communicate with your clients, you won’t be needing those unit tests or the rapid deployment. If you don’t communicate with your teammembers, you won’t be able to pass your tests regularly. Communication really is the glue that holds the project together.

I think Basil did a good job with these five practices. While I don’t think I could come up with a better list, I’m not sure about his ordering, which, honestly, could be a solid oratory practice.