SteveStreeting.com

Thanks John for the reminder to investigate S3 as a business media hosting service, it works like a charm!

Now that I have far fewer bandwidth worries (max $0.17 per GB), the Torus Knot site includes a nifty dynamic selector so you can pick low, medium or high quality - the latter is at a higher resolution too, clocking in at about 100Mb. I may well use S3 for future public commercial downloads in the future too. It’s altogether more convenient than the block bandwidth allocations you get with regular hosting packages, since it scales dynamically at a very fine level of detail depending on demand. And don’t be fooled by ‘unlimited’ bandwidth offers, all hosting companies have to pay for bandwidth and there’s no such thing as ‘unlimited’ resources; you’ll actually find your bandwidth being throttled or cut off via a ‘reasonable use’ clause in the small-print; ‘unlimited’ is simply a marketing lure. If you want truly scalable guaranteed bandwidth, you have to pay for it.

Getting S3 media hosting working wasn’t that hard, but did require a few discrete steps. Firstly, you need to create a bucket in your S3 account which is all in lower case, is globally unique and is DNS-compatible; so for example I created a bucket called ‘media.torusknot.com’.

Then to make it all look nice you need to create a DNS CNAME entry to map a sub-domain of your site to that S3 bucket; in my case I mapped ‘media.torusknot.com’ to ‘media.torusknot.com.s3.amazonaws.com’. That allows me to access any files I upload to that S3 bucket via ‘http://media.torusknot.com/somefile.jpg’. You do just need to set the ACLs on the files & the bucket to make sure public access is allowed.

Finally, if you want to stream video files via a Flash player from S3 to another domain, you also have to tell Flash that it’s ok for the content to be pulled in from a different domain. Create a file called ‘crossdomain.xml’ in the bucket, with these contents:

<cross-domain-policy>
<site-control permitted-cross-domain-policies="all"/>
</cross>

That allows the media to be accessed from anywhere - you can be more specific if you want but this is the simplest approach.

Once again I’m using the excellent FlowPlayer; my only issue with it is that the ‘buffering’ animation seems to not work all the time (so be patient if you’re viewing the high quality version).

Gotta love this cloud computing business

I came across Tim O’Reilly’s post entitled Open Source and Cloud Computing today, and I was pretty happy to see that his thoughts reinforced what I was saying a couple of months ago about how I thought isolated, corporate-owned islands in the ‘cloud’ were not a beneficial model for the Internet long-term, despite the short-term convenience in today’s society.

I was also very interested to see from his links in that article that some in the open source community are already forming plans to address it. It’s early days yet, but I look forward to a day when we can have all the convenience of sites like Facebook provide without having to cede control of our data to a centralised corporate entity searching desperately for a way to dissect that data & our browsing behaviour for revenue opportunities.

It’s funny because when talking about this issue to a friend recently, I made the comparison between sites like Facebook now, and the ‘gated communities’ of AOL / Compuserve in the past, before the Internet took off. They too used their sealed community to generate revenue, but when people finally had the option to become free of that and get the same functionality, users deserted in their droves to the liberated environment of the Internet. Tim used the same analogy in his article, so maybe I’m not crazy after all.

I consider LinkedIn to be one of the few genuinely useful things to come out of the whole Web 2.0 gold rush, since it’s a business-oriented, generally ‘fluffless’ site (if I see one more virtual gift or stupid time-wasting Facebook application, I’ll lose all will to live) - as such I actually do use it fairly regularly. I finally got around to creating an OGRE Group - feel free to join if you’re a professional OGRE user / contributor. I use LinkedIn to remind me who I can refer OGRE-related jobs to, so it might be worth your while

Now it’s out of beta, Firefox 3 has become my primary browser - it’s a nice speed upgrade and I like the little extras like the unobtrusive ‘remember password’ prompt, smart location bar and reduced memory usage. It’s a shame their servers went belly-up on the planned release day, but then they did paint a bullseye on their face.

There were a few hiccups- I have a few add-ons I generally feel I couldn’t live without and a couple of them didn’t want to work immediately.

• Firebug won’t upgrade itself, you have to switch to the beta 1.1 version if you want it to work with FF3. Seems to work fine.
• Google Browser Sync doesn’t work and Google have apparently dropped future development support for it - because they never released it as open source (why?) it’s essentially a dead product. Foxmarks is a nice quick alternative, although it does only synchronise bookmarks and not open tabs, cookies or passwords. It does have the advantage that you can sync it to your own server if you want though. Long term Mozilla Weave looks like it could be the best option, but it seems a little young right now.

Also oddly, I had no back / forward button block to begin with. I don’t know if that was linked to the fact that I’d been running the beta beforehand, but I had to customise the toolbar to add it back in.

Still, overall it definitely feels faster and slicker, so it’s a useful update to a major staple of my application toolset. I haven’t tried the native look and feel on OS X yet, I’ll be updating next time I’m on the Mac to see what it’s like.

Opera is still the fastest browser of course, but IMO they really missed the boat by holding on to the concept of being able to sell a browser for a little too long, and I’m not sure they’ll ever catch up in terms of the sheer breadth of available add-ons. I have Opera installed on my machine too (for testing) and although it’s good I always gravitate back to FF just because of all the useful add-ons & the more active community - the same applied to Safari on the Mac.

Whatever your preference, with all these options there’s really no excuse to still be using that buggy piece of trash called IE!

To my everlasting disgust I finally caved in and signed up to Facebook today. My singular reason was that a friend of mine has just moved to North America (to complete his years-long transition to the dark side) and he’d stated his intentions to publish most of his personal stuff there rather than blogging about it, so with much grumbling I now have a placeholder account to let me peer into that little den.

Now that I have my own account rather than peering quizzically at other people’s (over my horn-rimmed spectacles perhaps, as is implied when people hear I don’t use it), I still say Facebook has so few new ideas that it’s absolutely criminal that it’s valued so highly. It does the job sure, in a vaguely pedestrian way, but I see nothing more than a few tiny refinements on what everyone else has already done a ton of times before.  I still reckon it’s on borrowed time until someone else does something sexier and the student population move on to the next fad - just like they did the last 2 or 3 times. I’ve lost count of the number of social networking sites I’ve been invited to over the years (since about 2003/4 when they started getting popular), and after having gone through temporary membership of a few by 2005/6 I just got totally just bored with the whole thing. Now that I’ve finally joined Facebook, I’m fully expecting one or more of my friends / family / contacts to invite me to join the next thing that ‘everybody’s using’ in about 6 months - that’s normally how it works. Yawn. I certainly can’t be arsed to obsess over the content of my profile all over again. No, I’ll stick to LinkedIn where I can actually derive some (small) tangible business benefit, and my blog which I control and is thus doesn’t have to be moved / recreated every time fashion changes. And no, I don’t feel the need to start twittering, we all know where that can lead.

But the thought of being a part of something that that pompous, vacuous git Zuckerberg benefits from - ugh. I think I need a shower.

So, as we all know the whole MicroHoo! idea has been called off now, unless you believe the conspiracy theorists who believe this is all still part of Count Ballmer’s plan to devalue Yahoo! (as some of its shareholders go through a set of inevitable legal tantrums) and make it easier to pick up later. I’m not so sure about that myself - after all didn’t the rotund billionnaire say he wasn’t going to raise the original offer for Yahoo!, before doing exactly that? Doesn’t really sound like a bluff, unless you factor in that he knew they were going to reject that too, thus increasing the chances of annoying their shareholders. But then we start getting into the ‘he knew that they knew that he knew’ territory and it all gets dreadfully confusing. Whatever the case, the whole deal has always sounded like a recipe for total disaster to me. So much so, I was kinda hoping it would go ahead just so that I take some perverse pleasure in watching the train wreck later.

No-one I’ve spoken to ‘gets’ the Yahoo! bid at all, seemingly a constant across the entire spectrum of opinion about both companies. I can understand that Microsoft would love to get hold of Yahoo!’s ad customers, and to a lesser extent all the freeloading users required to eyeball said ads (and the services required to keep them sweet), but they can’t have seriously thought that it would be a smooth transfer. The philosophies of the companies couldn’t be much further apart, with Yahoo! very much invested in open source technology and service models, and spritually the culture of the company is very much of a younger, consumer-oriented, more agile and open thinking sort, very different from the business-focussed, shrink-wrapped and closely integrated, keep-it-in-the-company sort of vibe that Microsoft tends to exude. I’m sure Microsoft must have earmarked a bunch of money to use in encouraging key people to stay, but honestly in my experience the very best people aren’t swayed that much by that kind of offer. I think had the deal gone ahead, MicroHoo! would have haemorraged much of the best Yahoo! talent to other Valley companies like Google (or to new start-ups) in the first few weeks, whatever reassurances might be given.

And what about the technology? Yahoo! is again the antithesis of Microsoft here, running their core business on open source stacks. Quite whether the acquisition would have eventually led to that being replaced with equivalent Microsoft technology I don’t know, but a switch would seem like a pointless effort - regardless of your technology preference, the ‘if it ain’t broke, don’t fix it’ rule is universal. But at the same time an entire growth arm of the business running ‘competitors’ products would be somewhat jarring I would have thought, so perhaps their principles would have had them invest in that transition anyway - none of which sounds like a particularly efficient investment.

So actually, I think Microsoft has had a lucky escape here - I think they would have been hugely distracted (not to mention considerably poorer) trying to make a Yahoo! acquisition work. Sure it might have advanced them to some degree, but it surely would have been a messy, inefficient fight which would have given their main competitors - ok competitor - much amusement. The big question is whether they can achieve more on their own. The word is that Steve Ballmer is obsessed with beating Google (you would have thought he’d be content with all the billions he already has, but I guess there’s no pleasing some people), so it will be interesting to see what Plan B is. I hope they’ve screwed the chairs down in his office.

I’ve been a little busy for the usual diatribe these last couple of days (a fact for which no doubt the Intertubes will be grateful) so for the moment my spleen will have to simply tolerate the increased pressure in anticipation of future venting .

In the meantime, here’s an interesting site I found recently: TED. It’s made up of a ton of videos of presentations from quite interesting people on a variety of subjects including creativity, technology and politics. All round great bloke Johnny Lee even makes a short appearance with his Wii hacks in one of them - the head-tracking I’d already seen of course, the digital whiteboard & touchscreen I hadn’t. But you can pretty much grab any video from the site and get a 20 minute talk about something that’s usually mentally stimulating and perhaps a bit tangental to what you might normally watch.

I never touched Flash coding before this weeked - I’ve never really felt inclined to do so, I generally dislike Flash interfaces on websites and as such I never felt the need to buy expensive tools from MacromediaAdobe to develop in that environment. There was also the fact that forcing someone to install a plugin to view your content always left a bad taste in my mouth.

A few things have changed that attitude recently though. Firstly, the ascension of video on the web from the likes of YouTube has meant that everyone now has  a Flash player installed, far more than previously, which has mitigated the plugin concerns. Secondly, Niko brought to my attention that Adobe’s Flex SDK from Adobe is now open source, meaning that anyone can develop using their base tools now for no money down. Adobe do in fact seem to be embracing open source quite a lot these days, which I obviously applaud. Lastly, there’s one area beyond video playing where Flash does in fact justify itself among other web technologies, and and that’s game development. A lot of Flash games I’ve seen are pretty awful, but having discovered N+ on my 360 recently I was really impressed with the original N which was released as Flash. I have a bit of a penchant for stylish 2D gaming so that inspired me to give it a try myself this weekend.

I’ve been impressed. I’d heard all sorts of horror stories about ActionScript in the past, but I find AS3 really quite pleasant to use - it’s fast, pretty object-oriented now and I was comfortable using it in about an hour. The Flash environment in general is pretty friendly, and I managed to go from absolutely zero knowledge of Flash before the weekend, to developing my first (admittedly pretty crappy) game in only a handful of hours. So here it is

It’s a simple Lunar Lander clone where the idea is to land on the coloured platforms. You get more points for smaller platforms (the green one is a total bitch), and to score you have to put both ‘feet’ of your ship down on the surface and stop moving. You will die if you touch down too hard, and also if any part of your ship apart from the bottom of the feet touches the scenery.Click on the game to start, the controls are:

Left/Right arrow = turn   SPACE = thrust   R = restart

I know it’s not that impressive, but I was quite pleased to get this going in basically an afternoon and an evening with no prior experience. I used FlashDevelop as an IDE, which was surprisingly good - the only problems with it are no debugging support (which makes it quite hard to know what’s going wrong if you’re new) and that it only runs on Windows. I’ve now downloaded, but not tried yet, the absolutely enormous trial version of the Flex Builder 3 which is the ‘official’ dev tool, and has the advantage of being Eclipse based (awesome, multi-platform) and supporting debugging, so I’ll see what that’s like in comparison later. I also used a couple of libraries: Flint to do the smoke particle effect on death (which is great except that you don’t seem to be able to pause & restart just emissions rather than the entire system), and Box2DFlashAS3, a great 2D physics engine which is basically an AS3 port of the C++ Box2D.

On the whole then, I enjoyed myself with this stuff yesterday, it’s nice to be working in an environment that’s easy to use and you can get deployed in no time at all. I did a quick look around and didn’t find any ‘complete’ game engines for Flash, unusually, so I built my own simple one here - I did initially think about doing something clever with abstract property and event systems, but decided just to bash something specific out this time. Maybe I’ll write a more complete engine for it sometime

Edited 4 March: since some people have asked, here’s the source code, released under the Do What The Hell You Like With It Public License It’s quite shabby, so don’t laugh.

I’ve been thinking about subscribing to an online backup service for a little while; after all while a scheduled backup system to an external drive / NAS is all very well, if something should go seriously wrong (heaven forbid), you really need an offsite backup of your most critical data. There’s only so long you can go burning DVDs or removing hard drives and persuading friends / family to keep them at their houses before it starts to get unwieldy. As a small business, you often don’t have the option of fireproof safes and staff to do a proper offsite rotation system all the time, so with increasing broadband speeds online backup is starting to look more attractive.

There’s plenty of dedicated options there, like Mozy and Carbonite, and they’re fairly inexpensive, but most of them are aimed at home desktop PC users, and as such typically only have Windows and (sometimes) Mac GUI interfaces. Automated Linux server backups tend to be limited to the business versions, if they’re available at all, which come at a higher price. So I kept looking.

I happened across a number of services that piggy-back on Amazon’s Simple Storage Service (S3) - one of several web services that Amazon provide to developers who want to create online solutions without maintaining a scalable data centre of their own. It has the significant advantage of being rather cheap for small volumes, given that it uses a very fine-grained pricing structure (US hosting, EU hosting is 20% higher):

This means you’re never paying for more than you need - if you store 10GB and transfer the same amount in every month (unlikely), say with less than a thousand PUT requests, you’re going to be paying $2.51 per month, which isn’t half bad - some services like Carbonite have unlimited storage (per PC) for a fixed price though so if you go beyond a certain level you might be better with something like that, but again they’re aimed at desktop users. Personally, because I’m only using it to store my most critical documents, and I’ve already compressed them, I don’t expect to be paying more than about 50 cents per month. Services like JungleDisk, ElephantDrive and ElasticDrive all use S3 as a back-end storage, and provide the friendly front-ends which AWS/S3 doesn’t (it’s basically just an API). Of all of these, I liked JungleDisk the most, because it’s inexpensive ($20 for any number of PCs, over and above the S3 charges) and comes with a Linux version, which just requires you to install the FUSE module, whereby you can mount an S3-backed, locally cached drive onto your server an rsync directly to it. It also supports encryption so the files that are uploaded to S3 aren’t readable by anyone else.

In the end though, one thing struck me about all these services - useful though they were, you were dependent on a piece of software from a relatively new startup, and the file structures they create on S3 are typically encoded in some way - this is out of necessity, since S3 doesn’t support any notion of directories, it deals purely in terms of ‘buckets’ (which have to be globally unique) and ‘keys’ which actually store a block of data, ie a maximum of a 2-level system. JungleDisk have released some source code to show you how to get data back out of S3, should you want to do it independently of their software, which is a nice touch, but still, if the JungleDrive service was discontinued it would be a pain. In my travels, I’d come across a few open source solutions, so while they required a bit more effort, I thought it was worth investigating.

In the open source arena there’s a number of options, but out of those the ones I took most seriously were:

• Amanda now supports S3 as a target, 
• duplicity looks like a nice automated backup system with GPG encryption which has added support for S3 in recent versions
• s3sync is a Ruby script package which provides you with an rsync-style S3 implementation as well as a command-line tool for manipulating your S3 store (very useful in its own right since S3 don’t provide any tools)

Amanda would have been great, but it appears the S3 back-end is only available in the enterprise edition sold by Zmanda, not the open source version yet, which is disappointing. Duplicity also looked nice, but S3 support was only added in the recent versions, and it had a lot of dependencies and would have been a lot of trouble to set up manually on my server box, which is still chugging away on Debian Sarge and thus didn’t have recent enough versions in its stable list. An upgrade to Etch is definitely on the cards sometime soon, and I of course could have just grabbed the source packages, but I wanted to try out a simpler solution for now.

In the end, I went with s3sync. It has a few disadvantages over the other two, in that it doesn’t archive files up or compress them (leading to higher PUT/GET requests and transfers) and most significantly it doesn’t encrypt files - it can operate over SSL, but the files themselves are unencrypted, leaving them open to any Amazon employee or a server compromise once uploaded, which for my business documents was not acceptable. So I just scripted around that - I just use a standard incremental TAR backup of my most important directories, encrypt the results through GPG, and then use s3sync to upload the resulting data to S3. Works like a a charm - I get a small number of encrypted archives getting uploaded each time, and they’re small because they’re just diffs. The only problem I did have is that I didn’t check my Ruby version against the s3sync README - again Sarge was too old here, its Ruby 1.8.2 meant that all commands would work except uploading, which puzzled me for a while as I was still learning the nuances of the S3 system Luckily sarge-backports made it easy to grab a conforming version (and of libopenssl-ruby) which solved the problems.

Overall, I like S3 as an online backup back end - you can’t complain at the prices, and it’s nice to know that your data can grow as large as you like without crossing any major billing boundaries, and is taking advantage of Amazon’s infrastructure. I can see why startups are using these services a lot - being able to start small and grow without the infrastructural pain is a serious bonus (although its not perfect). For end-users I’d probably still recommend JungleDisk - it benefits from S3’s cheap prices while having a nice user interface - but if you’re a bit more picky about feeding your data through closed tools or having more control over the process, then you’ll want something else. If you have complex requirements duplicity looks worth your time, or if your requirements are fairly simple like mine, s3sync is a good low-level solution with very few prerequisites, so long as you remember to encrypt your own sensitive data.