The Overnightscape MetaFeed

Well it’s finally happened.

The stat-happy webhosting.info this month made it known that we now host over one million domains. A million!

That’s a lot!

I had a hard time visualizing “one million” so I turned to my old friend Google Image Search (with ‘resuse rights’ checked) and saw enough to make me realize that I really didn’t need to know what “one million” of anything looked like.

We wouldn’t be here without the support of our customers, employees (past and present!), and everyone in-between. So thank you!

By the way…if you worked here you would have found one of these on your desk this week:

Do you live near Los Angeles and enjoy pooping fire?

You’re in luck.

Next week a small contingent of DreamHost employees will be at Orochon Ramen to test your mettle!

Orochon is known for serving up big bowls of ramen that come in seven levels of spicy. In addition to those seven, there are two extreme levels of spiciness – known as Special #1, and Special #2 – the hottest.

You may have even seen the Special #2 featured on Food Network’s Man vs. Food.

If you can finish a bowl of Special #2 in less than 30 minutes they’ll put your picture on the wall.

And we’ll give you a year of free hosting!

Orochon Ramen doesn’t have a website (Really! In 2010!) but they do have a page on Yelp. Check out the menu and photos of what you’re in for.

We’ll see you there next Wednesday, June 2nd, at 7PM! If you plan to attend please RSVP on Facebook so that we’ll have a general idea of how many people to expect.

If you do manage to complete the challenge just look for anyone wearing a DreamHost shirt and let them know your account information – a year of free hosting will magically appear on your account shortly thereafter.

NOTE: DreamHost is not responsible for exploded organs, anal leakage, excess sweating, dry mouth, constricted airways, homicidal impulses, amnesia, decreased sexual interest, kidney failure, muscle weakness, erectile dysfunction, nerve damage, liver damage, male breast growth, or any other physical harm that may arise from consumption of food or drink during this event. Enter this contest at your own risk.

And please come prepared.

You may or may not have noticed that yesterday, www.dreamhost.com was offline and unreachable for the better part of 6 hours. We can’t let something like that go without an explanation.

I should note that during this time no customer sites were affected (other than one – which I’ll get to) and the main “www.dreamhost.com” domain. Customer sites were up, our web panel was up, everything was up…including the ire of some tech-savvy Muslims!

We’ve got a fairly liberal free-speech policy here which we’re quite proud of. Speech that is protected by the United States Constitution’s First Amendment is protected by DreamHost. While we don’t always agree with the content of the sites we host, we do support their right to host it in America!

Yesterday was Draw Mohammad Day.

This did not sit well with roughly 21% of the world’s population.

We happened to be hosting drawmuhammadday.com, a site that encouraged people to draw images of Mohammed. That’s kind of a no-no in the Muslim world.

Incidentally, did you know there’s like a million different ways to spell Mohammed?

In the spirit of yesterday’s event, but without the offensive parts, I’ve drawn some pictures to show you what you might have missed!

Some people weren’t too keen on the idea of the Draw Mohammad Day website and suddenly we were the target of the largest Distributed Denial of Service attack (DDoS) we’ve ever seen. drawmuhammadday.com was the first to fall. It was the main target and it didn’t take long…based on our stats it looked like almost the entire country of Pakistan was attacking us! Well not really. Butnobody in Pakistan could reach YouTube, Facebook, or Twitter yesterday, so what else were they gonna do?

These weren’t just random attacks from here and there. We saw several Pakistani groups targeting us on their blogs, often providing step-by-step directions and automated tools for launching e-assaults on dreamhost.com and drawmuhammadday.com.

They did not let up once the site was down. At one point dreamhost.com (the site itself) was handling around 20,000 requests per second. To put that number in perspective, when our customers’ sites have traffic surges a busy day might see that number get up to ten or even twenty.

Our load balancers, as great as they are, typically handle about 4,000 connections at any given moment. During the attack they made it up to 400,000 before they seized up and crapped out. We believe that even the most top-shelf battle-hardened load balancing options would not have been able to withstand an attack of this scale – a quick jump in traffic about 100x larger than normal traffic patterns we see on any given day.

Our fault-tolerant setup relied on those load balancers and they proved to be our undoing. Luckily only some services were affected by this for a very short time (webmail being one of them) before we got them going again a few minutes later

To restore services we had to take the site down altogether while we moved it to newer, stronger hardware, beyond the reach of our load balancers. We tuned the Linux kernel on this new machine aggressively to use less memory for TCP connections. We also abandoned Apache, favoring a specialized nginx installation.

When we flipped the switch to get dreamhost.com up and running again at around 2PM PDT, the attack load had dropped to 130,000 simultaneous connections with over 20,000 requests per second. The new setup took it like a champ and continues to perform well today – even while we’re still seeing elevated traffic as a result of lingering attacks.

We’re proud to say (and repeat!) that customer sites were not affected and our control panel was still reachable during this entire debacle. And of course if you ever suspect server problems with your DreamHost account be sure to check dreamhoststatus.com!

We learned some lessons yesterday and, moving forward, we’re going to put them into practice. Thanks for hangin’ in there.

(not Mohammed)

February through April we solicited donations from our customers for a worthy organization: Home for Women and Children of Shiprock, New Mexico.

Home for Women and Children is a domestic violence and sexual assault shelter located on the Navajo Reservation in Shiprock, New Mexico. They have served the entire Navajo reservation – which covers approximately 26,000 square miles – for 32 years. The program also serves all of the Four Corners area, New Mexico, Arizona, Colorado, and Utah.

32 of our customers made donations totaling $1193.60, which we then QUADRUPLED with our own money for a total donation of $4774.40. While our matching period may have ended, you can always submit donations to “HWC” directly. If you were one of those 32 generous souls – thank you.

Home for Women and Children joins a long list of non-profit organizations to which our customers have contributed over the past several years.

Current customers can visit the Home / Give to Charity section of their account control panel to submit donations at any time. The charge will appear on your next hosting bill and we’ll more-than-match anything you’re able to give, multiplying your own gift by a factor of four.

We’ve chosen to support The March of Dimes as our next donation target.

Their mission is to improve the health of babies by preventing birth defects, premature birth, and infant mortality. They carry out this mission through research, community services, education and advocacy to save babies’ lives. March of Dimes researchers, volunteers, educators, outreach workers and advocates work together to give all babies a fighting chance against the threats to their health: prematurity, birth defects, and low birthweight.

Please consider making a donation if you haven’t already.

Thanks!

…but not for long!

We’re giving them all away to twenty lucky entrants in our Apple-A-Day Twitter contest!

We had hoped to have these in our hands on the 3rd when everybody else did. Unfortunately Apple cancelled our first order, probably as a result of us ordering twenty when the stated limit was TWO. The credit card we originally used had also been cancelled thanks to the recent Monoprice fiasco by the time Apple was ready to charge it, so that probably didn’t go over very well either…

The good news is we were able to sneak in another order – and that one went through without a problem – and arrived today!

Bingo!

We unpacked 15 iPads, but the fourth box had a note…

Just kidding. I put that there. But it got your heart rate up a little, right?

So what can you actually do with 20 iPads? Quite a lot, actually!

There’s the obligatory doorstop.

Dominoes came to mind as well…

var swf = new SWFObject("https://media.dreamhost.com/mp4/player.swf", "mpl", "480", "379", 8); swf.addParam("allowfullscreen", "true"); swf.addParam("allowscriptaccess", "always"); swf.addVariable("file", "http://blog.dreamhost.com/wp-content/uploads/2010/04/dominoes_480x360.flv"); swf.write("v7152");

Playing chess on a touchscreen is like nothing you’ve ever experienced.

20 iPads have no problem giving someone bad news.

iPad/crowbar batting practice was a hit!

I’ve played a lot of bowling apps on my iPhone.

This one makes you feel like you’re really there!

var swf = new SWFObject("https://media.dreamhost.com/mp4/player.swf", "mpl", "480", "379", 8); swf.addParam("allowfullscreen", "true"); swf.addParam("allowscriptaccess", "always"); swf.addVariable("file", "http://blog.dreamhost.com/wp-content/uploads/2010/04/bowling_480x360.flv"); swf.write("v7274");

The iPad talks a good game, but when it comes time to play a little Call of Duty, he falls flat.

Trash talking only works when you’re good at something. PUNK.

I’ll be honest – the iPad was much better at playing drums in Rock Band than I was. Had some problems hitting the pedal though.

With the day wearing on I decided that maybe nobody would realize if a few iPads went missing. So I thought it might be a good idea to swallow a few and sneak them out that way.

It wasn’t easy. And it wasn’t successful.

var swf = new SWFObject("https://media.dreamhost.com/mp4/player.swf", "mpl", "480", "379", 8); swf.addParam("allowfullscreen", "true"); swf.addParam("allowscriptaccess", "always"); swf.addVariable("file", "http://blog.dreamhost.com/wp-content/uploads/2010/04/blahoo_480x360.flv"); swf.write("v5718");

The following Twitter users have won a 16GB Wifi iPad:
mrwickedd
enotsola
rafaldyrda
mike_Henderson
raimon49
feltpad
caoanroad4800
daniloeu
omtc
jbchan
twEapen
Onlera
emberna
mattmaher
MinaNJ
lisadrowe
LauraRogner
pkimbrel
benrwoodard
eminsker

If you’re listed above, CONGRATULATIONS! – and be on the lookout for a message from @dreamhost. We’re going to need some shipping information from you!

Winners must reply to @dreamhost on Twitter no later than May 14th at 11:59PM, PDT (one month from today) or they will forfeit their prize. At that time we will pick from the other entries received to select alternate winners. We don’t want to hold on to these things forever!

No iPads were harmed or removed from shrink wrap in the making of this blog post.

// <![CDATA[ digg_url = "http://blog.dreamhost.com/2010/04/14/weve-got-20-ipads/"; // ]]>

Today we’re revealing some never-before seen photos of some of the darkened corners of our datacenters.

It’s not something we’re proud of, but we intend to do something about it.

You network admins know how it is. Sometimes a server dies and it’s just easier to leave it there and reuse its network cable without rerouting everything. And sometimes it’s just too hard to put a new cable in the channel where it belongs so you end up draping it over the top of everything. It’s easy, it works, and your customers never need to know about it.

Before you know it your hardware is buried in a sea of cables and has become physically unreachable.

Let’s face it. Datacenters are kind of boring. The only major cosmetic change they’ve gotten over the past twenty years has been the size of the computers sitting in them. And maybe the colors of the cables running through them.

We’ve today announced a new initiative : Wireless Datacenters.

At our Los Angeles datacenter we’ve replaced literally thousands of miles of cat5, cat6, and fiber with a large supply of 802.11n USB wireless dongles. Everywhere you look you’ll find dongles dangling everywhere. Dangling dongles. (Internally this project was codenamed “Dang Dongs“.)

We’ve donated all of our old cabling to a local recycler, and the amount of clutter we’ve eliminated in our datacenter is phenomenal. You can now walk through the entire facility without tripping or fear of passing under a cable avalanche.

We’ve also reduced our carbon footprint by a huge amount. Do you have any idea how many trees it takes to make a 6 foot ethernet cable? Probably not very many…but still. Having less stuff generally means you’ve used fewer resources to make that stuff. It’s really a win-win all around.

Ethernet entanglements are now a thing of the past. 802.11n wireless is the future, and the future is now.

In fact it says right here on the box from Linksys that the new router we bought will “stream stored music to devices around your home” – no small feat. Imagine how fast it’ll serve your websites. Should be pretty great.

You can expect a white paper and some more photos of the new deployment soon.

As of today, Linus has decided to merge our Ceph distributed file system into 2.6.34!

Awesome.

Twitter’s okay I guess. 10 billion tweets is a big milestone.

But Twitter has one fatal flaw: it caters to people.

Animals are well-represented all over the Internet in wikipedia articles and blog posts, but those are all written by humans on behalf of animals.

Three quarters of Americans think that it’s “‘egotistical and a waste of time’ to develop a Facebook page for a pet”. Surely that means that 25% of Americans – over 75 million people – think it’s a great idea!

The evidence is clear. Pets deserve a voice of their own on the Internet. So today we’re launching…

PetStatus.com is just like Twitter…except it’s very clearly designed for pets, by pets.

Just look at the rich embeddable experience that awaits you and your pets:

{ "user":"conan", "server":"petstatus.com", "headerText":" and friends" }

And that’s just the half of it. Pets aren’t limited to submitting status updates and following others through the web – they can use a growing number of third-party micro-blogging clients to stay on top of their friends’ latest exploits.

If your pets are anything like mine, they don’t think too much of humans…

They’ve also got a lot of free time on their hands – much more than I do. Suddenly ten billion doesn’t seem so far away!

Remember, PetStatus.com is for non-humans only. That goes double for you, Samurai Unicorn guy.

Status.net, our new one-click software package, powers the entire operation. DreamHost customers can now install Status.net to their own domains with a single mouse click – making specialized Twitter clones at whim in a matter of seconds!

So start spreading the word – your pets will be heard!

What the…

Introducing the DREAMHOST TRADE SHOW BOOTH. Note the ample room for strolling, stretching, high-fiving, and macramé.

We’re not actually going to a trade show. And hosting conferences still aren’t for us. But we do need to hire people. Badly. Quickly.

We have a soft spot in our hearts for students of the Claremont Colleges of Southern California…particularly Harvey Mudd. So we’ll be there next Friday the 19th at their career fair!

If you’re a student at any of the Claremont Colleges, please stop by! You never know what you might find… DreamHost swag, new friends, maybe even a job!

If you’re not a student, don’t bother coming! You can apply online, right now, for any of our open positions.

November through January we solicited donations from our customers for a worthy organization: Therapeutic Living Centers for the Blind.

TLC is a community of learners and teachers who have come together to create a supportive and encouraging environment for those with developmental disabilities and blindness. More than a facility, TLC represents a family working together to support each individual in order to develop his or her unique strengths.

11 of our customers made donations totaling $378, which we then matched for a total of $756. While our matching period may have ended, you can always submit donations to TLC directly.

TLC joins a long list of non-profit organizations to which our customers have contributed over the past several years.

Current customers can visit the Home / Give to Charity section of their account control panel to submit donations at any time. The charge will appear on your next hosting bill and we’ll match anything you’re able to give. We host nearly a million domains now and know that far more than 11 of you are capable of supporting something larger than yourself, so please do!

We’ve chosen to support Home for Women & Children of Shiprock, New Mexico as our next donation target.

They’re a non-profit shelter providing safe harbor for victims of domestic violence and sexual assault. Please consider making a donation if you haven’t already.

Thanks!