Greetings,

The outbound mail service for "legacy" retail users (those using the old interface and system) is currently being blocked by Hotmail.

If you're trying to e-mail someone at Hotmail you've probably noticed it.

We're taking a multi-pronged approach to this problem so it will be cleared up shortly. When the fix is in place I will notify everyone via blog (and of course if you call in Support will know too!)

Thanks for your patience.
– Simon

As of late our mailout.easydns.com service has experienced some outages and downtime.

We've discovered the problem to be with the hardware we were using. To alleviate this we've deployed a new machine this morning to handle the mailout.easydns.com duties. Over the coming days we will also be adding more machines to handle the mailout load such that we can continue to deliver a reliable service.
If you notice any issues when using mailout.easydns.com please let us know.

As always , thanks for your continued support of easyDNS.

If you're a regular user of our easySMTP service at "mailout.easydns.com" you've probably noticed that it's been sketchy for the last couple of days, usually during peak times in the middle of the day!

The first part of the problem we've identified; we have had a couple of power users inadvertently pound our servers. We've managed to get that under control this afternoon, but the extra load highlighted an issue we've been working on for a while and are now ready to roll on.

The issue of course is server load! We've been getting ready to upgrade the operating capacity for "mailout.easydns.com" for a couple of weeks, and we've installed the extra capacity this morning.

These new servers will be in the pool by this evening after we iron out the usual operational quirks, and after we make sure that they won't munch your mail rather than deliver it.

Thanks for sticking with us during these load spikes. I know (all too personally, since I also use easySMTP) that trying to send an e-mail and having it sit in your outbox is one of the more frustrating Internetisms. And thank you for the feedback.

Thanks,
Simon

On a related note, we are dropping support for new .TEL registrations. At the time we added support we frankly didn't realize how broken the .TEL implementation is, but as a DNS hosting provider, we are ideologically opposed to a registry that won't let you set your own nameservers. Existing .TEL domains will continue to be supported.

The .CA space is unique in that it is one of the very few top-level domains that provide direct member input via the public consultations and the Board elections. I think all interested parties should avail themselves of that opportunity.

Every year the CIRA members (that's pretty well anybody who holds a .CA domain name) can put one candidate onto the ballot, in addition to the slate of candidates proffered via the CIRA NomCom (Nomination Committee). The member nominees this year are numerous, and I recognize a few names there. It's a shame we can only show our support for one member nominee at this stage of the game.

So, who should you support from the members' side of the slate this year?

My overall number one choice is Zak Muscovitch, a domain name lawyer and all around advocate for domainer rights. I've had numerous dealings with him in the past and he's very plugged in and attuned to the domain name space. He's also written some groundbreaking articles about reverse hijacking.

If your main concern about the .CA space is around technical stability and security, I would look at Andrew Sullivan, a long time DNS guru who I've turned to for advice and guidance in the past.

And if you're looking for an all around generalist with a good head for numbers and a down-to-earth grounding then I see that Rick Anderson is running on the member's side of the ledger this year. He's been on the Board before and I thought it was well served by his presence.

We had a brief blip on the outbound mail system for "legacy" retail users at the address mailout.easydns.com. We have restarted the responsible service and mail should be flowing again without issue.

Thanks for your patience.

[UPDATE - September 1 @ 1:40pm] – This issue has been resolved. More details about what steps are to be taken are located here:

http://blog.easydns.org/2010/09/01/mailout-easydns-com-upgrades-this-evening/

The "Greylisting" service on the new mail system has been disabled for the moment as we troubleshoot some odd results we have been getting with it.

Customers should expect a little bit more spam, but we hope to have the weirdness ironed out by this weekend.

[UPDATE - September 1 @ 1:39pm] – Greylisting has been re-enabled on the new mail system, and we're also incidentally increasing the capacity of this mail pool preemptively.

The article states "An attack on DNS is an attack on The Internet" and this much is true. As we always quip around here, "DNS is something nobody notices until it stops working".

I have to admit that in the early days of easyDNS I was oblivious to the possibility of DOS attacks. It simply never occurred to me. We were able to proclaim 100% DNS uptime since launching in1998 for a glorious 5 years and then on April 14th, 2003, it all ended as we got hit with a DOS that pancaked all four single-node nameservers and every domain on the system went dark for about 75 minutes. I nearly had a nervous breakdown, and then over the summer I thought long and hard about the ramifications and at the time surmised that the DNS hosting model was doomed.

Then we started looking at DNS anycasting but it took us another 5 years to get there. In the meantime we had another outage from another DOS: about an hour on Sept. 14/2005.  We added Prolexic DDoS mitigation within weeks of that attack and are happy to report we haven't had an outage since.

In the intervening years we also moved ourselves to a DNS Anycast architecture. While it is significantly harder to bring down an anycast architecture with a DOS attack, it can still happen. Usually instead of a complete and utter outage, you get "regional outages", which is basically a euphemism to deflect assertions of downtime: "Some users may experience regional outages….like North America and Europe" (credit to Steven Job for that bit of humour).

Some DNS Providers guarantee you that they will never go down and assert 100% DNS uptime in face of prior DOS attacks. In reality, every single DNS provider in existence for more than 5 years has had downtime. If the DOS attack that hit DNSMadeEasy last week really was 40 or 50 GIGS, and if it would have hit us, I hesitate to say "we would have stayed up".  In 2006  we got hit with an attack that was 20 to 25 gigs, and we didn't go down completely ("Some customers may have experienced regional outages"), but we sure felt it. Prolexic withstood the attacks and at the end of it we had to write a few enormous cheques to our providers to cover the bandwidth.

But I have long since backed away from my 2003 trepidations that the centralized DNS hosting model was doomed, for a few reasons:

1. DNS Anycast changes the game and drastically raises the bar for a DOS attack so that even if the resources can be mustered to do it, the duration of an outage is usually decreased as more numerous network carriers become aware of the problem and act to corral it.
2. DDoS Mitigation strategies have also improved. These days I think we are pretty well under a continuous state of low intensity DOS attack in one form or another. By low intensity I mean it doesn't bring us down anymore, but these attacks are about 10 to 20 times more powerful than the 2003 attack that did us in, so:
3. The DOS attacks that DNS providers routinely mitigate every day would probably level many non-professional, non-dedicated DNS setups.
4. The other benefits to using an specialized DNS hosting provider outweigh the isolated risks of DOS attacks. A good example of this is DNS Anycast: the DNS best practice that is simply not-viable for many organizations to implement on their own. Commercial DNS providers make viable through their economies of scale.

But this is the internet. If you elect to take part in it, there are certain unpleasant realities that will come home to roost. Like if you own a domain name, sooner or later it'll get joe-jobbed in a spam mailout. So to eventually you will get caught in the crossfire of a DOS attack against some target that has nothing to do with you but it's big enough to mess up one of your infrastructure suppliers. Like an empty bottle thrown at random into a crowd.

On the DNS side of things there are a few steps you can take to either not go down, even if your DNS provider does, or to make any impact minimal.

1. Use a DNS provider that allows third-party zone transfers. Either one that lets your slave your DNS zone from a primary nameserver outside of their own system (basically using a DNS provider as secondary DNS), or one that lets you designate other nameservers outside their system that can slave your DNS zone from it. Ideally, both.
2. Use two DNS providers. If you have the ability to setup  point #1 above with multiple DNS providers, then you are pretty redundant right there. I got an email from a large web services company after the DNSMadeEasy DOS who uses both theirs and our services. He said they experienced no downtime and using two DNS providers was still a lot less expensive than their previous setup.
3. Or, just use any third party nameserver, even one of your own. Have it slave your zone from your DNS host (or have your DNS host slave from it). Unless you are the actual target of the DOS, then, like a jet that can fly as long as one engine is firing, you'll be fine for the brief time your DNS provider may be down (or experiencing regional outages).

Being connected to the internet has varying degrees of importance to different organizations. For some, no downtime is acceptable (i.e. for DNS providers or web hosts, it's very very bad). Other organizations take a couple years to notice that their domain name expired.

Depending on the seriousness of your web presence you may want to also consider additional measures and be aware of a few things.

• Many top-level rootzones (.com, .net, .org, .biz and .info) make modifications in near realtime. People may be accustomed to things like nameserver delegation modifications to take a day to kick in. In fact a lot of user-interface verbiage probably still says as much ("please allow 24 to 48 hours for your nameserver delegation to take effect"). In these rootzones it's closer to 3 to 5 minutes. Use that. The bad guys (spammers, botnets, etc)  "fast-flux" their nameservers all the time to thwart tracing and reporting. It's a tactic you can take back from the black-hats and you can fast-flux your nameservers to provide a moving target in a DOS situation.
• Warm spares: have your DNS mirrored on third party nameservers, but do not add them to your nameserver delegation. If your DNS provider goes down, you then temporarily swap in your warm spares.
• For web hosts or other infrastructure suppliers that run DNS for their clients: do the above, except when you need to make a switch to your warm spares, you change the rootzone glue record for your nameservers: this way you do not need to make changes to each customer domain's nameserver delegation. The caveat here is you tend to only buy time: if the DOS is targeting you and you change-up your nameserver glue, the DOS may eventually (or sooner) follow you to the new IPs. Having said that, you can keep doing this and you may be able to diffuse the attack.
• Another overlooked fact: you can round-robin a nameserver glue record. We've tried it and don't find it near as effective as DNS anycast, but in a DOS situation, if you can add more warm spares to your nameserver glue records, then do it. Again, this diffuses the attack. "Regional outages" may indeed be a euphemism but it really is better than "everything is down hard".
• Here's one we learned the hard-way: don't have your nameservers in the same netblocks as your web interface and data storage, especially if you provide infrastructure services. If your nameservers are going to get clobbered you at least want to be able to get email and maybe provide a modicum of critical services to your users, something you can't do if your entire operation is within the same /24 that has been null routed by your upstream providers.

If I can perhaps add some comments to this theme: I would not wish a nameserver outage on any DNS provider. And you can believe me, when it happens, the people inside that company are tearing their hair out, suffering extreme mental anguish and pulling out all the stops to restore services. When I see a DNS provider taken out by a DOS attack and chatter on twitter, etc along the lines of "XYZDNS is down #fail #fail #fail" I want to thwap those people upside the head. Get a life. Do you think your DNS provider is out on the golf course while his business is being taken apart by a DOS?

While I am a businessman and we are a for-profit company, I do not relish gaining business at the expense of a DNS provider who's down because of a DOS attack. I'd rather gain customers on price, service offerings, customer support, our good looks, anything but a competitor going down because of a DOS. I guess because I've been there, I know how it feels. (Not all DNS providers take this view, in fact some of them pounce with glee when the opportunity presents itself, firing up the telemarketing crew to cold call the fallen provider's customers. If you're a customer of ours you have perhaps received such a call in the past).

DOS attacks are criminal acts. Get pissed off at the criminals who undertake them, not the people who are on the front lines of having to deal with them. Use these tips to stay online regardless of who your DNS provider is. I'm not advocating you stop using your existing DNS provider, but rather you modify your tactics so that instead of your DNS host becoming your single DNS host, it becomes more of a "DNS infrastructure management" role, that you use to setup and maintain multiple DNS structures (combining in-band nameservers from your DNS host with out-of-band nameservers outside their cloud), and warm spares.

During this time, access to the new easyDNS interface (Caprica) will be disabled. The old easyDNS interface (Retail) will not be affected.

We apologise for any inconvenience.

[UPDATE - August 13 @ 2:11am] – The database maintenance is complete, and the new easyDNS interface (Caprica) is accessible at this time. We thank you for your patience and understanding.

It's called Simcoe Day in Toronto.

Our telephone support hours are from 9am to 5pm Eastern.

You can always email us for help at support@easydns.com .

Thank you,

The easyDNS Support Team