Sunday, February 14, 2010

Migration Round-Up

Whew. The migration really went pretty well. Widget chats on queues were working by 9am Saturday. SMS, Google Talk, and Twitter gateways followed a bit later on Saturday. The remaining IM gateways came back Sunday morning. We believe that all of the pieces of the system are available. Load looks very good!

There is one thing about gateways you may need to know about: for now, you can only have one gateway on any given protocol on any single queue. Example: on one queue, you can have one AIM gateway, one Yahoo! gateway, one Meebo Me gateway, etc... You cannot have two AIM gateways on one queue currently. If you need to have more than one gateway on a given protocol, create additional queues, add your extra gateways, and assign the desired librarians. Or just chill out for two days while Eric fixes it.

This happened because when we migrated to the EC2, we found that our previous family of libpurple gateways were consuming far too many resources because of a bug involving updated versions of two core OS libraries. So, we switched our libpurple family gateways over to use the Spectrum XMPP transport, which runs far more efficiently. This just needs a bit more tweaking so that it will support multiple gateways of the same protocol per queue.

Other than that, we believe things are in good shape. Please let us know if you find bugs. We'll continue to use the Twitter feed to communicate issues as the week begins.

Monday, February 8, 2010

Migration to new host Saturday morning, Feb 13

Heads-up.

In an effort to increase system stability, we will be moving LibraryH3lp to a new host this coming Saturday morning, Feb 13, 2010. Saturday mornings (we're in the Eastern time zone) are very low-use.

While we can't provide an exact time frame, we expect the server to be completely unavailable for up to 1 or 2 hours as we migrate data.

We have already completed the work with the DNS, which means that the step with the most potential for causing lengthy server unavailability is done.

As previously discussed on our support group, in order to prevent embedded LibraryH3lp content from slowing down loading of your web pages in times of server outages, you can move the javascript to the very end of your web page. Ideally, this is right before the close body tag of your document. All the details, and a way to test your code, are here, in the docs.

Now, why are we moving? We've recently experienced a few failures completely beyond our control, coupled with some pretty poor communication from our hosting service. We can accept that a hardware failure can happen at any time, on any system. But we expect this kind of failure to be... rare. No one has made anything quite like LibraryH3lp before; we work hard to make this thing as stable as possible, even while we're building it and creating new features. It's no fun to put so much work into this system and get let down by the infrastructure that we (you) are paying for.

As always, we'll post system status updates as we go on our Twitter feed. If you want to get notices of activity on the Twitter feed over email, SMS, or some other format, check out notify.me.

We do apologize for the inconvenience. We'll do everything we can to make this migration as quick and painless as possible.