I know that some of you have been waiting for a while to get your beta invitation for playmoTV. The service has been running on a backup server at a reduced capacity ever since the hard disk crashed in one of our cloud servers and to maximise service quality we have had to limit the rate of invitations.
UPDATE 2012-03-29: The newly introduced redundancy server has not been receiving and sending network traffic reliably and we believe this is the reason for the non-deterministic behaviour some beta users have been experiencing. Therefore we have disabled the server until we have found a solution. Until then we have stopped inviting new users again.
UPDATE 2012-04-13: New redundancy server in place. Invites being sent out as we speak.
We are not there yet but this is a significant milestone
We have quietly been working on a failover system to eliminate the single-point-of-failure (SPOF) we previously had. There are to be honest still some SPOFs which we will continue to eliminate but this is a significant service redundancy milestone and it means we will be inviting most of you guys very soon.
The service infrastructure is quite complex and it is distributed between servers in three continents. These servers have different roles ranging from packet validation, munging flowing data bits, authenticating requests, monitoring server health, and the servers need to be able to communicate between themselves to accomplish their respective tasks. Because of this interconnectivity a server failure can affect the behaviour of other servers. By having more than one server serving each role we can minimise the disruptions.
On top of that the design principal we follow is serve-then-ask. This means that we try to err on the side of caution when it comes to restricting access to our service. So if the authentication servers are failing we allow the request to pass through until they start responding again and we are certain that we have to reject the request.
Our aim is to provide a reliable and robust service. One that our users are happy to pay a small monthly fee for.