Performance Advisory

Title: Terremark vCloud Express Outage 12/18/09               Document #: CPO031710
Advisory Class: Cloud Provider Outage                              Impact: Severe
Date Published:  3/18/10                                                  Date of Last Update:  3/18/10

Outage Description
Apparent Network's Cloud Performance Center (www.apparentnetworks.com/cpc) recently confirmed Terremark experienced connectivity loss which caused an outage in Terremark's vCloud Express services in their Miami data center.  The outage occurred on March 17, 2010 beginning at approximately 11:54am and ending 19:05pm EDT. There were intermittent periods of connectivity with high data packet loss between those times, and the number of connectivity loss events and duration varied slightly by location. According to Terremark's support forum page (https://community.vcloudexpress.terremark.com/en-us/discussion_forums/f/59/t/347.aspx_), their "network team has identified a packet loss between the Miami servers."

During that time, access to systems in the Miami data center was severely degraded and often unavailable.  Businesses utilizing Terremark's vCloud Express services from this datacenter were affected.

Testing Results

Apparent Networks' Cloud Performance Center (CPC), a free service that offers performance data on leading cloud services providers such as Amazon, Google and GoGrid, detected the connectivity loss on 3/17/10. The Cloud Performance Center provides real-time and historical data to help inform IT teams of network performance issues that could impact effective delivery of cloud services. It provides network path performance metrics -including bandwidth, jitter and latency-between the cloud providers and major cities throughout North America.

The Cloud Performance Center utilizes Apparent Networks' PathView Cloud service to test the performance of cloud service providers.  PathView Cloud software was configured to sample path performance to a series of pre-determined targets hosted at Terremark's vCloud Express Miami data center every 120 seconds.
   
PathView Cloud noted Terremark's vCloud Express outage-related connectivity loss events as follows:

According to Terremark's support user forum, their team acknowledged packet loss but they did not give an explanation for the outage. 

Click on the link below to access a report generated by PathView Cloud showing the outage reported from one of the PathView Cloud datacenters.

www.apparentnetworks.com/uploadDocs/3_17_10_Terremark_Outage_From_Dallas_Rackspace.pdf

Also visit the Cloud Performance Center (www.apparentnetworks.com/cpc) for live data showing the Terremark service interruption.

Cloud Providers Tested
Terremark's vCloud Express Services

Vendor Information, Solutions and Workarounds
No vendor information was provided to the public about this outage.
 
About PathView Cloud
PathView Cloud is a hosted network management tool that measures the performance of complete network paths from source to destination, including segments that pass through service providers' and carriers' networks. It enables IT teams and network managers to assess, troubleshoot and continuously monitor thousands of network paths simultaneously.  A free version of the tool allowing users to monitor and test five network paths simultaneously is available at www.apparentnetworks.com.

About the Cloud Performance Center
The Cloud Performance Center (www.apparentnetworks.com/cpc) provides real-time, region-by-region service delivery performance scores for cloud providers. Visitors can view an interactive map of the North American market and select locations that are relevant to their businesses. Based on their selection, the CPC provides a detailed Cloud Provider Scorecard for the major cloud service providers in those geographies. Service providers are ranked on an aggregate or overall performance score, as well as on a lengthy list of specific performance metrics, such as jitter, available bandwidth and packet loss. Users can also customize the performance metrics to suit their particular application or service needs. Armed with this unbiased performance data, companies can select the best provider or set of providers in the locations that matter most to them. 

About Apparent Networks
Apparent Networks is the only IT performance management provider that delivers the end-to-end service insight required for today's cloud applications. By experiencing network performance without affecting it, the company's patented path solutions (including PathView Cloud, PathView and AppCritical) assess network readiness, monitor service levels, and diagnose problems otherwise hidden from sight. Leading companies rely on Apparent Networks to assure application delivery and expand their service portfolios with confidence. For more information, visit www.apparentnetworks.com.

Disclaimer
The contents of this advisory are copyright (c) 2010 Apparent Networks Inc. and may be distributed freely as long as proper credit is given.