Wong Iseng's List: HPC in the Cloud

• • Lha kok bisa walaupun tidak komunikasi tapi tetep ada degradasi performance ? Virtual Machinenya sendiri bermasalah ?
• and this is true even when the nodes don’t communicate
• Figure 2 shows the run times of the benchmark programs. From the results, we see approximately 40%–1000% performance degradation in the EC2 runs compared to the NCSA runs. Greater then 200% performance degradation is seen in the programs CG, FT, IS, IU, and MG. Surprisingly, even EP (embarrassingly parallel), where no message-passing communication is performed during the computation and only a global reduction is performed at the end, exhibits approximately 50% performance degradation in the EC2 run.
• This article has shown that a performance gap exists between performing HPC computations on a traditional scientific cluster and on an EC2 provisioned scientific cluster. This performance gap is seen not only in the MPI performance of distributed-memory parallel programs but also in the single compute node OpenMP performance for shared-memory parallel programs.
• …While cloud computing provides an extensible and powerful computing environment for web services, our experiments indicate that the cloud (or Amazon’s EC2, at least) is not yet mature enough for HPC computations. We observe that the GFLOP/sec obtained per dollar spent decrease exponentially with increasing computing cores and correspondingly, the cost for solving a linear system increases exponentially with the problem size—very much in constrast to existing scalable HPC systems.
• If cloud computing vendors are serious about targeting the HPC market, different cost models must be explored. An obvious first step would be to offer better interconnects or nodes provisioned with more physical memory to overcome the slower network.

4 more annotations...

• One of the major questions the study hopes to answer is how well the DOE’s mid-range scientific workloads match up with various cloud architectures and how those architectures could be optimized for HPC applications. Today most public clouds lack the network performance, as well as CPU and memory capacities to handle many HPC codes. The software environment in public clouds also can be at odds with HPC, since little effort has been made to optimize computational performance at the application level. Purpose-built HPC clouds may be the answer, and much of the Magellan effort will be focused on developing these private “science clouds.”

• For the more traditional MPI applications there were significant slowdowns, over a factor of 10,”
• Since there has been some followup discussion, I wanted to clarify and add some context. The factor of 10 was a comparison between our unvirtualized Magellan hardware and Amazon’s Elastic Compute Cloud (EC2) using m1.xlarge instances. We ran the NERSC6 benchmarks to perform the comparison. For the seven applications we tested, the mean slowdown factor for EC2 relative to Magellan was 10.8. The best application, GAMESS, was 2.7 times slower, while the worst performance was with PARATEC, which was 51.8 times slower. Again, the Magellan results were on unvirtualized hardware with an Infiniband interconnect.

• How do people actually use multiple machines to solve a problem? – This is really the root question behind all of this work. The first scenario is high-end shared-memory machines (ala Cray supercomputers) and I’m going to eliminate that type of compute from the conversation due to the fact that it simply can’t be well-replicated in the cloud as we currently know it. The far opposite end of the spectrum is “manual” clustering or map reduce – someone figures out a problem they want to solve, divvy’s it up amongst N nodes, and then individually runs a program on each node with the appropriate settings and then manually aggregates the results. This extreme is most likely done by ad-hoc projects or those not familiar with traditional HPC technologies and approaches. Between the two extremes listed, there are Map/Reduce implementations and traditional MPI programs targeted at distributed memory systems.
• The first scenario is high-end shared-memory machines
• The far opposite end of the spectrum is “manual” clustering
• Between the two extremes listed, there are Map/Reduce implementations and traditional MPI programs targeted at distributed memory systems.
• very easy to utilize for lower-throughput MPI-based HPC - given you can get n Linux boxes for 0.10/cpu/hour and, because of the vast community that has grown up around it, there are pre-packaged clusters (via AIM)
• commercial vendors building businesses on top of providing HPC-style compute in EC2 in an “on-demand” fashion.
• but there currently isn’t any built-in infrastructure to bind those nodes into a single group/cluster.
• I’ve been chewing on whether or not it makes any sense to try to push HPC-style work into Azure, or if it should simply be relegated to the EC2’s of the world… One could conceivably build an implementation of MPI that, rather than relying on the underlying cluster would provide cloud-style/enabled communications between nodes… this could allow those most comfortable with (or with large existing code bases of) MPI-style apps to continue to utilize those libraries/applications, but one has to wonder if, unless the Microsoft pricing (to be announced later this summer) is incredibly cheaper than that of EC2, why would one bother (other than academic interest, of course) to build such?
• • nullThink together carefully along this line of thought. -- 2010-11-15
    The question is, how existing MPI user could make use of cloud ? Cloud enabled MPI ? Easily port into something else ? Nimbus ? -- 2010-11-15
    

7 more annotations...

• Microsoft is also providing an Azure resource for scientists that will not require an installation of Windows HPC Server. The service makes the National Center for Biotechnology Information’s BLAST technology, which lets scientists search the human genome, available on Azure. At SC10, Microsoft said it will demonstrate the NCBI BLAST application on Windows Azure performing 100 billion comparisons of protein sequences.

• The new EC2 cluster compute instance type is an excellent performing cloud server. Performance exceeded that of most of the "bare metal" cloud servers we benchmarked previously. Combined with 10 Gbps non-blocking clustering capabilities, and on-demand deployment & hourly billing, this new instance type provides exceptional value and capabilities for HPC applications.

• Evaluated the performance of resources from four production, commercial clouds. We have added to GrenchMark the C-Meter tool for evaluating the performance of cloud resources [3]. We have studied [2,6,9] the performance of resources from four production, commercial clouds: Amazon Elastic Compute Cloud (EC2), Mosso, Elastic Hosts, and GoGrid.
• Evaluated the variability of the performance delivered by production cloud services. We have collected year-long traces and, based on them, studied [8] the performance of over fifteen operations provided by nine services in two clouds, Amazon Web Services and Google App Engine.
• "The performance of the resources leased by four production clouds, including Amazon EC2, for running scientific applications is at the moment below the theoretical peak and below the maximum efficiency obtained in other specialized environments." [2][6][9]
• "The resource allocation strategy employed by the cloud user can be insignificant or can lead to 10-30% cost increase, when running long-term, grid-like workloads on clouds." [2][9
• Performance Analysis of Cloud Computing Services for Many-Tasks Scientific Computing,  IEEE Trans. on Parallel and Distributed Systems (TPDS)
• Epema,   A Performance Analysis of EC2 Cloud Computing Services for Scientific Computing,  In D.R. Avresky et al. (Eds.): Cloudcomp 2009, LNICST 34, pp. 115?131, 2010. 
    keywords cloud computing, scientific computing, performance evaluation.
• a,   A Performance Analysis of EC2 Cloud Computing Services for Scientific Computing,  TU Delft Technical Report PDS-2008-006, Dec 2008. 
    keywords cloud computing, performance evaluation, performance analysis, benchmarking, Bonnie, LMbench, CacheBench.
     

5 more annotations...

• During the past few years it has been no secret that EC2 has been best cloud provider for massive scale, but loosely connected scientific computing environments.
• pleasantly parallel, high-throughput computing workflows.
• difficult to run workloads which required high bandwidth or low latency communication within a collection of distinct worker nodes.

1 more annotation...

• The Magellan Cloud research team at the National Energy Research Scientific Computing Center (NERSC) was one of those beta customers and got a chance to test drive the new EC2 offering prior to this week's official launch. They reported that a series of HPC application benchmarks "ran 8.5 times faster on Cluster Compute Instances for Amazon EC2 than the previous EC2 instance types."

• “For perspective, in one of our pre-production tests, an 880 server sub-cluster achieved 41.82 TFlops on a LINPACK test run – we’re very excited that Amazon EC2 customers now have access to this type of HPC performance with the low per-hour pricing, elasticity, and functionality they have come to expect from Amazon EC2.” (Peter De Santis, General Manager of Amazon EC2)