Interview of Geffrey Noer – SGI Supercomputers, by Sander Olson

Here is an interview of Geoffrey Noer. (interview by Sander Olson) Mr. Noer is the Senior Director of Product Marketing for SGI, formerly Rackable. At the Supercomputing 2009 conference SGI unveiled the Altix UV supercomputer, which scales to petaflop levels and is capable of running standard software without recompilation. SGI is focused on becoming a force in the field of high-performance computing.

Technical Advances in the
SGI® Altix® UV Architecture (12 page pdf)

The key benefits of the Altix UV Xeon architecture include:
• Massive In-core Computation. The Altix UV allows much larger and more detailed models of physical systems

or any large data set to be entirely memory resident.
• Massively Memory Mapped I/O. For applications that are bound by random I/O accesses on large data sets, the Altix UV with Intel Xeon processors offers up to 1,000X performance increases by enabling entire datasets to be brought into main memory.

• Highly Efficient Application Scaling and Message Passing. Altix UV utilizes an array of advanced hardware and software features to offload thread synchronization, data sharing and message passing overhead from CPUs— accelerating critical tasks by up to 100x. These features benefit all programming styles, and for MPI applications, they are collectively referred to as the “MPI Offload Engine”, or MOE.

• Greatly Simplified Application Load Balancing. In cluster computing environments, each node completes its own threads and then waits until all other nodes complete their assigned tasks. The global shared memory available with the Altix UV for Intel Xeon processors allows processors that finish early to also work on other threads since each processor has access to all data and synchronization points through globally shared memory.

• Smooth Scaling of Application Size and Complexity. In most cluster environments, applications run on a fixed number of nodes that each have a fixed amount of CPU cores and memory. Applications run into a “wall” when they exceed the fixed amount of memory per core or node in the cluster. Conversely, applications running on a Altix UV do not run into a memory induced wall. Instead, they scale smoothly by drawing on additional memory distributed throughout the system.

• Petascale system and application scalability. In addition to global shared memory support where all resources are shared by a single copy of the operating system, the Altix UV provides an even larger Globally Addressable Memory (GAM) which enables systems to be built that extend beyond the shared-memory reach of any given processor or OS implementation. Advanced coherence functionality and atomic operations that increase efficiency of GSM environments also allow single MPI and PGAS applications to scale to over 256,000 cores and 8 PBytes of memory using efficient GAM capabilities.

• Efficient Application Development. Developing threaded applications, MPI applications, or PGAS applications on the Altix UV enables rapid development and large problem solution in the early stages of the parallelization process. Using these systems, applications that will ultimately run on thousands of CPU cores and access tens of terabytes of memory can be solved when only moderate levels of parallelism have been developed—proving algorithms early in the development cycle and shortening the time it takes to generate first results.

• Lower Cost Through Efficient Memory Utilization. In cluster systems, each node has a copy of the operating system, IO buffer caches, and additional space for message passing overhead and duplicated data structures. Each node is also typically configured to have a large amount of memory per core—just in case a large memory application is assigned to that node. These two factors combine to lead to large amounts of excess memory being purchased for cluster systems—greatly inflating their costs. In contrast, the global shared memory architecture in the Altix UV only requires a single O/S and a single buffer cache which reduces that amount of memory overhead. And since every application can access the entire memory, threads that need more memory than is available on their local nodes directly utilize memory resident on other nodes – greatly reducing the total amount of memory purchased.

• Simplified administration. The Altix UV platform enables large collections of compute, memory and storage resources to be managed together, significantly reducing the complexity and cost system administration

Question 1: You recently attended the supercomputing 2009 conference. How did that go?

Answer 1: SGI has been one of the leaders in the supercomputing space, and this year’s supercomputing conference was particularly important for the company. We launched a new scalable supercomputer called Altix UV, which is truly a unique machine. We believe that this supercomputer should have a major impact on the field of high-performance computing. Based on Intel’s upcoming NehalemEX processors, it will become available soon thereafter.

Question 2: The Altix UV is labeled as the world’s “fastest supercomputer”. Could this machine scale to petaflop levels?

Answer 2: Absolutely, architecturally this machine is capable of scaling to petaflop speeds. The NehalemEX can easily scale to 8 sockets/16 threads, but we have invested considerable resources in allow the UV system to scale up to 256 sockets/2048 cores and up to 16 TB of memory. But more importantly, it runs standard X86 software, so it can take full advantage of standard operating systems and applications. This is a departure from most supercomputers, which run on proprietary processor architectures.

Question 3: But supercomputers have to date primarily been used for scientific, as opposed to industrial uses.

Answer 3: Yes, and this is what makes the UV groundbreaking. Standard binaries run seamlessly on it, and no recompilations are necessary. This is a supercomputer that is designed for financial services, enterprise manufacturing, and product design companies. So this is truly a supercomputer designed as much for the business world as it was for scientific and government applications.

Question 4: What role will cloud-computing play in the high-performance computing field during the next decade? Will it obviate the need for companies to buy their own hardware?

Answer 4: SGI provides cloud computing infrastructures as well as more general purpose computers, and we see both businesses expanding. Cloud computing is growing and should continue to grow exponentially, but it won’t eliminate the need for organizations to have their own computers. Organizations that use hardware 24-7 for extended periods will still need their own hardware. Aggregate demand for computing power will inevitably increase, and we are well-positioned to benefit from that.

Question 5: How much does it currently cost to set up and operate a large data center? What measures can be taken to keep costs from spiraling upward?

Answer 5: Large data centers consume megawatts of power, and often have monthly power bills in the millions of dollars, so this an important issue from both an environmental and economic viewpoint. Data centers can be set up in areas with low electricity costs, such as the pacific northwest. A data center can minimize the number of power conversions, which needlessly waste electricity. In our server products we reduce the number of conversions to a minimum. The aim is to get efficiencies in the 90%-95% range. Still another method to reduce costs involves using evaporative cooling techniques, which only chills the cooling water when necessary. That alone can reduce cooling costs by 80%. Our ICE Cube modular data centers can easily take advantage of this type of approach. So data centers need to know that they can save significant sums of money and substantially reduce their carbon footprint by adopting these technologies.

Question 6: SGI now offers GPU computing solutions. What role do you anticipate GPUs playing in data centers and High Performance Computing?

Answer 6: The GPU will clearly play an increasingly important role in high performance computing deployments as time goes by. GPUs offer the opportunity to meet certain compute demands with a much smaller number of nodes. So for some specific types of applications GPU racks are becoming the ideal compute platform. But as useful as they are, we don’t expect to see a GPU in every server as many applications do not sufficiently benefit from GPU based acceleration.

Question 7: How many data centers currently exist? How many more do you anticipate will be built during the next decade?

Answer 7: There are currently thousands of operational data centers, and the number of data centers is growing dramatically. Moreover, data centers are denser than ever before – the servers are packed more tightly together. We are seeing increasing numbers of data centers consuming megawatts of electricity. So it is no surprise that energy-efficient servers have become such a priority.

Question 8: Solid-State drives offer advantages in speed, power consumption, and reliability over hard drives. Will SSD’s supplant hard drives in data centers and elsewhere?

Answer 8: SSDs are another important data center trend. Like so many new technologies, SSDs were overhyped early on. But substantially reduced costs and increased SSD capacities are making SSDs viable for many applications. We are seeing a tremendous amount of interest in SSDs, and the technology has clear advantages over hard drives. SSDs are more reliable, consume less energy, and are much faster than hard drives, all of which make them valuable to data centers. As time goes by we expect to see data centers making ever greater use of SSDs.

Question 9: Data centers are dependant upon the continuation of Moore’s law. Are you concerned that Moore’s law may end soon?

Answer 9: It seems like every year, someone in the industry comes out and warns that Moore’s Law is about to hit a ceiling because of a new inherent physical limit in semiconductor technologies. Then a new discovery is made, circumventing the issue and Moore’s Law continues. We certainly don’t see an end in sight at this point and have every confidence in the ingenuity of the semiconductor industry’s engineers.

Question 10: Some have suggested extreme solutions to heat problems, such as moving data centers to arctic locals or to offshore ships. Will such solutions ever be viable?

Answer 10: It depends on questions of latency – how long it takes a request to be fulfilled. The concept of an offshore data center ship is intriguing because such a ship could potentially be located in close proximity to a major city, thereby minimizing latency issues. So such a concept may eventually become feasible.

Question 11: SGI offers shipping containers filled with servers. Is this the future of the data center?

Answer 11: The traditional brick and mortar data center has been with us for decades. So it will take some time before containerized computing becomes completely mainstream. However, the containerized data center is gaining solid traction with essentially ever major server manufacturer now offering containerized solutions. We expect this paradigm will become increasingly important over time, whether it be to expand out brick and mortar facilities or to construct entire data centers consisting of a number of containers in a field completely replacing the need for lengthy and expensive construction projects.

Question 12: How do you see cloud computing and high-performance computing evolving over the next decade?

Answer 12: I see much more customization of clouds for particular markets. The high-performance computing field has been growing rapidly, and should continue to grow in importance for the next decade. High-performance computing tends to be a harbinger of things to come for mainstream computing. So I think that increasing numbers of corporations will take advantage of supercomputing power, whether it be from cloud computing or from supercomputers such as the Altix UV. Affordable petaflop computing will eventually become mainstream and an indispensable tool for both science and industry.

SGI Altix UV product page