Lenovo and Intel’s partnership is having a real-world effect on the scientific community, with New York’s Flatiron Institute leveraging HPC and AI solutions developed jointly by the two companies. According to the Institute, prior to these solutions, technology would often be the limiting factor in its scientific research, although since working with Lenovo and Intel, the researchers no longer feel that’s the case. 

Flatiron’s Scientific Computing Core supports scientific work across four disciplines, requiring an HPC infrastructure to handle a variety of extremely large data files, from giant genomic sequencing files to 100,000 small files in a single directory. Flatiron’s on-site cluster – a central HPC infrastructure – is comprised of 17,000 Intel computing cores in Lenovo ThinkSystem SD530 servers, running HPC and AI workloads. This combination allows the computers to analyse large, rich datasets and simulate complex physical processes.

Researchers at the Flatiron Institute are pursuing advances in biological sciences, astrophysics, quantum physics, and computational mathematics. Yet, before the large-scale computing clusters, scientists at the Flatiron Institute were limited in their scientific progress by system constraints that imposed practical limitations in the speed and scope of their system-based research. 

Lenovo and Intel partnered with the Flatiron Institute to improve and expand its HPC and AI capabilities so technologies are no longer the limiting factor in its research.

“Cosmological simulations like galaxy formation and black hole creation require hundreds of thousands of cores connected to each other to run. Science should dominate our researchers’ time, not computation,” said Dr. Ian Fisk, Scientific Computing Core co-director at the Flatiron Institute. 

“Our goal is that the limiting factor in research should be the carbon-based systems rather than the silicon-based systems.”

Solutions powered by HPC and AI convergence

Research at the Flatiron Institute is conducted with the help of Lenovo ThinkSystem SD530 Dense Rack servers with Lenovo Neptune Thermal Transfer Module (TTM) technology running leading edge Intel Xeon Platinum 8268 processors, part of the 2nd Gen Intel Xeon Scalable family featuring Intel Deep Learning Boost technology. 

Leveraging the joint Lenovo-Intel solution, researchers at the Flatiron Institute today are not only able to process petabytes of data faster and more efficiently to accelerate the pace of research. They can now carry forward workload-intensive research projects that otherwise would simply not have taken place. 

For example, in the past decade, the cost of full genome sequencing has become more affordable as HPC has become more attainable. Scientists previously could only sequence about 2% of genomic data, but they can now look at the entire genomic sequence of thousands of families at once. This progress can be the key to more effective discovery of genes that cause disease or the development of precision medicine.

Due to their unique computing demands and limited space constraints, Flatiron needed a dense, air-cooled system with vast amounts of memory. 

“With our Lenovo HPC supercomputer cluster, we have four or five times more memory than a typical HPC farm,” Fisk said referring to the additional RAM available with each node.

“Reading a 200 GB genomic sequencing file for a biology project or making 11,000 connections to other nodes for an astrophysics project requires a lot of memory. Now, when a researcher comes along with an interesting challenge that their equipment doesn’t support, we can achieve that.”

Dr. Fisk said the Lenovo – Intel systems were the fastest and densest he had, praising them as “a workhorse,” and noting the importance of the system’s improved thermals so those intense workloads can operate in a small space.

“We see high-performance computing and artificial intelligence as the keys to unlocking an entirely new era of untapped potential for customers,” added Scott Tease, general manager for HPC and AI at Lenovo’s Data Centre Group. 

“Seeing some of the greatest minds rely upon our technology to discover answers for some of humanity’s most pressing questions is rewarding. But pushing the boundaries of HPC and AI capabilities even further so more people can benefit is what drives us.”