IP developer IntelliProp is moving into the chip business with a software defined memory fabric based on the Compute Express Link (CXL) standard.
The company has been developing memory fabric technology for 20 years with tier one customers including Dell, Samsung, Sony and Intel. It was part of the Gen Z standards group which last month merged with CXL after the launch of the CXL3.0 standard.
“We believe we are at CXL4.0 and above,” John Spiers, CEO of IntelliProp told eeNews Europe.
“One of the problems we are solving is the DRAM bottle neck: as the number of cores increases the bandwidth per core decreases and that gap is continuing to grow,” he said. “At the same time AI exacerbates the problem. The AI models run faster when you process them in memory but servers lack the memory to do this and often the GPUs are not set up in peer-to-peer mode and can’t share the memory. This leads to stranded memory, servers with surplus DRAM that can’t share the memory with other servers that are over utilized.”
“We have built a fabric called Omega that allows you to connect CXL devices to hosts and outside the server. The first wave is to expand memory inside the server using CXL as a memory drive that plugs into the chassis and the CXL fabric in the server incorporates this into the memory pool.”
The Omega Memory Fabric eliminates memory bottleneck and allows for dynamic allocation and sharing of memory across compute domains both in and out of the server, providing Composable Disaggregated Infrastructure (CDI) that allows memory to scale across servers and across racks in the data centre.
“We think we are in good shape as our end points are 100% CXL compatible and we will continue to update the CXL portion of the chip over time,” said Spiers.
“CXL3.0 promises rack scale memory but it’s lacking the features to share memory outside the server. We started work on Gen Z and we built out a lot of the Gen Z fabric features that are being incorporated into CXL such as multipath and peer to peer computing so the GPUs can work on AI workloads without going back to the CPU. These things are not in the CXL spec and won’t be for some time. It something that customers will drive and the spec will support it, so we don’t see any risk areas – we think we are aligned and we are very active in the consortium,” he said.
IntelliProp has implemented CXL standard alongside its Fabric Management Software and Network Attached Memory (NAM) system in three field-programmable gate arrays (FPGA) and developed a chip using an FPGA-to-ASIC conversion process.
The company is currently evaluating the architecture of the next generation chip, whether to go with a monolithic device or a series of chiplets that could be combined with processors and memory systems from other suppliers.
“We are researching doing our ASIC as a series of chiplets using the UCI interface – its more expensive to do multiple chiplets rather a single chip but its simpler to do, we are looking at that and what the increase in cost will be,” said Hiren Patel, CTO of Intelliprop.
This is about reducing the latency of the memory access which is determined by the chip and the interface. “Today latency is 900ns for a 644byte cache read in the FPGA and we want to get to sub 200ns – we are looking at Gen 5 PCIe at 12nm and Gen 6 at 7nm,” he said.
Adding the technology to servers and solid state drives allows pools of memory to be created that can be accessed by multiple servers, all under software control. This addresses the problem that around half the memory in a data centre is under-used, as memory attached to one server can’t be used by another. This is costly and burns power.
“IntelliProp is on to something big. CXL disaggregation is key, as half of the cost of a server is memory. With CXL disaggregation, they are taking memory sharing to a whole new level,” said Marc Staimer, analyst at DragonSlayer. “IntelliProp’s technology makes large pools of memory shareable between external systems. That has immense potential to boost data center performance and efficiency while reducing overall system costs.”
“IntelliProp’s efforts in extending CXL connection beyond simple memory expansion demonstrates what is achievable in scaled out, composable data center resources,” said Jim Pappas, Chairman of the CXL Consortium. “Their advancements on both CXL and Gen-Z hardware and management software components has strengthened the CXL ecosystem.”
Today’s remote direct memory access (RDMA)-based disaggregation has too much overhead for most workloads and virtualization solutions are unable to provide transparent latency management. The CXL standard offers low-overhead memory disaggregation and provides a platform to manage latency.
“History tends to repeat itself. NAS and SAN evolved to solve the problems of over/under storage utilization, performance bottlenecks and stranded storage. The same issues are occurring with memory,” said Spiers.
“Our approach unlocks memory bottlenecks and enables next-generation performance, scale and efficiency for database and AI applications. For the first time, high-bandwidth, petabyte-level memory can be deployed for vast in-memory datasets, minimizing data movement, speeding computation and greatly improving utilization. We firmly believe IntelliProp’s technology will drive disruption and transformation in the data centre, and we intend to lead the adoption of composable memory.”
The Omega NAM is well suited for AI, ML, big data, HPC, cloud and hyperscale / enterprise data centre environments, specifically targeting applications requiring large amounts of memory and the chips are increasingly based on chiplets. This also helps to move the memory away from the servers, reducing the amount of cooling that is needed.
“In a survey IDC completed in early 2022, almost half of enterprise respondents indicated that they anticipate memory-bound limitations for key enterprise applications over time,” said Eric Burgener, research vice president, Infrastructure Systems, Platforms and Technologies Group fo market researcher IDC. “New memory pooling technologies will help to address this concern, enabling dynamic allocation and sharing of memory across servers with high performance and without hardware slot limitations. The composable disaggregated infrastructure market that IntelliProp is playing in is an exciting new market that is expected to grow at a 28.2 percent five-year compound annual growth rate to crest at $4.8 billion by 2025.”
The IntelliProp Omega Memory Fabric solutions are available as FPGA versions and will have the full features of the Omega Fabric architecture. The IntelliProp Omega ASIC based on CXL technology will be available in 2023.
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.