Huawei Announces the Open Sourcing of Its Data Virtualization Engine HetuEngine
November 19, 2019
Aiming for a Vibrant Kunpeng Computing Industry, Huawei Announces Data Infrastructure Strategy and the Open Sourcing of Its Data Virtualization Engine HetuEngine
[Shenzhen, China, November 19, 2019] Today, at the 2019 Global Intelligent Data Infrastructure Forum hosted by Huawei in Shenzhen, Huawei announced the launch of its data infrastructure strategy, as well as the decision to open source its data virtualization engine called HetuEngine. The aim is to allow partners to use big data with the simplicity of an ordinary database, and simplify data governance.
This is yet another significant move from the telecom giant in its all-out efforts to advance the computing industry, following the announcement of its “Kunpeng + Ascend” dual-engine computing strategy in September this year. Huawei’s data infrastructure strategy covers the entire data lifecycle (collection, storage, computing, management, and usage), and explains how Huawei’s converged, intelligent, and open data infrastructure can empower customers from all industries to unleash the value of data for pervasive intelligence.
In the intelligent era, computing power has become a crucial productive force, where data is the new factor of production, and 5G, AI, and Cloud become the new catalyst of productivity. The emergence of new applications such as autonomous driving, 4K/8K video, AR/VR, and the Internet of Things (IoT) has caused an explosive growth in data volumes as well as data analytical and processing needs that cannot be met with existing technologies.
“New technologies including 5G, AI, and cloud are transforming the way we live and work, but are generating huge volumes of data, bringing enormous pressure on the existing data infrastructure and making it increasingly difficult to efficiently locate, fetch, and utilize data.” Hou Jinlong, President of Huawei Cloud & AI Products and Services, said in his keynote speech titled “A Converged, Intelligent, Open Data Infrastructure for the Intelligent World”. He emphasized, “Aiming for a vibrant Kunpeng computing industry, Huawei launches its data infrastructure strategy and offers converged, intelligent, and open data infrastructure to streamline and optimize end-to-end data collection, storage, computing, and management. It is committed to delivering the industry’s best cost and value per bit through the entire data lifecycle, and allows partners to use big data with the simplicity of an ordinary database.”
Building a Converged, Intelligent, Open Data Infrastructure
With the best-of-breed computing power delivered by the “Kunpeng + Ascend” dual-engine, Huawei will continue to build computing, storage, and AI capabilities through continuous, intensive R&D investments, and technological innovation. Huawei’s data infrastructure includes data storage, processing, management systems and a data virtualization engine. It features the following:
Converged: Breakthroughs in storage, database, and big data technologies help ensure full data convergence at four levels: between storage systems; between databases and storage systems; between big data platform and storage systems; and between databases and the big data platform. This unprecedented level of convergence is expected to help customers slash TCO by over 30% and double data access, processing, and analytics performance.
Intelligent: A three-layer architecture—AI chipsets, storage, and HUAWEI CLOUD—allows efficient collaboration between cloud and on-premises resources, such as the flexible combination of on-cloud training and on-premises inference, delivering ever-faster performance and higher cost-efficiency as it adapts to the workloads. Huawei’s Ascend processors improve cache prefetching hit rate by automatically learning and identifying I/O flows, improving system performance by 20%. Huawei’s Kunpeng processors, with its multi-core architecture, automatically select the optimal data reduction algorithms in real time depending on the data types, reducing TCO by 25%. With HUAWEI CLOUD’s extensive, proven know-how in large system O&M, this intelligent infrastructure is able to predict disk faults 14 days in advance, performance bottlenecks 60 days in advance, and capacity insufficiency 365 days in advance. In addition, it allows self-healing for 30% of the faults.
Open: HetuEngine, Huawei’s data virtualization engine, makes it easier for users to locate, fetch, and utilize data. It makes data governance and use much simpler by offering a unified SQL interface, unified data catalogs, streamlined data, and unified security management, and hiding the differences in data types, localities, and syntaxes. It removes the technical complexity of the data infrastructure, allowing developers access to big data capabilities with the simplicity of an ordinary database. Reusing legacy ecosystems, tools, and skillsets improves development efficiency 2 to 10 times and saves on upfront investments.
At this forum, Huawei also announced the open-sourcing of HetuEngine, its proprietary data virtualization engine designed to help customers and partners better connect data and applications. The open-source HetuEngine is called openHetu, and its launch time is scheduled for June 2020. With an open-source kernel, developers will be able to customize this engine, including data source extensions and SQL execution policies, to allow fast interoperability and development.
Guided by the “Platform + Ecosystem” strategy and the principle of “open hardware, open-source software, and partner enablement”, Huawei is committed to developing an open, rich Kunpeng ecosystem together with its customers and partners for the intelligent world.
From November 18 to 19, 2019, Huawei hosted the 2019 Global Intelligent Data Infrastructure Forum themed “Pioneer Data Infrastructure for the Intelligent Era” in Shenzhen, China. Industry leaders and Huawei customers and partners attended this forum to discuss the future trends for computing in the intelligent era and ways to advance the computing industry, and also showcased the latest data infrastructure products and solutions. For more information, please visit https://e.huawei.com/topic/gidif2019/cn/index.html?source=corp_comm.