Huawei will open source data virtualization engine hetuengine next year to further improve big data development ecology
April 6, 2020
Hou Jinlong, President of Huawei cloud & AI products and services, announced at the 2019 global data infrastructure forum held last week that his data virtualization engine hetuengine will be open-source to help more users reduce the difficulty of data management and simplify the process of data use.
Hetuengine can reduce the complexity of data infrastructure, support the use of unified SQL statements for operation, reuse existing ecology, tools, skills, and improve development efficiency by 2 to 10 times.
The open source version of HetuEngine, named openHetu, is scheduled to go live in June next year. After Huawei opens the openhetu kernel code, community developers can carry out personalized and customized secondary development based on openhetu, including data source extension, SQL execution strategy adjustment, etc., which can realize rapid application docking and further improve development efficiency.
Hetuengine has four core competencies:
① A directory – through online metadata awareness, build a global virtual data view of 1000 + different heterogeneous data sources to improve data query efficiency.
② An entry – through the open connection framework and 5000 node SQL Engine, 30 kinds of heterogeneous data sources in different places can be accessed by unified SQL statements to obtain data quickly.
③ A piece of data – through the carbondata technology, a piece of data can be analyzed in multiple scenarios, shared in multiple applications, without data migration and easy to use.
④ Unified security: through fine-grained dynamic authorization and sensitive data auto sensing technology, centralized security configuration and control of heterogeneous data sources in different places are realized, data authorization time is greatly reduced, and data security and compliance are further strengthened.
Huawei has announced the open source of its server operating system, Euler OS and database operating system, Gauss, and has announced that it is based on “Kunpeng”+ Through cloud services, intelligent computing and big data, the Kunpeng computing industry layout of shengteng will build end-to-end data storage, computing and use capabilities, build a converged, intelligent and open data infrastructure, let the data system shift from isolation to integration, from closed to open, and then use unified SQL with the help of data virtualization engine Statement, using big data just like using a database.