Huawei's Data Virtualization Engine openLooKeng Goes Open Source

HUAWEI July 1, 2020


[Shenzhen, China, July 1, 2020] Yesterday, Huawei officially announced openLooKeng, the data virtualization engine, goes open source at openLooKeng marks a huge step forward to power data analytics experience with the ultimate simplicity, enabling users to harness big data with the ease of database usage.

openLooKeng is previously named openHetu, which was based on the HetuEngine (now LooKengEngine) and first announced on November 19, 2019. Positioned as an open source high-performance data virtualization engine, openLooKeng provides unified interfaces for structured query language (SQL), analysis across data sources and centers, and converged queries for interactive, batch, and stream scenarios. In addition, openLooKeng is enhanced with pre-scheduling, cross-source indexing, dynamic filtering, cross-source collaboration, and scale-out capabilities.

The drastic growth of big data technologies is increasing data diversity and distribution, and greatly complicating query scenarios and big data usage. Huawei started the openLooKeng project to address these issues with a data virtualization engine.

openLooKeng features:

  • Unified SQL interfaces for accessing disparate data sources

  • Local data processing without data migration, and processing across data centers and hybrid clouds

  • Interactive, batch, and stream querying (The first version supports interactive queries)

openLooKeng also provides a new level of convenience with coordinator active-active (AA) for high availability, and a scalable data source connector framework.

openLooKeng uses Presto, an established engine for open source SQL, to provide basic interactive queries and analysis. openLooKeng continues to improve capabilities in converged scenarios for queries, processing across data centers and clouds, data source expansion, performance, reliability, and security. openLooKeng makes data governance and usage easier.

Based on openLooKeng, Huawei establishes an open source community with the business-friendly Apache License 2.0 for free use, modify and redistribution of software among users, developers, and partners. We welcome more big data developers to join the community with sharing resources, and promoting a more prosperous ecosystem.

Homepage of the openLooKeng community:

openLooKeng code hosting:

Huawei Partners with Open Source Organizations for a Thriving Community Worldwide

1. Huawei has remained dedicated to promoting open source ecosystem around the world.

Huawei serves as top level members of several major open source foundations. It is a Platinum Member of the Linux Foundation, Platinum Sponsor of the Apache Foundation, Platinum Member of the OpenStack Foundation, and Strategic Member of the Eclipse Foundation.

2. Huawei is an active contributor to open source communities and projects. Developers from Huawei currently are holding more than 200 roles in major open source community, including TSC, PMC, PTL, Maintainer, and Committer. Huawei also has board member representatives for more than 10 open source foundations and serve for many projects.

Huawei’s contributions rank:

  • No. 4 in the Linux kernel community, which is the world’s largest open source community

  • No. 5 in the Cloud Native Computing Foundation (CNCF)

  • No. 5 in the Docker community,

3. Huawei contributes extensively with basic software to various projects in the open source community, including:
  • openEuler: Linux distribution

  • MindSpore: Deep learning framework

  • SODA: Intelligent platform for open data autonomy

  • Apache CarbonData: Big data

  • Apache ServiceComb: Microservice

  • CNCF KubeEdge: Edge computing framework

  • CNCF Volcano: High-performance scheduling engine