Release 1.2.0

Key Features

AreaFeaturePR #s
Heuristic IndexSeveral Hindex improvements:
1. Preload index when server starts via config
2. Clean up index record if index creation fails (previously, some records may be left behind and required manual deletion)
3. Determine index creation level dynamically (previously, for BtreeIndex user had to specify level=table or level=partition, this is no longer required)
4. Use HetuMetastore for index metadata management (removes file based metadata management)
5. BTreeIndex: refactoring and support for AND, OR operators
6. BitmapIndex: support for additional operators (>, >=, <, <=, IN, Between, see docs for complete list)
7. Improved functional automated testing via TestingPrestoServer
646,627,604,
537,609,601,
529,611,600
Star Tree IndexA Pre-aggregation technique built to optimize latency for icebergs queries.
1. Sql support to manage cubes
2. Support for creating partitioned cubes
3. Predicate support to create cubes for a partial data
4. Incremental insert support for building large cubes
330
PerformanceOptimize the plan so that the CTE(queries having with clauses) are planned in such a way that they are executed only once and streamed to other references in the query there by reducing the computational resources required for that query. Reducing computational resources aids in improving the concurrency support of the system in general623
several code and engineering optimizations to speed up the data mutation operations614,625,645
Task RecoveryRecover from query failures resulted from task and worker errors:
1. Capture snapshots during query execution
2. When a recoverable failure is detected, resume query execution from most recent successful snapshot

There are limitations about when this feature can be used. Please refer to relevant documentation under “Administration Reliable Execution”. When an unsupported scenario is detected, a warning is produced and the feature is turned off automatically.
This is currently a beta version release. It is recommended to not use it in production environments. Some known issues has been listed in the bottom table.
663
ConnectorThe HBase performance is further optimized. New sharding algorithm is added in to improve single table query performance. In addition, this optimization supports HBase access to snapshots, improving multi-concurrent query performance.522,573
The engine supports the registration of external functions (UDFs) and pushing them down to JDBC data sources. To improve user experience, users can use existing UDFs without migrating the UDFs within data source. This improves UDF usability.647
Provide a new push-down framework to simplify the operator push-down, we hope that each connector can participate in the execution plan optimization. That is, the engine can apply the optimization rules of the Connector when optimizing the execution plan.633
Web UI1. The metadata tree supports asynchronous loading. The UI page freeze issue caused by scheduled full loading is optimized.
2. The query history and query results are displayed in pagination mode.
3. The parameter configuration mode of the new connector is optimized, and configuration file is not required any more.
4. The node status is displayed.
575,621,641,
679
Security1. Add the permission control interfaces to support row filtering, column masking, and authentication user simulation. And integrate them into the openLooKeng’s Ranger permission management plug-in to implement permission management.
2. Optimize the permission control of the login user on the Web UI. The user can only view the query records executed by the same user.
3. Implement authentication user mapping: define mapping rules from users in the authentication system to openLooKeng users. This is especially important for Kerberos authentication with complex usernames such as alice/server1@example;
539,586,667

Known Issues

CategoryDescriptionGitee issue
Heuristic IndexIn multi-CN scenarios, the number of rows in show index is incorrect, but the index function is not affected.I3E3T6
WEB UIIn the schema search bar, only the schema names that can be matched under the catalog are displayed.I3E3Q3
In the high-concurrency scenario, there is a possibility that the task is completed but the UI displays freeze to progress 100%.I3E3OK
Task RecoveryDuring Task Recovery, if an abnormal worker node is restored, the recovery of task fails.I3CEWB
When a worker node is killed during bucket table creation via CTAS, the recovery task occasionally fails.I3E3D8
When a worker node is killed during table creation via CTAS, the recovery of task occasionally fails and error 503 is reported.I3E3EH
When a worker node is killed when using CTAS creates a partition transaction table, the recovery task occasionally fails.I3E3F9
After a CTAS table is successfully created using Task Recovery, the number of inserted rows doubles occasionally.I3E3IG
The Number of Inserted Rows Increases Occasionally After the Task Recovery Is Successfully Executed in the CTAS TableI3E3JP
In the cross-DC scenario, if a worker node is killed during table creation via CTAS, the recovery task fails.I3E3T6

Obtaining the Document

For details, see https://gitee.com/openlookeng/hetu-core/tree/1.2.0/hetu-docs/en