Biweekly Report (Apr.10 - Apr.23) – Impl Time-Series Forecasting function in PromQL

Summary ​

Together with all our contributors worldwide, we are glad to see GreptimeDB making remarkable progress for the better. Below are some highlights:

• Impl show create table [table] statement
• Impl Time-Series Forecasting function in PromQL
• Support parsing local timestamp
• Impl COPY for cluster

Contributor list: (in alphabetical order) ​

For the past two weeks, our community has been super active with a total of 7 PRs from 5 contributors merged successfully and lots pending to be merged. Congrats on becoming our most active contributors in the past 2 weeks:

• @DiamondMofeng (dashboard#199)
• @etolbakov (db#1342)
• @haohuaijin (db#1362 db#1407 db#1438)
• @lizhemingi (db#1440)
• @nearsyh (db#1405)

👏 Let's welcome @nearsyh as the new contributor to join our community and with their first PR merged.

A big THANK YOU for the generous and brilliant contributions! It is people like you who are making GreptimeDB a great product. Let's build an even greater community together.

Highlights of Recent PR ​

Migrate our substrait implementation to the datafusion-substrait ​

It provides more plans and is tighter upstream. Our implementation will be kept as a fallback for now, but will be removed in the future.

Impl show create table [table] statement ​

mysql> show create table monitor;
+---------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table   | Create Table                                                                                                                                                                                                     |
+---------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| monitor | CREATE TABLE IF NOT EXISTS monitor (
  host STRING NULL,
  ts TIMESTAMP NOT NULL,
  cpu DOUBLE NULL DEFAULT 0,
  memory DOUBLE NULL,
  TIME INDEX (ts),
  PRIMARY KEY (host)
)
ENGINE=mito
WITH(
  regions = 1
) |
+---------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.01 sec)

Main changes:

• Impl Display for create table statements.
• Create CreateTable statement from table info.
• Impl show create table handlers.
• Fixed column comment lost.

Impl Time-Series Forecasting function in PromQL ​

We have implemented holt_winters and predict_linear functions in promql and added some unit tests. Extract linear_regression from deriv.rs to functions.rs to reuse it in predict_linear.

holt_winters is a "Triple exponential smoothing" method used to smooth and predicate time series data. Unlike other range functions, holt_winters accepts two extra parameters sf and tf. predict_linear is also a function that predicts the future value. It's based on simple linear regression like deriv(). predict_linear also accepts one extra parameter t to specify the time.

#[range_fn]util macro doesn't support these two formats, thus we need to implement the entire structure manually. A similar case is the idelta under promql/functions/idelta.rs.

Support parsing local timestamp ​

This PR changes the behavior of parsing timestamp/datetime literals without explicit time zone info, according to system time zone (inherently from TZ env variable when GreptimeDB instance runs). For example:

• 1970-01-01 08:00:00+0000 will be parsed to 28800 in unix timestamp, since it has explicit time offset 0
• 1970-01-01 08:00:00+0800 will be parsed to 0 in unix timestamp, since it has explicit time offset +0800
• 1970-01-01 08:00:00 will be parsed to 0 if GreptimeDB instance is running in CST time zone (UTC+8) and 28800 if running in Asia/Dubai time zone (UTC) This PR also adds time zone identifier to DateTime formatted string.

mysql> CREATE table demo (ts TIMESTAMP(6) TIME INDEX, cnt INT);
Query OK, 0 rows affected (0.05 sec)

# Insert timestamp without time zone info, it will be convert to local timestamp.
mysql> insert into demo(ts,cnt) values ('2023-04-04 08:00:00.52', 1);
Query OK, 1 row affected (0.00 sec)

# queries also format time into local time zone.
mysql> select * from demo;
+------------------------------+------+
| ts                           | cnt  |
+------------------------------+------+
| 2023-04-04 08:00:00.520+0800 |    1 |
+------------------------------+------+

Impl COPY for cluster ​

Implement COPY FROM/TO for distributed mode. The GreptimeDB cluster requires this feature because the "copy" function is a highly convenient way for individuals to begin using GreptimeDB, both in standalone mode and in distributed mode. We moved the original implementation from Datanode to Frontend, and updated the tests.

New things ​

• We have upgraded the documentation for GreptimeDB, fine-tuned the directory structure and fixed some usability issues in the original user documentation to bring a better experience to developers.
• Greptime Play is now online! Play is our interactive document environment where users can execute SQL statements and receive real-time feedback in a Play session. Play is built on GreptimeCloud, which we previewed privately earlier. We will gradually launch more interactive documents to help users get familiar with GreptimeDB's features. Access Play through the top right corner of the home page on our website.