Append-only Mode Goes Live - Effortless Duplication Data Storage | Greptime Biweekly Report

Summary ​

Together with our global community of contributors, GreptimeDB continues to evolve and flourish as a growing open-source project. We are grateful to each and every one of you.

Below are the highlights among recent commits:

• Officially added support for Append Mode. Users can now create tables in Append-only Mode by setting Append Mode (append_mode = true) during table creation. Append-only tables will not deduplicate incoming data, allowing for the convenient storage of duplicate data.

• Support for using the VictoriaMetrics Remote Write protocol to write into GreptimeDB.

• PromQL now supports cross-Schema queries, with the addition of the special matcher __schema__, which is used to select the Schema in which the table is located. This allows users to perform cross-Schema queries within PromQL.

Contributors ​

For the past two weeks, our community has been super active with a total of 54 PRs merged. 8 PRs from 5 individual contributors merged successfully and lots pending to be merged.

Congrats on becoming our most active contributors in the past 2 weeks:

• @dimbtp（#3627)

• @J0HN50N133（#3602)

• @CookiePieWw（#3682 #3675 #3626)

• @DiamondMofeng（#3668)

• @etolbakov（#3620 #3507)

👏 Welcome contributor @CookiePieWw join to the community as the new individual contributor, and congratulations on successfully merging their first PR, more PRs are waiting to be merged.

A big THANK YOU to all our members and contributors! It is people like you who are making GreptimeDB a great product. Let's build an even greater community together.

Highlights of Recent PRs ​

#3558 #3598 #3624 Added support for Append-only tables ​

• Users can create tables in Append-only Mode by setting Append Mode (create table ... engine=mito with('append_mode'='true');).

• Append-only tables support insert operations only and do not support delete or update operations.

• Append-only tables do not deduplicate data upon insertion, making it easier to store duplicate data.

#3452 #3507 The TQL (Time-Series Query Language) parser now accepts SQL functions and expressions such as now(),now() - '10 seconds'::interval for the start, end, and step parameters. ​

We've enhanced the tql_parse logic by introducing small pre-planning and pre-execution phases to convert the start, end, and step parameters into numbers (timestamps) if these parameters were passed as SQL functions.

The logic is as follows:

1. Convert the function into a “logical” expression.
2. Use a simplifier to evaluate the expression.

#3635 Corrected the error handling logic for Compaction failures ​

Fixed a bug caused by an incorrect variable name.

#3604 Added support for Grafana's 2+2 and /status/buildinfo requests ​

Added support for Grafana v9.0's 2+2 and /status/buildinfo requests, removing error messages when adding GreptimeDB as a Prometheus data source in Grafana.

#3641 Added support for VictoriaMetrics Remote Write ​

This PR enhances support for VictoriaMetrics, allowing users to write data into the GreptimeDB using the VictoriaMetrics Remote Write protocol.

#3602 Added support for multiple PostgreSQL date output formats ​

Enhanced support for PostgreSQL, allowing users to use more date formats.

#3620 Added support for Unix timestamp numerical type input to to_timezone ​

Previously, to_timezone only supported TIMESTAMP and VARCHAR inputs. This PR adds support for Unix timestamp numerical type (BIGINT).

Good First Issue ​

#2618 Added tests for the implementation of the KvBackend Trait ​

Added behavioral tests for the KvBackend implementation to ensure all implementations behave consistently.

Keywords: testing

Difficulty: medium

#3511 Supported specifying a time range in COPY FROM statements to avoid importing unnecessary data ​

OPY FROM statements are often used to restore table data from backup files. Supporting the filtering of specific time ranges can avoid importing unnecessary data and speed up data recovery.

Keywords: backup/recovery, filtering, timestamp

Difficulty: medium

#3597 Migrated from orc-rs to datafusion-orc ​

The crate orc-rs is no longer maintained, necessitating the migration of dependencies to the alternative crate datafusion-orc.

Keywords: dependency update

Difficulty: medium

New Things ​

GreptimeCloud ​

• AI Solution Launch

  GreptimeAI is now built-in and can be accessed from the console, offering better observability of the status of LLM applications.

• Console supports uploading data in InfluxDB Line Protocol format

  Allows for quick data import from InfluxDB, facilitating user data migration.

• GreptimePlay

  The official Playground now supports automatic demo data import, providing a smoother user experience.

Website: ​

Light-themed blog launched to offer a better reading experience for users.