[#209]feat(catalog): Hive table entity serde and store support #233

mchades · 2023-08-14T07:13:51Z

What changes were proposed in this pull request?

SerDe Hive table entity as Table Proto in Graviton store while operating Hive table

Why are the changes needed?

we could store the additional entity information to our own storage.

Fix: #209

Does this PR introduce any user-facing change?

no

How was this patch tested?

UTs added

github-actions · 2023-08-14T07:17:14Z

Code Coverage Report

Overall Project	59.23% `-0.81%`	🟢
Files changed	54.34%	🔴

Module	Coverage
core	66.65% `-0.97%`	🔴
catalog-hive	58.28% `-3.17%`	🟢

Files

Module	File	Coverage
core	ProtoEntitySerDe.java	65.2%	🟢
	BaseTable.java	57.67% `-6.41%`	🔴
	TableEntitySerde.java	0%	🔴
catalog-hive	HiveCatalogOperations.java	66.15% `-6.67%`	🟢

yuqi1129 · 2023-08-14T07:25:18Z

catalog-hive/src/main/java/com/datastrato/graviton/catalog/hive/HiveCatalogOperations.java

@@ -402,26 +403,34 @@ public boolean dropSchema(NameIdentifier ident, boolean cascade) throws NonEmpty
    EntityStore store = GravitonEnv.getInstance().entityStore();
    Namespace schemaNamespace =
        Namespace.of(ArrayUtils.add(ident.namespace().levels(), ident.name()));
+    List<BaseTable> tables = Lists.newArrayList();


Unnecessary initiation

use addAll in line 416

yuqi1129 · 2023-08-14T07:26:47Z

catalog-hive/src/main/java/com/datastrato/graviton/catalog/hive/HiveCatalogOperations.java

    }

    try {
+      List<BaseTable> finalTables = tables;


Why do you introduce a new local variant? if you want to handle final variant in lambda function, I advice you can use addAll in line 416

### What changes were proposed in this pull request? Add diqiu50 to maintainer ### Why are the changes needed? Fix: #755 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? No

### What changes were proposed in this pull request? (Please outline the changes and how this PR fixes the issue.) ### Why are the changes needed? (Please clarify why the changes are needed. For instance, 1. If you propose a new API, clarify the use case for a new API. 2. If you fix a bug, describe the bug.) Fix: # (issue) ### Does this PR introduce _any_ user-facing change? (Please list the user-facing changes introduced by your change, including 1. Change in user-facing APIs. 2. Addition or removal of property keys.) ### How was this patch tested? (Please test your changes, and provide instructions on how to test it: 1. If you add a feature or fix a bug, add a test to cover your changes. 2. If you fix a flaky test, repeat it for many times to prove it works.)

)  ### What changes were proposed in this pull request? close #1724 ### Why are the changes needed? Add server-side REST API support for fileset Fix: #1724 ### Does this PR introduce _any_ user-facing change? (Please list the user-facing changes introduced by your change, including 1. Change in user-facing APIs. 2. Addition or removal of property keys.) ### How was this patch tested? - add unit tests

…2043)  ### What changes were proposed in this pull request? Unify the style of configuration options ### Why are the changes needed? Fix: #2042 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Just doc. Co-authored-by: Heng Qin <[email protected]>

…2045)  ### What changes were proposed in this pull request? Unify the style of configuration options ### Why are the changes needed? Fix: #2042 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Just doc. Co-authored-by: qqqttt123 <[email protected]> Co-authored-by: Heng Qin <[email protected]>

…uri (#2136)  ### What changes were proposed in this pull request? Change `gravitino.url` to `gravitino.uri`. ### Why are the changes needed? The example in trino-connector/installation.md uses different keys for gravitino link. https://github.com/datastrato/gravitino/blob/edfcdbea89dbf16d8e48833573c8917f9bcd5972/docs/trino-connector/installation.md?plain=1#L75-L81 From source code, it should be `gravitino.uri`. https://github.com/datastrato/gravitino/blob/edfcdbea89dbf16d8e48833573c8917f9bcd5972/trino-connector/src/main/java/com/datastrato/gravitino/trino/connector/GravitinoConfig.java#L22 We also need to fix the one in trino-connector/configuration.md. https://github.com/datastrato/gravitino/blob/edfcdbea89dbf16d8e48833573c8917f9bcd5972/docs/trino-connector/configuration.md?plain=1#L13 Fix: #2118 ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? Check there is no `gravitino.url` in the document. Signed-off-by: PoAn Yang <[email protected]>

…uri (#2142)  ### What changes were proposed in this pull request? Change `gravitino.url` to `gravitino.uri`. ### Why are the changes needed? The example in trino-connector/installation.md uses different keys for gravitino link. https://github.com/datastrato/gravitino/blob/edfcdbea89dbf16d8e48833573c8917f9bcd5972/docs/trino-connector/installation.md?plain=1#L75-L81 From source code, it should be `gravitino.uri`. https://github.com/datastrato/gravitino/blob/edfcdbea89dbf16d8e48833573c8917f9bcd5972/trino-connector/src/main/java/com/datastrato/gravitino/trino/connector/GravitinoConfig.java#L22 We also need to fix the one in trino-connector/configuration.md. https://github.com/datastrato/gravitino/blob/edfcdbea89dbf16d8e48833573c8917f9bcd5972/docs/trino-connector/configuration.md?plain=1#L13 Fix: #2118 ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? Check there is no `gravitino.url` in the document. Signed-off-by: PoAn Yang <[email protected]> Co-authored-by: PoAn Yang <[email protected]>

### What changes were proposed in this pull request? Dependencies follow alphabetical order ### Why are the changes needed? For better readability Fix: #738 ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? N/A --------- Co-authored-by: Qi Yu <[email protected]>

### What changes were proposed in this pull request? Replace LinkedList with ArrayList. ### Why are the changes needed? Address linter issues mentioned in #2163. Fix #2163 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Covered by existing tests.

…#2138)  ### What changes were proposed in this pull request? close #1955 ### Why are the changes needed? (Please clarify why the changes are needed. For instance, 1. If you propose a new API, clarify the use case for a new API. 2. If you fix a bug, describe the bug.) Fix: #1955 ### Does this PR introduce _any_ user-facing change? - no ### How was this patch tested? - unit

…n atomic to keep thread safe (#2139)  ### What changes were proposed in this pull request? close #1953 #1946 #1941 #1920 #1910 ### Why are the changes needed? Fix: #1953 #1946 #1941 #1920 #1910 ### Does this PR introduce _any_ user-facing change? - no ### How was this patch tested? - ut

### What changes were proposed in this pull request? - close #2154 ### Why are the changes needed? - Fix: #2154 ### Does this PR introduce _any_ user-facing change? - no ### How was this patch tested? `./gradlew build` and UTs

…2403)  ### What changes were proposed in this pull request? - close #2367 ### Why are the changes needed? - Fix: #2367 ### Does this PR introduce _any_ user-facing change? - no ### How was this patch tested? - `./gradlew build` --------- Co-authored-by: mchades <[email protected]>

…for consistency (#2437)  ### What changes were proposed in this pull request? Make all Spark SQL keywords used in spark connector integration test uppercase ### Why are the changes needed? Fix: #2424 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? `./gradlew build -PskipTests`

…le clean (#2531)  ### What changes were proposed in this pull request? Remove directory `spark-warehouse` when exectue gradle clean in moudle catalog-lakehouse-iceberg. Remove directory `src/main/resources` when execute gradle clean in module server, since which is generated by gradle build. ### Why are the changes needed? Fix: #2508 ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? `./gradlew :catalogs:catalog-lakehouse-iceberg:clean` `./gradlew :server:clean` --------- Co-authored-by: Qi Yu <[email protected]>

…rdedByInstance error-prone (#2514)  ### What changes were proposed in this pull request? `StaticGuardedByInstance` amis to avoid `public static` fields. Because this field is not thread-safe. See more details at https://errorprone.info/bugpattern/StaticGuardedByInstance. In this MR, I remove some test code in `com.datastrato.gravitino.trino.connector.GravitinoPlugin` and `com.datastrato.gravitino.trino.connector.GravitinoConnectorFactory`. At the same time, I enable `StaticGuardedByInstance` error-prone. ### Why are the changes needed? - Fix: #2167 ### Does this PR introduce _any_ user-facing change? - no ### How was this patch tested? - original unit tests.

…ticGuardedByInstance error-prone (apache#2514)  ### What changes were proposed in this pull request? `StaticGuardedByInstance` amis to avoid `public static` fields. Because this field is not thread-safe. See more details at https://errorprone.info/bugpattern/StaticGuardedByInstance. In this MR, I remove some test code in `com.datastrato.gravitino.trino.connector.GravitinoPlugin` and `com.datastrato.gravitino.trino.connector.GravitinoConnectorFactory`. At the same time, I enable `StaticGuardedByInstance` error-prone. ### Why are the changes needed? - Fix: apache#2167 ### Does this PR introduce _any_ user-facing change? - no ### How was this patch tested? - original unit tests.

…ult (#2764)  ### What changes were proposed in this pull request? When listing metalakes they get sorted by their createdTime value ### Why are the changes needed? Its requested in the issue Fix: #2754 (issue) ### Does this PR introduce _any_ user-facing change? metalakes will be shown in ASC order ### How was this patch tested? N/A --------- Co-authored-by: CHEYNE <[email protected]>

…y environment variant (#4229)  ### What changes were proposed in this pull request? Read the environment variable (if exists) when -PskipDockerTests is not specified. ### Why are the changes needed? Fix: #4181 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? ``` export SKIP_DOCKER_TESTS=false ./gradlew build # check docket tests are running ```

… mutable (#4379)  ### What changes were proposed in this pull request? switch catalog properties value input disable to enable hive catalog before: <img width="623" alt="Pasted Graphic" src="https://github.com/user-attachments/assets/77472113-ddd8-4051-825a-21bbaaf07bdd"> hive catalog after: <img width="708" alt="Pasted Graphic 1" src="https://github.com/user-attachments/assets/d08634da-0044-452d-9ded-ff186aae3507"> Iceberg catalog before: <img width="623" alt="Pasted Graphic 2" src="https://github.com/user-attachments/assets/3ecca557-6b2d-42ab-8d0f-115044fe97b3"> iceberg catalog after: <img width="588" alt="Pasted Graphic 3" src="https://github.com/user-attachments/assets/5585d55c-4bec-42c3-b496-eaf75dc55e9f"> pg catalog before: <img width="580" alt="Pasted Graphic 6" src="https://github.com/user-attachments/assets/334abb54-ad33-42b5-9d96-6f3d7fdded19"> pg catalog after: <img width="594" alt="Pasted Graphic 7" src="https://github.com/user-attachments/assets/4aca0e85-8ba2-4bbe-a19b-58c52685ae42"> kafka catalog before: <img width="605" alt="Pasted Graphic 4" src="https://github.com/user-attachments/assets/37f1938d-90df-45ff-81bf-7081238921db"> kafka catalog after: <img width="592" alt="Pasted Graphic 5" src="https://github.com/user-attachments/assets/4dd7e321-dd6a-4e86-8e5d-dae6d601fbf7"> fileset catalog before: <img width="607" alt="Pasted Graphic 8" src="https://github.com/user-attachments/assets/1b430711-f494-4b87-89a2-ea9c774c381c"> fileset catalog after: <img width="618" alt="Pasted Graphic 9" src="https://github.com/user-attachments/assets/04ce17cb-5eac-4f25-b1e0-e309db2a1103"> ### Why are the changes needed? make the change by the [PR](#4262) merged Fix: #4351 ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? manual Co-authored-by: Qian Xia <[email protected]>

### What changes were proposed in this pull request? Add `**/*.out` to the list of exclusions will skip the licence check on the *.out file. ### Why are the changes needed? Fix: #4439 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? No Co-authored-by: Saker <[email protected]>

#4667)  ### What changes were proposed in this pull request? The documents should be updated. ### Why are the changes needed? To correct reference from MySQL to PostgreSQL in JDBC catalog documentation. Fix: #4666 ### Does this PR introduce _any_ user-facing change? No; Just the markdown documents. ### How was this patch tested? Manual review.

### What changes were proposed in this pull request? `TestHiveTableOperations` can be tested before `TestHiveTable` initialized. Make `TestHiveTableOperations::steup()` run alone. ### Why are the changes needed? When we try to just run `gradle :catalogs:catalog-hive:test --tests 'org.apache.gravitino.catalog.hive.TestHiveTableOperations'`, `TestHiveTableOperations` needs static variables which must be initialized ahead, and it can't be run alone. Fix: #4851 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Refactor of unit test setup, local unit tests have passed.

### What changes were proposed in this pull request? In `DorisTableOperations`, handled a possible Null Pointer Exception scenario ### Why are the changes needed? Fix: (#4605) ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Local Test Run - Passed --------- Co-authored-by: Avinash <[email protected]>

### What changes were proposed in this pull request? Remove Key-Value storage backend storage implementations ### Why are the changes needed? Fix: #5633 ### Does this PR introduce _any_ user-facing change? yes ### How was this patch tested? Remove tests for the Key-Value storage implementation code

### What changes were proposed in this pull request? Remove Key-Value storage backend storage implementations ### Why are the changes needed? Fix: apache#5633 ### Does this PR introduce _any_ user-facing change? yes ### How was this patch tested? Remove tests for the Key-Value storage implementation code

### What changes were proposed in this pull request? Fix irregular variable names: tablename -> tableName ### Why are the changes needed? Variable name `tablename` is irregular ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Modify variable name naming convention, no testing required.

### What changes were proposed in this pull request? This is a small improvement, previously, typing `entity help` will displays the help information for that entity. Now, the `entity --help` command will have the same effect. ### Why are the changes needed? Fix: #5822 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? ```bash bin/gcli.sh metalake [--help|help] # metalake help information bin/gcli.sh catalog [--help|help] # catalog help information bin/gcli.sh schema [--help|help] # schema help information bin/gcli.sh table [--help|help] # table help information bin/gcli.sh column [--help|help] # column help information bin/gcli.sh fileset [--help|help] # fileset help information bin/gcli.sh group [--help|help] # group help information bin/gcli.sh role [--help|help] # role help information bin/gcli.sh topic [--help|help] # topic help information bin/gcli.sh user [--help|help] # user help information bin/gcli.sh catalog -m demo_metalake --name Hive_catalog # correct details output ``` --------- Co-authored-by: Justin Mclean <[email protected]>

…x a performance regression (#6379)  ### What changes were proposed in this pull request? bump log4j dependency from 2.22.0 to 2.24.3 to fix a performance regression ### Why are the changes needed? Fix: #6378 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Ran in internal workload.

…om gravitino-env.sh (#6385)  ### What changes were proposed in this pull request? Update Gravitino JVM memory default. ### Why are the changes needed? Fix: #6384 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? No test

…x a performance regression (apache#6379)  ### What changes were proposed in this pull request? bump log4j dependency from 2.22.0 to 2.24.3 to fix a performance regression ### Why are the changes needed? Fix: apache#6378 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Ran in internal workload.

…nts from gravitino-env.sh (apache#6385)  ### What changes were proposed in this pull request? Update Gravitino JVM memory default. ### Why are the changes needed? Fix: apache#6384 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? No test

* [apache#6384] fix: Gravitino default JVM config mismatches with comments from gravitino-env.sh (apache#6385)  ### What changes were proposed in this pull request? Update Gravitino JVM memory default. ### Why are the changes needed? Fix: apache#6384 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? No test * [apache#6279] feat (gvfs-fuse): Add gvfs-fuse integration tests for big files and open-file flag test cases (apache#6280) ### What changes were proposed in this pull request? Add gvfs-fuse integration tests for big files and open-file flag test cases ### Why are the changes needed? Fix: apache#6279 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? IT * [apache#6380] fix(postgres-sql): Fix errors for PG backend about `delete...limit..` clause. (apache#6393) ### What changes were proposed in this pull request? PostgreSQL does not support SQL sentences like `DELETE FROM xxxx_table where xxxx limit 10` , Cluase `limit xxx` is not allowed in the `Delete syntax` ### Why are the changes needed? it's a bug. Fix: apache#6380 ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? UT * [apache#6356] improve(CLI): Add tag support for model in CLI (apache#6360) ### What changes were proposed in this pull request? Add tag support for model in CLI. 1. `UntagEntity` 2. `TagEntity` 3. `ListEntityTags` The logic for handling models in these three methods has been added. need to add the processing logic to the `RemoveAllTags` method. ### Why are the changes needed? Fix: apache#6356 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test. * [MINOR] fix(gvfs): expose the nested exception for better understanding (apache#6398) ### What changes were proposed in this pull request? Improve the gvfs to expose the nested exception for better understanding. ### Why are the changes needed? The exception message may not be enough to understand the problem why the fs initialization is failed. So we should expose the whole stack for better understanding. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Existing tests. * [apache#6375] improvment(catalog-hadoop): Remove `protobuf-java` to avoid conflict with authorization module (apache#6376) ### What changes were proposed in this pull request? Remove jar `protobuf-java.jar` from the distribution package to avoid conflicts ### Why are the changes needed? To make authorization works for GCS fileset. Fix: apache#6375 ### Does this PR introduce _any_ user-facing change? N/A. ### How was this patch tested? N/A * [apache#6394] feat(iceberg): upgrade Iceberg version from 1.5.2 to 1.6.1 (apache#6374) ### What changes were proposed in this pull request? Upgrade Iceberg version from 1.5.2 to 1.6.1 In 1.6, the `token` endpoint is deprecated and there're no new endpoint is added. ### Why are the changes needed? Fix: apache#6394 ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? run pass existing ITs * [apache#6320] feat (gvfs-fuse): Support mount and umount command for gvfs-fuse command line tools (apache#6321) ### What changes were proposed in this pull request? 1. Support mount and umount command 2. Make mount create a daemon process to run background ### Why are the changes needed? Fix: apache#6320 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? IT --------- Co-authored-by: Qiming Teng <[email protected]> * [apache#6357] feat (gvfs-fuse): Passing file type argument in the Filesystem::stat() to improve the performance of open_dal_filesystem (apache#6358) ### What changes were proposed in this pull request? Passing file type argument in the Filesystem::stat() to improve the performance of open_dal_filesystem ### Why are the changes needed? Fix: apache#6357 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? UI --------- Co-authored-by: Qiming Teng <[email protected]> * [apache#6326] Refactor to add a command context to CLI (apache#6343) ### What changes were proposed in this pull request? Refactor to add a command context and simple wrappers on output to make it easy for "global" command like a "--quiet" option to be added. Note this in progress as this only has changed Metlake. Some duplicate code can be removed once everything is done. ### Why are the changes needed? For maintainability. Fix: apache#6326 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Tested locally. --------- Co-authored-by: Shaofeng Shi <[email protected]> * [apache#6407] improve(client-python): Reorganize the python package (apache#6408) ### What changes were proposed in this pull request? Move the `Fileset` and `Model` related classes to the sub-package for better organized. ### Why are the changes needed? For better organized. Fix: apache#6407 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Existing tests. * [apache#5900] feat(tag): support tag pre-event to Gravitino server (apache#5980) ### What changes were proposed in this pull request? Implemented pre-event handling for all tag-related operations, including: - listTags - listTagsInfo - getTag - createTag - alterTag - deleteTag - listMetadataObjectsForTag - listTagsForMetadataObject - listTagsInfoForMetadataObject - associateTagsForMetadataObject - getTagForMetadataObject ### Why are the changes needed? Fix: apache#5900 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? add UT * [apache#6409] fix(client-python): Change to use `GenericFileset` for the return value of fileset catalog (apache#6410) ### What changes were proposed in this pull request? This PR changes the return value to `GenericFileset` for fileset operation. ### Why are the changes needed? Previously, it returns the plain fileset dto, so that it cannot support some operations like get credentials. Fix: apache#6409 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Existing tests. * [apache#5932] improve(CLI): Fix the CLI delete the anonymous user (apache#6359) ### What changes were proposed in this pull request? Fix the CLI delete the anonymous user ### Why are the changes needed? Fix: apache#5932 ### Does this PR introduce _any_ user-facing change? (Please list the user-facing changes introduced by your change, including 1. Change in user-facing APIs. 2. Addition or removal of property keys.) ### How was this patch tested? local test ```bash gcli user delete -m demo_metalake --user anonymous # Can't delete anonymous user. This will cause unexpected behavior. ``` <img width="536" alt="image" src="https://github.com/user-attachments/assets/479ec7eb-2ad3-42bd-b70a-a6f2ce05f794" /> * [apache#6419] improve(CLI): Add table command context CLI (apache#6428) ### What changes were proposed in this pull request? Add table command context CLI. ### Why are the changes needed? Fix: apache#6419 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test. * [apache#6415] improve(CLI): Add Fileset command context CLI (apache#6431) ### What changes were proposed in this pull request? Add Fileset command context CLI ### Why are the changes needed? Fix: apache#6415 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test * [apache#6417] improve(CLI): Add model command context CLI (apache#6430) ### What changes were proposed in this pull request? Add model command context CLI ### Why are the changes needed? Fix: apache#6417 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test * [apache#6416] improve(CLI): Add schema command context CLI (apache#6427) ### What changes were proposed in this pull request? Add schema command context CLI ### Why are the changes needed? Fix: apache#6416 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test. * [apache#6414] improve(CLI): Add catalog command context CLI (apache#6425) ### What changes were proposed in this pull request? Add catalog command context CLI ### Why are the changes needed? Fix: apache#6414 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test. * [apache#6418] improve(CLI): Add stream command context CLI (apache#6434) ### What changes were proposed in this pull request? Add stream command context CLI ### Why are the changes needed? Fix: apache#6418 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test. * [apache#6420] improve(CLI): Add column command context CLI (apache#6436) ### What changes were proposed in this pull request? Add column command context CLI ### Why are the changes needed? Fix: apache#6420 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test. * [MINOR] docs: Fix the description of `Multi-engine support` in `overview.md` (apache#6439) ### What changes were proposed in this pull request? This PR fixes the description of the `Multi-engine support` section in `overview.md`, as Gravitino now supports connectors for Apache Spark and Apache Flink. ### Why are the changes needed? Gravitino now supports connectors for Apache Spark and Apache Flink. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? manual review * [apache#6423] improve(CLI): Add tags command context CLI (apache#6435) ### What changes were proposed in this pull request? Add tags command context CLI ### Why are the changes needed? Fix: apache#6423 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test * [apache#6421] improve(CLI): Add user and group command context CLI (apache#6437) ### What changes were proposed in this pull request? Add user and group command context CLI ### Why are the changes needed? Fix: apache#6421 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test. * [apache#6422] improve(CLI): Add roles and owner command context CLI (apache#6438) ### What changes were proposed in this pull request? Add roles and owner command context CLI ### Why are the changes needed? Fix: apache#6422 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? local test. * [apache#6424] improve(CLI): Refactor getURL in CLI and add context to simple commands. (apache#6440) ### What changes were proposed in this pull request? Refactor getURL in CLI and add context to simple commands. ### Why are the changes needed? Fix: apache#6424 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test. * [apache#6429] Fixed Wrong log format codes (apache#6433) ### What changes were proposed in this pull request? The PR updates incorrect log format codes from `%s` to `{}` in the following lines: - Line 420 - Line 483 - Line 504 - Line 525 The original code in these lines used the `%s` placeholder, which is invalid for the logging framework used. It has been corrected to use `{}` to match the proper syntax for log messages. ### Why are the changes needed? This change is necessary to fix the incorrect log format codes in the codebase. The `%s` placeholder is not valid and can lead to incorrect log formatting or errors when the logs are generated. The correct placeholder syntax for this logging framework is `{}`. Fix: apache#6429 ### Does this PR introduce _any_ user-facing change? No user-facing changes are introduced. This is an internal fix for the logging format. ### How was this patch tested? The changes were verified manually by reviewing the code and ensuring that all instances of `%s` were replaced with `{}` in the affected lines. No new functionality was added, so no additional tests were required. * [apache#6361] feat(paimon):Support specifying primary keys during create paimon table by flink (apache#6362) ### What changes were proposed in this pull request? Support specifying primary keys during create paimon table by flink ### Why are the changes needed? Fix: apache#6361 ### Does this PR introduce _any_ user-facing change? None ### How was this patch tested? Add testCreateTableWithPrimaryKey case in org.apache.gravitino.flink.connector.integration.test.FlinkCommonIT * [MINOR] docs: polish Iceberg REST server document (apache#6444) ### What changes were proposed in this pull request? multi catalog and view operation should shown in document side bars. ### Why are the changes needed? User could find multi catalog configuration easily ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? just document * [MINOR] improve(CLI): Remove redundant code in CLI (apache#6443) ### What changes were proposed in this pull request? Remove redundant code in CLI, and make context argument final in Command class. ### Why are the changes needed? The constructor should take only one argument, CommandContext. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? local test. * [apache#6097] improve(CLI): Add --quiet option to the Gravitino CLI (apache#6447) ### What changes were proposed in this pull request? Add --quiet option to the Gravitino CLI ### Why are the changes needed? Fix: apache#6097 ### Does this PR introduce _any_ user-facing change? user can use --quiet option suppress the output. ### How was this patch tested? local test ```bash gcli tag create -m cli_demo --tag tagA tagB -i # Tags tagA,tagB created gcli tag create -m cli_demo --tag tagC tagD -i --quiet # no output ``` * [apache#6449] improve(CLI): Refactor getAuth method in CLI (apache#6451) ### What changes were proposed in this pull request? Refactor getAuth method in CLI. ### Why are the changes needed? Fix: apache#6449 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test. * [apache#6450] fix(web): fix authentication.type issue (apache#6452) ### What changes were proposed in this pull request? authentication.type of catalog property should send correct request to server ### Why are the changes needed? N/A Fix: apache#6450 ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? manually --------- Co-authored-by: Pucheng Yang <[email protected]> Co-authored-by: Yuhui <[email protected]> Co-authored-by: Qi Yu <[email protected]> Co-authored-by: Lord of Abyss <[email protected]> Co-authored-by: Jerry Shao <[email protected]> Co-authored-by: FANNG <[email protected]> Co-authored-by: Qiming Teng <[email protected]> Co-authored-by: Justin Mclean <[email protected]> Co-authored-by: Shaofeng Shi <[email protected]> Co-authored-by: Lychee <[email protected]> Co-authored-by: YangJie <[email protected]> Co-authored-by: Aryan Mahawar <[email protected]> Co-authored-by: yangyang zhong <[email protected]> Co-authored-by: Qian Xia <[email protected]>

…nts from gravitino-env.sh (apache#6385)  ### What changes were proposed in this pull request? Update Gravitino JVM memory default. ### Why are the changes needed? Fix: apache#6384 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? No test

### What changes were proposed in this pull request? fix uri title of version detail ### Why are the changes needed? N/A Fix: #6478 ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? manually Co-authored-by: Qian Xia <[email protected]>

…TAINERS.md (#6572) Update unknowntpo's organization due to career change in MAINTAINERS.md.  ### What changes were proposed in this pull request? Update unknowntpo's organization due to career change in MAINTAINERS.md. ### Why are the changes needed? Career change. Fix: #(issue) ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? None.

…TAINERS.md (apache#6572) Update unknowntpo's organization due to career change in MAINTAINERS.md.  ### What changes were proposed in this pull request? Update unknowntpo's organization due to career change in MAINTAINERS.md. ### Why are the changes needed? Career change. Fix: #(issue) ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? None.

…n connecting to JDBC catalog (#6639)  ### What changes were proposed in this pull request? jdbc.pool.test-on-borrow does not work when connecting to JDBC catalog ### Why are the changes needed? Fix: #6630 ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? N/A

…n connecting to JDBC catalog (#6643)  ### What changes were proposed in this pull request? jdbc.pool.test-on-borrow does not work when connecting to JDBC catalog ### Why are the changes needed? Fix: #6630 ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? N/A Co-authored-by: Xiaojian Sun <[email protected]>

Hive table entity serde and store support

c901df5

mchades requested review from jerryshao and yuqi1129 August 14, 2023 07:13

mchades self-assigned this Aug 14, 2023

yuqi1129 reviewed Aug 14, 2023

View reviewed changes

handle final variant in lambda function

84d797d

jerryshao approved these changes Aug 14, 2023

View reviewed changes

jerryshao merged commit bb29195 into apache:main Aug 14, 2023

liujinhui1994 mentioned this pull request Mar 21, 2024

[#2625] fix: cannot compile under docker bash #2626

Closed

yuqi1129 mentioned this pull request Aug 14, 2024

[#4445] feat(docker): Publish Docker image to Apache official DockerH… #4522

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[#209]feat(catalog): Hive table entity serde and store support #233

[#209]feat(catalog): Hive table entity serde and store support #233

mchades commented Aug 14, 2023

github-actions bot commented Aug 14, 2023 •

edited

Loading

yuqi1129 Aug 14, 2023

mchades Aug 14, 2023

yuqi1129 Aug 14, 2023

mchades Aug 14, 2023

[#209]feat(catalog): Hive table entity serde and store support #233

[#209]feat(catalog): Hive table entity serde and store support #233

Conversation

mchades commented Aug 14, 2023

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

github-actions bot commented Aug 14, 2023 • edited Loading

Code Coverage Report

yuqi1129 Aug 14, 2023

Choose a reason for hiding this comment

mchades Aug 14, 2023

Choose a reason for hiding this comment

yuqi1129 Aug 14, 2023

Choose a reason for hiding this comment

mchades Aug 14, 2023

Choose a reason for hiding this comment

github-actions bot commented Aug 14, 2023 •

edited

Loading