make Geometry comparable #25225

justin2004 · 2025-03-05T18:04:41Z

In PostGIS, instances of Geometry are comparable (that is, you can run select distinct on those columns).
This PR allows Trino to do the same and addresses #24961.

e.g. with the patch applied:

trino> describe postgis.public.movement;                            
     Column      |     Type     | Extra | Comment 
-----------------+--------------+-------+---------
 id              | integer      |       |         
 thing_id        | integer      |       |         
 geometry        | Geometry     |       |         
 geometry_string | varchar(50)  |       |         
 dt              | timestamp(6) |       |          
 speed_mph       | integer      |       |         
(6 rows)                                                           

Query 20250305_174904_00016_abpcz, FINISHED, 1 node                                                                                    
Splits: 19 total, 19 done (100.00%)
0.40 [6 rows, 364B] [14 rows/s, 901B/s]
trino> select geometry from postgis.public.movement;
      geometry       
---------------------
 POINT (-90 38.5)    
 POINT (-90 38.501)  
 POINT (-90 38.5)    
 POINT (-90.1 38.58) 
(4 rows)

Query 20250305_174913_00017_abpcz, FINISHED, 1 node
Splits: 1 total, 1 done (100.00%)
0.23 [4 rows, 0B] [17 rows/s, 0B/s]

trino> select distinct geometry from postgis.public.movement;
      geometry       
---------------------
 POINT (-90 38.501)  
 POINT (-90 38.5)    
 POINT (-90.1 38.58) 
(3 rows)

Query 20250305_174919_00018_abpcz, FINISHED, 1 node
Splits: 1 total, 1 done (100.00%)
0.26 [3 rows, 0B] [11 rows/s, 0B/s]

EDIT:
I got the formatter to run (using intellij).

cla-bot · 2025-03-05T18:04:44Z

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Justin Dowdy.
This is most likely caused by a git client misconfiguration; please make sure to:

check if your git client is configured with an email to sign commits git config --list | grep email
If not, set it up using git config --global user.email [email protected]
Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

cla-bot · 2025-03-05T21:23:55Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

so we can do `select distinct` on columns of that type. postgis allows that.

used google-java-format

using intellij

cla-bot · 2025-03-05T21:43:10Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

justin2004 · 2025-03-05T21:59:08Z

I've emailed the signed CLA.

cla-bot · 2025-03-05T22:21:58Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

electrum · 2025-03-06T01:33:33Z

plugin/trino-geospatial/src/main/java/io/trino/plugin/geospatial/GeometryType.java

 public class GeometryType
        extends AbstractVariableWidthType
 {
    public static final GeometryType GEOMETRY = new GeometryType();

+    private static final TypeOperatorDeclaration TYPE_OPERATOR_DECLARATION =


I think this can be

private static final TypeOperatorDeclaration TYPE_OPERATOR_DECLARATION = extractOperatorDeclaration(GeometryType.class, lookup(), Slice.class);

See UuidType for an example.

I tried this but then the test failed:

[ERROR] Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 34.16 s <<< FAILURE! -- in io.trino.plugin.geospatial.TestGeoSp atialQueries [ERROR] io.trino.plugin.geospatial.TestGeoSpatialQueries.testDistinctGeometry -- Time elapsed: 9.717 s <<< ERROR! io.trino.testing.QueryFailedException: io.trino.spi.TrinoException: Geometry READ_VALUE operator can not be adapted to convention (([FL AT])FAIL_ON_NULL). Available implementations: [([BLOCK_POSITION_NOT_NULL])FAIL_ON_NULL, ([NEVER_NULL])BLOCK_BUILDER] at io.trino.testing.AbstractTestingTrinoClient.execute(AbstractTestingTrinoClient.java:138) at io.trino.testing.DistributedQueryRunner.executeInternal(DistributedQueryRunner.java:565) at io.trino.testing.DistributedQueryRunner.execute(DistributedQueryRunner.java:548) at io.trino.sql.query.QueryAssertions$QueryAssert.lambda$new$1(QueryAssertions.java:317) at com.google.common.base.Suppliers$NonSerializableMemoizingSupplier.get(Suppliers.java:200) at io.trino.sql.query.QueryAssertions$QueryAssert.result(QueryAssertions.java:436) at io.trino.plugin.geospatial.TestGeoSpatialQueries.testDistinctGeometry(TestGeoSpatialQueries.java:58)

electrum · 2025-03-06T01:34:35Z

plugin/trino-geospatial/src/main/java/io/trino/plugin/geospatial/GeometryType.java

+    }
+
+    @Override
+    public boolean isComparable()


This could implement isOrderable() = true as well.

electrum · 2025-03-06T01:37:51Z

Can you add tests for this in TestGeoSpatialQueries?

cla-bot · 2025-03-06T20:36:46Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

cla-bot · 2025-03-06T20:38:45Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

justin2004 · 2025-03-06T20:39:38Z

Can you add tests for this in TestGeoSpatialQueries?

@electrum, done.

before this change the test says:

[ERROR]   TestGeoSpatialQueries.testDistinctGeometry:55 ? QueryFailed line 1:1: DISTINCT can only be applied to comparable types (actua
l: Geometry): geom

- Reuse new module as core for default tarball - Avoid duplication of artifact set definitions - Most plugins are not included in the core - trino-server tarball is essentially unchanged - Improve content of README in binaries

Looks like a copy-paste error. The latter parts of this assertion use this one.

Remove nested loop to find ambiguous tables. This optimizes compute from O(n^2) to 2*O(n) when bigquery.case-insensitive-name-matching is enabled

The table column reference was registered incorrectly with the original case taken from the view definition. It then failed to match the column schema returned from `SystemAccessControl` and the mask was not applied. Instead of sprinkling `toLowerCase()` here and there, we will associate the original `Field` with the column mask and use `Field#canResove` to do the matching. The problem with this is that there's no way to do efficient lookups by name in a case-insensitive way, so we have to iterate the list of `Field`s to find a match. Fixes trinodb#24054.

In the plugins page in the documentation for now. In a follow up we can add these links on the doc pages for specific plugins that we no longer ship with the default binaries.

…o geometry-comparable

cla-bot · 2025-03-06T20:53:36Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

justin2004 added 3 commits March 5, 2025 15:42

make Geometry comparable

2a89d4d

so we can do `select distinct` on columns of that type. postgis allows that.

format

2da75f2

used google-java-format

format

1417e95

using intellij

justin2004 force-pushed the geometry-comparable branch from f4ee92f to 1417e95 Compare March 5, 2025 21:43

Merge branch 'master' into geometry-comparable

be677f6

electrum reviewed Mar 6, 2025

View reviewed changes

Merge branch 'master' into geometry-comparable

f12a721

add test

eebd2ef

wendigo and others added 13 commits March 6, 2025 14:50

Use correct action to merge arguments

c2903d1

Exclude dependency in trino-pinot

d9d5d6b

Improve DuckDB connector doc for requirements and connection

204cb27

Add minimal tarball trino-server-core

46a6c92

- Reuse new module as core for default tarball - Avoid duplication of artifact set definitions - Most plugins are not included in the core - trino-server tarball is essentially unchanged - Improve content of README in binaries

Use a proper variable in assertion

e1d0631

Looks like a copy-paste error. The latter parts of this assertion use this one.

Optimize BigQuery listing tables

93e98e6

Remove nested loop to find ambiguous tables. This optimizes compute from O(n^2) to 2*O(n) when bigquery.case-insensitive-name-matching is enabled

Add Trino 472 release notes

6254a6f

[maven-release-plugin] prepare release 472

472cabb

[maven-release-plugin] prepare for next development iteration

2a1b4c0

Remove Kudu connector

fbbb672

Remove Phoenix connector

90663cf

Add download links for plugins

e5a000f

In the plugins page in the documentation for now. In a follow up we can add these links on the doc pages for specific plugins that we no longer ship with the default binaries.

justin2004 added 2 commits March 6, 2025 14:50

add test

aef2473

Merge branch 'geometry-comparable' of github.com:justin2004/trino int…

c2bab74

…o geometry-comparable

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

make Geometry comparable #25225

make Geometry comparable #25225

justin2004 commented Mar 5, 2025 •

edited

Loading

cla-bot bot commented Mar 5, 2025

cla-bot bot commented Mar 5, 2025

cla-bot bot commented Mar 5, 2025

justin2004 commented Mar 5, 2025

cla-bot bot commented Mar 5, 2025

electrum Mar 6, 2025

justin2004 Mar 6, 2025

electrum Mar 6, 2025

electrum commented Mar 6, 2025

cla-bot bot commented Mar 6, 2025

cla-bot bot commented Mar 6, 2025

justin2004 commented Mar 6, 2025 •

edited

Loading

cla-bot bot commented Mar 6, 2025

make Geometry comparable #25225

Are you sure you want to change the base?

make Geometry comparable #25225

Conversation

justin2004 commented Mar 5, 2025 • edited Loading

cla-bot bot commented Mar 5, 2025

cla-bot bot commented Mar 5, 2025

cla-bot bot commented Mar 5, 2025

justin2004 commented Mar 5, 2025

cla-bot bot commented Mar 5, 2025

electrum Mar 6, 2025

Choose a reason for hiding this comment

justin2004 Mar 6, 2025

Choose a reason for hiding this comment

electrum Mar 6, 2025

Choose a reason for hiding this comment

electrum commented Mar 6, 2025

cla-bot bot commented Mar 6, 2025

cla-bot bot commented Mar 6, 2025

justin2004 commented Mar 6, 2025 • edited Loading

cla-bot bot commented Mar 6, 2025

justin2004 commented Mar 5, 2025 •

edited

Loading

justin2004 commented Mar 6, 2025 •

edited

Loading