DH-5413/add clickhouse support (#391)

* DH-5413/add clickhouse support

* DH-5413/reformat with black

README.md

@@ -160,7 +160,7 @@ Once the engine is running, you will want to use it by:
 3. Querying the data in natural language
 
 ### Connecting to your data warehouses
-We currently support connections to Postgres, DuckDB, BigQuery, Databricks, Snowflake and AWS Athena. You can create connections to these warehouses through the API or at application start-up using the envars.
+We currently support connections to Postgres, DuckDB, BigQuery, ClickHouse, Databricks, Snowflake and AWS Athena. You can create connections to these warehouses through the API or at application start-up using the envars.
 
 #### Connecting through the API
 

dataherald/db_scanner/sqlalchemy.py

@@ -3,6 +3,7 @@
 from typing import Any, List
 
 import sqlalchemy
+from clickhouse_sqlalchemy import engines
 from overrides import override
 from sqlalchemy import Column, ForeignKeyConstraint, MetaData, Table, inspect
 from sqlalchemy.schema import CreateTable
@@ -196,7 +197,12 @@ def get_table_schema(
             for col in valid_columns
         ]
 
-        new_table = Table(original_table.name, MetaData(), *new_columns)
+        if "clickhouse" not in str(db_engine.engine.url).split(":")[0]:
+            new_table = Table(original_table.name, MetaData(), *new_columns)
+        else:
+            new_table = Table(
+                original_table.name, MetaData(), *new_columns, engines.MergeTree()
+            )
 
         for fk in original_table.foreign_keys:
             new_table.append_constraint(

docs/api.create_database_connection.rst

@@ -219,6 +219,17 @@ Example::
 
 "connection_uri": snowflake://jon:123456@foo-bar/my-database/public
 
+ClickHouse
+^^^^^^^^^^^^
+
+Uri structure::
+
+"connection_uri": clickhouse+http://<user>:<password>@<host>:<port>/<db_name>?protocol=https
+
+Example::
+
+"connection_uri": clickhouse+http://jon:123456@foo-bar:8443/my-database?protocol=https
+
 AWS Athena
 ^^^^^^^^^^^^
 

docs/api.refresh_table_description.rst

@@ -1,7 +1,7 @@
 .. api.scan_database:
 
 Refresh a table description
-=======================
+==============================
 
 This endpoint establishes a database connection, retrieves the table names, and merges these records with the existing
 table-descriptions. It updates the status accordingly: if the table no longer exists, it changes the status to

requirements.txt

@@ -40,3 +40,4 @@ tiktoken==0.5.2
 duckdb-engine==0.9.1
 duckdb==0.9.1
 PyMySQL==1.1.0
+clickhouse-sqlalchemy==0.2.2