Add oauth support to dbx connection #2220
Open
+54
−1
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Only had access to databricks via oauth so needed to implement more connection methods.
Based on the official databricks sql connector docs: https://docs.databricks.com/aws/en/dev-tools/python-sql-connector
Solves the issue: #2087
This adds no new properties but the databricks connection will now make use of
auth_method
andconfiguration
dict properties.Token authentication:
If supplying
token
will work as normal and use personal access token authentication.user to machine oauth
If supplying
auth_method
and setting it to "databricks-oauth" will be u2m authenticationmachine to machine oauth
If neither the above are supplied it will work with the docs m2m authentication method. This configuration property gets supplied to the config class in which the user can supply the host/client_id/client_secret.
I tried to set this up in a way that enables flexability. The config class has a very large amount of options so if a more tech savvy user wants to change these settings they are not blocked from doing so and can supply them to the configuration property.
The oauth_type also has an azure oauth method that can be switched to instead.
If neither oauth_method or token are supplied, and configuration is empty, it will still use the empty Config object. This can be beneficial as the config class, if not supplied a parameter value, will look for environment variables instead. See the source code.
A caveat of how this is configured is that the if using m2m you will probably need to input host twice. once outside configuration and once inside. A way of resolving this is if host isn't set as a property, it will try and get the hostname from the instantiated config object and set it as that instead. This mean host comes from config meaning it can be from the environment variable or configuration property and doesn't have to be supplied twice.