Create External Table without schema [Spark]

[osopardo1]

Description

This PR fixes #165

Type of change

When creating an external table with an already populated directory, Qbeast was missing the schema in the final Catalog entry.
We add a method to detect this specific situation and update the service accordingly.

Checklist:

Here is the list of things you should do before submitting this pull request:

• New feature / bug fix has been committed following the Contribution guide.
• Add comments to the code (make it easier for the community!).
• Add tests.
• Your branch is updated to the main branch (dependent changes have been merged).

How Has This Been Tested? (Optional)

A new test is added in QbeastCatalogIntegrationTest in which we try to create an external table without schema.

[osopardo1]

We need to test this solution on the EMR clusters with Glue Catalog, see if we miss something else in the implementation and if everything works correctly.

[codecov]

Codecov Report

Merging #168 (318ee28) into main (a425a22) will increase coverage by 0.00%.
The diff coverage is 94.11%.

❗ Current head 318ee28 differs from pull request most recent head b49b63b. Consider uploading reports for the commit b49b63b to get more accurate results

@@           Coverage Diff           @@
##             main     #168   +/-   ##
=======================================
  Coverage   93.71%   93.71%           
=======================================
  Files          81       81           
  Lines        1925     1941   +16     
  Branches      145      154    +9     
=======================================
+ Hits         1804     1819   +15     
- Misses        121      122    +1     

Impacted Files	Coverage Δ
.../internal/sources/catalog/QbeastCatalogUtils.scala	96.42% <94.11%> (-0.64%)	⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[osopardo1]

Hello @Adricu8 !! Do you think this is ready to merge? Did you find any bug creating external tables with Qbeast? (We still missing pre-load the columnsToIndex too, but we will develop it in another PR).

[Adricu8]

Hi Paola! I did not find any issue with this. Creating external tables worked fine for me :)

[osopardo1]

Hi Paola! I did not find any issue with this. Creating external tables worked fine for me :)

Perfect! Could you approve the PR? 🙏

[Adricu8]

Changes look good to me, thanks Paola :)