Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DRAFT ingestion(ingestion/fivetran) - support for standard version #12832

Draft
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

acrylJonny
Copy link
Collaborator

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@acrylJonny acrylJonny marked this pull request as draft March 10, 2025 13:12
@github-actions github-actions bot added the ingestion PR or Issue related to the ingestion of metadata label Mar 10, 2025
Copy link

codecov bot commented Mar 10, 2025

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
2287 1 2286 47
View the top 1 failed test(s) by shortest run time
tests.unit.test_packages::test_check_str_enum_usage
Stack Traces | 0.545s run time
pytestconfig = <_pytest.config.Config object at 0x7f12a2752b40>

    def test_check_str_enum_usage(pytestconfig: pytest.Config) -> None:
        root = pytestconfig.rootpath
    
>       ensure_no_enum_mixin([root / "src", root / "tests"])

tests/unit/test_packages.py:28: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

dirs = [PosixPath('.../datahub/metadata-ingestion/src'), PosixPath('.../datahub/metadata-ingestion/tests')]

    def ensure_no_enum_mixin(dirs: List[pathlib.Path]) -> None:
        # See the docs on the StrEnum implementation for why this is necessary.
    
        bad_lines = {
            "(str, Enum)",
            "(str, enum.Enum)",
            # We don't have any int enums right now, but this will catch them if we add some.
            "(int, Enum)",
            "(int, enum.Enum)",
        }
    
        ignored_files = {
            "datahub/utilities/str_enum.py",
            "datahub/testing/check_str_enum.py",
        }
    
        for dir in dirs:
            for file in dir.rglob("*.py"):
                if any(str(file).endswith(ignored_file) for ignored_file in ignored_files):
                    continue
    
                with file.open() as f:
                    for line in f:
                        if any(bad_line in line for bad_line in bad_lines):
>                           raise ValueError(
                                f"Disallowed enum mixin found in {file}: `{line.rstrip()}`. "
                                "This enum mixin's behavior changed in Python 3.11, so it will work inconsistently across versions."
                                "Use datahub.utilities.str_enum.StrEnum instead."
                            )
E                           ValueError: Disallowed enum mixin found in .../datahub/metadata-ingestion/src/datahub/ingestion/source/fivetran/fivetran_constants.py: `class FivetranMode(str, Enum):`. This enum mixin's behavior changed in Python 3.11, so it will work inconsistently across versions.Use datahub.utilities.str_enum.StrEnum instead.

.../datahub/testing/check_str_enum.py:29: ValueError

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ingestion PR or Issue related to the ingestion of metadata
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant