Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new-style 2.0 command names for Airflow 1.10.x #12725

Merged
merged 9 commits into from
Dec 2, 2020

Conversation

ashb
Copy link
Member

@ashb ashb commented Dec 1, 2020

This introduces the new style commands for 1.10, but keeps all the
existing commands working, they just don't show in the top level
airflow --help anymore.

image

image

This was done by taking the argparse groups/sub-commands from master
(along with supporting functions) and copying it here, then providing
needed shims for compat.

I have not added any new sub-commands, nor changed any existing
behaviours, so the 1.10 commands won't change in behaviour.


^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

This introduces the new style commands for 1.10, but keeps all the
existing commands working, they just don't show in the top level
`airflow --help` anymore.

This was done by taking the argparse groups/sub-commands from master
(along with supporting functions) and copying it here, then providing
needed shims for compat.

I have not added any new sub-commands, nor changed any existing
behaviours, so the 1.10 commands won't change in behaviour.
@ashb ashb added this to the Airflow 1.10.14 milestone Dec 1, 2020
Copy link
Member

@kaxil kaxil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@potiuk potiuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great idea!

@github-actions
Copy link

github-actions bot commented Dec 1, 2020

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest master at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

@github-actions github-actions bot added the full tests needed We need to run full set of tests for this PR to merge label Dec 1, 2020
@ashb
Copy link
Member Author

ashb commented Dec 1, 2020

Ohh crap. This might not be possible with anything less than python 3.7. The required option to subparsers wasn't added until 3.7.

No, I think something else is going on here.

@ashb
Copy link
Member Author

ashb commented Dec 1, 2020

Tests are failing due to some interaction/side effect between tests.

@ashb
Copy link
Member Author

ashb commented Dec 1, 2020

Ah, side-effect of deleting elements out of the class-level dictionary meant we couldn't create more than a single parser object per process. Fixed in 4bd9f91 I hope

@kaxil kaxil merged commit 5fbd0ed into apache:v1-10-stable Dec 2, 2020
@kaxil kaxil deleted the new-style-commands-2.0 branch December 2, 2020 17:20
ashb added a commit to astronomer/airflow that referenced this pull request Dec 3, 2020
In apache#12725 I mistakenly _removed_ `airflow scheduler` command. Oops.

(This was caught by our K8s tests, but those weren't run on the PR.
Separate issue)
ashb added a commit that referenced this pull request Dec 3, 2020
In #12725 I mistakenly _removed_ `airflow scheduler` command. Oops.

(This was caught by our K8s tests, but those weren't run on the PR.
Separate issue)
AntonyRileyAtVerto pushed a commit to vertoanalytics/incubator-airflow that referenced this pull request Feb 2, 2021
- BugFix: Tasks with ``depends_on_past`` or ``task_concurrency`` are stuck (apache#12663)
- Fix issue with empty Resources in executor_config (apache#12633)
- Fix: Deprecated config ``force_log_out_after`` was not used (apache#12661)
- Fix empty asctime field in JSON formatted logs (apache#10515)
- [AIRFLOW-2809] Fix security issue regarding Flask SECRET_KEY (apache#3651)
- [AIRFLOW-2884] Fix Flask SECRET_KEY security issue in www_rbac (apache#3729)
- [AIRFLOW-2886] Generate random Flask SECRET_KEY in default config (apache#3738)
- Add missing comma in setup.py (apache#12790)
- Bugfix: Unable to import Airflow plugins on Python 3.8 (apache#12859)
- Fix setup.py missing comma in ``setup_requires`` (apache#12880)
- Don't emit first_task_scheduling_delay metric for only-once dags (apache#12835)

- Update setup.py to get non-conflicting set of dependencies (apache#12636)
- Rename ``[scheduler] max_threads`` to ``[scheduler] parsing_processes`` (apache#12605)
- Add metric for scheduling delay between first run task & expected start time (apache#9544)
- Add new-style 2.0 command names for Airflow 1.10.x (apache#12725)
- Add Kubernetes cleanup-pods CLI command for Helm Chart (apache#11802)
- Don't let webserver run with dangerous config (apache#12747)
- Replace pkg_resources with importlib.metadata to avoid VersionConflict errors (apache#12694)

- Clarified information about supported Databases
cfei18 pushed a commit to cfei18/incubator-airflow that referenced this pull request Mar 5, 2021
This introduces the new style commands for 1.10, but keeps all the
existing commands working, they just don't show in the top level
`airflow --help` anymore.

This was done by taking the argparse groups/sub-commands from master
(along with supporting functions) and copying it here, then providing
needed shims for compat.

I have not added any new sub-commands, nor changed any existing
behaviours, so the 1.10 commands won't change in behaviour.
cfei18 pushed a commit to cfei18/incubator-airflow that referenced this pull request Mar 5, 2021
In apache#12725 I mistakenly _removed_ `airflow scheduler` command. Oops.

(This was caught by our K8s tests, but those weren't run on the PR.
Separate issue)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:CLI full tests needed We need to run full set of tests for this PR to merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants