Asset event is not created for asset decorator Dag

[atul-astronomer]

Apache Airflow version

main (development)

If "Other Airflow 2 version" selected, which one?

No response

What happened?

Asset event should be created when user triggers a Dag which is defined using asset decorator.

What you think should happen instead?

An asset event should be created since asset defined using asset decorator is outlet asset by default.

How to reproduce

Trigger the below Dag and check for asset events.

@asset(schedule=None)
def asset_decorator():
    pass

Operating System

Linux

Versions of Apache Airflow Providers

No response

Deployment

Other

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[atul-astronomer]

@vatsrahul1001

[Lee-W]

just verified it and it does not relates to the inlet event PR

[uranusjr]

This is caused by an AssetRef is not currently handled correctly in the execution model, but the asset decorator uses AssetRef to define asset relationships between dags. So… many layers to fix—collecting events after execution, sending events with ref (not full assets) to the execution API, and resolving them in the scheduler. Going to take a while…

[Lee-W]

This is caused by an AssetRef is not currently handled correctly in the execution model, but the asset decorator uses AssetRef to define asset relationships between dags. So… many layers to fix—collecting events after execution, sending events with ref (not full assets) to the execution API, and resolving them in the scheduler. Going to take a while…

The last time I tried, it worked fine with #47292. but yep, we still have many issue on the execution end

[uranusjr]

I think #47292 is not enough if we go through the task sdk; it’d still fail since the execution api can’t serialise things correctly.

[Lee-W]

just take a deeper look. The logic looks very wrong... asset_type handling is also incorrect

[Lee-W]

I added yet another draft PR as a tmp workaround to make asset subclasses (including @ asset) works ok. #47598

[uranusjr]

Yeah I noticed that too. The way I changed it is to always just use Asset.__name__ AssetNameRef.__name__ etc. like when those values are handled in register_asset_changes_in_db. I think that’s probably cleaner.

[Lee-W]

Yeah I noticed that too. The way I changed it is to always just use Asset.__name__ AssetNameRef.__name__ etc. like when those values are handled in register_asset_changes_in_db. I think that’s probably cleaner.

If handle it that way, we still need to check whether subclasses like Dataset and Model are Asset?

[uranusjr]

User-defined asset subclasses is only useful at execution time (if the user wants to introspect), and the UI (where we show asset_type as a string). But in register_asset_changes_in_db we don’t actually care about those subclasses, they are all assets and go through the exact same code path without any distinction. So these two probably should be two separate fields (and the one used in register_asset_changes_in_db should not be named asset_type).

[phanikumv]

Targeting to complete this by EOD today, so that we can get this included in tomorrow's beta release

[uranusjr]

This will be fixed by the combination of #47659 and #47677.