Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ts_ingest push to multiple endpoints? #28

Open
7yl4r opened this issue Jun 3, 2021 · 0 comments
Open

ts_ingest push to multiple endpoints? #28

7yl4r opened this issue Jun 3, 2021 · 0 comments
Labels
enhancement New feature or request tool-airflow

Comments

@7yl4r
Copy link
Member

7yl4r commented Jun 3, 2021

I'm setting up an instance of this on IMaRS servers to hopefully solve #26. Two ways to do this:

  1. airflow ts_ingest pushes data from IMaRS server to influxDB on FWC google cloud
  2. airflow ts_ingest pushes data from IMaRS server to influxDB at IMaRS. Then grafana running on FWC google cloud connects to IMaRS influxDB.

Either is fine. 1 is easier so I will do that. It is probably more performant too. But this did give me an idea:

perhaps ts_ingest should push data to a list of endpoints? That should be an easy feature addition and could be useful. For example: in the future we may want to push this same data from IMaRS into the FWC and FKNMS dashboards. This will require either:

  1. two airflow tasks. ie ts_ingest_fwc and ts_ingest_fknms with some duplication in filepaths
  2. ts_ingest tasks organized based on our data structures that push to multiple influxdb endpoints
@7yl4r 7yl4r added enhancement New feature or request tool-airflow labels Jun 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request tool-airflow
Projects
None yet
Development

No branches or pull requests

1 participant