Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PLATFORM-562] feat: Implement connectors commands #134

Merged
merged 22 commits into from
Apr 30, 2021

Conversation

raulb
Copy link
Member

@raulb raulb commented Apr 28, 2021

Description of change

Fixes https://meroxa.atlassian.net/browse/PLATFORM-562.

This pull request contains all commands related to connectors while also migrating them to use the new builder process adding tests to some commands that didn't have them before.

  • list
  • create
  • describe
  • logs
  • remove
  • update
  • connect migrated to new builder.

The idea to ultimately release this will be with the use of MEROXA_V2 env variable.

example:

  • export MEROXA_V2=only would make only v2 commands to be available
  • unset MEROXA_V2 would leave the CLI as v1 only.
  • export MEROXA_V2=any_value would make both v1 and v2 commands coexist. We could decide to remove this restriction once we're feature complete on v2.

Type of change

  • New feature
  • Bug fix
  • Refactor
  • Documentation

How was this tested?

  • Unit Tests
  • Tested in staging

Demo

Before this pull-request

❯ .m help
The Meroxa CLI allows quick and easy access to the Meroxa data platform.

Using the CLI you are able to create and manage sophisticated data pipelines
with only a few simple commands. You can get started by listing the supported
resource types:

meroxa list resource-types

Usage:
  meroxa [command]

Available Commands:
  add         Add a resource to your Meroxa resource catalog
  api         Invoke Meroxa API
  billing     Open your billing page in a web browser
  completion  Generate completion script
  connect     Connect two resources together
  create      Create Meroxa pipeline components
  describe    Describe a component
  help        Help about any command
  list        List components
  login       login or sign up to the Meroxa platform
  logout      logout of the Meroxa platform
  logs        Print logs for a component
  open        Open in a web browser
  remove      Remove a component
  update      Update a component
  version     Display the Meroxa CLI version
  whoami      Display the current logged in user


Flags:
      --config string      config file
      --debug              display any debugging information
  -h, --help               help for meroxa
      --json               output json
      --timeout duration   set the client timeout (default 10s)

Use "meroxa [command] --help" for more information about a command

After this pull-request

❯ MEROXA_V2=true .m help
The Meroxa CLI allows quick and easy access to the Meroxa data platform.

Using the CLI you are able to create and manage sophisticated data pipelines
with only a few simple commands. You can get started by listing the supported
resource types:

meroxa list resource-types

Usage:
  meroxa [command]

Available Commands:
  api         Invoke Meroxa API
  billing     Open your billing page in a web browser
  connectors  Manage connectors on Meroxa
  help        Help about any command
  login       login or sign up to the Meroxa platform
  logout      logout of the Meroxa platform
  version     Display the Meroxa CLI version
  whoami      Display the current logged in user


Flags:
      --config string      config file
      --debug              display any debugging information
  -h, --help               help for meroxa
      --json               output json
      --timeout duration   set the client timeout (default 10s)

Use "meroxa [command] --help" for more information about a command.

Manage connectors on Meroxa

Usage:
  meroxa connectors [command]

Aliases:
  connectors, connector

Available Commands:
  create      Create a connector
  describe    Describe connector
  list        List connectors
  logs        Print logs for a connector
  remove      Remove connector
  update      Update connector state

Flags:
  -h, --help   help for connectors

Global Flags:
      --config string      config file
      --debug              display any debugging information
      --json               output json
      --timeout duration   set the client timeout (default 10s)

Use "meroxa connectors [command] --help" for more information about a command.


❯ MEROXA_V2=true .m help connectors create
Use `connectors create` to create a connector from a source (--from) or to a destination (--to)

Usage:
  meroxa connectors create [NAME] [flags]

Examples:

meroxa connectors create [NAME] --from pg2kafka --input accounts
meroxa connectors create [NAME] --to pg2redshift --input orders # --input will be the desired stream
meroxa connectors create [NAME] --to pg2redshift --input orders --pipeline my-pipeline


Flags:
      --from string       resource name to use as source
  -h, --help              help for create
      --input string      command delimited list of input streams
      --pipeline string   pipeline name to attach connector to
      --to string         resource name to use as destination

Global Flags:
      --config string      config file
      --debug              display any debugging information
      --json               output json
      --timeout duration   set the client timeout (default 10s)

❯ .m help connectors describe
Describe connector

Usage:
  meroxa connectors describe [NAME] [flags]

Flags:
  -h, --help   help for describe

Global Flags:
      --config string      config file
      --debug              display any debugging information
      --json               output json
      --timeout duration   set the client timeout (default 10s)

❯ .m help connectors logs
Print logs for a connector

Usage:
  meroxa connectors logs NAME [flags]

Flags:
  -h, --help   help for logs

Global Flags:
      --config string      config file
      --debug              display any debugging information
      --json               output json
      --timeout duration   set the client timeout (default 10s)

❯ .m help connectors remove
Remove connector

Usage:
  meroxa connectors remove NAME [flags]

Aliases:
  remove, rm, delete

Flags:
  -h, --help   help for remove

Global Flags:
      --config string      config file
      --debug              display any debugging information
      --json               output json
      --timeout duration   set the client timeout (default 10s)

❯ .m help connectors update
Update connector state

Usage:
  meroxa connectors update NAME --state pause | resume | restart [flags]

Flags:
  -h, --help           help for update
      --state string   connector state

Global Flags:
      --config string      config file
      --debug              display any debugging information
      --json               output json
      --timeout duration   set the client timeout (default 10s)

Additional references

Design document

@raulb raulb force-pushed the raul/subject-verb-object-connectors branch from 1c1552f to ed35401 Compare April 28, 2021 18:59
@raulb raulb force-pushed the raul/subject-verb-object-connectors branch from d929905 to 8bc712e Compare April 29, 2021 11:57
@raulb raulb force-pushed the raul/subject-verb-object-connectors branch from dc0aa3c to c1af420 Compare April 29, 2021 13:06
@raulb raulb marked this pull request as ready for review April 29, 2021 13:50
@raulb raulb requested a review from lovromazgon April 29, 2021 13:50
@raulb raulb changed the title feat: Implement connectors commands [PLATFORM-562] feat: Implement connectors commands Apr 29, 2021
Copy link
Member

@lovromazgon lovromazgon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a couple of suggestions, but all in all, I'm quite happy with the PR, I think the structure is nice and clear.

@raulb raulb requested a review from lovromazgon April 30, 2021 13:35
@raulb raulb requested a review from lovromazgon April 30, 2021 13:55
Copy link
Member

@lovromazgon lovromazgon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙌 Nicely done!

@raulb raulb merged commit 6071574 into master Apr 30, 2021
@raulb raulb deleted the raul/subject-verb-object-connectors branch April 30, 2021 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants