[flytepropeller][flyteidl][flyteconnector][flytagent] Connector Service and Connnector IDL

[Future-Outlier]

Tracking issue

#3936

Why are the changes needed?

Given there's a name collision between LLM agent and flyte agent, we are going to rename flyte agent to flyte connectors.

What changes were proposed in this pull request?

1. you can't enable agent-service and connector-service now in the same time, propeller will crash because the task type is ambiguous.
   https://github.com/flyteorg/flyte/blob/new-proto-agent/flytepropeller/pkg/controller/nodes/task/handler.go#L360-L363
2. new connector IDL succeed when sending gRPC request to agent server with old flytekit.

How was this patch tested?

from flytekit import task, workflow
from flytekit.sensor.file_sensor import FileSensor

sensor = FileSensor(name="test_file_sensor")

@workflow
def wf():
    sensor(path=".")

new flyte and old flytekit

single binary with this branch and flytekit with master branch at 5/11.
flytekit commit hash: 0214448e96c18ecfcbc050d767be77966f7191fb

1. enable connector-service plugin.

tasks:
  task-plugins:
    enabled-plugins:
      - container
      - sidecar
      - K8S-ARRAY
      - connector-service
      - echo
  connector-service:
    defaultConnector:
      endpoint: "localhost:8000"
      insecure: true
      defaultTimeout: 100s

2. enable agent-service plugin.

tasks:
  task-plugins:
    enabled-plugins:
      - container
      - sidecar
      - K8S-ARRAY
      - agent-service
      - echo
  agent-service:
    defaultAgent:
      endpoint: "localhost:8000"
      insecure: true
      defaultTimeout: 100s

Labels

Please add one or more of the following labels to categorize your PR:

• added: For new features.
• changed: For changes in existing functionality.
• deprecated: For soon-to-be-removed features.
• removed: For features being removed.
• fixed: For any bug fixed.
• security: In case of vulnerabilities

This is important to improve the readability of release notes.

Setup process

Screenshots

Check all the applicable boxes

• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.

Related PRs

Docs link

[flyte-bot]

Code Review Agent Run Status

• Limitations and other issues: ❌ Failure - Bito Code Review Agent didn't review this pull request automatically because it exceeded the size limit. No action is needed if you didn't intend for the agent to review it. Otherwise, you can initiate the review by typing /review in a comment below.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 59.46%. Comparing base (a2331bd) to head (25a1e75).
Report is 3 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6325      +/-   ##
==========================================
+ Coverage   58.50%   59.46%   +0.96%     
==========================================
  Files         937      553     -384     
  Lines       71088    37859   -33229     
==========================================
- Hits        41587    22514   -19073     
+ Misses      26349    13689   -12660     
+ Partials     3152     1656    -1496     

Flag	Coverage Δ
unittests-datacatalog	59.06% <ø> (ø)
unittests-flyteadmin	56.27% <ø> (-0.03%)	⬇️
unittests-flytecopilot	30.99% <ø> (ø)
unittests-flytectl	64.70% <ø> (-0.06%)	⬇️
unittests-flyteidl	76.12% <ø> (ø)
unittests-flyteplugins	?
unittests-flytepropeller	?
unittests-flytestdlib	64.04% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[flyte-bot]

Code Review Agent Run Status

• Limitations and other issues: ❌ Failure - Bito Code Review Agent didn't review this pull request automatically because it exceeded the size limit. No action is needed if you didn't intend for the agent to review it. Otherwise, you can initiate the review by typing /review in a comment below.

[flyte-bot]

Code Review Agent Run Status

• Limitations and other issues: ❌ Failure - Bito Code Review Agent didn't review this pull request automatically because it exceeded the size limit. No action is needed if you didn't intend for the agent to review it. Otherwise, you can initiate the review by typing /review in a comment below.

[flyte-bot]

Code Review Agent Run Status

• Limitations and other issues: ❌ Failure - Bito Code Review Agent didn't review this pull request automatically because it exceeded the size limit. No action is needed if you didn't intend for the agent to review it. Otherwise, you can initiate the review by typing /review in a comment below.

[flyte-bot]

Code Review Agent Run Status

• Limitations and other issues: ❌ Failure - Bito Code Review Agent didn't review this pull request automatically because it exceeded the size limit. No action is needed if you didn't intend for the agent to review it. Otherwise, you can initiate the review by typing /review in a comment below.