Explore removing @jupyter/chat by merging it into other packages

[dlqqq]

Problem

I believe this repo can be simplified to just 2 NPM packages: jupyterlab-chat should provide all the tokens, and jupyterlab-chat-extension should provide all the implementations. This would follow the provider-consumer pattern everywhere else in JupyterLab.

The original motivation behind a 3rd package was that developers may use @jupyter/chat to build their own extensions. However, I think this needs to be reconsidered for the reasons below.

1. @jupyter/chat is confusing for developers: It includes some tokens & some implementations, but not all of them. It's not clear when source code should be defined in @jupyter/chat v.s. the other packages.
2. @jupyter/chat may bring no benefit: If @jupyter/chat were merged into the other packages, developers could install jupyterlab-chat-extension from NPM to use our frontend components. They would still be able to develop their own chat labextension with our components without installing jupyterlab-chat from PyPI.
3. @jupyter/chat conflicts with the goal of Jupyter Chat as a whole: Jupyter Chat is being built to provide a modular chat extension that can be modified & extended by other extensions like Jupyter AI. I'm not sure we should recommend other developers use @jupyter/chat to build their own chat applications. IMO, we should mention that it's possible, but recommend that users extend / contribute to Jupyter Chat instead.

Proposed Solution

1. Merge tokens from @jupyter/chat into jupyterlab-chat.
2. Merge implementations from @jupyter/chat into jupyterlab-chat-extension.
3. Remove @jupyter/chat and deprecate it on NPM.

Additional context

Original discussion occurred in #161.

[dlqqq]

cc @jtpio @brichet

[brichet]

Thanks @dlqqq for opening this discussion.

The main idea behind this packaging is to provide:

• a frontend package that can be used in any extension
• a package using the frontend package, based on collaborative file for the exchange system
• a package exporting the plugins for the extension

1. @jupyter/chat is confusing for developers: It includes some tokens & some implementations, but not all of them. It's not clear when source code should be defined in @jupyter/chat v.s. the other packages.

Maybe we could clarify it in the contributing documentation.
In my opinion, source code should be defined in @jupyter/chat when it is used by the package itself.

About the tokens, defining it in @jupyter/chat ensures that the same object is not defined twice, in case several extension provide a chat. Perhaps we could change this logic and define all the token in jupyterlab-chat extension (and let every extension define its own token).

2. @jupyter/chat may bring no benefit: If @jupyter/chat were merged into the other packages, developers could install jupyterlab-chat-extension from NPM to use our frontend components. They would still be able to develop their own chat labextension with our components without installing jupyterlab-chat from PyPI.

It's true, but it would also bring some unnecessary packages, like @jupyter/collaborative-drive, @jupyter/ydoc, y-protocols, yjs.

3. @jupyter/chat conflicts with the goal of Jupyter Chat as a whole

I don't get this point, what do you mean by "goal of Jupyter Chat as a whole" ?
jupyterlab-chat relies on shared document, which does not work for example in jupyterlite.
@jupyter/chat can be used to build an extension with a chat for jupyterlite (like in the lite deployment of the documentation https://jupyter-chat.readthedocs.io/en/latest/lite/lab/index.html).

For example, jupyterlite-ai is an extension using the chat in jupyterlite. It depends only on @jupyter/chat.