Define a new framework for chat commands

[dlqqq]

Description

• Implements a new framework for chat commands, following the design proposed in Design proposal: Chat Completions API (rev. 1.1) #144.

• Closes Design proposal: Chat Completions API (rev. 1.1) #144.

• Implements a new chat commands menu UI, which shows commands matching the current word the user is typing.

• Allows consumer extensions to provide multiple commands that have multiple patterns.

• Allows consumer extensions to define how commands are handled from the frontend. No server code necessary.

Extended description

This PR introduces a modular and extensible framework for defining chat commands. To do so, this PR implements a ChatCommandRegistry singleton, which is provided to other lab extensions. Extensions can request this to add one or more chat command providers to the registry. Providers implement the IChatCommandProvider interface. Once added, each provider provides a set of commands to the command menu, and defines how those commands are handled.

This PR only aims to provide a minimal implementation of this design for further iteration. There are several known shortcomings with this PR currently, which I've documented as follow-up items after the demo section.

Demo

In this PR, 2 command providers are defined for demo purposes: one for slash commands, and one for emojis.

Screen.Recording.2025-02-10.at.4.05.15.PM.mov

The demo shows how this PR implements new features previously not possible in Jupyter AI or Jupyter Chat:

• Commands that may come from multiple extensions are all shown together in the commands menu.
• Command suggestions are shown for both, even though they have different patterns / syntax.
• When an emoji command is run, it replaces the current word with the emoji.

Follow-up items

Roughly ordered in descending priority, i.e. most important first, least important last.

• Subissue: Lift chat input state to ChatModel #163

• Subissue: Show icons & descriptions in chat commands menu #175

• Subissue: Remove old autocomplete framework #168

• Subissue: Allow command providers to be initialized async #174

• Subissue: Improve support for async command providers #167

• Subissue: Do not show commands menu when user is typing literally #176

• Subissue: Consider triggering commands on type #177

[github-actions]

👈 Launch a Binder on branch dlqqq/jupyter-chat/chat-commands

[brichet]

Thanks @dlqqq for working on it.

I left some comments below.

[brichet]

In a more general test, I saw 2 issues in usages, that can be seen in the following screencast:

• after replacing an input text using a commands, a new line replace again the text with undefined
• the completion menu does not open if the word is not at the beginning of the input. Strangely, when printing the autocompleteProps, the open props is set to true, and the options array is not empty.

record-2025-02-14_14.00.16.webm

[dlqqq]

@brichet @jtpio Thank you both for the review! I've addressed your feedback and fixed the bugs that @brichet noticed.

I'll close the remaining threads once I finish opening issues. I have something really urgent to attend to at work right now, so I'll finish that on Monday hopefully.

[dlqqq]

Demo of latest branch state.

Screen.Recording.2025-02-14.at.2.12.21.PM.mov

[brichet]

Thanks @dlqqq for updating the PR, I can't reproduce the issues anymore 👍 .

[dlqqq]

@brichet I've removed the old files. 👍

[brichet]

Thanks @dlqqq.
The failing test are related to the old autocompletion framework.
The best option would be to update them.
Otherwise we should at least skip them, and maybe add a fix to #168 (or another issue).

[dlqqq]

@brichet Since CI is already failing on main, can we go ahead and merge this while tracking the CI failure in a separate issue? Merging this now would make it a lot easier to work on #171.

[dlqqq]

Finally finished opening all of the follow-up issues raised by other reviewers and me! Thank you all for taking the time to review this & ask questions. 🤗

[jtpio]

Since CI is already failing on main, can we go ahead and merge this while tracking the CI failure in a separate issue?

Not sure it's the same failure. Based on @brichet's comment above (#161 (comment)) it seems related to the previous autocompletion registry implementation:

Ideally we should be able to update these tests? If it's too much effort for now, maybe worth disabling / removing them for now, but track adding them back in an issue?

[dlqqq]

@jtpio Thanks for the reminder! I've opened a new issue: #179

[jtpio]

Nice thank!

So let's remove or disable the outdated test code in this PR?

After #148 the CI should normally be passing more consistently again (with skipped tests). And to lower maintenance burden it's nice to have a green check even though we know some tests have been disabled (it's easier to merge other PRs).

[dlqqq]

@jtpio Good idea, I'll do that now.

[dlqqq]

@jtpio Done! 👍

[brichet]

Thanks @dlqqq and @jtpio