Added experimental language server #3635

Tomatower · 2021-01-27T21:44:15Z

I have implemented a base structure for Microsofts Language Server Protocol (https://langserver.org).

Alongside the basic protocol parsing and error message generation it currently only supports 2 things:

Click-to-code: with Visual Studio Code (Clicking in Openscad opens and selects the respective file in VSCode)
Start a preview: (only the protocol - not the internas yet)

This PR has to be completed with a lot more implementations: such as utilizing the AST-Trees for context sensitive autocomplete information and keeping track of open and modified fieles in vscode in order to render them without saving.

I would like to get some feedback on how I integrated it into the remaining codebase so far, and maybe somebody has more ideas on how we can control the openscad window from an external editor.

Open Questions

Currently It will attach the LanguageServerInterface to the first opened window. Is that a limitation?
Should the featureset of an openscad window opened that way (most likely from an editor) be limited (no editor / no saving / no multiple windows in a process)
What information should be transferred to the editor?

ToDo

followed up by #3648
closes #2629

Tomatower · 2021-02-02T23:35:01Z

❎ Netbeans

Language Server on Netbeans (currently no preview support, this will unfortunately most likely require a custom plugin):

search field in the top right corner (close all open files if it does not show up)
search for Connect
select Connect to Language Server
root: project root, port the port you used to start openscad with (--lsp-port 23725) extension: .scad
Open a SCAD file, and keep an eye on the terminal output of openscad for issues

State of integration

not supported Text Document sync (textDocument/didOpen etc)
not supported Go To Code (window/showDocument)
not supported Does not send updates, so we dont know what is open (textDocument/publishDiagnostics)
test pending OpenSCAD log messages (window/logMessage)

means the default LanguageServer implementation of Netbeans is very limited for the Moment.

Quirks

Netbeans does not send a initialized message after sucessful setup.
Does have a very limited capability set (only "go to definition" and tracking the workspace files)

Tomatower · 2021-02-03T01:42:16Z

✔️ Visual Studio Code

checkout & compile
start openscad --lsp-port 23725 (23725 is 0x5CAD)
get the prototype VSCode Extension from https://github.com/Tomatower/vscode-openscad (which is just the example implementation modified for TCP)
Launch with the extension (Please extension authors - integrate that)
Open a model in this openscad window
Right click somewhere, and jump to the code. Both VSCode and Openscad should now have opened the file.

State of Integration

Working Text Document sync (textDocument/didOpen etc)
Working Go To Code (window/showDocument)
Working Diagnostics: creates red squigglies (textDocument/publishDiagnostics)
Working test pending OpenSCAD log messages (window/logMessage)

Tomatower · 2021-02-03T01:44:03Z

✔️ vim

Installation: Using vim Ale (https://github.com/dense-analysis/ale) with added SCAD language (https://github.com/Tomatower/ale/tree/lsp-openscad NOT MAINTAINED! - will have to do a PR after this is merged.)
do set filetype=scad if ALEInfo does not detect the filetype automatically.

State of Integration

Working Text Document sync (textDocument/didOpen etc)
🟨 Not Supported Go To Code (window/showDocument)
Working Diagnostics (textDocument/publishDiagnostics)
Working OpenSCAD log messages (window/logMessage) (altought only one line at a time in the vim log)

Tomatower · 2021-02-03T01:44:14Z

✔️ emacs

Installation: Use this fork of lsp-mode: https://github.com/Tomatower/lsp-mode/tree/client-openscad (upstream is pending to merging this PR)

State of Integration

Works Text Document sync (textDocument/didOpen etc)
Unsupported Go To Code (window/showDocument) ([3.16] Support handle Show Document Request emacs-lsp/lsp-mode#2459)
Works Diagnostics (textDocument/publishDiagnostics)
Pending OpenSCAD log messages (window/logMessage)

Thank you @Lenbok for the emacs integration!

Tomatower · 2021-02-03T01:44:31Z

✔️ atom

Installation: Somehow hack together a minimal LSP server, either as host or as client mode. If you ask me nicely I can give some source code. But it is barely working.

State of Integration

Working Text Document sync (textDocument/didOpen etc)
🟨 Not Supported Go To Code (window/showDocument) [ might be because my plugin did not implement it?]
Working Diagnostics (textDocument/publishDiagnostics)
Working OpenSCAD log messages (window/logMessage)

Tomatower · 2021-02-09T20:16:08Z

🟡 Netbeans (with plugin)

Install the plugin from https://github.com/Tomatower/netbeans-openscad (Contributions & Maintainer welcome)
Open a *.scad file - it will launch openscad for you automatically.

State of integration

Working Text Document sync (textDocument/didOpen etc)
Not Supported Go To Code (window/showDocument)
Not showing up Diagnostics (textDocument/publishDiagnostics)
Not showing up OpenSCAD log messages (window/logMessage)

means the LanguageServer implementation of Netbeans is very limited for the Moment.
The Editor might also benefit form putting more effort into the Plugin, if someone has the time to spare.

Quirks

Netbeans does not send a initialized message after sucessful setup.
Netbeans ignores our capability set and announces a bogus capability set, which has no connection to the actually implemented features.
Does have a very limited capability set (only "go to definition" and tracking the workspace files)

Lenbok · 2021-02-11T21:38:31Z

I can help test any emacs lsp integration

Lenbok · 2021-02-11T21:44:01Z

BTW, I've currently been using this small emacs extension I wrote for controlling the openscad gui from within emacs: https://github.com/Lenbok/scad-dbus - it uses DBUS for the IPC mechanism (linux only), but if your LSP server provides for equivalent control of the scad GUI that would probably be a step forward for other platforms. Is that within scope of an LSP server?

Tomatower · 2021-02-11T21:49:59Z

UI control, which is provided via DBUS is not scope of the LSP.
The LSP is designed for things like autocomplete, context information and error reporting/highlighting.

The dbus interface will still be usable if you use the LSP.

I would be very happy if someone can help me out in testing / developing the extension for emacs lsp-mode (so far I have no working LSP integration for emacs)
My current state is: https://github.com/Tomatower/lsp-mode/tree/client-openscad

Lenbok · 2021-02-12T00:49:39Z

Yeah, I thought that UI manipulation would probably be out of scope for an LSP server, but saw you had some comments about it in the PR.

I will try and have a play with your preliminary code over the weekend, thanks for the pointer!

Lenbok · 2021-03-06T21:39:51Z

@Tomatower I figured how how to send the $openscad/preview message and that it wanted the uri of the file to preview. However, I get the response: "File was not opened!", see below for the captured stderr - note that the document was opened immediately previously.

Method mapping:
	initialize 	 --> InitializeRequest
	initialized 	 --> InitializedNotifiy
	shutdown 	 --> ShutdownRequest
	exit 	 --> ExitNotification
	textDocument/didOpen 	 --> DidOpenTextDocument
	textDocument/didChange 	 --> DidChangeTextDocument
	textDocument/didClose 	 --> DidCloseTextDocument
	textDocument/documentSymbol 	 --> DocumentSymbolRequest
	$openscad/preview 	 --> OpenSCADRender
Listening on port 10007
Received connection from :54092
RECEIVED: [2193]: {"jsonrpc":"2.0","method":"initialize","params":{"processId":null,"rootPath":"/home2/len/objects/scad/mine/foo","clientInfo":{"name":"emacs","version":"GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.20)\n of 2020-09-20"},"rootUri":"file:///home2/len/objects/scad/mine/foo","capabilities":{"workspace":{"workspaceEdit":{"documentChanges":true,"resourceOperations":["create","rename","delete"]},"applyEdit":true,"symbol":{"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]}},"executeCommand":{"dynamicRegistration":false},"didChangeWatchedFiles":{"dynamicRegistration":true},"workspaceFolders":true,"configuration":true},"textDocument":{"declaration":{"linkSupport":true},"definition":{"linkSupport":true},"implementation":{"linkSupport":true},"typeDefinition":{"linkSupport":true},"synchronization":{"willSave":true,"didSave":true,"willSaveWaitUntil":true},"documentSymbol":{"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]},"hierarchicalDocumentSymbolSupport":true},"formatting":{"dynamicRegistration":true},"rangeFormatting":{"dynamicRegistration":true},"rename":{"dynamicRegistration":true,"prepareSupport":true},"codeAction":{"dynamicRegistration":true,"isPreferredSupport":true,"codeActionLiteralSupport":{"codeActionKind":{"valueSet":["","quickfix","refactor","refactor.extract","refactor.inline","refactor.rewrite","source","source.organizeImports"]}},"resolveSupport":{"properties":["edit","command"]},"dataSupport":true},"completion":{"completionItem":{"snippetSupport":true,"documentationFormat":["markdown"],"resolveAdditionalTextEditsSupport":true},"contextSupport":true},"signatureHelp":{"signatureInformation":{"parameterInformation":{"labelOffsetSupport":true}}},"documentLink":{"dynamicRegistration":true,"tooltipSupport":true},"hover":{"contentFormat":["markdown","plaintext"]},"foldingRange":{"dynamicRegistration":true},"callHierarchy":{"dynamicRegistration":false},"publishDiagnostics":{"relatedInformation":true,"tagSupport":{"valueSet":[1,2]},"versionSupport":true}},"window":{"workDoneProgress":true}},"initializationOptions":null,"workDoneToken":"1"},"id":169}

Handling Message [id 169] with method initialize
SENDING: [204]: {"id":169,"jsonrpc":"2.0","result":{"capabilities":{"documentSymbolProvider":true,"openscad":{"preview":true},"textDocumentSync":{"change":1,"openClose":true},"window":{"showDocument":{"support":true}}}}}
RECEIVED: [53]: {"jsonrpc":"2.0","method":"initialized","params":{}}

Handling Message [id <UNSET>] with method initialized
RECEIVED: [273]: {"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"file:///home2/len/objects/scad/mine/foo/bar.scad","languageId":"openscad","version":16,"text":"\nuse</home2/len/objects/scad/mine/foo/baz.scad>\n\nbaz();\nbaz();\n\ncube([10, 10, 10]);\n"}}}

Handling Message [id <UNSET>] with method textDocument/didOpen
SENDING: [145]: {"jsonrpc":"2.0","method":"textDocument/publishDiagnostics","params":{"diagnostics":[],"uri":"file:///home2/len/objects/scad/mine/foo/bar.scad"}}
RECEIVED: [141]: {"jsonrpc":"2.0","method":"$openscad/preview","params":{"textDocument":{"uri":"file:///home2/len/objects/scad/mine/foo/bar.scad"}},"id":170}

Handling Message [id 170] with method $openscad/preview
SENDING: [83]: {"error":{"code":-32602,"message":"File was not opened!"},"id":170,"jsonrpc":"2.0"}

Tomatower · 2021-03-06T23:05:25Z

thanks for the test, I will look into it!

Lenbok · 2021-03-10T09:40:41Z

I saw your force push and had another try - the extra logging helped. Looks like I was sending the preview with the incorrect message structure (you can see in my paste above, I had the uri inside a textDocument element). Once I rearranged the params correctly it previews.

However, even though it shows the preview, I noticed that the openscad app itself doesn't seem to know the file is loaded. By which I mean if I then switch to the GUI and tell it to do a final render (and export stl, etc) - it still thinks it's looking at an empty "Untitled.scad" and the console warns: "No top level geometry to render". Do you have a suggestion?

BTW, You can update the "Emacs" section above to indicate that TextDocument sync, and Diagnostics are working. Go To Code isn't currently implemented in lsp-mode (emacs-lsp/lsp-mode#2459). Log Messages seem to be currently disabled in your PR? (I also noticed that your initial checkbox in the PR description mentions a --lsp-port, which doesn't exist, which some may find confusing).

This is coming along nicely, and is already quite handy!

Tomatower · 2021-03-10T12:17:15Z

Thanks - glad it helped, I did not have the time to test this enough to tell you!

As you can see in the message log, the language server client (emacs) is sending the document content to openscad in a textDocument/didChange message. This content is then kept inside of the language server implementation and is not in the openscad editor.
All openscad functionality, like openscad-side F5/F6/export/... is done with the editor - and since I never feed a textDocument/didOpen into the editor, the whole front end does not know about stuff.

This was a deliberate decision, since opening the file in openscad opens pandoras box of parallel editing the same document in two editors - which I wanted to avoid.
I think i have to split the openscad internal logic from the editor, so I can force the LSP-document for rendering/preview if triggered from the application itself, I already have ideas.

Lenbok · 2021-03-11T07:17:50Z

I should also mention that I don't like the fact that when you run openscad as an LSP server, it permanently overwrites the user's preference for whether to show the editor and use automatic reload and preview. This makes it hard to switch between using the LSP server and using openscad standalone. It should only alter those settings in a non-persistent manner.

Lenbok · 2021-06-20T00:24:27Z

@Tomatower Have you had any time to take this feature any further?

Tomatower · 2021-06-20T00:35:12Z

I did not yet have any real issues with my own setup (where I use openscad as LSP + vscode), but I did not yet have the time to fix up the preferences-issue you pointed out.

Lenbok · 2021-06-20T01:30:12Z

Can you describe your workflow in more detail, in terms of where openscad gets started, when and how you trigger previews / renders / STL exports etc, and how that works for multi-file projects?

Here's my workflow prior to this PR:

I open a scad file in emacs, and from there I have a key bind that can start openscad for the current file. I have openscad to auto-reload, so that whenever I save the file (or other ones referenced by it), openscad updates. In addition, I use https://github.com/Lenbok/scad-dbus so that for most operations I don't even need to leave emacs.

Here's the workflow I've been using with this PR:

I open the scad file in emacs, and lsp-mode starts the openscad GUI with lsp server. I like the automatic preview on save (this handles knowing when to redisplay for multi-file projects) so rather than issuing the $openscad/preview command on a "master scad file", I minimize the GUI that was opened by lsp and start a new one as above. (But currently I have to re-enable the option that the lsp-server has disabled).

With this workflow it would be nice if the lsp-server didn't mess with the user preferences (and even better if it could start headless without a GUI at all). Alternatively I could just use the GUI started by lsp-server and hook into emacs so that it had a notion of the "primary scad file" and whenever any scad file in the project was saved it issued the $openscad/preview to get the latest update. That doesn't help with the inability to do render / export etc though.

Do you have any suggestions?

ochafik · 2022-03-12T15:33:29Z

Not sure what's missing here, but would be amazing to be able to use this in the OpenSCAD Web Demo (which uses the Monaco editor + some dirty regexps for completion right now).

Lenbok · 2022-03-12T18:56:45Z

As an alternative take on a language server for openscad, see https://github.com/dzhu/openscad-language-server (it doesn't require openscad itself).

Tomatower force-pushed the language-server branch 2 times, most recently from bc3011e to 8c3cd79 Compare February 1, 2021 02:09

Tomatower changed the title ~~RFC: added experimental language server~~ Added experimental language server Feb 7, 2021

This was referenced Feb 7, 2021

Enhance Language Server Interface #3648

Open

OpenSCAD Language Server Antyos/vscode-openscad#16

Closed

Tomatower force-pushed the language-server branch from ad486b5 to 17c6573 Compare February 9, 2021 20:18

Tomatower added 14 commits February 25, 2021 10:01

added experimental language server

a4364fa

LSP: log, cleanup

d4a655b

AbstractNode: add getNodesByLocation

d109ac9

cleanup to focus

c39e553

clean up log messages

c57adae

added logging and diagnostig messages

6ff0cb6

Added AST parsing, error messages and log messages to lsp

4243feb

Hide editor when starting with LSP

da765ae

disable autoreload and preview in LSP mode, add welcome msg

6d1de3c

rename interface to lspinterface

7649ac3

fixed qt project: add lsp headers

bbeae38

LSP: Add ImplementationRequest

28e5ae8

deactivate go to implementation (for now)

7e8cbf5

Refactor Connection: create tcp / stdio connection

52254c3

Tomatower added 14 commits February 25, 2021 10:02

cleanup stdio, stdio interface

a96bfa0

robustified Qt stream reading for netbeans

206579e

Fixed diagnostics on document change

f301c0c

LSP: cleanup and documentation

e9b042a

removed unneccessary prints

6edceb0

add handleDependencies to handle use<> and include()s

7723ab3

clean up diagnostics

cc3d180

removed last constexpr, won the sfinae fight

4092ff4

Add lsp-listen mode (as specified by the spec)

104f4f4

Add $openscad/preview command

3b688a8

Go to code after lsp preview

82e463a

Cleanup LSP Initialialize

f940981

LSP: Add textDocument/documentSymbol dummy

a14f72c

added openscad capabilities

4ff41af

Tomatower force-pushed the language-server branch from f420c33 to 4ff41af Compare February 25, 2021 09:03

open file if it could not be found for preview

cd72c07

Tomatower force-pushed the language-server branch from c5d9e67 to cd72c07 Compare March 8, 2021 12:36

Tomatower mentioned this pull request Dec 13, 2021

Hover Information Antyos/vscode-openscad#28

Open

t-paul mentioned this pull request May 1, 2022

Implement OpenSCAD LSP (Language Server Protocol) in editor Irev-Dev/cadhub#610

Open

t-paul mentioned this pull request Mar 31, 2024

Language support for popular code editors via Language Server Protocol #2629

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added experimental language server #3635

Added experimental language server #3635

Tomatower commented Jan 27, 2021 •

edited

Loading

Tomatower commented Feb 2, 2021 •

edited

Loading

Tomatower commented Feb 3, 2021 •

edited

Loading

Tomatower commented Feb 3, 2021 •

edited

Loading

Tomatower commented Feb 3, 2021 •

edited

Loading

Tomatower commented Feb 3, 2021 •

edited

Loading

Tomatower commented Feb 9, 2021 •

edited

Loading

Lenbok commented Feb 11, 2021

Lenbok commented Feb 11, 2021

Tomatower commented Feb 11, 2021

Lenbok commented Feb 12, 2021

Lenbok commented Mar 6, 2021

Tomatower commented Mar 6, 2021

Lenbok commented Mar 10, 2021

Tomatower commented Mar 10, 2021

Lenbok commented Mar 11, 2021

Lenbok commented Jun 20, 2021

Tomatower commented Jun 20, 2021

Lenbok commented Jun 20, 2021

ochafik commented Mar 12, 2022

Lenbok commented Mar 12, 2022

Added experimental language server #3635

Are you sure you want to change the base?

Added experimental language server #3635

Conversation

Tomatower commented Jan 27, 2021 • edited Loading

Tomatower commented Feb 2, 2021 • edited Loading

❎ Netbeans

State of integration

Quirks

Tomatower commented Feb 3, 2021 • edited Loading

✔️ Visual Studio Code

State of Integration

Tomatower commented Feb 3, 2021 • edited Loading

✔️ vim

State of Integration

Tomatower commented Feb 3, 2021 • edited Loading

✔️ emacs

State of Integration

Tomatower commented Feb 3, 2021 • edited Loading

✔️ atom

State of Integration

Tomatower commented Feb 9, 2021 • edited Loading

🟡 Netbeans (with plugin)

State of integration

Quirks

Lenbok commented Feb 11, 2021

Lenbok commented Feb 11, 2021

Tomatower commented Feb 11, 2021

Lenbok commented Feb 12, 2021

Lenbok commented Mar 6, 2021

Tomatower commented Mar 6, 2021

Lenbok commented Mar 10, 2021

Tomatower commented Mar 10, 2021

Lenbok commented Mar 11, 2021

Lenbok commented Jun 20, 2021

Tomatower commented Jun 20, 2021

Lenbok commented Jun 20, 2021

ochafik commented Mar 12, 2022

Lenbok commented Mar 12, 2022

Tomatower commented Jan 27, 2021 •

edited

Loading

Tomatower commented Feb 2, 2021 •

edited

Loading

Tomatower commented Feb 3, 2021 •

edited

Loading

Tomatower commented Feb 3, 2021 •

edited

Loading

Tomatower commented Feb 3, 2021 •

edited

Loading

Tomatower commented Feb 3, 2021 •

edited

Loading

Tomatower commented Feb 9, 2021 •

edited

Loading