Too broad pattern for Ansible Var Files results in spurious errors working with pandoc defaults file

[chrisgrieser]

I suspect that the problem is not with ansiblelint but rather with the schemastore (where I created an issue: SchemaStore/schemastore#2919 ), but in case the problem is indeed created by ansiblelint rather than the schemastore, I am also opening an issue here.

---

The problem: Apparently, schemas defined in this repo get pulled by the schemastore. The yaml-lsp, in turn, by default pulls yaml schemas from the schemastore and applies them to the yaml files I am working with.

Now this would be totally fine, if it wasn't that some schemas seem to have a pattern that is so broad that it also matches yaml files that aren't ansible files and results in tons of spurious errors there. In particular, I am working with pandoc defaults file, which must be stored in a directory called defaults, which matches the pattern defined in the ansible vars file schema. This results in the LSP treating completely unrelated pandoc defaults files as ansible files, leading to tons of spurious errors, which I, as a user cannot even really disable on my end.

It took me quite a bit of trail-following to figure out that an LSP error was actually created by a file in this repo. Since the broad pattern defaults/*.yml probably makes sense for this linter, I assume that the root problem is caused by the schemastore. But in case someone from this repo is responsible for pushing the schemas to the schema store, I am also reporting this here.

[ssbarnea]

@chrisgrieser That conflict may prove tricky to address as involves 4-5 repositories from multiple projects. AFAIK, the examples block from within the schema is used only our test suite, as nobody else is reading it.

First, I want to see a repository example where the pandoc yaml is incorrectly identified, as I need to see the full path of the file. Once we establish these, we might know how to narrow down the matching patterns and update:

• the examples in schema (this repo)
• the patterns in schemastore project
• possible YAML extension and Ansible extension too

I will try to help you, but I first need to example as I need to play a little bit with the pattern changes in order to update them.

[chrisgrieser]

thanks for the quick response!

So one of the files is in my dotfile repo., the repo-relative path being pandoc/defaults/md2pdf.yaml.

The errors disappear as soon as I rename the parent folder to anything other than "defaults". I also confirmed the behavior anywhere else: ~/Desktop/defaults/file.yaml also makes yaml-ls treat the file as an ansible file. Renaming the directory to ~/Desktop/foobar/file.yaml makes yaml-ls stop treating it as an ansible file.

However, while the pandoc data-dir may be placed anywhere, the subfolder name defaults is fixed:

All options for document conversion, including input and output files, can be set using a defaults file. The file will be searched for first in the working directory, and then in the defaults subdirectory of the user data directory (pandoc docs)

effectively, this means that the corresponding pandoc files have to placed in folder named defaults, and the every pandoc defaults file will always match the pattern **/defaults/*.yml

[ssbarnea]

That seems like an unfortunate decision on pandoc side, basically they will never be able to get good support for their yaml because it is not unique enough.

[chrisgrieser]

well, to be honest, I think the issue lies somewhere along the trail of schemastore and yaml-ls, since somewhere there they apply a schema for ansible files to all yaml files in "defaults" directory? Like, I literally do not even use ansible, and yet a get an ansible schema applied to my yaml files

I mean, basically any application that stores yaml files in such a directory will get spurious LSP errors, not just pandoc

[ssbarnea]

After talking about this issue in our team meeting, we decided to not do anything about it, mainly because we don't think there is something doable other than changing the configuration of your editor to force it to use a difference JSON Schema for these documents. You can even specify the schema in a comment.

You should also contact pandoc folks and ask them to consider switching to a different file-naming pattern (less generic) to avoid conflicts like this.

[chrisgrieser]

changing the configuration of your editor to force it to use a difference JSON Schema for these documents.

only problem with that solution is that there is no schema for pandoc default files available (I checked in the schema store), and yaml-ls in turn is only able to change the schema of a document manually, but not able to disable schemas for a file 🙈
redhat-developer/yaml-language-server#446

You should also contact pandoc folks and ask them to consider switching to a different file-naming pattern (less generic) to avoid conflicts like this.

yeah, I'll try that, but to be honest, I again don't blame them to choose a generic file-naming pattern, but rather the fact that somewhere between yaml-ls and the schemastore the schema gets applied too broadly.

[jgm]

basically they will never be able to get good support for their yaml because it is not unique enough.

Pandoc maintainer here. We're not particularly concerned with getting highlighting support for the YAML. But it's quite annoying that a generic name like defaults/*.yml gets automatically linted as ansible, leading users to get spurious recommendations. Ansible is squatting on a generic name. Something beyond that name should be required to trigger specific linting suggestions.

[ssbarnea]

@jgm Hi! The linter can be configured to treat specific default folders as something else than ansible-vars file. Still, this would require user configuration. Same applies for YAML extension for vscode and for Ansible extension for vscode.

In fact, due to the listing in schemastore, even other editors might recognize similar folders as Ansible.

This puts pandon project at at disadvantage, as it will keep receiving false bug reports about broken schema.

Not sure what we can do, especially as there is no specific change we can to narrow down ansible defaults files. I can go into details if you want but the idea is that defaults folder can be hosted either inside a an ansible role (any folder), or inside a folder with playbooks. It common to have a {project_root}/defaults/*.yml pattern.

There is a feature request at redhat-developer/vscode-yaml#401 for allowing setting the schema using the document itself. Still, this is not fixed, even if fixed it will likely not be supported by all editors and it is not part of YAML spec itself.

The way I see it is that the only escape route is to start using a more specific pattern for pandoc itself, like pandoc_defaults or pandoc/defaults (afaik, we could implement negating patterns, linter already supports them).

Ansible ecosystem is too big and established in order to even consider changes on this side. I suspect that the defaults folder may even be like 10 years old. That is only a practical observation, I don't mind advocating for changes, but when there is chance of success.

Maybe someone can do a little bit of research using https://sourcegraph.com/search to see how many of the files stored in defaults are pandon vs ansible around the the indexed base. Getting some data here might help us find a way to improve the current situation.

[jgm]

We can't really change either due to backwards compatibility. However, if on your end you're able to exclude cases of the form
.pandoc/defaults/*.{yml,yaml}
pandoc/defaults/*.{yml,yaml}
that would take care of it, I think.