Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding git-modes to NonGNU ELPA #143

Closed
phikal opened this issue Aug 26, 2021 · 19 comments
Closed

Adding git-modes to NonGNU ELPA #143

phikal opened this issue Aug 26, 2021 · 19 comments

Comments

@phikal
Copy link

phikal commented Aug 26, 2021

Hi,

I am working on adding a few major modes to NonGNU ELPA, including the major modes packaged in git-modes. As you certainly know from having have added Magit, ELPA requires a version tag in the package header. My question is therefore, would it be possible to add such a tag and update it for future versions?

@tarsius
Copy link
Member

tarsius commented Aug 27, 2021

To quote git-modes.el's commentary:

;; Each mode is defined in its own library by the same name. Loading
;; `git-modes' causes all three libraries to be loaded, but you could
;; also load the libraries individually. On Melpa, the libraries are
;; distributed as separate packages.

It seems a bit silly that we distribute this as three packages on Melpa. We probably do that for "historic reasons".

If we add this to NonGNU Elpa, then we should probably switch to distributing just one package git-modes. I assume that someone who wants to use one of these modes would not mind having the others installed as well. In any case we should distribute it the same way on NGE and Melpa.

Switching from 3 to 1 package means that some existing users won't migrate for years to come, but I think that's okay because these packages aren't actually under active development. Mentioning @purcell in case he would like to comment.

@phikal
Copy link
Author

phikal commented Aug 29, 2021

That is fine, it will certainly be easier to package.

@purcell
Copy link

purcell commented Aug 31, 2021

Yeah, separate packages don't seem necessary at all in this case.

@phikal
Copy link
Author

phikal commented Aug 31, 2021

Ok, in that case only git-modes.el would require a Version attribute.

@tarsius
Copy link
Member

tarsius commented Aug 31, 2021

Please point me to the emacs-devel thread where you proposed to add git-modes. I searched but didn't find anything. Maybe you didn't mention it by name? Even so it might still make sense for me to reply to a message from a "add various packages" thread, instead of opening a new one.

@phikal
Copy link
Author

phikal commented Aug 31, 2021

I am currently using this thread to make suggestions, but have not formally proposed git-modes yet. The "elpa-packages" attachment in this message mentions git-modes, but my intention is to add git-modes along with a few other major modes in the near future.

@tarsius
Copy link
Member

tarsius commented Aug 31, 2021

my intention is to add git-modes along with a few other major modes in the near future.

If you don't mind I would prefer to add it myself. In fact I would imagine that many authors would prefer to do it themselves. Proposing packages you are not involved with is fine, some authors might need a little shove to get it done, or even appreciate it.

But adding a package is only a little part of the work involved in having a package in [Non]GNU Elpa. Unlike with Melpa authors also have to actively keep the elpa.git/nongnu.git repository and the upstream repository up-to-date and in-sync.

Once a package is in [Non]GNU Elpa, there are two official repositories where it is published. IMO the design decisions that have lead to this are problematic but that's what we have to deal with now.

I have looked at a subset packages on GNU Elpa and for more than one third (sic!) the two repositories have differed in ways that require manual intervention. Furthermore it is easy to do the merging "wrong" (and Stefan has done that a few times already). There is no documentation how to deal with diverging histories.

I plan to publish some documentation(/opinion piece) on what the problems are and how I would deal with them. But I wasn't planning to do it soon.

So I have mixed feelings about the recent rush to add more packages. More packages is good -- more packages whose two publications more or less instantly start to diverge... not so much. That's why I am not so happy about your push to add more third-party packages: if a package author makes the move themselves, then they probably have thought about the involved version control work and feel up to the task. If they are "encouraged" by a third-party and (falsely) assured that the involved vc work is trivial, then that increases the odds of the package quickly diverging.

@phikal
Copy link
Author

phikal commented Aug 31, 2021 via email

@tarsius
Copy link
Member

tarsius commented Aug 31, 2021

And I am bumping "write elpa.git documentation with examples of problematic state and concrete steps how to recover and avoid issues in the future" up on my todo list.

But the vague answer to most of your questions is "once the two repositories have diverged, then the automatic syncing does not work anymore". Also even if the repositories have not diverged yet, then syncing is only done in one direction, upstream to elpa.git. Many authors are not aware of that so they never pull any changes that (usually) Stefan has made to their packages in elpa.git, and Stefan doesn't have push rights to their upstream repositories so he cannot do anything about it either.

@phikal
Copy link
Author

phikal commented Aug 31, 2021 via email

@tarsius
Copy link
Member

tarsius commented Sep 4, 2021

I am a little sad throwing the 644,050 Melpa downloads statistic away. ☹️

@phikal
Copy link
Author

phikal commented Sep 4, 2021

Just to be sure, you don't intend to remove the git-modes from MELPA, right? I don't think enough people use NonGNU ELPA yet to legitimize moves like that.

@tarsius
Copy link
Member

tarsius commented Sep 4, 2021

No, I am gonna add git-modes, replacing the three separate packages.

@tarsius
Copy link
Member

tarsius commented Oct 18, 2021

For a moment I though my mind was going...

I remembered we had talked about this somewhere but couldn't find anything. After digging around in the Melpa issue tracker and emacs-devel for quite some time, I finally realized it must have been here. Piuuu oO

I'll try to do it soon, probably today.

tarsius added a commit to melpa/melpa that referenced this issue Oct 20, 2021
We now bundle all libraries in a single package, available from both
Melpa and NonGNU ELPA.  Previously each mode library was available as
a separate package on Melpa only.

This was discussed in magit/git-modes#143.
idcrook added a commit to idcrook/.emacs.d that referenced this issue Oct 23, 2021
[Adding git-modes to NonGNU ELPA · Issue #143 · magit/git-modes](magit/git-modes#143)
- melpa/melpa@cc3a0ce
@tarsius
Copy link
Member

tarsius commented Oct 31, 2021

@tarsius tarsius closed this as completed Oct 31, 2021
@quentinmit
Copy link

Something about package.el's resolver is very confused by this package rename. I'm no longer able to install helm-gitignore which has a dependency on gitignore-mode because Melpa claims gitignore-mode no longer exists. (Even though the web UI on Melpa redirects to git-modes.) Can you consider reverting the redirection part and just leaving the old versions available as individual packages? That way you don't break any existing dependencies.

@tarsius
Copy link
Member

tarsius commented Sep 24, 2022

Please open a pr at https://github.com/jupl/helm-gitignore to update the dependency.

@quentinmit
Copy link

Please open a pr at https://github.com/jupl/helm-gitignore to update the dependency.

helm-gitignore hasn't had a release or commit in 6 years. There's already a PR to update this dependency but I don't have much hope of a quick merge (jupl/helm-gitignore#6).

Nevertheless, this is still relevant for any other packages that also have a dependency on any of the split modes. Please consider a smoother migration path.

@tarsius
Copy link
Member

tarsius commented Sep 26, 2022

Please consider a smoother migration path.

The path you suggested leads to other issues, which is why we never do that on Melpa. If two packages provided the same feature, then a user might end up with both versions installed, possibly without even being aware of it (because, as would be the case here, one version would get pulled in as a dependency). Most often when that happens, it is because one package bundles a third-party library, here the situation is a bit different, but that doesn't change the fact that we never want conflicting packages.

Feature conflicts are a big problem, e.g., after a bug is fixed in the upstream version. A user might report the bug again and be told to updated, but after doing that the bug is still there because the other version gets loaded (due to the order of load-path). Such "impossible" bugs are very frustrating to debug.

helm-gitignore hasn't had a release or commit in 6 years. There's already a PR to update this dependency but I don't have much hope of a quick merge (jupl/helm-gitignore#6).

Still, the best course of action is to once more ping the author to merge. This package may be "done" but a change in a dependency can always happen, so maintaining such a package, means to respond to such changes.

Nevertheless, this is still relevant for any other packages that also have a dependency on any of the split modes.

I would assume that I checked for such packages before making the change but overlooked this package. Anyway, as part of my work on the Emacsmirror, I have tools that look for inconsistent metadata and if any is found I try to get these issues fixed. If they are not fixed, then the offending packages are eventually removed from the Emacsmirror and Melpa. An alternative is to move them to the Emacsorphanage, which I also maintain.

  • I am going once more try to get the maintainer to merge (and create a new release). I will also suggest to alternatively move the package to the orphanage.

  • At some point I will likely add additional checks to the Emacsmirror tooling to detect issues in Package-Requires. (That's not needed for the Emacsmirror itself because it builds the dependency based on require forms in each package, instead of relying on Package-Requires. But since this is also used to detect issues concerning Melpa/package.el, it makes sense to add that anyway.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants