Too many tiny repositories

[emidoots]

From @slimsag on November 2, 2014 9:40

Something I've thought about for a while now, is that we have too many tiny repositories, which makes the project hard to track. Here is a complete list of all the repositories we currently have:

• audio
• audio-wav
• binpack
• clock
• dstarlite
• dstarlite-grid
• examples
• gfx
• gfx-window
• gfx-gl2
• keyboard
• lmath
• mouse
• native-freetype
• native-cp
• native-al
• native-glfw

These ones are unstable, i.e. no version 1 release yet:

• audio-flac
• tmx

These are used for the website and/or issue tracker (they must be kept):

• issues (issue tracker)
• semver (semantic versioning package)
• azul3d.github.io (website file hosting)
• cmd-webgen (website generator)
• appengine (website versioned package proxy, uses semver and serves azul3d.github.io files)

These are deprecated and we have recommended not to use them for a while now:

• chippy (replaced by GLFW)
• native-gles2 (replaced by Glow)
• thirdparty-resize (never intended to be used)
• cmd-glwrap (replaced by Glow)
• native-gl (replaced by Glow)
• cmd-azulfix (not deprecated, but probably not useful anymore.)

What can we do?

I fear that the problem will only grow worse as more packages are added. Before on Google Code, we had one single repository and it made it annoying to utilize single packages (what if I want to use audio but not anything else?), now we have an inverse problem: we have so many repositories it's hard to sort through them all or even watch them.

I've contacted GitHub support regarding the ability to "watch" an organization the same way you would any other user, and they said it's on their TODO but there is no ETA.

The Plan - audio

• Merge the audio repositories together, making the audio/wav and audio/flac packages become subpackages. This would be done in the upcoming version 2 of the audio package:

Before	After
azul3d.org/audio.v1	azul3d.org/audio.v2
azul3d.org/audio/wav.v2	azul3d.org/audio.v2/wav
azul3d.org/audio/flac.v0	azul3d.org/audio.v2/flac

The Plan - gfx

• Merge the gfx repositories together, making the gfx/window and gfx/gl2 packages become subpackages. This would be done in the upcoming version 2 of the gfx package:

Before	After
azul3d.org/gfx.v1	azul3d.org/gfx.v2
azul3d.org/gfx/window.v2	azul3d.org/gfx.v2/window
azul3d.org/gfx/gl2.v2	azul3d.org/gfx.v2/gl2
azul3d.org/clock.v1	azul3d.org/gfx.v2/clock

The Plan - dstarlite

• The dstarlite/grid package should be removed -- and the grid.Data type would be merged into the dstarlite package instead (224 LOC).

Before	After
azul3d.org/dstarlite.v1	azul3d.org/dstarlite.v2
azul3d.org/dstarlite/grid.v1	azul3d.org/dstarlite.v2

The Plan - native

Nothing changing yet.

native-glfw and native-al will most likely always exist.

native-freetype: maybe we can deprecate it in the future in favor of freetype-go. I want to wait until I have some text rendering stuff going to know for certain.

native-cp: might break with Go 1.5 GC changes, it uses an insane amount of callbacks. At the very least it will be a hateful kind of joy to update properly. Maybe a full rewrite in Go is in order? Who knows.

The Plan - deprecated packages

I do want to make sure that Azul3D has a nice amount of backwards compatibility. Since we moved to GitHub not a single API-incompatible change has been made. As far as I am aware of, a program wrote against Azul3D packages almost a year ago will still build fine today. I'd like to keep this mindset unless it's a serious problem.

Aftermath

In the aftermath, the repositories we have (excluding deprecated ones and website ones) should look like:

• audio (audio, audio/wav and audio/flac packages).
• binpack
• dstarlite (dstarlite package with grid support merged)
• examples
• gfx (gfx, gfx/window, gfx/clock, and gfx/gl2 packages)
• keyboard
• lmath
• mouse
• native-freetype
• native-cp
• native-al
• native-glfw
• tmx
• semver

Before any action is taken I'd like to hear your suggestions/concerns? Please let me know what you think!

Copied from original issue: azul3d-legacy/issues#33

[emidoots]

From @mewmew on November 2, 2014 17:32

In general I agree that the project feels a bit messy with so many repositories and sub-repositories. It's difficult to get a good overview and general idea of which packages are related. The main benefit of keeping each package contained within a dedicated repository is that it prevents unnecessary version bumping. If too many packages were placed in the same repo, we would have to bump the major version any time one of them made a backwards incompatible change. However, grouping strongly related packages into a single dedicated repository makes a lot of sense to me. Specifically for the audio packages it seems that if the audio interface changes, the flac and wav packages have to be updated to use the new interface. Therefore it makes sense to simply bump the major version once and update all strongly related packages. I can't really say if this applies to the gfx packages etc, as I'm mostly familiar with the audio packages.

All in all, I like the proposed plan. At least for the audio packages you have my +1 :)

P.S. thanks for referencing this issue from the audio issue, otherwise I may have overlooked it.

[emidoots]

Thank you for the feedback!

thanks for referencing this issue from the audio issue, otherwise I may have overlooked it.

Ironically, this right here is another reason that placing highly-coupled packages (the audio packages, the gfx packages) into one single repository makes sense. =)

Then all audio issues are in, the audio repository. New features (decoders/encoders), bug reports, feature requests, etc are all in one place. Rather than spread out across so many repositories.

EDIT: this also closes #28 I think.

[emidoots]

From @mdlayher on November 3, 2014 15:21

👍 for audio as well. It will make it much easier to discover when new functionality (such as a decoder) is added.

[emidoots]

In the spirit of making things concise and understandable: For each repository that is removed I am creating a moved branch and removing all the files in that separate branch.

I am also adding a README.md talking about where the repository has moved to.

You can see this in effect already here and here for instance.

(Of course, the actual code still exists in the other branches but I think this makes it obvious that it's no longer where the code lives.)

[emidoots]

Since the azul3d.org/clock.v1 package is purposely just for graphics (mostly frame-rate statistics) it will also be moved into the gfx repository. I've updated the original post to reflect this.

[emidoots]

Fixed/merged as part of #1