Flexible font define in style

[alanchenboy]

Design Proposal: Flexible font define in style

Motivation

MapLibre inherits the font pbf design from Mapbox, this is a limit to rendering complex text. We must let the core reach the font file to finish shaping and rendering complex text like Indic, Khmer, Burmese, etc.

Proposed Change

The design looks like CSS unicode-range with font-face,

"font-faces": [
      {
          "text-font": ["Noto Sans Regular"],
          "font-files": [
              {
                  "url": "https://alanchenboy.github.io/harfbuzzResource/fonts/Noto%20Sans%20Regular/khmer.ttf",
                  "unicode-range": [
                      "U+1780-17FF"
                  ]
              },
              {
                  "url":"https://alanchenboy.github.io/harfbuzzResource/fonts/Noto%20Sans%20Regular/myanmar.ttf",
                  "unicode-range": [
                      "U+1000-109F", "U+A9E0-0xA9FF", "U+AA60-0xAA7F"
                  ]
              },
              {
                  "url":"https://alanchenboy.github.io/harfbuzzResource/fonts/Noto%20Sans%20Regular/devanagari.ttf",
                  "unicode-range": [
                      "U+0900-097F", "U+A8E0-0xA8FF"
                  ]
              }
              
          ]
      }
  ],

full style:
https://alanchenboy.github.io/harfbuzzResource/hindi.json

API Modifications

A new feature for complex font rendering, it's transparent for the map users.

Here is an example of how to render complex text on the native side.
maplibre/maplibre-native#1439.

Migration Plan and Compatibility

N/A

Rejected Alternatives

N/A

[HarelM]

Thanks for taking the time to open this design proposal.
Can you clarify the relation between this property and glyphs? It is not clear when one should be selected over the other.
Also, will using this require some configuration in the rendering part? special build? a plugin?
We don't have other style properties that has external requirements in order to work as far as I know...

[louwers]

We would use Harfbuzz for text shaping in MapLibre Native by default, so no special plugins would be required.

[HarelM]

Wouldn't this require a special build? Or is the default build comes with Harfbuzz and you can reduce build size if you need, but this would be the special build - i.e without it?

[louwers]

The latter: we would enable it by default, although it would be possible to drop out if you have binary size constraints and you make your own builds. The default build is the 'kitchen sink' build. Most users don't care that much about binary size.

[wipfli]

Thank you for the proposal @alanchenboy. Can we add to the proposal a way to load glyph SDFs from a server?

When rasterizing glyphs on the client with TinySDF, i.e., make gray-scale image then convert to SDF with Felzenszwalb/Huttenlocher distance transform, the visual quality is lower than when using font maker or similar which operates directly on Bezier curves to get the SDFs...

[alanchenboy]

This approach means that if a font is defined here by font-face, then the selected Unicode point will be shaped with Harfbuzz and then rasterized with FreeType.
In addition to rendering complex fonts, this also has the implicit benefit of saving traffic by the font files actually smaller than the pre-rendered pbf files.

[wipfli]

Exactly! Additionally, if we have an API to define server-generated SDF glyphs we can benefit from:

• Clients MapLibre GL JS and MapLibre Native do not have to use FreeType
• Bezier-based SDF generation which is more precision and looks better than TinySDF-based SDF generation

The API could maybe look something like this:

"font-faces": [
      {
          "text-font": ["Noto Sans Regular"],
          "font-files": [
              {
                  "url": "https://alanchenboy.github.io/harfbuzzResource/fonts/Noto%20Sans%20Regular/khmer.ttf",
                  "unicode-range": [
                      "U+1780-17FF"
                  ],
                  "glyphs": "https://alanchenboy.github.io/harfbuzzResource/glyphs//Noto%20Sans%20Regular/khmer/{range}.pbf"
              },
              {
                  "url":"https://alanchenboy.github.io/harfbuzzResource/fonts/Noto%20Sans%20Regular/myanmar.ttf",
                  "unicode-range": [
                      "U+1000-109F", "U+A9E0-0xA9FF", "U+AA60-0xAA7F"
                  ],
                  "glyphs": "https://alanchenboy.github.io/harfbuzzResource/glyphs//Noto%20Sans%20Regular/myanmar/{range}.pbf"
              },
              {
                  "url":"https://alanchenboy.github.io/harfbuzzResource/fonts/Noto%20Sans%20Regular/devanagari.ttf",
                  "unicode-range": [
                      "U+0900-097F", "U+A8E0-0xA8FF"
                  ],
                  "glyphs": "https://alanchenboy.github.io/harfbuzzResource/glyphs//Noto%20Sans%20Regular/devanagari/{range}.pbf"
              }
              
          ]
      }
  ],

Some fonts have several hundred glyphs so it might make sense to split the glyphs up into ranges of 256 like we do with normal MapLibre fonts. The difference to normal fonts would be that the index is actually the glyph index from the font .ttf file and not simply the unicode codepoint.

[1ec5]

Bezier-based SDF generation which is more precision and looks better than TinySDF-based SDF generation

FYI, maplibre/maplibre-gl-js#4564 would lean more heavily on TinySDF, and I have a half-baked idea in maplibre/maplibre-gl-js#4564 (comment) to use it exclusively if the top-level glyphs property is missing (which is currently an error). The idea would be to rely on Web Fonts (or the native equivalent) for any custom fonts, without requiring any MapLibre-specific server-side changes.

[ianthetechie]

A few thoughts from me as we discussed this on the MapLibre Eastern call a few days ago.

Opt-in / not breaking backward compatibility

The only major concern I remember during the Eastern call was ensuring we don't break existing styles. Meaning, it should be possible to add this behavior to existing styles, so that renderers which support it can upgrade, but older renderers will still be able to get the glyphs as before. Without this property, I doubt it will get wide adoption, as commercial tile providers are unlikely to publish a style that doesn't preserve backward compatibility.

I think this is fine as-is. I assume the behavior would be along the lines of "if we have font-faces for the font and the code points are from a matched range, use that. If loading fails or there are no matches, fall back to glyphs. Else fail gracefully."

Supporting externally generated glyphs

In my opinion, how the glyphs are generated is an implementation detail. We could swap out the precise implementation on native. We just need to ensure that the style spec is flexible enough.

That said, I don't at all mind @wipfli's proposal to add glyphs here in case you wanted to have those pre-generated externally. Some follow-ups from this:

1. I guess url and glyphs would both be optional; you just need to provide (at least) one? What happens if you have both from a spec perspective?
2. I suggest renaming to font_url and glyphs_url or something if the community decides to add both.

Valid URLs

Would this would work with local URLs (native only, I assume?). I could see wanting to ship a font file or glyphs locally on mobile.

unicode-range format

I'm guessing this was just an oversight, but some example ranges above were "U+0900-097F", "U+A8E0-0xA8FF". The first one looks reasonable, but the second one introduces a 0x hex prefix at the start of the upper bound, and I'm not sure why.

It's too bad JSON doesn't support hexadecimal numbers, but alas... The following seem logical enough to me to standardize on (in rough order of personal preference; discussion welcome):

A: Ranges as pairs with EITHER numbers or hex strings (no U+ prefix)

"unicode-range": [
    [4096, "109F"], ["A9E0", "A9FF"], ["AA60", "AA7F"]
]

B: Ranges as a hex range string (no prefix)

"unicode-range": [
    "1000-109F", "A9E0-A9FF", "AA60-AA7F"
]

C: Ranges as a hex range string (single prefix)

"unicode-range": [
    "U+1000-109F", "U+A9E0-A9FF", "U+AA60-AA7F"
]

[HarelM]

How would this proposal work for native?
I know @1ec5 is working on a branch to support this in web, but I want to understand how can this use native fonts from the OS if possible for native?
I think the range part is important to define, but I'm good with most of the solutions above, I prefer the array with two items in it (start and end as separate array entries), also it should probably be called unicode-ranges (plural).
Besides that, aligning to CSS definitions is a good idea as it will create clarity from people familiar with this area and would like to apply the same idea here.

[1ec5]

The only major concern I remember during the Eastern call was ensuring we don't break existing styles.

I believe this was in response to the approach I started with maplibre/maplibre-gl-js#4564 (comment) and continued in 1ec5/maplibre-gl-js#1. Graceful degradation isn’t the only way to achieve the goal of maintaining compatibility with existing stylesheets. My approach relies on relaxing an existing “optional” requirement in the style specification.

In order to adopt extended TinySDF, a style author would intentionally break compatibility with existing versions of MapLibre on all platforms by removing the glyphs property from the stylesheet. Similar to incrementing the version, a well-formed client would recognize upfront whether it’s compatible with the stylesheet. However, the style will continue to load, just without any text. This is about as severe as if you author a stylesheet that uses a new feature such as terrain that an older client doesn’t support.

If we really want, we could implement a fallback strategy on platforms that don’t support extended TinySDF to fall back to a last resort fontstack, either bundled or distributed from maplibre.org. But I don’t think this is necessary, because omitting glyphs will be a very intentional step on the part of the style author. Doing so can come with caveats. You could even think of TinySDF as a fallback strategy for when glyphs is missing, as I did in maplibre/maplibre-gl-js#4564.

The alternative would be that style authors have to continue specifying glyph PBFs that they have to generate but hope will never be used. This will become a much larger burden on developers once we expand character support beyond the Basic Multilingual Plane. I don’t think developers will see this as a good tradeoff.

how can this use native fonts from the OS if possible for native?

This is infeasible on the Web and unlikely to ever be feasible. Browser vendors are trying to lock down access to native fonts to prevent fingerprinting. The experimental Local Font Access API is only supported by Chromium and requires user permissions. I don’t think MapLibre should require the user to give privacy-related permissions just to view a map. More likely, we’d continue to improve upon the TinySDF technique on the Web to the point that it’s closer to true native text rendering on the native platforms using Harfbuzz.

[HarelM]

I was asking about using OS fonts on native devices using the maplibre-native implantation and how would this spec proposal cater for it. Downloading fonts is less likely for native apps as far as I know (although not my expertise).

[1ec5]

I think it’s much more feasible on native platforms (though you may need to perform expensive reconstructive surgery). It would be inherently native-only, permanently, as opposed to the situation with extended TinySDF, which would only be a temporary situation so that we wouldn’t have to synchronize releases across all three platforms.

I suspect direct system font access is kind of redundant on iOS anyways, since you can already get the vectors of the glyph if you really want. And both Android and iOS now provide direct native access to text shaping.

[1ec5]

The other aspect of direct font access is licensing. Mapbox’s original idea behind glyph PBFs is that they’re just collections of screenshots of individual letters, which under U.S. law are ineligible for copyright protection. As soon as you depend on actual font data, the font’s license kicks in. Fortunately, these days, there’s a wealth of fonts licensed for distribution as Web fonts (think Google Fonts), while the landscape for “desktop” fonts is more restrictive. Presumably system fonts are fair game, because you don’t have to embed anything, but then you’re at the mercy of the device’s font selection, which can be highly variable even on iOS, let alone Android.

[HarelM]

I'm not sure I'm following to be honest.
If the defined spec change in this thread is only relevant for web and can't be applied to native, we should consider adding options to map construction in web and not change the spec.
If this is applicable to native, how would this work in theory, can native use tinysdf as well? Does this mean native will download the file and apply it to the map?

[louwers]

@HarelM I agree, the spec should be cross-platform and should not have web-specific or native-specific extensions.

Would this would work with local URLs (native only, I assume?). I could see wanting to ship a font file or glyphs locally on mobile.

MapLibre Native is able handle asset:// and file:// URLs. Although as @1ec5 says default system fonts are inaccessible on iOS. You would need to bundle a font with your app (or download it to the filesystem).

If this is applicable to native, how would this work in theory, can Native use tinysdf as well? Does this mean native will download the file and apply it to the map?

Native is already using TinySDF for CJK characters. But the PR from @alanchenboy implements rasterization using FreeType.

---

So this may be a stupid suggestion, I don't know much about the complexities of font rendering. But can we push the complexity to the server and update the glyphs template?

{
  "glyphs": "https://maplibre.org/font/{fontstack}/{range}.{font_format}"
}

For the {font_format} MapLibre GL JS can substitute .pbf to request generated SDFs (or ignore the glyphs property and use TinySDF when configured with some map property), MapLibre Native can also request a .ttf.

When a .ttf is requested {range} would be the unicode block containing the character that is needed. Maybe instead of a numeric range we can request the official block name, i.e. https://maplibre.org/font/Noto%20Sans/Khmer.ttf

instead of https://maplibre.org/font/Noto%20Sans/6016-6143.ttf or https://maplibre.org/font/Noto%20Sans/1780-17FF.ttf.

[1ec5]

Although as @1ec5 says default system fonts are inaccessible on iOS. You would need to bundle a font with your app (or download it to the filesystem).

Right, I think this would be a limitation of anything that needs to access the actual font data. By default, on every platform, the TinySDF-based local font rendering mechanism for CJK only needs rasterized characters and metrics, which the standard library already provides. Extended TinySDF wouldn’t change much about this; the main difference is just how we’d segment the string to send to TinySDF.

Incidentally, the existing CJK mechanism on the native platforms relies on a separate configuration file that overrides what’s in the stylesheet, equivalent to the map constructor option suggested by @HarelM. Putting the option in the style specification would’ve posed a barrier to some cross-platform applications: developers generally want to use system-provided CJK fonts rather than bundling huge custom fonts, but each platform comes with a different set of CJK fonts.