[MPEG-OTSPEC] lookup processing and 'chws', 'vchw'

Thu Aug 20 06:25:51 CEST 2020

Thanks very much, Ken

I hadn't ever noticed that detail in the 'kern' or 'vkrn' descriptions — probably I never read them in detail because I believed I knew what the features were for and how they worked without needing to read the details. 

The 'kern' feature predates OpenType, having been defined for TrueType Open; and the description for TTO doesn't mention that bit at all.

Here's a TTO snapshot:

https://web.archive.org/web/20140721071009/http:/www.microsoft.com/typography/tt/ttoreg.htm

I no longer have access to any OT versions prior to OT 1.4. In OT 1.4, the 'kern' description did include this.

https://docs.microsoft.com/en-us/typography/opentype/otspec140/features_ko

Evidently, this was added some time after TTO (i.e., no earlier than OT 1.0) and before OT 1.4.

I find it puzzling since it seems to me to be at odds with the specification for how lookups are to be processed (text from OTL Common Formats quoted below); and the OT spec and OFF nowhere (apart from these feature descriptions) allow for tailoring of the generalized description of lookup processing. Moreover, I strongly suspect that Microsoft's OTL processing code (developed prior to OT 1.4) doesn't make any feature-specific exceptions in lookup processing, and I'm pretty certain it wasn't revised for this any time since 2004; though I may be wrong about what was in place prior to 2004.

My main concern in asking about this is that it seems suspicious to have feature-specific exceptions in lookup processing, particularly given that the core spec doesn't seem to allow for it. That's not to say that the spec couldn't be changed to allow for it. But unless it is, this strikes me as the kind of thing that could lead to NON-interoperability of fonts / layout implementations. And in my mind, interoperability has always been a primary concern.

Peter

-----Original Message-----
From: Ken Lunde <ken.lunde at gmail.com> 
Sent: Wednesday, August 19, 2020 8:58 PM
To: Peter Constable <pgcon6 at msn.com>
Cc: MPEG OT Spec list (mpeg-otspec at lists.aau.at) <mpeg-otspec at lists.aau.at>
Subject: Re: [MPEG-OTSPEC] lookup processing and 'chws', 'vchw'

Peter and others,

As the person who originally submitted the 'chws' and 'vchw' features for registration, I think that I am best suited to answering your questions. Apologies for the delay in responding.

The background and use cases for these features can be found in the following CJK Type Blog articles that I published exactly one year apart, in 2018 and 2019:

https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblogs.adobe.com%2FCCJKType%2F2018%2F04%2Fcontextual-spacing.html&data=02%7C01%7C%7C6edd5f88d26a473915c308d844bd466c%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637334926999398251&sdata=ognn%2F2hIrRIJ4A9d0k3e1iWidg3uyfgQq9kQUg0nV2E%3D&reserved=0
https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblogs.adobe.com%2FCCJKType%2F2019%2F04%2Fcontextual-spacing-gpos-features-redux.html&data=02%7C01%7C%7C6edd5f88d26a473915c308d844bd466c%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637334926999398251&sdata=VTdqJQYtwWK6tQWf3CXU1UuaJoL6SzMeGvt8gGYG6aU%3D&reserved=0

The entire premise of these features is for them to behave like the 'kern' and 'vkrn' features, but to be completely independent from them, because it is entirely possible -- and somewhat expected -- for a font to include the 'kern', 'vkrn', 'chws', and 'vchw' features. The wording that you observed was therefore taken from the "Application interface" section of the 'kern' and 'vkrn' features, so it shouldn't be a stranger to implementers:

https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Ftypography%2Fopentype%2Fspec%2Ffeatures_ko%23tag-kern&data=02%7C01%7C%7C6edd5f88d26a473915c308d844bd466c%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637334926999398251&sdata=xDzfH5YUDm1AVNzvdxHai130%2BP8XyQS9dSpFPZFubfE%3D&reserved=0
https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Ftypography%2Fopentype%2Fspec%2Ffeatures_uz%23tag-vkrn&data=02%7C01%7C%7C6edd5f88d26a473915c308d844bd466c%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637334926999398251&sdata=oWek7atOua0KE7IIOjXg9oFCWSHo1okde27FbIGm8ZY%3D&reserved=0

I had planned to add these features to the fonts for the open source "Noto CJK" and "Source Han" Pan-CJK typefaces, but my involvement in those projects unexpectedly ended last September.

I still firmly believe that these features are completely worthwhile and helpful, but they require a vehicle in the form of a broadly-distributed set of fonts that support said features. In case someone wants to pick up the proverbial torch, I already blueprinted the font implementation for these features in the second of the two CJK Type Blog articles:

https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblogs.adobe.com%2FCCJKType%2Ffiles%2F2019%2F04%2Ffeatures.txt&data=02%7C01%7C%7C6edd5f88d26a473915c308d844bd466c%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637334926999398251&sdata=8k%2FZjqvXFpO3OjHlRZ4g%2ByyG5jmyIjUa2OfqKyyM1Jk%3D&reserved=0

As to non-font implementations, someone from W3C, such as Koji Ishii, will need to chime in with details. My involvement was strictly on the font-development side.

Regards...

-- Ken

> On Aug 19, 2020, at 08:55, Peter Constable <pgcon6 at msn.com> wrote:
> 
> While reviewing the content of Amendment 1 of ISO/IEC 14496-22:2019, I noticed the addition of the ‘chws’ and ‘vchw’ feature tags (which are not yet listed in the OT feature tag registry). This was proposed December 8, 2018 as Vlad was wrapping up content to go into Amendment 1. 
>  
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.aau.at%2Fpipermail%2Fmpeg-otspec%2F2018-December%2F000060.html&data=02%7C01%7C%7C6edd5f88d26a473915c308d844bd466c%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637334926999408257&sdata=Adhe4GDJYitNo01iAr7Tlp%2F1UveEJ3lg6PuzO%2FzqdLg%3D&reserved=0
>  
> December, of course, is not the best time of year to get eyes on things. On January 2, 2019, Vlad asked once more for comments.
>  
> “We need to finalize this and prepare an input contribution for the upcoming WG11 meeting (coming up on January 14-18, 2019); the submission deadline for input contribution is end of day Jan. 7, 2019 (Monday next week). If any of you have an objection to the proposed changes, or if you would like to suggest any additional changes - please respond on this email list no later than end of day Friday, January 4th.
> “As usual, silence is treated as approval so please speak up!”
>  
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.aau.at%2Fpipermail%2Fmpeg-otspec%2F2019-January%2F000066.html&data=02%7C01%7C%7C6edd5f88d26a473915c308d844bd466c%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637334926999408257&sdata=soDuDrjARmwab46HGjxgIFbtzZAc%2F6JaQ6cQhonoKmw%3D&reserved=0
>  
> Then on January 4, Vlad circulated his updated draft of Amd 1 and again solicited comments. Rob McKaughan (MS) was the only one to comment, raising a concern regarding use of ‘vert’ in GPOS and asking for time to investigate. Ken Lunde gave a reply, and nothing further was said. The changes went in and were balloted in ISO process.
>  
> One shortcoming of the process is that none of us font nerdy engineer types (Dave, you’re spot on!) are really engaged directly or through the various companies in the formal ISO balloting process. Effectively, I think that meant that the addition of ‘chws’ and ‘vchw’ went in without much review.
>  
> Now, in the descriptions for ‘chws’ and ‘vchw’, I noticed this:
>  
> “When using the [GPOS] type 2 lookup on a run of glyphs, it’s critical to remember to not consume the last glyph, but to keep it available as the first glyph in a subsequent run (this is a departure from normal lookup behavior).”
>  
> Now, a GPOS type 2 evaluates pairs of glyphs and can make positioning adjustments on one or both. If only the first glyph is acted on (the second is still matched for context), then when processing of a lookup subtable is complete the second glyph with be the next glyph in lookup subtable processing. But if the left glyph’s positioning is adjusted, then it is “consumed”—i.e., processing will advance to the following glyph. This is in accordance with the following:
>  
> “During text processing, a client applies a lookup to each glyph in the string before moving to the next lookup. A lookup is finished for a glyph after the client makes the substitution/positioning operation. To move to the “next” glyph, the client will typically skip all the glyphs that participated in the lookup operation: glyphs that were substituted/positioned as well as any other glyphs that formed a context for the operation. However, in the case of pair positioning operations (i.e., kerning), the “next” glyph in a sequence may be the second glyph of the positioned pair (see pair positioning lookup for details).”
>  
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Ftypography%2Fopentype%2Fspec%2Fchapter2%23lookupTbl&data=02%7C01%7C%7C6edd5f88d26a473915c308d844bd466c%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637334926999408257&sdata=OINhyBhXkO0EUFOJ7%2BD9DTrwdTrpI%2Bt2G14TRfv2c50%3D&reserved=0
> See also ISO/IEC 14496-22:2019, clause 6.2.5.
>  
> So, the “not consume” clause in the description of ‘chws’ and ‘vchw’ would seem to be consistent with the rest of the spec _provided that the GPOS type 2 lookups do not act on the second glyph_. However, the descriptions for these features don’t give any indication that that constraint should be followed.
>  
> Thus, I’d like to know if such a constraint on type 2 lookups used for ‘chws’/’vchw’ was intended but somehow left out of the descriptions (hence should be added)? Or was it really intended (as it appears on the surface) that type 2 lookups for ‘chws’/’vchw’ should be allowed to act on the second glyph but that clients should _exceptionally_ not consume the second glyph in this case?
>  
> If the latter, I’d like to know which products, or which text layout or shaping libraries supports this? Is this compatible with CoreText? With DWrite? With Harfbuzz? Other…?
>  
>  
>  
> Thanks
> Peter
>  
>  
>  
>  
> _______________________________________________
> mpeg-otspec mailing list
> mpeg-otspec at lists.aau.at
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.aau.at%2Fmailman%2Flistinfo%2Fmpeg-otspec&data=02%7C01%7C%7C6edd5f88d26a473915c308d844bd466c%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637334926999408257&sdata=q9KnIJWAbV%2BZ6m422PavQo2JY8aoYxoZjgoslfhuGDQ%3D&reserved=0