[mpeg-OTspec] RE: [OpenType] RE: Proposal: deprecate ReqFeatureIndex

[Behdad Esfahbod]

Peter,

Can you please clarify how Microsoft implements ReqFeatureIndex?  In
particular, for Arabic and Indic shapers, is ReqFeatureIndex applied before
script-specific features or after?

IIUC, 'locl' is applied before script-specific features.  If ReqFeatureIndex
is applied after them, then it is providing functionality that is not easy to
replace right now.

behdad

On 08/22/2012 01:46 PM, Peter Constable wrote:
>  
> 
> I don’t think it’s necessary to nail down every script implementation spec to
> decide on this.
> 
>  
> 
> It seems clear what this mechanism does: specify a feature that must be
> applied when in the context of a given language system. And it also seems
> clear (to me, at least) as to essential aspects of how to interpret that: this
> should be used in conjunction with features that are not mandatory for the
> given script, and applied as to the entire run. You may have questions about
> how that feature and associated lookups should be processed in relation to
> other features and lookups, but as I’ve indicated I think that’s an orthogonal
> question that may exist independently.
> 
>  
> 
> I think the key question is whether there are scenarios in which this
> mechanism would be a useful complement (or supplement) to the ‘locl’ feature,
> and I think that can be considered here without first needing the last script
> implementation to be nailed down.
> 
>  
> 
> I don’t have a strong argument to say that this is a useful mechanism that
> should be kept. I just wouldn’t want to see it deprecated without first having
> had appropriate questions asked.
> 
>  
> 
>  
> 
> Peter
> 
>  
> 
>  
> 
> *From:*mpeg-OTspec at yahoogroups.com [mailto:mpeg-OTspec at yahoogroups.com] *On
> Behalf Of *Sairus Patel
> *Sent:* August 22, 2012 10:32 AM
> *To:* opentype-migration-list at indx.co.uk; mpeg-OTspec at yahoogroups.com
> *Subject:* [mpeg-OTspec] RE: [OpenType] RE: Proposal: deprecate ReqFeatureIndex
> 
>  
> 
>  
> 
> Peter,
> 
>> [...] a priori, nothing is determined in this regard.
>> If under analysis it turns out that the 'locl' feature is entirely adequate
> and that ReqFeatureIndex doesn't provide any other benefits for this scenario
> [...]
> 
> All of this ("nothing is determined," "If under analysis," etc) sounds like OT
> still hasn't come up with a layout model it's comfortable with. This may be
> the root of the issue, and the ReqFeatureIndex discussion has simply surfaced it.
> 
> The layout script specifications ("Script-specific Development" section at
> http://www.microsoft.com/typography/SpecificationsOverview.mspx) is the best
> we have, and they are completely under the control of Microsoft, as far as one
> can tell.
> 
> However, they are really an essential part of OT/OFF, inasmuch as vendors need
> to follow them in order to create fonts that actually work.
> 
> Wasn't there an attempt a few years ago to form some kind of committee to fill
> out the gaps in the MS layout script specifications? (I believe John Hudson
> was involved or expressed interest, IIRC.)
> 
> Would you suggest the ReqFeatureIndex deprecation be put on hold until the
> gaps have been addressed? If so, how would you propose the gaps be addressed?
> (Examples of gaps: need for a CJK layout spec, lack of mention of 'locl' in
> the standard scripts section.)
> 
> Thanks,
> Sairus
> 
> 
> -----Original Message-----
> From: listmaster at indx.co.uk <mailto:listmaster%40indx.co.uk>
> [mailto:listmaster at indx.co.uk <mailto:listmaster%40indx.co.uk>] On Behalf Of
> Peter Constable
> Sent: Tuesday, August 21, 2012 4:01 PM
> To: multiple.recipients.of.OpenType at inbound-smtp-1.corp.adobe.com
> <mailto:multiple.recipients.of.OpenType%40inbound-smtp-1.corp.adobe.com>
> Subject: RE: [OpenType] RE: Proposal: deprecate ReqFeatureIndex
> 
> Message from OpenType list:
> 
> 
> Sairus:
> 
> Wrt #2, I'm not sure I completely get the need. E.g., in creating a font
> specimen chart, if you're trying to capture all of the glyphs, then the most
> reliable approach would be to specify by glyph index; and if not, then it
> seems like the most useful thing would be to present things to end users in
> terms of the mechanisms that end users would normally use to control variations.
> 
> Wrt #3, I don't see an issue here: all that the field is indicating is that
> application of a particular feature is required; it doesn't say anything about
> ordering or how many times lookups are processed. Any ambiguity about how many
> times or in what order lookups may be processed is a completely independent
> issue. (It's also usually the case that lookups are not self-feeding and that
> repeated application is a problem only in relation to performance, but as
> stated, I think it's an unrelated issue.)
> 
> Wrt #1, you're right: the 'locl' feature can and should be used to get
> language-specific typography. But that does not necessarily imply that
> ReqFeatureIndex might not also have an appropriate role for this scenario.
> (E.g., perhaps there are ways the two are complementary -- a priori, nothing
> is determined in this regard.) If under analysis it turns out that the 'locl'
> feature is entirely adequate and that ReqFeatureIndex doesn't provide any
> other benefits for this scenario, then it may be reasonable to deprecate it.
> But your proposal did not discuss this.
> 
> 
> 
> Peter
> 
> 
> -----Original Message-----
> From: listmaster at indx.co.uk <mailto:listmaster%40indx.co.uk>
> [mailto:listmaster at indx.co.uk <mailto:listmaster%40indx.co.uk>] On Behalf Of
> Sairus Patel
> Sent: August 21, 2012 1:55 PM
> To: multiple recipients of OpenType
> Subject: RE: [OpenType] RE: Proposal: deprecate ReqFeatureIndex
> 
> Message from OpenType list:
> 
> 
> Peter,
> 
> Thanks for your comments.
> 
> The 'locl' feature is OpenType's way of expressing language-specific behavior
> (at least for localized forms). Many Adobe, Microsoft, and other fonts have
> been using it in this way for years.
> 
> ('ccmp' is another pretty-much-required feature that engines should almost
> always apply. The limitation is that 'locl' applies only to GSUB. If there is
> a need for it, someone could propose a GPOS sibling for it -- or extend its
> definition to say it applies to GSUB and/or GPOS. Of course, language-specific
> behavior can also be added to other features.)
> 
> This idea of a named feature that is documented as required or not in separate
> layout specifications is the model that OpenType has had in place for a while
> now. One advantage of this model is that a layout engine can choose to turn
> off specific features, including those deemed as "required" in the layout
> specifications, for expert users, e.g. users who may need to create font
> specimen charts or language educational material. (In fact, this was
> specifically the objection to the recent 'rclt' proposal; see recent thread
> "rclt feature -- minor concern"
> http://tech.groups.yahoo.com/group/mpeg-OTspec/message/803.)
> 
> So:
> 
> 1. ReqFeatureIndex isn’t needed to express language-specific behavior; there
> is an already-established model for that in OT, that of named features
> documented as required or not in separate layout specifications.
> 
> 2. ReqFeatureIndex can't be turned off, and layout engines need a way to turn
> even "required" features off for expert users. (At the very least, a new
> internal API would need to be added so that the layout engine can communicate
> "do/don’t apply any ReqFeatureIndices" to the guts of the GSUB/GPOS engine,
> which is inelegant and, given (1) above, unnecessary.)
> 
> 3. ReqFeatureIndex has the unpleasant property that it could be re-applied
> multiple times on the same glyph run when an engine applies features one by
> one in the prescribed order (e.g. for Arabic layout
> http://www.microsoft.com/typography/otfntdev/arabicot/features.aspx). This is
> inefficient and may even produce undesirable results if the required feature
> was constructed in certain ways.
> 
> We've discussed deprecating ReqFeatureIndex since 2010 on these lists, so I
> thought I'd move things forward and propose something.
> 
> ReqFeatureIndex (and the preceding "reserved" field in the LangSys table,
> LookupOrder) were invented before the current OT layout model, so it's not
> surprising that the LookupOrder field is indicated as "reserved". It's
> probably time to do the same for ReqFeatureIndex.
> 
> If someone would like to propose a specific amount of additional time to
> research this in, e.g. 3 more months, I'd be happy to put the proposal on hold
> for that amount of time. But as it stands right now, the ReqFeatureIndex model
> interferes with the already-established model and so something has to address
> this.
> 
> Best,
> Sairus
> 
> 
> -----Original Message-----
> From: listmaster at indx.co.uk <mailto:listmaster%40indx.co.uk>
> [mailto:listmaster at indx.co.uk <mailto:listmaster%40indx.co.uk>] On Behalf Of
> Peter Constable
> Sent: Tuesday, August 21, 2012 10:59 AM
> To: multiple.recipients.of.OpenType at inbound-smtp-1.corp.adobe.com
> <mailto:multiple.recipients.of.OpenType%40inbound-smtp-1.corp.adobe.com>
> Subject: [OpenType] RE: Proposal: deprecate ReqFeatureIndex
> 
> Message from OpenType list:
> 
> 
>>****** Attachments to this email message have been removed ******
> 
> The ReqFeatureIndex field is really inadequate for the general problem of
> forcing a shaping engine to apply certain features, such as 'rlig': as we
> know, there are several features that should always be mandatory. This field
> can only specify one, and those details should be documented in the feature
> registry and then hard coded into shaping engine implementations.
> 
> Now, there is a different potential use for that field: indicating a feature
> that may not be mandatory for the script generally, but that should always be
> applied for a particular language. (Note that the field is in a langsys table,
> not a script table.) Now, any examples I've seen of attested usage (_very_
> few) do not appear to be of that nature, and so I would not cite them as
> precedents. Also, even for this scenario, one could imagine a better data
> structure. For instance, it might have been useful if one could specify a
> feature and also an index for a type 3 (alternates) lookup, or to specify more
> than one feature. Even so, it can still be useful for this scenario in its
> present form by allowing a cvnn or ssnn feature to be specified.
> 
> Thus, even though it hasn't been used much (or, in those few cases, used
> well), it's not clear to me that it warrants deprecation. We're just now
> getting to a point in implementations at which langsys tables can actually
> become useful.
> 
> 
> Peter
> 
> From: mpeg-OTspec at yahoogroups.com <mailto:mpeg-OTspec%40yahoogroups.com>
> [mailto:mpeg-OTspec at yahoogroups.com <mailto:mpeg-OTspec%40yahoogroups.com>] On
> Behalf Of Sairus Patel
> Sent: August 13, 2012 3:20 PM
> To: opentype-migration-list at indx.co.uk
> <mailto:opentype-migration-list%40indx.co.uk>; mpeg-OTspec at yahoogroups.com
> <mailto:mpeg-OTspec%40yahoogroups.com>
> Subject: [mpeg-OTspec] Proposal: deprecate ReqFeatureIndex
> 
> 
> 
> I propose that this be considered for the 3rd edition of OFF that's being
> worked on right now.
> 
> --- Rationale:
> 
> Currently, each language system table in a GSUB or GPOS can specify the index
> of a "required" feature. A layout engine is always to apply this feature when
> performing layout in that language system.
> 
> However, this doesn't quite fit in a layout model wherein features are to be
> applied one at a time, and wherein certain feature tags are designated by a
> separate layout specification (and not in the font itself) as being required.
> For example, see MS's Arabic and Indic layout specifications at
> http://www.microsoft.com/typography/SpecificationsOverview.mspx.
> 
> Adobe's CoolType font engine (used by most Creative Suite products) removed
> support for ReqFeatureIndex in 2005, and FTE (Flash Text Engine) has never
> supported it.
> 
> Deprecating ReqFeatureIndex has come up a couple times on the OT and OFF
> lists, most recently Nov 29, 2011 (Subject "ReqFeatureIndex: obsolete?"), and
> I have not heard from anyone objecting to this. I have heard only supportive
> comments.
> 
> Thus, this proposal.
> 
> 
> --- Proposal:
> 
> { In section "Language System Table" at
> http://www.microsoft.com/typography/otspec/chapter2.htm, replace the following
> paragraphs: }
> 
> Optionally, a LangSys table may define a Required Feature Index
> (ReqFeatureIndex) to specify one feature as required within the context of a
> particular language system. For example, in the Cyrillic script, the Serbian
> language system always renders certain glyphs differently than the Russian
> language system.
> 
> Only one feature index value can be tagged as the ReqFeatureIndex. This is not
> a functional limitation, however, because the feature and lookup definitions
> in OpenType Layout are structured so that one feature table can reference many
> glyph substitution and positioning lookups. When no required features are
> defined, then the ReqFeatureIndex is set to 0xFFFF.
> 
> All other features are optional. For each optional feature, a zero-based index
> value references a record (FeatureRecord) in the FeatureRecord array, which is
> stored in a Feature List table (FeatureList). The feature indices themselves
> (excluding the ReqFeatureIndex) are stored in arbitrary order in the
> FeatureIndex array. The FeatureCount specifies the total number of features
> listed in the FeatureIndex array.
> 
> Features are specified in full in the FeatureList table, FeatureRecord, and
> Feature table, which are described later in this chapter. Example 2 at the end
> of this chapter shows a Script table, LangSysRecord, and LangSys table used
> for contextual positioning in the Arabic script.
> 
> { by (a modified version of the last two paragraphs above): }
> 
> For each feature in this table, a zero-based index value references a record
> (FeatureRecord) in the FeatureRecord array, which is stored in a Feature List
> table (FeatureList). The feature indices themselves are stored in arbitrary
> order in the FeatureIndex array. The FeatureCount specifies the total number
> of features listed in the FeatureIndex array.
> 
> Features are specified in full in the FeatureList table, FeatureRecord, and
> Feature table, which are described later in this chapter. Example 2 at the end
> of this chapter shows a Script table, LangSysRecord, and LangSys table used
> for locale-specific numeric forms in the Arabic script.
> 
> { In the entry for ReqFeatureIndex in the "LangSys table" table, replace the
> "Name" value with: }
> 
> Reserved
> 
> { and replace the "Description" value with: }
> 
> Set to 0xFFFF. (This used to be ReqFeatureIndex in some previous versions of
> the specification.)
> 
> { In the "Example 2" section further down the page, replace: }
> 
> Then the text-processing client uses a required OpenType Layout glyph
> substitution feature, defined in the Urdu LangSys table, to access the correct
> Urdu glyphs for the 4, 6, and 7 numerals.
> 
> { by: }
> 
> Then the text-processing client uses the Localized Forms ("locl") feature
> referenced in the Urdu LangSys table to access the correct Urdu glyphs for the
> 4, 6, and 7 numerals.
> 
> { In the table in "Example 2":
> 
> Under DefLangSys, replace the comment "ReqFeatureIndex, no required features"
> by "Reserved".
> 
> Under UrduLangSys, replace the ReqFeatureIndex line with: FFFF | 0xFFFF |
> Reserved
> 
> Under UrduLangSys, change the FeatureCount from 3 to 4
> 
> Under UrduLangSys, add an additional line: 0003 | 3 | FeatureIndex[3], "locl"
> feature (localized forms) }
> 
> Sairus
> 
> 
> 
> 
>>****** Attachments to this email message have been removed ******
> 
> 
> 
> List archive: http://www.indx.co.uk/biglistarchive/
> 
> subscribe: opentype-migration-sub at indx.co.uk
> <mailto:opentype-migration-sub%40indx.co.uk>
> unsubscribe: opentype-migration-unsub at indx.co.uk
> <mailto:opentype-migration-unsub%40indx.co.uk>
> messages: opentype-migration-list at indx.co.uk
> <mailto:opentype-migration-list%40indx.co.uk>
> 
> 
> 
> 
> 
> List archive: http://www.indx.co.uk/biglistarchive/
> 
> subscribe: opentype-migration-sub at indx.co.uk
> <mailto:opentype-migration-sub%40indx.co.uk>
> unsubscribe: opentype-migration-unsub at indx.co.uk
> <mailto:opentype-migration-unsub%40indx.co.uk>
> messages: opentype-migration-list at indx.co.uk
> <mailto:opentype-migration-list%40indx.co.uk>
> 
> 
> 
> 
> 
> 
> 
> 
> List archive: http://www.indx.co.uk/biglistarchive/
> 
> subscribe: opentype-migration-sub at indx.co.uk
> <mailto:opentype-migration-sub%40indx.co.uk>
> unsubscribe: opentype-migration-unsub at indx.co.uk
> <mailto:opentype-migration-unsub%40indx.co.uk>
> messages: opentype-migration-list at indx.co.uk
> <mailto:opentype-migration-list%40indx.co.uk>
> 
>