[MPEG-OTSPEC] checksum / 4-byte table offset weasel wording

[Peter Constable]

If we don’t make that a requirement, then the stated algorithm for calculating a checksum must explicitly and unambiguously allow for data in which tables are not four-byte aligned. Otherwise software implementers (“consumers”) are faced with ambiguity.

From: Behdad Esfahbod <behdad at behdad.org>
Sent: Monday, August 31, 2020 10:36 AM
To: Peter Constable <pgcon6 at msn.com>
Cc: MPEG OT Spec list (mpeg-otspec at lists.aau.at) <mpeg-otspec at lists.aau.at>
Subject: Re: [MPEG-OTSPEC] checksum / 4-byte table offset weasel wording

I suggest we *recommend* that tables start at 4byte offsets, since that makes access faster on some hardware. But otherwise no requirement at all, since all implementations have to be prepared to handle non-aligned tables.

behdad
http://behdad.org/<https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fbehdad.org%2F&data=02%7C01%7C%7C0c63f7c7416e4cc9f99f08d84dd45c70%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637344921757739154&sdata=xdZzVqn%2Bb%2BOwMbnPKs2QIgjfYtAlrG9dLAeJdnoOOgo%3D&reserved=0>


On Mon, Aug 31, 2020 at 9:41 AM Peter Constable <pgcon6 at msn.com<mailto:pgcon6 at msn.com>> wrote:
Currently the OT spec / OFF as well as Apple’s spec are somewhat ambiguous wrt the need for top-level tables to begin at offsets that are integral multiples of four bytes.

Apple’s spec<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.apple.com%2Ffonts%2FTrueType-Reference-Manual%2FRM06%2FChap6.html&data=02%7C01%7C%7C0c63f7c7416e4cc9f99f08d84dd45c70%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637344921757739154&sdata=ermVJuT%2F4oXstF1DZsR3yNutdfZ2pm9R6yTwo8qqxU0%3D&reserved=0> gives the algorithm to calculate a checksum for a table, which sums 32-bit chunks. This doesn’t necessarily imply that table lengths must be multiples of four (it could consume the first one to three bytes of a following table) except in the case of the last table in the file—there’s no allowance in the algorithm for reaching EOF before reaching the table length. But allowing checksum for one table to count bytes from a following table would seem odd. It could alco create confusion when calculating a checksum for the file as a whole: Do overlapping bytes get counted twice?

The section on calculating checksums in the OT spec<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Ftypography%2Fopentype%2Fspec%2Fotff%23calculating-checksums&data=02%7C01%7C%7C0c63f7c7416e4cc9f99f08d84dd45c70%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637344921757749151&sdata=GU0CWCNElMkxRoE26oHHJ%2Biv9jhpJTnmDq3IonZMIV8%3D&reserved=0> and in OFF give the same algorithm, but add a following note:

“This function implies that the length of a table must be a multiple of four bytes. In fact, a font is not considered structurally well-formed without the correct padding. All tables must begin on four-byte boundaries, and any remaining space between tables is padded with zeros. The length of all tables should be recorded in the table record with their actual length (not their padded length).”

That seems like a clearly-stated requirement. But if four-byte alignment is a requirement, it shouldn’t be stated here as an implication of the checksum adjustment; it should be stated in the description of table records within the table directory.

But then in the Recommendations section<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Ftypography%2Fopentype%2Fspec%2Frecom%23table-alignment-and-length&data=02%7C01%7C%7C0c63f7c7416e4cc9f99f08d84dd45c70%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637344921757749151&sdata=dF2nHCQfXihSoS0omH6AD6C66yHgbGjWpC5HmglP5LU%3D&reserved=0> of OT/OFF, four-byte alignment is mentioned as a _recommendation_, not a requirement:


“All tables should be aligned to begin at offsets which are multiples of four bytes. While this is not required by the TrueType rasterizer, it does prevent ambiguous checksum calculations and greatly speeds table access on some processors.

“All tables should be recorded in the table directory with their actual length. To ensure that checksums are calculated correctly, it is suggested that tables begin on 32-bit boundaries. Any extra space after a table (and before the next 32-bit boundary) should be padded with zeros.”

This is all weasel wording. Either four-byte alignment (with zero padding as needed) should be required, and the checksum calculation be stated is now—easily. Or, four-byte alignment should not be required, but the description for calculation of checksums should be elaborated to cover edge cases unambiguously.

As allowing for the edge cases adds non-trivial complication, both in the description and in implementations, I’m inclined to make it a requirement that tables begin at four-byte-multiple offsets (and remove the weasel wording in the Recommendations section).

Either way, there would be one possible edge case that should be covered: the last table in a file, and how to calculate a checksum if the table length is not a multiple of four. My inclination would be to cover this —which would also cover the general case — by saying that, if a table length is not a multiple of four, then it must be padded with zero bytes to the next four-byte boundary.

Would this break any assumptions in current runtime or tool implementations to make this a clear requirement?



Thanks
Peter
_______________________________________________
mpeg-otspec mailing list
mpeg-otspec at lists.aau.at<mailto:mpeg-otspec at lists.aau.at>
https://lists.aau.at/mailman/listinfo/mpeg-otspec<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.aau.at%2Fmailman%2Flistinfo%2Fmpeg-otspec&data=02%7C01%7C%7C0c63f7c7416e4cc9f99f08d84dd45c70%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637344921757759143&sdata=P5b61S%2FFCc2s%2FsHVl%2FMX3uQnTs01CgaU%2FeEGvRG76O0%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.aau.at/pipermail/mpeg-otspec/attachments/20200831/9c1f1a89/attachment.html>