Updates | Frédéric Wang2018-07-12T15:13:58+02:00http://frederic-wang.fr/Frédéric WangJekyll v3.1.6Review of Igalia's Web Platform activities (H1 2018)2018-07-10T00:00:00+02:00http://frederic-wang.fr/review-of-igalia-s-web-platform-activities-H1-2018<p>This is the semiyearly report to let people know a bit more about Igalia’s activities around <a href="https://en.wikipedia.org/wiki/Web_platform">the Web Platform</a>,
focusing on the activity of the first semester of year 2018.</p>
<h2 id="projects">Projects</h2>
<h3 id="javascript">Javascript</h3>
<p>Igalia has proposed and developed the specification for <a href="https://github.com/tc39/proposal-bigint/">BigInt</a>, enabling math on arbitrary-sized integers in JavaScript. Igalia has been developing implementations in <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1366287">SpiderMonkey</a> and <a href="https://bugs.webkit.org/show_bug.cgi?id=179001">JSC</a>, where core patches have landed. Chrome and Node.js shipped implementations of BigInt, and the proposal is at Stage 3 in TC39.</p>
<p>Igalia is also continuing to develop several features for JavaScript classes, including <a href="github.com/tc39/proposal-class-fields">class fields</a>. We developed a prototype implementation of class fields in JSC. We have maintained Stage 3 in TC39 for our specification of class features, including static variants.</p>
<p>We also participated to <a href="https://webassembly.org/">WebAssembly</a> (now at First Public Working Draft) and <a href="https://github.com/tc39/proposals/blob/master/ecma402/README.md">internationalization features</a> for new features such as Intl.RelativeTimeFormat (currently at Stage 3).</p>
<p>Finally, we have written more tests for JS language features, performed maintenance and optimization and participated to other spec discussions at TC39. Among performance optimizations, we have contributed a significant optimization to Promise performance to V8.</p>
<h3 id="accessibility">Accessibility</h3>
<p>Igalia has continued the standardization effort at the W3C. We are pleased to
announce that the following milestones have been reached:</p>
<ul>
<li><a href="https://www.w3.org/TR/graphics-aria-1.0/">WAI-ARIA Graphics Module</a> and <a href="https://www.w3.org/TR/graphics-aam-1.0/">Graphics Accessibility API Mappings</a> moved to proposed recommendation.</li>
<li><a href="https://www.w3.org/TR/accname-1.1/">Accessible Name and Description Computation</a> moved to Candidate Recommendation.</li>
</ul>
<p>A new <a href="https://www.w3.org/2015/10/aria-charter.html">charter</a> for the ARIA WG as well as drafts for <a href="https://w3c.github.io/aria/">ARIA 1.2</a> and <a href="https://w3c.github.io/core-aam/">Core Accessibility API Mappings 1.2</a> are in preparation and are expected to be published this summer.</p>
<p>On the development side, we implemented new ARIA features and fixed several bugs in WebKit and Gecko. We have refined platform-specific tools that are needed to automate accessibility Web Platform Tests (examine the accessibility tree, obtain information about accessible objects, listen for accessibility events, etc) and hope we will be able to integrate them in Web Platform Tests. Finally we continued maintenance of the <a href="https://help.gnome.org/users/orca/stable/">Orca</a> screen reader, in particular fixing some accessibility-event-flood issues in Caja and Nautilus that had significant impact on Orca users.</p>
<h3 id="web-platform-predictability">Web Platform Predictability</h3>
<p>Thanks to support from <a href="http://www.bloomberg.com/company/">Bloomberg</a>, we were
able to improve interoperability for various Editing/Selection use cases.
For example when <a href="https://bugs.chromium.org/p/chromium/issues/detail?id=731000">using backspace to delete text content just after a table</a> (<a href="https://github.com/w3c/editing/issues/164">W3C issue</a>) or <a href="https://bugs.chromium.org/p/chromium/issues/detail?id=731913">deleting a list item inside a content cell</a>.</p>
<p>We were also pleased to continue our collaboration with <a href="https://www.ampproject.org/">the AMP project</a>.
They provide us a list of bugs and enhancement requests
(mostly for the WebKit iOS port) with concrete use cases and repro cases.
We check the status and plans in WebKit, do
debugging/analysis and of course actually submit patches to address
the issues. That’s not always easy (e.g. when it is touching proprietary code
or requires to find some specific reviewers) but at least we make discussions move
forward. The topics are very diverse, it can be about
<a href="https://bugs.webkit.org/show_bug.cgi?id=186593">MessageChannel API</a>,
<a href="https://bugs.webkit.org/show_bug.cgi?id=106133">CSSOM View</a>,
<a href="https://bugs.webkit.org/show_bug.cgi?id=177684">CSS transitions</a>,
<a href="https://bugs.webkit.org/show_bug.cgi?id=170784">CSS animations</a>,
<a href="https://bugs.webkit.org/show_bug.cgi?id=149264">iOS frame scrolling</a>
<a href="https://bugs.webkit.org/show_bug.cgi?id=183586">custom elements</a> or
<a href="https://bugs.webkit.org/show_bug.cgi?id=165627">navigating special links</a>
and many others.</p>
<p>In general, our projects are always a good opportunity to write new
<a href="https://github.com/web-platform-tests/wpt">Web Platform Tests</a> import them
in WebKit/Chromium/Mozilla or improve the testing infrastructure. We have been able
to work on tests for several specifications we work on.</p>
<h3 id="css">CSS</h3>
<p>Thanks to support from <a href="http://www.bloomberg.com/company/">Bloomberg</a> we’ve been pursuing our activities around CSS:</p>
<ul>
<li>CSS Grid Layout: we <a href="https://blogs.igalia.com/mrego/2018/04/04/getting-rid-of-grid-prefix-on-css-grid-layout-gutter-properties/">removed “grid-“ prefix from gutter properties</a> and added support percentage to column-gap property in Multicolumn. We worked on <a href="https://blogs.igalia.com/jfernandez/2017/05/03/can-i-use-css-box-alignment/">baseline alignment</a>. Finally we are now improving the memory structures of the implementation in order to make bigger grids possible.</li>
<li>Composition underline is no longer always black in Chromium, it uses now the <a href="https://twitter.com/regocas/status/978638089781465088">current text color for by default</a></li>
<li>CSS Containment: We updated Chromium implementation to align with the spec and fixed several bugs.</li>
<li>CSS Text 3: We started to implement the new <a href="https://drafts.csswg.org/css-text-3/#valdef-overflow-wrap-break-spaces">CSS value ‘break-spaces’</a></li>
</ul>
<p>We also got more involved in the CSS Working Group, in particular participating to the <a href="https://blogs.igalia.com/mrego/2018/04/16/csswg-f2f-berlin-2018/">face-to-face meeting in Berlin</a> and will attend <a href="https://www.w3.org/wiki/TPAC/2018">TPAC’s meeting in October</a>.</p>
<h3 id="webkit">WebKit</h3>
<p>We have also continued improving the web platform implementation of some Linux ports of WebKit (namely <a href="https://webkitgtk.org/">GTK</a> and <a href="https://www.igalia.com/wpe/">WPE</a>). A lot of this work was possible thanks to the financial support of <a href="https://www.metrological.com/index.html">Metrological</a>.</p>
<ul>
<li>We improved <a href="https://developer.mozilla.org/en-US/docs/Web/WebDriver">Web Driver</a> support and made it work with WPE too.</li>
<li><a href="https://developer.mozilla.org/en-US/docs/Web/API/Web_Animations_API">Web Animations</a> are now enabled.</li>
<li><a href="https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API">Service Workers</a> are now <a href="https://bugs.webkit.org/show_bug.cgi?id=178576">enabled behind an experimental features flag</a>.</li>
<li><a href="https://bugs.webkit.org/show_bug.cgi?id=182424">We implemented most</a> of the missing functionality of <a href="https://developer.mozilla.org/en-US/docs/Web/API/ImageBitmap">ImageBitmap</a>.</li>
<li><a href="https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas">OffscreenCanvas</a> is now <a href="https://bugs.webkit.org/show_bug.cgi?id=183720">mostly implemented</a>, but pending review.</li>
<li><a href="https://github.com/web-platform-tests/wpt">Web Platform Tests</a> integration: We landed a patch to <a href="https://bugs.webkit.org/show_bug.cgi?id=183356">make possible to run tests against the official Web Platform Tests repository</a>.</li>
<li><a href="https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API">WebRTC</a>: We made the raspberry pi work with Firefox and Chrome properly and started pushing upstream all the branch. We have already upstreamed all the changes required in the current WebKit classes, the libwebrtc compilation and the getUserMedia support. We have two main pending patches: PeerConnection support and EnumerateDevices support.</li>
<li><a href="https://developer.mozilla.org/en-US/docs/Web/API/Media_Source_Extensions_API">Media Source Extensions</a>: We refined our implementation and fixed several bugs ; dealing with the corresponding issues on the gstreamer side.</li>
<li><a href="https://developer.mozilla.org/en-US/docs/Web/API/WebVR_API">WebVR</a>: Most of the API (85%) is now implemented using OpenVR and we can already get the <a href="https://www.youtube.com/watch?v=QS5uJdqvuD8">tracking information</a>.</li>
</ul>
<h2 id="other-activities">Other activities</h2>
<h3 id="preparation-of-web-engines-hackfest-2018">Preparation of Web Engines Hackfest 2018</h3>
<p>Igalia has been organizing and hosting the <a href="https://webengineshackfest.org/">Web Engines Hackfest</a> since 2009, a three days event where Web Platform developers can meet, discuss and work together. We are still working on the list of invited, <a href="https://webengineshackfest.org/#sponsors">sponsors</a> and talks but you can already save the date: It will happen from 1st to 3rd of October in A Coruña!</p>
<h3 id="new-igalians">New Igalians</h3>
<p>This semester, new developers have joined Igalia to pursue the Web platform effort:</p>
<ul>
<li>
<p><a href="https://www.igalia.com/igalia-247/igalian/item/rbuis/">Rob Buis</a>, a Dutch developer currently living in Germany. He is a well-known member of the Chromium community and is currently helping on the web platform implementation in WebKit.</p>
</li>
<li>
<p><a href="https://www.igalia.com/igalia-247/igalian/item/qzhang/">Qiuyi Zhang (Joyee)</a>, based in China is a prominent member of the Node.js community who is now also assisting our compilers team on V8 developments.</p>
</li>
<li>
<p><a href="https://www.igalia.com/igalia-247/igalian/item/dinfuehr/">Dominik Infuer</a>, an Austrian specialist in compilers and programming language implementation who is currently helping on our JSC effort.</p>
</li>
</ul>
<h3 id="coding-experience-programs">Coding Experience Programs</h3>
<p>Two students have started a <a href="https://www.igalia.com/about-us/coding-experience">coding experience program</a> some weeks ago:</p>
<ul>
<li>
<p>Oriol Brufau, a recent graduate in math from Spain who has been an active collaborator of the CSS Working Group and a contributor to the Mozilla project. He is working on the CSS Logical Properties and Values specification, implementing it in Chromium implementation.</p>
</li>
<li>
<p>Darshan Kadu, a computer science student from India, who contributed to GIMP and Blender. He is working on Web Platform Tests with focus on WebKit’s infrastructure and the GTK & WPE ports in particular.</p>
</li>
</ul>
<p>Additionally, <a href="https://github.com/caiolima">Caio Lima</a> is continuing his coding experience in Igalia and is among other things working on implementing BigInt in JSC.</p>
<h2 id="conclusion">Conclusion</h2>
<p>Thank you for reading this blog post and we look forward to more work on the web platform this semester!</p>
Review of Igalia's Web Platform activities (H2 2017)2018-01-12T00:00:00+01:00http://frederic-wang.fr/review-of-igalia-s-web-platform-activities-H2-2017<p>Last september, I published a first <a href="http://frederic-wang.fr/review-of-igalia-s-web-platform-activities-H1-2017.html">blog post</a> to let people know a bit more about Igalia’s activities around the Web platform, with a plan to repeat such a review each semester. The present blog post focuses on the activity of the second semester of 2017.</p>
<h2 id="accessibility">Accessibility</h2>
<p>As part of <a href="https://www.igalia.com/nc/igalia-247/news/item/our-commitment-to-diversity-and-inclusion">Igalia’s commitment to diversity and inclusion</a>, we continue our effort to standardize and implement accessibility technologies. More specifically, Igalian <a href="https://www.igalia.com/nc/igalia-247/igalian/item/jdiggs/">Joanmarie Diggs</a> continues to serve as chair of the W3C’s <a href="https://www.w3.org/WAI/ARIA/">ARIA working group</a> and as an editor of <a href="https://www.w3.org/TR/wai-aria/">Accessible Rich Internet Applications (WAI-ARIA) 1.1</a>, <a href="https://www.w3.org/TR/core-aam/">Core Accessibility API Mappings 1.1</a>, <a href="https://www.w3.org/TR/dpub-aria/">Digital Publishing WAI-ARIA Module 1.0</a>, <a href="https://www.w3.org/TR/dpub-aam/">Digital Publishing Accessibility API Mappings 1.0</a> all of which <a href="https://lists.w3.org/Archives/Public/w3c-wai-ig/2017OctDec/0239.html">became W3C Recommandations in December</a>! Work on versions 1.2 of ARIA and the Core AAM will begin in January. Stay tuned for the First Public Working Drafts.</p>
<p>We also contributed patches to fix several issues in the ARIA implementations of WebKit and Gecko and implemented support for the new DPub ARIA roles. We expect to continue this collaboration with Apple and Mozilla next year as well as to resume more active maintenance of <a href="https://help.gnome.org/users/orca/stable/">Orca</a>, the screen reader used to access graphical desktop environments in GNU/Linux.</p>
<p>Last but not least, progress continues on switching to <a href="https://github.com/w3c/web-platform-tests">Web Platform Tests</a> for ARIA and “Accessibility API Mappings” tests. This task is challenging because, unlike other aspects of the Web Platform, testing accessibility mappings cannot be done by solely examining what is rendered by the user agent. Instead, an additional tool, an “<a href="https://spec-ops.github.io/atta-api/index.html">Accessible Technology Test Adapter</a>” (ATTA) must be also be run. ATTAs work in a similar fashion to assistive technologies such as screen readers, using the implemented platform accessibility API to query information about elements and reporting what it obtains back to WPT which in turn determines if a test passed or failed. As a result, the tests are currently officially <a href="http://web-platform-tests.org/writing-tests/manual.html">manual</a> while the platform ATTAs continue to be developed and refined. We hope to make sufficient progress during 2018 that ATTA integration into WPT can begin.</p>
<h2 id="css">CSS</h2>
<p>This semester, we were glad to receive <a href="http://www.bloomberg.com/company/">Bloomberg</a>’s support again to pursue our activities around CSS. After a long commitment to CSS and a lot of feedback to Editors, <a href="https://twitter.com/regocas/status/908604005999357952">several of our members finally joined the Working Group</a>! Incidentally and as mentioned in a <a href="http://frederic-wang.fr/recent-browser-events.html">previous blog post</a>, during the <a href="https://wiki.csswg.org/planning/paris-2017">CSS Working Group face-to-face meeting in Paris</a> we got the opportunity to answer Microsoft’s questions regarding <a href="https://alistapart.com/article/the-story-of-css-grid-from-its-creators">The Story of CSS Grid, from Its Creators</a> (see also the <a href="https://www.youtube.com/watch?v=J9uaT9dggZE">video</a>). You might want to take a look at our own videos for CSS Grid Layout, regarding <a href="https://www.dropbox.com/s/co24b12wd81zaau/201709_gridlayout_alignment.mp4?dl=0">alignment and placement</a> and <a href="https://matrix.igalia.com/_matrix/media/v1/download/igalia.com/AEMEmqJXZvPAQGjlXFiCyceW">easy design</a>.</p>
<p>On the development side, we maintained and fixed bugs in Grid Layout implementation for Blink and WebKit. We also implemented alignment of positioned items in Blink and WebKit. We have several improvements and bug fixes for editing/selection from Bloomberg’s downstream branch that we’ve already upstreamed or plan to upstream. Finally, it’s worth mentioning that the <a href="https://blogs.igalia.com/mrego/2018/01/11/display-contents-is-coming/">work done on <code class="highlighter-rouge">display: contents</code></a> by our former coding experience student <a href="https://twitter.com/ecbos_">Emilio Cobos</a> was taken over and completed by antiik (for WebKit) and rune (for Blink) and is now enabled by default! We plan to pursue these developments next year and have various ideas. One of them is improving the way grids are stored in memory to allow huge grids (e.g. spreadsheet).</p>
<h2 id="web-platform-predictability">Web Platform Predictability</h2>
<p>One of the area where we would like to increase our activity is <a href="https://www.chromium.org/blink/platform-predictability">Web Platform Predictability</a>. This is obviously essential for our users but is also instrumental for a company like Igalia making developments on all the open source Javascript and Web engines, to ensure that our work is implemented consistently across all platforms. This semester, we were able to put more effort on this thanks to financial support from <a href="http://www.bloomberg.com/company/">Bloomberg</a> and <a href="https://www.ampproject.org/">Google AMP</a>.</p>
<p>We have implemented more
<a href="https://www.dropbox.com/s/k9y5wxw4mdrr41p/201708-sandboxing.mp4">frame sandboxing attributes WebKit</a>
to improve user safety and make control of sandboxed documents more flexible.
We improved the
<a href="https://html.spec.whatwg.org/#sandboxed-navigation-browsing-context-flag">sandboxed navigation browser context flag</a> and implemented the new
<a href="https://html.spec.whatwg.org/#attr-iframe-sandbox-allow-popups-to-escape-sandbox">allow-popup-to-escape-sandbox</a>, <a href="https://html.spec.whatwg.org/#attr-iframe-sandbox-allow-top-navigation-by-user-activation">allow-top-navigation-without-user-activation</a>, and <a href="https://html.spec.whatwg.org/#attr-iframe-sandbox-allow-modals">allow-modals</a> values for the <code class="highlighter-rouge">sandbox</code> attribute.</p>
<p>Currently, <a href="https://bugs.webkit.org/show_bug.cgi?id=149264">HTML frame scrolling</a> is not implemented in WebKit/iOS. As a consequence, one has to use the non-standard <code class="highlighter-rouge">-webkit-overflow-scrolling: touch</code> property on overflow nodes to emulate scrollable elements. In parallel to the progresses toward using more standard HTML frame scrolling we have also worked on annoying issues related to overflow nodes, including
<a href="https://bugs.webkit.org/show_bug.cgi?id=175135">flickering/jittering of “position: fixed” nodes</a> or <a href="https://bugs.webkit.org/show_bug.cgi?id=178789">broken Find UI</a> or a <a href="https://github.com/ampproject/amphtml/issues/11829">regression causing content to disappear</a>.</p>
<p>Another important task as part of our CSS effort was to address compatibility issues between the different browsers. For example we fixed editing bugs related to HTML List items: <a href="https://webkit.org/b/174593">WebKit’s Bug 174593</a>/<a href="https://crbug.com/744936">Chromium’s Issue 744936</a> and <a href="https://webkit.org/b/173148">WebKit’s Bug 173148</a>/<a href="https://crbug.com/731621">Chromium’s Issue 731621</a>. Inconsistencies in web engines regarding selection with floats have also been detected and we submitted the first patches for <a href="https://webkit.org/b/176096">WebKit</a> and <a href="https://crbug.com/643106">Blink</a>. Finally, we are currently improving line-breaking behavior in <a href="https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/fP1jDcQu68g/qSjd8EI8BQAJ">Blink</a> and <a href="https://bugs.webkit.org/show_bug.cgi?id=177327">WebKit</a>, which implies the implementation of new CSS values and properties defined in the last draft of the CSS Text 3 specification.</p>
<p>We expect to continue this effort on Web Platform Predictability next year and we are discussing more ideas e.g. <a href="https://github.com/WICG/webpackage">WebPackage</a> or <a href="https://drafts.csswg.org/css-flexbox/">flexbox</a> compatibility issues. For sure, <a href="https://github.com/w3c/web-platform-tests">Web Platform Tests</a> are an important aspect to ensure cross-platform inter-operability and we would like to help improving synchronization with the conformance tests of browser repositories. This includes the accessibility tests mentioned above.</p>
<h2 id="mathml">MathML</h2>
<p>Last November, we launched a <a href="https://mathml.igalia.com/">fundraising Campaign</a> to implement MathML in Chromium and presented it during Frankfurt Book Fair and TPAC. We have gotten very positive feedback so far with encouragement from people excited about this project. We strongly believe the native MathML implementation in the browsers will bring about a huge impact to STEM education across the globe and all the incumbent industries will benefit from the technology. As <a href="https://twitter.com/RickByers/status/928305865555296256">pointed out by Rick Byers</a>, the web platform is a commons and we believe that a more collective commitment and contribution are essential for making this world a better place!</p>
<p>While waiting for progress on Chromium’s side, we have provided minimal maintenance for MathML in WebKit:</p>
<ul>
<li>We fixed all the debug ASSERTs reported on Bugzilla.</li>
<li>We did follow-up code clean up and refactoring.</li>
<li>We imported Web Platform tests in WebKit.</li>
<li>We performed review of MathML patches.</li>
</ul>
<p>Regarding the last point, we would like to thank Minsheng Liu, a new volunteer who has started to contribute patches to WebKit to fix issues with MathML operators. He is willing to continue to work on MathML development in 2018 as well so stay tuned for more improvements!</p>
<h2 id="javascript">Javascript</h2>
<p>During the second semester of 2017, we worked on the design, standardization and implementation of several JavaScript features thanks to sponsorship from <a href="http://www.bloomberg.com/company/">Bloomberg</a> and <a href="https://www.mozilla.org/">Mozilla</a>.</p>
<p>One of the new features we focused on recently is <a href="https://github.com/tc39/proposal-bigint">BigInt</a>. We are working on an implementation of BigInt in SpiderMonkey, which is currently feature-complete but requires more optimization and cleanup. We wrote corresponding <a href="https://github.com/tc39/test262/">test262</a> conformance tests, which are mostly complete and upstreamed. Next semester, we intend to finish that work while our coding experience student <a href="https://github.com/caiolima">Caio Lima</a> continues work on a BigInt implementation on JSC, which has already started to land. Google also decided to implement that feature in V8 based on the specification we wrote. The BigInt specification that we wrote reached Stage 3 of TC39 standardization. We plan to keep working on these two BigInt implementations, making specification tweaks as needed, with an aim towards reaching Stage 4 at TC39 for the BigInt proposal in 2018.</p>
<p>Igalia is also proposing <a href="https://github.com/tc39/proposal-class-fields">class fields</a> and <a href="http://github.com/tc39/proposal-private-methods">private methods</a> for JavaScript. Similarly to BigInt, we were able to move them to Stage 3 and we are working to move them to stage 4. Our plan is to write test262 tests for private methods and work on an implementation in a JavaScript engine early next year.</p>
<p>Igalia implemented and shipped <a href="https://jakearchibald.com/2017/async-iterators-and-generators/">async iterators and generators</a> in Chrome 63, providing a convenient syntax for exposing and using asynchronous data streams, e.g., HTML streams. Additionally, we shipped a major performance optimization for Promises and async functions in V8.</p>
<p>We implemented and shipped two internationalization features in Chrome, <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/PluralRules">Intl.PluralRules</a> and <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/NumberFormat/formatToParts">Intl.NumberFormat.prototype.formatToParts</a>. To push the specifications of internationalization features forwards, we have been editing various other internationalization-related specifications such as <a href="https://github.com/tc39/proposal-intl-relative-time/">Intl.RelativeTimeFormat</a>, <a href="https://github.com/tc39/proposal-intl-locale/">Intl.Locale</a> and <a href="https://github.com/tc39/proposal-intl-list-format">Intl.ListFormat</a>; we also convened and led the first of what will be a monthly meeting of internationalization experts to propose and refine further API details.</p>
<p>Finally, Igalia has also been formalizing WebAssembly’s <a href="https://littledan.github.io/spec/document/js-api/">JavaScript API specification</a>, which reached the W3C first public working draft stage, and plans to go on to improve testing of that specification as the next step once further editorial issues are fixed.</p>
<h2 id="miscellaneous">Miscellaneous</h2>
<p>Thanks to sponsorship from <a href="http://mozilla.org/">Mozilla</a> we have continued our involvement in the <a href="https://wiki.mozilla.org/Platform/GFX/Quantum_Render">Quantum Render project</a> with the goal of using Servo’s WebRender in Firefox.</p>
<p>Support from <a href="https://www.metrological.com/index.html">Metrological</a> has also given us the opportunity to implement more web standards from some Linux ports of WebKit (<a href="https://webkitgtk.org/">GTK</a> and <a href="https://www.igalia.com/wpe/">WPE</a>, including:</p>
<ul>
<li>WebRTC</li>
<li>WebM</li>
<li>WebVR</li>
<li>Web Crypto</li>
<li>Web Driver</li>
<li>WebP animations support</li>
<li>HTML interactive form validation</li>
<li>MSE</li>
</ul>
<h2 id="conclusion">Conclusion</h2>
<p>Thanks for reading and we look forward to more work on the web platform in 2018. Onwards and upwards!</p>
Recent Browser Events2017-10-23T00:00:00+02:00http://frederic-wang.fr/recent-browser-events<h2 id="tldr">TL;DR</h2>
<p>At Igalia, we attend many browser events. This is a quick summary of some
recents conferences I participated to… or that gave me the opportunity to meet
Igalians in Paris 😉.</p>
<h2 id="week-31-paris---css-wg-f2f---w3c">Week 31: Paris - CSS WG F2F - W3C</h2>
<p>My teammate <a href="https://www.igalia.com/igalia-247/igalian/item/svillar/">Sergio</a> attended the <a href="https://wiki.csswg.org/planning/paris-2017">CSS WG F2F meeting</a> as an observer. On Tuesday morning, I also made an appearance
(but it was so brief that
<a href="https://fr.wikisource.org/wiki/Une_saison_en_enfer">ceux que j’ai rencontrés ne m’ont peut-être pas vu</a>).
Together with other browser vendors and WG members,
Sergio gave an interview regarding <a href="https://alistapart.com/article/the-story-of-css-grid-from-its-creators">the successful story of CSS Grid Layout</a>.
By the way, given our
implementation work in WebKit and Blink, Igalia finally decided
to <a href="https://twitter.com/regocas/status/908604005999357952">join the CSS Working Group</a> 😊.
Of course, during that week I had dinner with
Sergio and it was nice to chat with my colleague in a French restaurant
of <a href="https://en.wikipedia.org/wiki/Montmartre">Montmartre</a>.</p>
<h2 id="week-38-tokyo---blinkon-8---google">Week 38: Tokyo - BlinkOn 8 - Google</h2>
<p><a href="https://www.igalia.com/igalia-247/igalian/item/jaragunde/">Jacobo</a>,
<a href="https://www.igalia.com/igalia-247/igalian/item/gkim/">Gyuyoung</a> and I
attended <a href="https://www.youtube.com/playlist?list=PL9ioqAuyl6UK7Z0HHswBM5JgAp-izn_3S">BlinkOn 8</a>. I had nice discussions and listened to
interesting talks about a wide range of topics (Layout NG,
Accessibility, CSS, Fonts,
Web Predictability & Standards, etc). It was a pleasure to finally meet in persons
some developers I had been in touch with during my projects on
<a href="https://github.com/Igalia/chromium#chromium-for-wayland">Ozone/Wayland</a> and
WebKit/iOS.
For the lightning talks, we presented our activities
on <a href="https://blogs.igalia.com/jaragunde/files/2017/09/chromium-on-embedded-linux.pdf">embedded linux platforms</a> and the <a href="https://people.igalia.com/fwang/blink-on-8/lightning-talk-frederic-wang-igalia-contribution-to-the-web-platform.pdf">Web Platform</a>. Incidentally, it was great to see Igalia’s work
mentioned during the
<a href="https://docs.google.com/presentation/d/11rr_vo7UNS6icihnWZMx45O4y5JiSegZKYLQKA1LAdo/present">Next Generation Rendering Engine session</a>.
Obviously, I had the opportunity to visit places and taste Japanese food in
<a href="https://en.wikipedia.org/wiki/Asakusa">Asakusa</a>, <a href="https://en.wikipedia.org/wiki/Ueno">Ueno</a> and <a href="https://en.wikipedia.org/wiki/Roppongi">Roppongi</a> 😋.</p>
<h2 id="week-40-a-coruña---web-engines-hackfest---igalia">Week 40: A Coruña - Web Engines Hackfest - Igalia</h2>
<p>I attended
<a href="https://webengineshackfest.org/">one of my favorite events</a>, that gathers
the whole browser community during three days for
<a href="https://github.com/Igalia/webengineshackfest/wiki#web-engines-hackfest-2017">technical presentations, breakout sessions, hacking and galician food</a>. This year, we had many
<a href="https://webengineshackfest.org/#sponsors">sponsors</a>
and attendees. It is good to see that the event is becoming more
and more popular! It was long overdue, but I was finally able to make
<a href="https://github.com/google/brotli">Brotli</a>
and <a href="https://github.com/google/woff2">WOFF2</a> installable as system libraries
on Linux and usable by WebKitGTK+ 😊. I opened <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1405703">similar</a> <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1405704">bugs</a> in Gecko and the same could be done in Chromium. Among the things I enjoyed,
I met <a href="https://mozillians.org/en-US/u/jfkthame">Jonathan Kew</a> in person and heard more about <a href="https://www.igalia.com/igalia-247/igalian/item/agomes/">Antonio</a> and <a href="https://www.igalia.com/igalia-247/igalian/item/msisov/">Maksim</a>’s <a href="https://www.youtube.com/watch?v=efmYPFHnaSM">progress on Ozone/Wayland</a>.
As usual, it was nice to share time with colleagues, attend the assembly meeting,
play <a href="https://en.wikipedia.org/wiki/Association_football">football</a> matches, have meals, visit <a href="https://en.wikipedia.org/wiki/Asturias">Asturias</a>… and tell one’s story 😉.</p>
<h2 id="week-41-san-jose---webkit-contributors-meeting---apple">Week 41: <del>San Jose - WebKit Contributors Meeting - Apple</del></h2>
<p>In the past months, I have mostly been working on WebKit at Igalia and
I would have been happy to see my fellow WebKit developers.
However, given the events in Japan and Spain, I was not willing to make another
trip to the USA just after. Hence I had to miss the
WebKit Contributors Meeting again this year 😞.
Fortunately,
my colleagues <a href="https://www.igalia.com/igalia-247/igalian/item/alex/">Alex</a>,
<a href="https://www.igalia.com/igalia-247/igalian/item/mcatanzaro/">Michael</a> and
<a href="https://www.igalia.com/igalia-247/igalian/item/zdobersek/">Žan</a>
were present. Igalia is an important contributor
to WebKit and we will continue to send people and propose some talks next year.</p>
<h2 id="week-42-paris---monthly-speaker-series---mozilla">Week 42: Paris - Monthly Speaker Series - Mozilla</h2>
<p>This Wednesday, I attended a conference on
<a href="https://air.mozilla.org/privacy-as-a-competitive-advantage-with-gry-hasselbalch/">Privacy as a Competitive Advantage</a> in Mozilla’s office. It was
nice
to hear about the increasing interest on privacy and to see the
regulation made by the European Union in that direction.
My colleague <a href="https://www.igalia.com/igalia-247/igalian/item/pnormand/">Philippe</a> was visiting the office
to work with some Mozilla developers on one of our project, so I was also able
to meet him in the conference room. Actually, Mozilla employees were kind enough
to let me stay at the office after the conference… Hence I was able to work on Apple’s Web Engine on a <a href="http://frederic-wang.fr/amp-and-igalia-working-together-to-improve-the-web-platform.html">project sponsored by Google</a> at the Mozilla office… probably something you can only do at Igalia 😉. Last but not least, <a href="https://www.igalia.com/igalia-247/igalian/item/gemont/">Guillaume</a> was also in holidays in Paris this week, so I let you imagine
what happens when three French guys meet (hint: it involves food 😋).</p>
Review of Igalia's Web Platform activities (H1 2017)2017-09-07T00:00:00+02:00http://frederic-wang.fr/review-of-igalia-s-web-platform-activities-H1-2017<h2 id="introduction">Introduction</h2>
<p>For many years <a href="http://igalia.com/">Igalia</a> has been committed to and dedicated efforts to the improvement of Web Platform in all open-source Web Engines (Chromium, WebKit, Servo, Gecko) and JavaScript implementations (V8, SpiderMonkey, ChakraCore, JSC). We have been working in the implementation and standardization of some important technologies (CSS Grid/Flexbox, ECMAScript, WebRTC, WebVR, ARIA, MathML, etc). This blog post contains a review of these activities performed during the first half (and a bit more) of 2017.</p>
<h2 id="projects">Projects</h2>
<h3 id="css">CSS</h3>
<p>A few years ago <a href="http://www.bloomberg.com/company/">Bloomberg</a> and Igalia started a collaboration to implement a new layout model for the Web Platform. Bloomberg had complex layout requirements and what the Web provided was not enough and caused performance issues. <a href="https://drafts.csswg.org/css-grid/">CSS Grid Layout</a> seemed to be the right choice, a feature that would provide such complex designs with more flexibility than the currently available methods.</p>
<p><a href="https://blogs.igalia.com/mrego/2017/03/16/css-grid-layout-is-here-to-stay/">We’ve been implementing CSS Grid Layout in Blink and WebKit</a>, initially behind some flags as an experimental feature. This year, after some coordination effort to ensure interoperability (talking to the different parties involved like browser vendors, the CSS Working Group and the web authors community), it has been shipped by default in Chrome 58 and Safari 10.1. This is a huge step for the layout on the web, and modern websites will benefit from this new model and enjoy all the features provided by CSS Grid Layout spec.</p>
<p>Since the CSS Grid Layout shared the same alignment properties as the CSS Flexible Box feature, a new spec has been defined to generalize alignment for all the layout models. We started implementing this new spec as part of our work on Grid, <a href="https://blogs.igalia.com/jfernandez/2017/05/03/can-i-use-css-box-alignment/">being Grid the first layout model supporting it</a>.</p>
<p>Finally, we worked on other minor CSS features in Blink such as <a href="https://blogs.igalia.com/mrego/2017/01/09/coloring-the-insertion-caret/">caret-color</a> or <a href="https://blogs.igalia.com/mrego/2017/05/03/adding-focus-within-selector-to-chromium/">:focus-within</a> and also several interoperability issues related to Editing and Selection.</p>
<h3 id="mathml">MathML</h3>
<p><a href="https://www.w3.org/Math/">MathML</a> is a W3C recommendation to represent mathematical formulae that has been included in many other standards such as ISO/IEC, HTML5, ebook and office formats. There are many tools available to handle it, including various assistive technologies as well as generators from the popular LaTeX typesetting system.</p>
<p>After the improvements we performed in <a href="https://webkit.org/blog/6803/improvements-in-mathml-rendering/">WebKit’s MathML implementation</a>, we have regularly been in contact with Google to see how we can implement MathML in Chromium.
Early this year, we had several meetings with Google’s layout team to discuss this in further details. We agreed that MathML is an important feature to consider for users and that the right approach would be to rely on the new <a href="https://docs.google.com/document/d/1uxbDh4uONFQOiGuiumlJBLGgO4KDWB8ZEkp7Rd47fw4/">LayoutNG</a> model currently being implemented. We created a prototype for a <a href="https://github.com/emilio/chromium/commits/mathml-ng">small LayoutNG-based MathML implementation</a> as a proof-of-concept and as a basis for future technical discussions. We are <a href="https://twitter.com/regocas/status/869871891628126209">going to follow-up on this after the end of Q3</a>, once Chromium’s layout team has made more progress on LayoutNG.</p>
<h3 id="servo">Servo</h3>
<p><a href="https://servo.org/">Servo</a> is Mozilla’s next-generation web content engine based on Rust, a language that guarantees memory safety. Servo relies on a Rust project called WebRender which replaces the typical rasterizer and compositor duo in the web browser stack. WebRender makes extensive use of GPU batching to achieve very exciting performance improvements in common web pages. Mozilla has decided to make WebRender part of the <a href="https://wiki.mozilla.org/Platform/GFX/Quantum_Render">Quantum Render</a> project.</p>
<p>We’ve had the opportunity to collaborate with Mozilla for a few years now, focusing on the graphics stack. Our work has focused on bringing full support for CSS stacking and clipping to WebRender, so that it will be available in both Servo and Gecko. This has involved creating a data structure similar to what WebKit calls the “scroll tree” in WebRender. The scroll tree divides the scene into independently scrolled elements, clipped elements, and various transformation spaces defined by CSS transforms. The tree allows WebRender to handle page interaction independently of page layout, allowing maximum performance and responsiveness.</p>
<h3 id="webrtc">WebRTC</h3>
<p><a href="https://webrtc.org/">WebRTC</a> is a collection of communications protocols and APIs that enable real-time communication over peer-to-peer connections. Typical use cases include video conferencing, file transfer, chat, or desktop sharing. Igalia has been working on the WebRTC implementation in WebKit and this development is currently sponsored by <a href="https://www.metrological.com/index.html">Metrological</a>.</p>
<p>This year we have continued the implementation effort in WebKit for the <a href="https://webkitgtk.org/">WebKitGTK</a> and <a href="https://www.igalia.com/wpe/">WebKit WPE</a> ports, as well as the maintenance of two test servers for WebRTC: Ericsson’s p2p and Google’s <a href="https://github.com/webrtc/apprtc">apprtc</a>. Finally, a lot of progress has been done to add support for <a href="https://jitsi.org/">Jitsi</a> using the existing OpenWebRTC backend.</p>
<p>Since OpenWebRTC development is not an active project anymore and given libwebrtc is gaining traction in both Blink and the WebRTC implementation of WebKit for Apple software, we are taking the first steps to replace the original WebRTC implementation in WebKitGTK based on OpenWebRTC, with a new one based on libwebrtc. Hopefully, this way we will share more code between platforms and get more robust support of WebRTC for the end users. GStreamer integration in this new implementation is an issue we will have to study, as it’s not built in libwebrtc. libwebrtc offers many services, but not every WebRTC implementation uses all of them. This seems to be the case for the Apple WebRTC implementation, and it may become our case too if we need tighter integration with GStreamer or hardware decoding.</p>
<h3 id="webvr">WebVR</h3>
<p><a href="https://w3c.github.io/webvr/">WebVR</a> is an API that provides support for virtual reality devices in Web engines. Implementation and devices are currently actively developed by browser vendors and it looks like it is going to be a huge thing. Igalia has started to investigate on that topic to see how we can join that effort. This year, we have been in discussions with Mozilla, Google and Apple to see how we could help in the implementation of WebVR on Linux. We decided to start experimenting an implementation within WebKitGTK. We <a href="https://lists.webkit.org/pipermail/webkit-dev/2017-August/029372.html">announced our intention on the webkit-dev mailing list</a> and got encouraging feedback from Apple and the WebKit community.</p>
<h3 id="aria">ARIA</h3>
<p><a href="https://www.w3.org/WAI/intro/aria">ARIA</a> defines a way to make Web content and Web applications more accessible to people with disabilities. Igalia strengthened its ongoing committment to the W3C: Joanmarie Diggs joined Richard Schwerdtfeger as a co-Chair of the W3C’s <a href="https://www.w3.org/WAI/ARIA/">ARIA working group</a>, and became editor of the <a href="https://w3c.github.io/aria/core-aam/core-aam.html">Core Accessibility API Mappings</a>, [Digital Publishing Accessibility API Mappings] (https://w3c.github.io/aria/dpub-aam/dpub-aam.html), and <a href="https://w3c.github.io/aria/accname-aam/accname-aam.html">Accessible Name and Description: Computation and API Mappings</a> specifications. Her main focus over the past six months has been to get <a href="https://w3c.github.io/aria/aria/aria.html">ARIA 1.1</a> transitioned to Proposed Recommendation through a combination of implementation and bugfixing in WebKit and Gecko, creation of automated testing tools to verify platform accessibility API exposure in GNU/Linux and macOS, and working with fellow Working Group members to ensure the platform mappings stated in the various “AAM” specs are complete and accurate. We will provide more information about these activities after ARIA 1.1 and the related AAM specs are further along on their respective REC tracks.</p>
<h3 id="web-platform-predictability-for-webkit">Web Platform Predictability for WebKit</h3>
<p><a href="https://www.ampproject.org/">The AMP Project</a> has recently sponsored Igalia to improve WebKit’s implementation of the Web platform. We have worked on many issues, the main ones being:</p>
<ul>
<li>Frame sandboxing: Implementing sandbox values to allow trusted third-party resources to open unsandboxed popups or restrict unsafe operations of malicious ones.</li>
<li>Frame scrolling on iOS: Addressing issues with scrollable nodes; trying to move to a more standard and interoperable approach with scrollable iframes.</li>
<li>Root scroller: Finding a solution to the old interoperability issue about how to scroll the main frame; considering a new rootScroller API.</li>
</ul>
<p>This project aligns with <a href="https://www.chromium.org/blink/platform-predictability">Web Platform Predictability</a> which aims at making the Web more predictable for developers by improving interoperability, ensuring version compatibility and reducing footguns. It has been a good opportunity to collaborate with Google and Apple on improving the Web. You can find further details in <a href="http://frederic-wang.fr/amp-and-igalia-working-together-to-improve-the-web-platform.html">this blog post</a>.</p>
<h3 id="javascript">JavaScript</h3>
<p>Igalia has been involved in design, standardization and implementation of several JavaScript features in collaboration
with Bloomberg and Mozilla.</p>
<p>In implementation, Bloomberg has been sponsoring implementation of modern JavaScript features in V8, SpiderMonkey, JSC and ChakraCore, in collaboration with the open source community:</p>
<ul>
<li>Implementation of many ES6 features in V8, such as <a href="https://wingolog.org/archives/2013/05/08/generators-in-v8">generators</a>, destructuring binding and arrow functions</li>
<li><a href="https://blogs.igalia.com/compilers/2016/05/23/awaiting-the-future-of-javascript-in-v8/">Async/await</a> and <a href="https://jakearchibald.com/2017/async-iterators-and-generators/">async iterators and generators</a> in V8 and some work in JSC</li>
<li>Optimizing SpiderMonkey generators</li>
<li>Ongoing implementation of <a href="https://github.com/tc39/proposal-bigint">BigInt</a> in SpiderMonkey and class field declarations in JSC</li>
</ul>
<p>On the design/standardization side, Igalia is active in TC39 and with Bloomberg’s support</p>
<ul>
<li>Arbitrary precision integers (BigInt)</li>
<li><a href="https://github.com/tc39/proposal-class-fields">Class field declarations</a>, <a href="http://github.com/tc39/proposal-private-methods">private methods</a> and decorators</li>
<li>New RegExp features such as <a href="https://github.com/tc39/proposal-regexp-named-groups/">named capture groups</a></li>
</ul>
<p>In partnership with Mozilla, Igalia has been involved in the specification of various JavaScript standard library features for internationalization, in specification, implementation in V8, code reviews in other JavaScript engines,
as well as working with the underlying ICU library.</p>
<h2 id="other-activities">Other activities</h2>
<h3 id="preparation-of-web-engines-hackfest-2017">Preparation of Web Engines Hackfest 2017</h3>
<p>Igalia has been organizing and hosting the <a href="https://webengineshackfest.org/">Web Engines Hackfest</a> since 2009. This event under an unconference format has been a great opportunity for Web Engines developers to meet, discuss and work together on the web platform and on web engines in general. We announced <a href="https://blogs.igalia.com/mrego/2017/04/05/announcing-a-new-edition-of-the-web-engines-hackfest/">the 2017 edition</a> and <a href="https://webengineshackfest.org/#attendees">many developers already confirmed their attendance</a>. We would like to thank <a href="https://webengineshackfest.org/#sponsors">our sponsors</a> for supporting this event and we are looking forward to seeing you in October!</p>
<h3 id="coding-experience">Coding Experience</h3>
<p><a href="http://emiliocobos.me/about/">Emilio Cobos</a> has completed his <a href="https://www.igalia.com/about-us/coding-experience">coding experience program</a> on implementation of web standards. He has been working in the implementation of “display: contents” in <a href="https://bugs.chromium.org/p/chromium/issues/detail?id=657748">Blink</a> but some work is pending due to unresolved CSS WG issues. He also started the corresponding work in WebKit but implementation is still very partial. It has been a pleasure to mentor a skilled hacker like Emilio and we wish him the best for his future projects!</p>
<h3 id="new-igalians">New Igalians</h3>
<p>During this semester we have been glad to welcome new igalians who will help us to pursue Web platform developments:</p>
<ul>
<li><a href="https://www.igalia.com/nc/igalia-247/igalian/item/dehrenberg/">Daniel Ehrenberg</a> joined Igalia in January. He is an active contributor to the V8 JavaScript engine and has been representing Igalia at the ECMAScript TC39 meetings.</li>
<li><a href="https://www.igalia.com/nc/igalia-247/igalian/item/aboya/">Alicia Boya</a> joined Igalia in March. She has experience in many areas of computing, including web development, computer graphics, networks, security, and software design with performance which we believe will be valuable for our Web platform activities.</li>
<li><a href="https://www.igalia.com/nc/igalia-247/igalian/item/ms2ger/">Ms2ger</a> joined Igalia in July. He is a well-known hacker of the Mozilla community and has wide experience in both Gecko and Servo. He has noticeably worked in DOM implementation and web platform test automation.</li>
</ul>
<h1 id="conclusion">Conclusion</h1>
<p>Igalia has been involved in a wide range of Web Platform technologies going from Javascript and layout engines to accessibility or multimedia features. Efforts have been made in all parts of the process:</p>
<ul>
<li>Participation to standardization bodies (<a href="https://www.w3.org/Consortium/Member/Testimonial/#t1506">W3C</a>, <a href="https://www.ecma-international.org/memento/TC39.htm">TC39</a>).</li>
<li>Elaboration of conformance tests (<a href="https://github.com/w3c/web-platform-tests">web-platform-tests</a> <a href="https://github.com/tc39/test262">test262</a>).</li>
<li>Implementation and bug fixes in all open source web engines.</li>
<li>Discussion with users, browser vendors and other companies.</li>
</ul>
<p>Although, some of this work has been sponsored by Google or Mozilla, it is important to highlight how external companies (other than browser vendors) can make good contributions to the Web Platform, playing an important role on its evolution. <a href="https://twitter.com/alanstearns">Alan Stearns</a> already pointed out the <a href="https://www.youtube.com/watch?v=4ggNcqdwT-Y">responsibility of the Web Plaform users on the evolution of CSS</a> while <a href="https://rachelandrew.co.uk/">Rachel Andrew</a> emphasized <a href="https://rachelandrew.co.uk/speaking/event/cssconfeu-2017">how any company or web author can effectively contribute to the W3C in many ways</a>.</p>
<p>As mentioned in this blog post, <a href="http://www.bloomberg.com/company/">Bloomberg</a> is an important contributor of <a href="https://www.techatbloomberg.com/blog/bloombergs-2016-open-source-contributions-5-top-projects/">several open source projects</a> and they’ve been a key player in the development of CSS Grid Layout or Javascript. Similarly, <a href="https://www.metrological.com/index.html">Metrological</a>’s support has been instrumental for the implementation of WebRTC in WebKit. We believe others could follow their examples and we are looking forward to seeing more companies sponsoring Web Platform developments!</p>
The AMP Project and Igalia working together to improve WebKit and the Web Platform2017-08-30T00:00:00+02:00http://frederic-wang.fr/amp-and-igalia-working-together-to-improve-the-web-platform<h2 id="tldr">TL;DR</h2>
<p><a href="https://www.ampproject.org/">The AMP Project</a> and
<a href="https://www.igalia.com/">Igalia</a>
have recently been collaborating to
improve <a href="http://webkit.org/">WebKit</a>’s implementation of
the <a href="https://platform.html5.org/">Web platform</a>.
Both teams are committed to make the Web better and we expect
that all developers and users will benefit from this effort.
In this blog post, we review some of the bug fixes and features currently being
considered:</p>
<ul>
<li>
<p><strong>Frame sandboxing</strong>: Implementing sandbox values to allow trusted
third-party resources to open unsandboxed popups
or restrict unsafe operations of malicious ones.</p>
</li>
<li>
<p><strong>Frame scrolling on iOS</strong>: Trying to move to a more standard and
interoperable approach via <code class="highlighter-rouge">iframe</code> elements; addressing miscellaneous
issues with scrollable nodes (e.g. visual artifacts while scrolling, view
not scrolled when using “Find Text”…).</p>
</li>
<li>
<p><strong>Root scroller</strong>: Finding a solution to the
<a href="https://bugs.webkit.org/show_bug.cgi?id=5991">old interoperability issue</a>
about how to scroll the main frame; considering a
<a href="https://github.com/bokand/NonDocumentRootScroller">new rootScroller API</a>.</p>
</li>
</ul>
<p>Some demo pages for <a href="https://webkit.org/demos/frames/">frame sandboxing and scrolling</a> are also available if you wish to test features discussed in this blog post.</p>
<h2 id="introduction">Introduction</h2>
<p>AMP is an open-source project to enable websites and ads that are consistently fast, beautiful and high-performing across devices and distribution platforms.
Several interoperability bugs and missing features in WebKit have
caused problems to AMP users and to Web developers in general. Although
it is possible to add platform-specific workarounds to AMP, the best way to
help the Web Platform community is to directly fix these issues in WebKit,
so that everybody can benefit from these improvements.</p>
<p>Igalia is a consulting company with a team dedicated to Web Platform
developments in
all open-source Web Engines (Chromium, WebKit, Servo, Gecko) working
in the implementation and standardization of miscellaneous technologies
(CSS Grid/flexbox, ECMAScript, WebRTC, WebVR, ARIA, MathML, etc).
Given this expertise,
the AMP Project sponsored Igalia so that they can lead these
developments in WebKit. It is worth noting that this project aligns with the
<a href="https://www.chromium.org/blink/platform-predictability">Web Predictability effort</a> supported by both Google and Igalia, which aims at making the Web more
predictable for developers. In particular, the following aspects are considered:</p>
<ul>
<li>Interoperability: Effort is made to write
<a href="https://github.com/w3c/web-platform-tests">Web Platform Tests</a> (WPT), to follow
<a href="https://platform.html5.org/">Web standards</a> and ensure consistent behaviors
between web engines or operating systems.</li>
<li>Compatibility: Changes are carefully analyzed using telemetry techniques or
user feedback in order to avoid breaking compatibility with previous versions
of WebKit.</li>
<li>Reducing footguns: Removals of non-standard features (e.g. CSS vendor prefixes) are attempted while new features are carefully introduced.</li>
</ul>
<p>Below we provide further description of the WebKit
improvements, showing concretely how the above principles are followed.</p>
<h2 id="frame-sandboxing">Frame sandboxing</h2>
<p>A <a href="https://html.spec.whatwg.org/#attr-iframe-sandbox">sandbox</a> attribute can
be specified on the <code class="highlighter-rouge">iframe</code> element in order to enable a set of
restrictions on any content it hosts.
These conditions can be relaxed by specifying a list of values such as
<code class="highlighter-rouge">allow-scripts</code> (to allow javascript execution in the frame) or <code class="highlighter-rouge">allow-popups</code>
(to allow the frame to open popups). By default, the same restrictions apply
to a popup opened by a sandboxed frame.</p>
<div style="text-align: center;">
<div style="width: 300px; margin-left: auto; margin-right: auto;"><img src="http://frederic-wang.fr/images/iframe-sandboxing.svg" alt="iframe sandboxing" width="300" height="200" /></div>
<div><small>Figure 1: Example of sandboxed frames (Can they navigate their top frame or open popups? Are such popups also sandboxed?)</small></div>
</div>
<p>However, sometimes this behavior is not wanted. Consider for example the case
of an
advertisement inside a sandboxed frame. If a popup is opened from this frame
then it is likely that a non-sandboxed context is desired on the landing page.
In order to handle this use case, a new <a href="https://html.spec.whatwg.org/#attr-iframe-sandbox-allow-popups-to-escape-sandbox">allow-popups-to-escape-sandbox</a> value
has been introduced. The value is now supported in <a href="https://developer.apple.com/safari/technology-preview/release-notes/#r34">Safari Technology Preview 34</a>.</p>
<p>While performing that work, it was noticed that some WPT tests for the sandbox
attribute were still failing. It turns out that WebKit does not really
follow the <a href="https://html.spec.whatwg.org/multipage/browsers.html#allowed-to-navigate">rules to allow navigation</a>. More specifically, navigating a top context
is never allowed when such context corresponds to an opened popup.
<a href="https://trac.webkit.org/changeset/218921">We have made some changes to WebKit</a>
so that it behaves more closely to the specification. This is integrated into <a href="https://developer.apple.com/safari/technology-preview/release-notes/#r35">Safari Technology Preview 35</a> and you can for example
try <a href="http://w3c-test.org/html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_escaping-3.html">this W3C test</a>. Note that this test requires to change preferences to allow popups.</p>
<p>It is worth noting that web engines may slightly depart from the specification
regarding the previously mentioned rules. In particular, WebKit checks a
same-origin condition to be sure that one frame is allowed to navigate another
one.
WebKit always has contained
a special case to ignore this condition when a sandboxed frame with
the
<a href="https://html.spec.whatwg.org/#attr-iframe-sandbox-allow-top-navigation">allow-top-navigation</a> flag
tries and navigate its top frame. This feature, sometimes known
as “frame busting,” has been used by third-party resources
to perform malicious auto-redirecting.
As a consequence, Chromium developers proposed to restrict
frame busting to the case where the navigation is triggered by a user gesture.</p>
<p>According to <a href="https://www.chromestatus.com/metrics/feature/timeline/popularity/1752">Chromium’s telemetry</a> frame busting without a user gesture is very rare.
But when experimenting with the <a href="https://github.com/WICG/interventions/issues/16">behavior change of <code class="highlighter-rouge">allow-top-navigation</code></a> several regressions were reported.
Hence it was instead decided to introduce the
<a href="https://html.spec.whatwg.org/#attr-iframe-sandbox-allow-top-navigation-by-user-activation">allow-top-navigation-by-user-activation</a> flag in order to provide this improved
safety context while still preserving backward compatibility.
We implemented this feature in WebKit and it is now available in <a href="https://developer.apple.com/safari/technology-preview/release-notes/#r37">Safari Technology Preview 37</a>.</p>
<p>Finally, another proposed security improvement is to use an
<a href="https://html.spec.whatwg.org/#attr-iframe-sandbox-allow-modals">allow-modals</a>
flag
to explicitly allow sandboxed frames to display modal dialogs (with <code class="highlighter-rouge">alert</code>,
<code class="highlighter-rouge">prompt</code>, etc). That is, the default behavior for sandboxed frames will be
to forbid such modal dialogs. Again, such a change of behavior must be done with
care. Experiments in Chromium showed that the
<a href="https://www.chromestatus.com/metrics/feature/timeline/popularity/767">usage of modal dialogs in sandboxed frames</a> is very low and no users complained.
Hence we
<a href="https://trac.webkit.org/changeset/221193">implemented that behavior in WebKit</a>
and the feature should arrive in Safari Technology Preview soon.</p>
<p>Check out the
<a href="https://webkit.org/demos/frames/sandboxing/">frame sandboxing demos</a> if
if you want to test the new <code class="highlighter-rouge">allow-popup-to-escape-sandbox</code>,
<code class="highlighter-rouge">allow-top-navigation-without-user-activation</code> and <code class="highlighter-rouge">allow-modals</code> flags.</p>
<h2 id="frame-scrolling-on-ios">Frame scrolling on iOS</h2>
<p>Apple’s UI choice was to (almost) always “flatten” (expand) frames so that
users do not require to scroll them. The rationale for this is that it avoids to
be trapped into hierarchy of nested frames. Changing that behavior is likely
to cause a big backward compatibility issue on iOS
so for now we proposed a less radical solution:
Add a heuristic to support the case of “fullscreen” iframes used by the AMP
Project. Note that such exceptions already exist in WebKit, e.g. to
avoid making <a href="http://accessibilitytips.com/2008/03/04/positioning-content-offscreen/">offscreen content</a> visible.</p>
<p>We thus <a href="https://trac.webkit.org/changeset/218480">added the following heuristic</a> into
WebKit Nightly: do not flatten
out-of-flow iframes (e.g. <code class="highlighter-rouge">position: absolute</code>) that have viewport units
(e.g. <code class="highlighter-rouge">vw</code> and <code class="highlighter-rouge">vh</code>). This includes the case of the “fullscreen” iframe previously
mentioned. For now it is still under a developer flag so that WebKit developers
can control when they want to enable it. Of course, if this is successful we
might consider more advanced heuristics.</p>
<p>The fact that <a href="https://bugs.webkit.org/show_bug.cgi?id=149264">frames are never scrollable in iOS</a> is an obvious interoperability
issue. As a workaround, it is possible to emulate such “scrollable nodes”
behavior using
<code class="highlighter-rouge">overflow: scroll</code> nodes with the <code class="highlighter-rouge">-webkit-overflow-scrolling: touch</code>
property set. This is not really ideal for our Web Predictability goal as we
would like to get rid of browser vendor prefixes. Also, in practice such
workarounds lead to even more problems in AMP as explained in these
<a href="https://medium.com/@dvoytenko/amp-ios-scrolling-and-position-fixed-b854a5a0d451">blog</a>
<a href="https://hackernoon.com/amp-ios-scrolling-and-position-fixed-redo-the-wrapper-approach-8874f0ee7876">posts</a>. That’s why implementing scrolling of frames is
one of the main goals of this project and
significant steps have already been made in that direction.</p>
<div style="text-align: center;">
<div style="width: 400px; margin-left: auto; margin-right: auto;"><a href="http://frederic-wang.fr/images/scrolling-classes.svg"><img src="http://frederic-wang.fr/images/scrolling-classes.svg" alt="Class Hierarchy" width="400" /></a></div>
<div><small>Figure 2: C++ classes involved in frame scrolling</small></div>
</div>
<p>The (relatively complex) class hierarchy involved in frame scrolling is
summarized in Figure 2. The frame flattening heuristic mentioned above is
handled in the <code class="highlighter-rouge">WebCore::RenderIFrame</code> class (in purple).
The <code class="highlighter-rouge">WebCore::ScrollingTreeFrameScrollingNodeIOS</code> and
<code class="highlighter-rouge">WebCore::ScrollingTreeOverflowScrollingNodeIOS</code> classes from the
scrolling tree (in blue) are used to scroll, respectively,
the main frame and overflow nodes on iOS. Scrolling of
non-main frames will obviously have some code to share with the former, but
it will also have some parts in common with the latter. For example,
passing an extra <code class="highlighter-rouge">UIScrollView</code> layer is needed instead of relying on the one
contained in the <code class="highlighter-rouge">WKWebView</code> of the main frame. An important
step is thus to introduce a special class for scrolling inner frames that
would share some logic from the two other classes and
<a href="https://bugs.webkit.org/show_bug.cgi?id=174097">some refactoring to</a> <a href="https://bugs.webkit.org/show_bug.cgi?id=174130">ensure optimal code reuse</a>.
Similar refactoring has been done for scrolling node states (in red)
<a href="https://bugs.webkit.org/show_bug.cgi?id=174134">to move the scrolling layer parameter into <code class="highlighter-rouge">WebCore::ScrollingStateNode</code></a> instead of having separate members
for <code class="highlighter-rouge">WebCore::ScrollingStateOverflowScrollingNode</code> and <code class="highlighter-rouge">WebCore::ScrollingStateFrameScrollingNode</code>.</p>
<p>The scrolling coordinator classes (in green) are also important, for example to
handle hit testing. At the moment, this is not really implemented for
overflow nodes but it might be important to have it for scrollable frames.
Again, one sees that some logic is shared for asynchronous scrolling on macOS (<code class="highlighter-rouge">WebCore::ScrollingCoordinatorMac</code>) and iOS (<code class="highlighter-rouge">WebCore::ScrollingCoordinatorIOS</code>)
in ancestor classes. Indeed, our effort to make frames scrollable on iOS
is also opening the possibility of
<a href="https://bugs.webkit.org/show_bug.cgi?id=171667">asynchronous scrolling of frames on macOS</a>, something that is currently not implemented.</p>
<div style="text-align: center;">
<div style="width: 500px; margin-left: auto; margin-right: auto;"><a target="_blank" href="https://player.vimeo.com/video/225557385"><img src="http://frederic-wang.fr/images/video-ios-frame-scrolling.png" alt="Class Hierarchy" width="500" /></a></div>
<div><small>Figure 4: Video of this <a href="https://webkit.org/demos/frames/scrollable-iframes.html">demo page</a> on WebKit iOS with experimental patches to make frame scrollables (2017/07/10)</small></div>
</div>
<p>Finally, some more work is necessary in the render classes (purple) to ensure
that the layer hierarchies are correctly built. Patches
<a href="https://bugs.webkit.org/show_bug.cgi?id=173833">have been uploaded</a>
and you can view the result on the video of Figure 4.
Notice that this work has not been reviewed yet and
there are known bugs, for example with
overlapping elements (hit testing not implemented) or <code class="highlighter-rouge">position: fixed</code>
elements.</p>
<p>Various other scrolling bugs were reported, analyzed and sometimes fixed by
Apple. The switch from overflow nodes to scrollable iframes is unlikely to
address them. For example, the “Find Text” operation in iOS has
advanced features done by the UI process (highlight, smart magnification) but
the <a href="https://bugs.webkit.org/show_bug.cgi?id=163911">scrolling operation needed only works for the main frame</a>. It looks like this could be fixed
by unifying a bit the scrolling code path with macOS. There are also
several <a href="https://bugs.webkit.org/show_bug.cgi?id=154399">jump and flickering bugs</a> with <code class="highlighter-rouge">position: fixed</code> nodes. Finally, Apple fixed
<a href="https://bugs.webkit.org/show_bug.cgi?id=162499">inconsistent scrolling inertia</a>
used for the main frame and the one used for inner scrollable nodes by
making the former the same as the latter.</p>
<h2 id="root-scroller">Root Scroller</h2>
<p>The CSSOM View specification <a href="https://drafts.csswg.org/cssom-view/#extension-to-the-element-interface">extends the DOM element with some scrolling properties</a>.
That specification indicates that the element to consider to scroll the main view
is <code class="highlighter-rouge">document.body</code> in quirks mode while it is <code class="highlighter-rouge">document.documentElement</code>
in no-quirks mode. This is the behavior that has always been followed
by browsers like Firefox or Interner Explorer. However, WebKit-based browsers
always treat <code class="highlighter-rouge">document.body</code> as the root scroller. This interoperability issue
has been a big problem for web developers.
One convenient workaround was to introduce the
<a href="https://drafts.csswg.org/cssom-view/#dom-document-scrollingelement">document.scrollingElement</a> which returns the element to use for scrolling the main
view (<code class="highlighter-rouge">document.body</code> or <code class="highlighter-rouge">document.documentElement</code>) and was recently
<a href="https://bugs.webkit.org/show_bug.cgi?id=143609">implemented in WebKit</a>.
Use this
<a href="https://webkit.org/demos/frames/scrollingElement.html">test page</a>
to verify whether your browser supports the
<code class="highlighter-rouge">document.scrollingElement</code> property and which DOM element is used to scroll
the main view in no-quirks mode.</p>
<p>Nevertheless, this does not solve the issue with existing web pages.
Chromium’s Web Platform Predictability team has made a huge communication effort
with Web authors and developers which has
drastically reduced the use of <code class="highlighter-rouge">document.body</code> in no-quirks mode. For instance,
Chromium’s telemetry on Figure 3 indicates that the percentage of
<code class="highlighter-rouge">document.body.scrollTop</code> in no-quirks pages has gone
from 18% down to 0.0003% during the past three years. Hence the Chromium team
is now considering <a href="https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/X64Sg16RhT4">shipping the standard behavior</a>.</p>
<div style="text-align: center;">
<div style="width: 400px; margin-left: auto; margin-right: auto;"><a href="https://www.chromestatus.com/metrics/feature/popularity#ScrollTopBodyNotQuirksMode"><img width="400" src="http://frederic-wang.fr/images/ScrollTopBodyNotQuirksMode.png" alt="UseCounter for ScrollTopBodyNotQuirksMode" /></a></div>
<div><small>Figure 3: Use of <code>document.body.scrollTop</code> in no-quirks mode over time (Chromium's UseCounter)</small></div>
</div>
<p>In WebKit, the issue has been known for a long time and an
<a href="https://bugs.webkit.org/show_bug.cgi?id=106133">old attempt to fix it was reverted for causing regressions</a>. For now, we imported
the <a href="https://trac.webkit.org/changeset/215726/">CSSOM View tests</a> and just
marked the one related to the scrolling element as failing. An analysis
of the situation has been left on
<a href="https://bugs.webkit.org/show_bug.cgi?id=5991#c14">WebKit’s bug</a>; Depending on
how things evolve on Chromium’s side we could consider the discussion and
implementation work in WebKit.</p>
<p>Related to that work, a
<a href="https://github.com/bokand/NonDocumentRootScroller">new API</a> is being proposed
to set the root scroller to an arbitrary scrolling element, giving
more flexibility to authors of Web applications. Today, this is unfortunately
not possible
without losing some of the special features of the main view (e.g. on iOS,
Safari’s URL bar is hidden when scrolling the main view
to maximize the screen space). Such
API is currently being experimented in Chromium and we plan to
investigate whether this can be implemented in WebKit too.</p>
<h2 id="conclusion">Conclusion</h2>
<p>In the past months, The AMP Project and Igalia have worked on analyzing some
interoperability issue and fixing them in WebKit. Many improvements for
frame sandboxing are going to be available soon.
Significant progress has also been
made for frame scrolling on iOS and collaboration continues with
Apple reviewers to ensure that the work will be integrated in future versions of
WebKit. Improvements to “root scrolling” are also being considered although they
are pending on the evolution of the issues on Chromium’s side. All these
efforts are expected to be useful for WebKit users and the Web platform
in general.</p>
<div>
<div style="width: 364px; margin-left: auto; margin-right: auto;"><a href="http://www.igalia.com"><img src="http://frederic-wang.fr/images/igalia-logo-364x130.png" alt="Igalia Logo" width="364" height="130" /></a></div>
<div style="width: 400px; margin-left: auto; margin-right: auto;"><a href="https://www.ampproject.org/"><img src="http://frederic-wang.fr/images/amp-logo-400x210.png" alt="AMP Logo" width="400" height="210" /></a></div>
</div>
<p>Last but not least, I would like to thank Apple engineers Simon Fraser,
Chris Dumez, and Youenn Fablet for their reviews and help, as well as Google
and the AMP team for supporting that project.</p>
¡Igalia is hiring!2016-12-22T00:00:00+01:00http://frederic-wang.fr/igalia-is-hiring<p>If you read this blog, you probably know that I joined <a href="https://www.igalia.com/">Igalia</a> early this year, where I have been involved in projects related to free software and web engines. You may however not be aware that Igalia has a flat & cooperative structure where all decisions (projects, events, recruitments, company agreements etc) are voted by members of an assembly. In my opinion such an organization allows to take better decisions and to avoid frustrations, compared to more traditional hierarchical organizations.</p>
<p>After several months as a staff, I finally applied to become an assembly member and my application was approved in November! Hence I attended my first assembly last week where I got access to all the internal information and was also able to vote… In particular, we approved the opening of two <a href="https://www.igalia.com/nc/igalia-247/news/item/we-are-hiring/">new job positions</a>. If you are interested in state-of-the-art free software projects and if you are willing to join a company with great human values, you should definitely consider applying!</p>
STIX Two in Gecko and WebKit2016-12-10T00:00:00+01:00http://frederic-wang.fr/stix-two-in-gecko-and-webkit<p>On the 1st of December, the <a href="http://stixfonts.org/">STIX Fonts project</a>
announced the release of STIX 2. If you never heard about this project, it is
described <a href="http://stixfonts.org/abt_geninfo.html">as follows</a>:</p>
<blockquote>
<p>The mission of the Scientific and Technical Information Exchange (STIX) font creation project is the preparation of a comprehensive set of fonts that serve the scientific and engineering community in the process from manuscript creation through final publication, both in electronic and print formats.</p>
</blockquote>
<p>This sounds a very exciting goal but the way it has been achieved has
made the STIX project infamous for its
<a href="https://en.wikipedia.org/wiki/STIX_Fonts_project#Development_process">numerous delays</a>,
for its <a href="https://sourceforge.net/p/stixfonts/tracking/2/">poor</a> or
<a href="https://sourceforge.net/p/stixfonts/tracking/45/">confusing</a> packaging,
for delivering math fonts with
<a href="https://sourceforge.net/p/stixfonts/tracking/59/">too</a>
<a href="https://sourceforge.net/p/stixfonts/tracking/49/">many</a>
<a href="https://sourceforge.net/p/stixfonts/tracking/33/">bugs</a> to be usable,
for its lack of
openness & <a href="https://sourceforge.net/p/stixfonts/tracking/74/">communication</a>,
for its bad handling of third-party
feedback & <a href="https://github.com/khaledhosny/xits-math">contribution</a>…</p>
<p>Because of these laborious travels towards unsatisfactory releases, some
snarky people claim that the project was actually named after
<em>Styx</em> (<a href="https://en.wiktionary.org/wiki/%CE%A3%CF%84%CF%8D%CE%BE">Στύξ</a>)
the river from Greek mythology that one has to cross to enter the Underworld.
Or that the <a href="http://stixfonts.org/proj_timeline.html">story of the project</a>
is summarized by Baudelaire’s verses from
<a href="http://fleursdumal.org/poem/163">L’Irrémédiable</a>:</p>
<div lang="fr">
<blockquote>
Une Idée, une Forme, un Être<br />
Parti de l’azur et tombé<br />
Dans un Styx bourbeux et plombé<br />
Où nul œil du Ciel ne pénètre ;<br />
</blockquote>
</div>
<p>More seriously, the good news is that the STIX
Consortium <em>finally</em> released text fonts with a beautiful design and a
companion math font that is usable in math rendering engines such as
Word Processors, LaTeX and Web Engines. Indeed,
WebKit and Gecko have supported OpenType-based MathML layout for more
than three years (with
<a href="http://frederic-wang.fr/mathml-improvements-in-webkit.html">recent</a>
<a href="http://frederic-wang.fr/fonts-at-the-web-engines-hackfest-2016.html">improvements</a> by Igalia) and STIX Two now has correct OpenType data and metrics!</p>
<p>Of course, the STIX Consortium did not
address all the technical or organizational issues
that have made its reputation but I count on
<a href="https://twitter.com/KhaledGhetas">Khaled Hosny</a> to maintain
his more open <a href="https://github.com/khaledhosny/xits-math">XITS fork</a>
with enhancements that have been ignored for STIX Two
(e.g. Arabic and RTL features) or
with fixes of
<a href="https://sourceforge.net/p/stixfonts/tracking/milestone/Release%202.0.0/">already reported bugs</a>.</p>
<p>As <a href="https://en.wikipedia.org/wiki/Jacques_Distler">Jacques Distler</a> wrote in a <a href="https://golem.ph.utexas.edu/~distler/blog/archives/002926.html#MMLf1">recent blog post</a>,
OS vendors should ideally bundle the STIX Two fonts in their default
installation. For now, users can
<a href="http://sourceforge.net/projects/stixfonts/files/Current%20Release/STIXv2.0.0.zip/download">download</a> and install the OTF fonts themselves.
Note however that the STIX Two archive contains WOFF and WOFF2 fonts that page
authors can <a href="https://fred-wang.github.io/MathFonts/">use as web fonts</a>.</p>
<p>I just landed patches in <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1322743">Gecko</a>
and <a href="https://bugs.webkit.org/show_bug.cgi?id=165676">WebKit</a> so that future
releases will
try and find STIX Two on your system for MathML rendering. However, you
can already do the proper font configuration via the preference
menu of your browser:</p>
<ul>
<li>For Gecko-based applications (e.g. Firefox, Seamonkey or Thunderbird),
go to the font preference and select <code class="highlighter-rouge">STIX Two Math</code> as the
“font for mathematics”.</li>
<li>For WebKit-based applications (e.g. Epiphany or Safari)
add the following rule to your user stylesheet:
<code class="highlighter-rouge">math { font-family: "STIX Two Math"; }</code>.</li>
</ul>
<p>Finally, here is a screenshot of MathML formulas rendered by
<a href="https://www.mozilla.org/en-US/firefox/">Firefox</a> 49 using STIX Two:</p>
<div style="text-align: center; width: 500px; margin-left: auto; margin-right: auto;">
<a href="http://frederic-wang.fr/images/stix-two-firefox-49.png"><img width="500" src="http://frederic-wang.fr/images/stix-two-firefox-49.png" alt="Screenshot of MathML formulas rendered by Firefox using STIX 2" /></a>
<small></small>
</div>
<p>And the same page rendered by
<a href="https://wiki.gnome.org/Apps/Web/">Epiphany</a> 3.22.3:</p>
<div style="text-align: center; width: 500px; margin-left: auto; margin-right: auto;">
<a href="http://frederic-wang.fr/images/stix-two-epiphany-3.22.3.png"><img width="500" src="http://frederic-wang.fr/images/stix-two-epiphany-3.22.3.png" alt="Screenshot of MathML formulas rendered by Epiphany using STIX 2" /></a>
</div>
Fonts at the Web Engines Hackfest 20162016-10-06T00:00:00+02:00http://frederic-wang.fr/fonts-at-the-web-engines-hackfest-2016<p>Last week I travelled to
<a href="https://en.wikipedia.org/wiki/Galicia_%28Spain%29">Galicia</a> for one of the
regular gatherings organized by <a href="http://igalia.com/">Igalia</a>. It was a great
pleasure
to meet again all the Igalians and friends. Moreover, this time was a bit
special since we celebrated our <a href="https://www.igalia.com/nc/igalia-247/news/item/igalia-celebrates-our-15th-anniversary/">15th anniversary</a> :-)</p>
<div style="width: 600px; margin-left: auto; margin-right: auto; text-align: center;">
<div style="width: 500px; margin-left: auto; margin-right: auto;"><a href="https://secure.flickr.com/photos/bertogg/29962370082/" title="Igalia Summit October 2016"><img src="https://c3.staticflickr.com/9/8554/29962370082_024203fd6f.jpg" width="500" height="375" alt="Igalia Summit October 2016" /></a></div> <small>Photo by <a href="https://secure.flickr.com/photos/bertogg/">Alberto Garcia</a> licensed under <a href="http://creativecommons.org/licenses/by-sa/2.0/">CC BY-SA 2.0</a>.</small></div>
<p><a data-flickr-embed="true" href="https://www.flickr.com/photos/bertogg/29962370082/in/photostream/" title="Igalia Summit October 2016"></a><script async="" src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script></p>
<p>I also attended the third edition of the
<a href="http://www.webengineshackfest.org/">Web Engines Hackfest</a>,
sponsored by Igalia,
<a href="https://www.collabora.com/">Collabora</a> and <a href="https://www.mozilla.org/">Mozilla</a>.
This year, we had various participants from the Web Platform including folks from
Apple, Collabora, Google, Igalia, Huawei, Mozilla or Red Hat.
For my first hackfest as an Igalian, I invited some experts on fonts &
math rendering to collaborate
on OpenType MATH support HarfBuzz and its use in math rendering engines.
In this blog post, I am going to focus on the work I have made
with <a href="http://behdad.org/">Behdad Esfahbod</a> and
<a href="http://khaledhosny.org/">Khaled Hosny</a>. I think it was again a great and
productive hackfest and I am looking forward to attending the next edition!</p>
<h2 id="opentype-math-in-harfbuzz">OpenType MATH in HarfBuzz</h2>
<div style="width: 600px; margin-left: auto; margin-right: auto; text-align: center;">
<div style="width: 500px; margin-left: auto; margin-right: auto;"><a href="https://www.flickr.com/photos/webhackfest/30112185606/in/album-72157673583191111/" title="Behdad talking about HarfBuzz"><img src="https://c7.staticflickr.com/6/5279/30112185606_c3c14fcd66.jpg" width="500" height="281" alt="Web Engines Hackfest, main room" /></a></div> <small>Photo by <a href="https://www.flickr.com/photos/webhackfest/">@webengineshackfest</a> licensed under <a href="http://creativecommons.org/licenses/by-sa/2.0/">CC BY-SA 2.0</a>.</small></div>
<p>Behdad <a href="https://github.com/Igalia/webengineshackfest/wiki#scheduling">gave a talk</a> with a nice overview of the work accomplished in
<a href="https://freedesktop.org/wiki/Software/HarfBuzz/">HarfBuzz</a> during ten years.
One thing appearing recently in HarfBuzz is the need for APIs to parse
OpenType tables on all platforms. As part of my job at Igalia, I had started
to experiment adding support for the
<a href="http://frederic-wang.fr/opentype-math-in-harfbuzz.html">MATH table</a>
some months ago and it was
nice to have Behdad finally available to review, fix and improve commits.</p>
<p>When I talked to Mozilla employee
<a href="http://blog.karlt.net">Karl Tomlinson</a>, it became apparent that
the simple shaping API for stretchy operators proposed
<a href="http://frederic-wang.fr/opentype-math-in-harfbuzz.html">in my blog post</a>
would not cover all the special cases currently implemented in Gecko. Moreover,
this shaping API is also very similar to another one existing in HarfBuzz for
non-math script so we would have to decide the best way to share the logic.</p>
<p>As a consequence, we decided for now to focus on providing an API to access all
the data of the MATH table. After the Web Engines Hackfest,
<a href="https://github.com/behdad/harfbuzz/blob/master/src/hb-ot-math.h">such a math API</a> is now integrated into the development
repository of HarfBuzz and will available in version 1.3.3 :-)</p>
<h2 id="mathml-in-web-rendering-engines">MathML in Web Rendering Engines</h2>
<p>Currently, several math rendering engines have their own code to parse the data
of the OpenType MATH table. But many of them actually use HarfBuzz for normal
text shaping and hence could just rely on the new math API the math rendering
too.
Before the hackfest, Khaled already had tested my work-in-progress branch with
<a href="https://github.com/khaledhosny/libmathview/">libmathview</a> and I had done the
same for <a href="https://github.com/fred-wang/chromium.src/tree/mathml">Igalia’s Chromium MathML branch</a>.</p>
<div style="text-align: center;">
<div style="width: 600px; margin-left: auto; margin-right: auto;"><img src="http://frederic-wang.fr/images/mathml-fraction-parameters-test.png" width="600" height="464" alt="MathML Fraction parameters test" /></div>
<small>MathML test for OpenType MATH Fraction parameters in Gecko, Blink and WebKit.</small>
</div>
<p>Once the new API landed into HarfBuzz, Khaled was also able to use it for the
<a href="https://en.wikipedia.org/wiki/XeTeX">XeTeX</a> typesetting system.
I also started to experiment this
for <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1305977">Gecko</a> and
<a href="https://bugs.webkit.org/show_bug.cgi?id=162671">WebKit</a>.
This seems to work pretty well and we get consistent results for
Gecko, Blink and WebKit! Some random thoughts:</p>
<ul>
<li>The math data is exposed through a <code class="highlighter-rouge">hb_font_t</code> which contains the text size.
This means that the various values are now directly resolved and returned as a
<a href="https://en.wikipedia.org/wiki/Fixed-point_arithmetic">fixed-point number</a>
which should allow to avoid rounding errors we may currently have in
Gecko or WebKit when multiplying by float factors.</li>
<li>HarfBuzz has some magic to automatically handle invalid offsets and sizes
that greatly simplifies the code, compared to what exist in Gecko and WebKit.</li>
<li>Contrary to Gecko’s implementation, HarfBuzz does not cache the latest
result for glyph-specific data. Maybe we want to keep that?</li>
<li>The WebKit changes were tested on the GTK port, where HarfBuzz is enabled.
Other ports may still need to use the existing parsing code from the
WebKit tree. Perhaps
Apple should consider adding support for the OpenType MATH table to
<a href="https://developer.apple.com/reference/coretext">CoreText</a>?</li>
</ul>
<h2 id="brotliwoff2ots-libraries">Brotli/WOFF2/OTS libraries</h2>
<div style="width: 600px; margin-left: auto; margin-right: auto; text-align: center;">
<div style="width: 500px; margin-left: auto; margin-right: auto;"><a href="https://www.flickr.com/photos/webhackfest/30147116385/in/album-72157673583191111/" title="Web Engines Hackfest, main room"><img src="https://c2.staticflickr.com/6/5232/30147116385_06a307626b.jpg" width="500" height="281" alt="Web Engines Hackfest, main room" /></a></div> <small>Photo by <a href="https://www.flickr.com/photos/webhackfest/">@webengineshackfest</a> licensed under <a href="http://creativecommons.org/licenses/by-sa/2.0/">CC BY-SA 2.0</a>.</small></div>
<p>We also updated the copies of WOFF2 and OTS
libraries in <a href="https://bugs.webkit.org/show_bug.cgi?id=162608">WebKit</a> and
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1305944">Gecko</a> respectively.
This improves one <a href="https://www.w3.org/TR/WOFF2/#conform-mustNotRejectIncorrectTotalSize">requirement from the WOFF2 specification</a> and allows to
<a href="http://test.csswg.org/harness/test/woff2_dev/single/header-totalsfntsize-001/">pass the corresponding conformance test</a>.</p>
<p>Gecko, WebKit and Chromium bundle their own copy of the Brotli, WOFF2
or OTS libraries in their source repositories. However:</p>
<ul>
<li>
<p>We have to use more or less automated mechanisms to keep these bundled copies
up-to-date. This is especially annoying for Brotli and WOFF2 since they are
still in development and we must be sure to always integrate the latest security
fixes. Also, we get compiler warnings or coding style errors that do not exist
upstream and that must be disabled or patched until they are fixed upstream
and imported again.</p>
</li>
<li>
<p>This obviously is not an optimal sharing of system library and may increase
the size of binaries.
Using shared libraries is what maintainers of Linux (or other
FLOSS systems) generally ask and this was raised during the WebKitGTK+
session. Similarly, we should really use the system Brotli/WOFF2 bundled in
future releases of Apple’s operating systems.</p>
</li>
</ul>
<p>There are several issues that make hard for package maintainers to
provide these libraries: no released binaries or release tags, no proper build
system to generate shared libraries, use of git submodule to include one library source
code into another etc Things have
gotten a bit better for Brotli and I was able to
<a href="https://github.com/google/brotli/pull/421">tweak the CMake script to produce shared libraries</a>. For WOFF2,
<a href="https://github.com/google/woff2/issues/40">issue 40</a> and
<a href="https://github.com/google/woff2/issues/49">issue 49</a> have been inactive but
hopefully these will be addressed in the future…</p>
MathML Improvements in WebKit2016-07-13T00:00:00+02:00http://frederic-wang.fr/mathml-improvements-in-webkit<p>In a <a href="http://frederic-wang.fr/mathml-refactoring-in-webkit.html">previous blog post</a>, I explained the work made by <a href="http://igalia.com/">Igalia</a>’s
web platform team to refactor WebKit’s
MathML layout classes. I stated that although some rendering improvements were
a nice side effect, the main goal of <a href="https://trac.webkit.org/wiki/MathML/Early_2016_Refactoring#Phase1">the first phase</a> was really to clean the code up so
that it is easier for developers to work on MathML in the future. Indeed this
<em>really</em> made things easier to review: Quite unexpectedly to me,
<a href="https://trac.webkit.org/wiki/MathML/Early_2016_Refactoring#Phase2">the second phase</a>
only took 4 days to be upstreamed… Kudos to
<a href="http://whtconstruct.blogspot.fr/">Brent Fulgham</a> for having reviewed so many
patches in such a short period of time!</p>
<p>In this blog post, I am going to give an overview of the improvements made
during these two first phases taking <a href="https://trac.webkit.org/changeset/203109">changeset r203109</a> as a reference. The changes will be available in <a href="https://webkitgtk.org/">WebKitGTK+ 2.14</a> in September and are likely to be included this month in the next <a href="https://developer.apple.com/safari/technology-preview/">Safari Technology Preview</a>.
It definitely remains more work to do such as
<a href="https://trac.webkit.org/wiki/MathML/Early_2016_Refactoring#Phase3">the third phase</a> or other rendering improvements, but I believe we have already made a big
step forward!</p>
<h3 id="mathematical-fonts">Mathematical Fonts</h3>
<p>Two years ago, basic support for operator stretching via the OpenType MATH
table was added to WebKit. During the refactoring, we improved that support
and also made use of more parameters to improve the math layout (see section
about OpenType MATH parameters below). While
<a href="http://www.gust.org.pl/projects/e-foundry/lm-math">Latin Modern Math</a> will be used in most screenshots, the following one shows that you can
use any <a href="http://trac.webkit.org/wiki/MathML/Fonts">math fonts</a>. By default
WebKit will try and use one of these fonts but if none are available or if you
force a non-math <code class="highlighter-rouge">font-family</code> then the rendering quality may not be good.</p>
<p>The following screenshot gives the rendering for various fonts.
For the last one
we used the value <code class="highlighter-rouge">sans-serif</code> to illustrate issues with non-math fonts
(displaystyle integral too small, mathvariant italic glyphs taken from another
font, missing italic correction etc).</p>
<div style="text-align: center; width: 500px; margin-left: auto; margin-right: auto;">
<a href="http://frederic-wang.fr/images/webkitgtk-r203109-fonts.png"><img width="500" src="http://frederic-wang.fr/images/webkitgtk-r203109-fonts.png" alt="Screenshots of a formula with various fonts" /></a>
<small>
Integral obtained by <a href="https://en.wikipedia.org/wiki/Methods_of_contour_integration">contour integration</a><br />
WebKitGTK+ r203109 with various font-family values.<br />
From top to bottom:
<a href="http://www.gust.org.pl/projects/e-foundry/tg-math/download/index_html#Schola_Math">TeX Gyre Schola</a>, <a href="http://www.gust.org.pl/projects/e-foundry/lm-math">Latin Modern</a>, <a href="https://github.com/dejavu-fonts/dejavu-fonts">DejaVu Math</a>, <a href="http://stixfonts.org/">STIX</a>,
and <code>sans-serif</code>.</small>
</div>
<h3 id="the-href-attribute">The <code class="highlighter-rouge">href</code> Attribute</h3>
<p>This new feature is obvious: You can now create a hyperlink for any part
of a mathematical formula! Here is a screenshot of the MathML Torture Test 21
with additional links, as displayed in WebKit r203109. Click the image to load
the test case in your browser and test it.</p>
<div style="text-align: center; width: 378px; margin-left: auto; margin-right: auto;">
<a href="https://bug-85733-attachments.webkit.org/attachment.cgi?id=274756"><img width="378" src="http://frederic-wang.fr/images/r203109-latin-modern-math-href.png" alt="Screenshot of MathML Torture test 21 with hyperlinks" /></a>
</div>
<h3 id="the-mathvariant-attribute">The <code class="highlighter-rouge">mathvariant</code> Attribute</h3>
<p>Unicode contains <a href="https://en.wikipedia.org/wiki/Mathematical_Alphanumeric_Symbols">Mathematical Alphanumeric Symbols</a> to convey special meaning such as <a href="https://en.wikipedia.org/wiki/Blackboard_bold">double-struck</a> or specific <a href="https://www.w3.org/TR/arabic-math/#N10951">Arabic styles</a>. Characters for these symbols are generally provided by
<a href="http://fred-wang.github.io/MathFonts/">math fonts</a>.
In MathML, mathematical variables are automatically rendered using the
italic characters from this Unicode block. One can also access these characters
via the <code class="highlighter-rouge">mathvariant</code> attribute and that attribute is actually used by many
LaTeX-to-MathML converters.</p>
<p>In the following screenshot, you can see that the letters f, x and y are now
drawn with this special mathematical italic glyphs and that WebKit uses the
conventional fraktur style for the Lie algebra g. Note that the prime is still too
small because WebKit does not make use of the <a href="https://www.microsoft.com/typography/otspec/features_pt.htm#ssty">ssty feature</a> yet.</p>
<div style="text-align: center; width: 500px; margin-left: auto; margin-right: auto;">
<a href="http://frederic-wang.fr/images/safari9-vs-r203109-latin-modern-math-lie-algebra-mathvariant.png"><img width="500" src="http://frederic-wang.fr/images/safari9-vs-r203109-latin-modern-math-lie-algebra-mathvariant.png" alt="Screenshot of Wikipedia article on Lie Algebra" /></a>
<small>Homomorphism of <a href="https://en.wikipedia.org/wiki/Lie_algebra">Lie algebra</a><br />
Top: Safari 9.1.1. Bottom: Safari r203109.</small>
</div>
<h3 id="operators-and-spacing">Operators and Spacing</h3>
<p>As said in my previous blog post, the rendering of large and stretchy operators
have been rewritten a lot and as a consequence the rendering has improved.
Also, I mentioned that <a href="https://wikimedia.org/api/rest_v1/media/math/render/svg/5d166c08a30ad598edc84d0bbe0223be8bcc5a6d">the width of operators may depend on their height</a>. This may cause accumulated approximations
during the computation of preferred widths. The old flexbox-based implementation
<a href="https://bugs.webkit.org/show_bug.cgi?id=107613">incorrectly forced layout during preferred computation</a> to avoid that but a quick
workaround for that security concern caused the approximate
preferred widths to be used for the logical widths. With our new implementation,
the logical width is now correctly calculated.
Finally, we added partial support for the <code class="highlighter-rouge">mpadded</code> element
which is often used to tweak spacing in mathematical formulas.</p>
<p>The screenshot below illustrates the fix for a
<a href="https://bugs.webkit.org/show_bug.cgi?id=151916">serious regression with large operator</a> (summation symbol) as well as improvements in the rendering of
stretchy operators (horizontal braces). Note that the formula has a hack with
a zero-width <code class="highlighter-rouge">mpadded</code> element which used to cause improper spacing
(large gap between the group of a’s and the group of b’s).</p>
<div style="text-align: center; width: 500px; margin-left: auto; margin-right: auto;">
<a href="http://frederic-wang.fr/images/safari9-vs-r203109-latin-modern-math-torture-test-operator-and-spacing.png"><img width="500" src="http://frederic-wang.fr/images/safari9-vs-r203109-latin-modern-math-torture-test-operator-and-spacing.png" alt="Screenshot from Mozilla's MathML torture test using Safari 9.1.1 or the the current refactoring" /></a>
<small>Tests 21 and 22 from the <a href="https://developer.mozilla.org/en-US/docs/Mozilla_MathML_Project/MathML_Torture_Test">MathML torture test</a><br />
Left: Safari 9.1.1. Right: Safari r203109.</small>
</div>
<p>The following screenshot shows how incorrect width computations used to cause
excessive spacing after the stretchy radical and slash symbols:</p>
<div style="text-align: center; width: 484px; margin-left: auto; margin-right: auto;">
<a href="http://frederic-wang.fr/images/safari9-vs-r203109-latin-modern-math-trigonometric-functions-width.png"><img width="484" src="http://frederic-wang.fr/images/safari9-vs-r203109-latin-modern-math-trigonometric-functions-width.png" alt="Screenshot of Wikipedia article on Trigonometric Functions" /></a>
<small><a href="https://en.wikipedia.org/wiki/Trigonometric_functions">Sine of 1 degree</a><br />
Top: Safari 9.1.1. Bottom: Safari r203109.</small>
</div>
<h3 id="the-displaystyle-property">The <code class="highlighter-rouge">displaystyle</code> Property</h3>
<p>Mathematical formulas can be integrated inside a paragraph of text (inline math
in TeX terminology) or displayed in its own horizontally centered paragraph
(display math in TeX terminology). In the latter case, the formula is in
<code class="highlighter-rouge">displaystyle</code> and does not have any restrictions on vertical spacing.
In the former case, the layout of the mathematical formula is modified a bit to
optimize this vertical spacing and to better integrate within the surrounding text.
The <code class="highlighter-rouge">displaystyle</code> property can also be set using the corresponding attribute or
can change automatically in subformulas (e.g. in fractions).</p>
<p>In the following screenshot the fix for the large operator regression is
obvious but you can also notice that the summation is now slightly different
for the definition of a Bézier curve (top) and for the one of
a rational Bézier curve
(bottom). For example, to save some vertical space in the fractions, the
sigma symbol is drawn smaller and the scripts attached to it are moved on
its right. However, the script size could still be improved when we implement
the <a href="https://bugs.webkit.org/show_bug.cgi?id=151916">scriptlevel</a> property.</p>
<div style="text-align: center; width: 500px; margin-left: auto; margin-right: auto;">
<a href="http://frederic-wang.fr/images/safari9-vs-r203109-latin-modern-math-bezier-displaystyle.png"><img width="500" src="http://frederic-wang.fr/images/safari9-vs-r203109-latin-modern-math-bezier-displaystyle.png" alt="Screenshot of Wikipedia article on Bézier Curves" /></a>
<small><a href="https://en.wikipedia.org/wiki/B%C3%A9zier_curve">Definitions of Bézier Curve</a><br />
Left: Safari 9.1.1. Right: Safari r203109.</small>
</div>
<h3 id="opentype-math-parameters">OpenType MATH Parameters</h3>
<p>Many new <a href="https://trac.webkit.org/wiki/MathML/Fonts#OpenFontFormatfeatures">OpenType MATH features</a> have been implemented following the
<a href="http://www.mathml-association.org/MathMLinHTML5/">MathML in HTML5 Implementation Note</a>:</p>
<ul>
<li>Use of the <code class="highlighter-rouge">AxisHeight</code> parameter to set vertical position of fractions,
tables and symmetric operators.</li>
<li>Use of layout constants for radicals (some of them were already used),
scripts and fractions. This improves math spacing and positioning and allows
to adjust them according to value of the <code class="highlighter-rouge">displaystyle</code> property discussed
in the previous section.</li>
<li>Use of the italic correction of large operator glyph to set the position of
subscripts.</li>
</ul>
<p>The screenshots below illustrate some of these improvements. For the first one,
the use of <code class="highlighter-rouge">AxisHeight</code> allows to better align the fraction bar with the plus
sign. For the second one, the use of layout constants for scripts as well
as the italic correction of the integral improves the placement of limits.</p>
<div style="text-align: center; width: 498px; margin-left: auto; margin-right: auto;">
<a href="http://frederic-wang.fr/images/safari9-vs-r203109-latin-modern-math-continued-fraction-axis-height.png"><img width="498" src="http://frederic-wang.fr/images/safari9-vs-r203109-latin-modern-math-continued-fraction-axis-height.png" alt="Screenshot of Wikipedia article on the Continued Fractions" /></a>
<small><a href="https://en.wikipedia.org/wiki/Generalized_continued_fraction">Generalized Continued Fraction</a><br />
Top: Safari 9.1.1. Bottom: Safari r203109.</small>
</div>
<div style="text-align: center; width: 378px; margin-left: auto; margin-right: auto;">
<a href="http://frederic-wang.fr/images/safari9-vs-r203109-latin-modern-math-gamma-function-scripts.png"><img width="378" src="http://frederic-wang.fr/images/safari9-vs-r203109-latin-modern-math-gamma-function-scripts.png" alt="Screenshot of Wikipedia article on the Gamma Function" /></a>
<small><a href="https://en.wikipedia.org/wiki/Gamma_function">Definition of the Gamma Function</a><br />
Top: Safari 9.1.1. Bottom: Safari r203109.</small>
</div>
<h3 id="right-to-left-radicals">Right-to-left radicals</h3>
<p>One of the advantage of the old flexbox-based implementation is that
right-to-left layout was available for free. This support has of course been
preserved in the new implementation. We also added a simple workaround to mirror
radicals using a scale transform
as shown in the screenshot below. However, general
<a href="https://bugs.webkit.org/show_bug.cgi?id=130840">glyph-level mirroring</a> is still
missing.</p>
<div style="text-align: center; width: 500px; margin-left: auto; margin-right: auto;">
<a href="http://frederic-wang.fr/images/safari9-vs-r203109-latin-modern-math-torture-test-rtl.png"><img width="500" src="http://frederic-wang.fr/images/safari9-vs-r203109-latin-modern-math-torture-test-rtl.png" alt="Screenshot of Wikipedia article on Lie Algebra" /></a>
<small>Test 13 from the <a href="https://developer.mozilla.org/en-US/docs/Mozilla_MathML_Project/MathML_Torture_Test">MathML torture test (Machrek Style)</a><br />
Top: Safari 9.1.1. Bottom: Safari r203109.</small>
</div>
<h3 id="conclusion">Conclusion</h3>
<p>Igalia’s web platform team has been able to follow the
<a href="http://www.mathml-association.org/MathMLinHTML5/">MathML in HTML5 Implementation Note</a> in order to significantly improve the rendering of mathematical
expressions in WebKit. More work remains to do but
we will definitely appreciate any feedback that can
help improving native MathML support in web engines.
We are also excited to continue work and collaboration at the next
<a href="http://www.webengineshackfest.org/">Web Engines Hackfest</a>!</p>
MathML Refactoring in WebKit2016-07-09T00:00:00+02:00http://frederic-wang.fr/mathml-refactoring-in-webkit<h2 id="introduction">Introduction</h2>
<p>If you follow WebKit developments, you are certainly aware that
<a href="http://igalia.com/">Igalia</a> has been working on WebKit’s
MathML implementation for some time. More recently, effort has been made to
<a href="https://trac.webkit.org/wiki/MathML/Early_2016_Refactoring">write a clean implementation</a>
addressing issues reported by WebKit reviewers in the past.
After joining Igalia in March, I have been in charge of
getting this work reviewed and merged into WebKit’s development branch.
In the past four months, we have been successful in upstreaming the
<a href="https://trac.webkit.org/wiki/MathML/Early_2016_Refactoring#Phase1">first phase</a>
of the refactoring and the work accomplished is described in this blog post.</p>
<div style="text-align: center; width: 500px; margin-left: auto; margin-right: auto;" id="safari9VSrefactoring">
<a href="http://frederic-wang.fr/images/safari9-vs-refactoring-latin-modern-math.png"><img width="500" src="http://frederic-wang.fr/images/safari9-vs-refactoring-latin-modern-math.png" alt="Screenshot from Mozilla's MathML torture test using Safari 9.1.1 or the the current refactoring" /></a><br />
<small><a href="https://developer.mozilla.org/en-US/docs/Mozilla_MathML_Project/MathML_Torture_Test">MathML torture tests</a> with the <a href="http://www.gust.org.pl/projects/e-foundry/lm-math">Latin Modern Math font</a><br />
Left: Safari 9.1.1. Right: Current MathML branch.</small>
</div>
<p>Note that the focus was on code refactoring so the improvement may not be obvious
for non-developers. Nevertheless many issues have already been fixed as a
consequence of that work: math italic, position of scripts,
stretchy and large operators, rendering update and more.
More importantly, this preliminary step opens the way for
<a href="http://www.mathml-association.org/MathMLinHTML5/">beautiful math rendering based on TeX rules and the OpenType MATH table</a>. Rendering improvements and
implementation of new features have already started in the
<a href="https://trac.webkit.org/wiki/MathML/Early_2016_Refactoring#Phase2">next phase</a>
of the refactoring, so stay tuned…</p>
<h2 id="design-issues">Design Issues</h2>
<p>As explained in a <a href="http://frederic-wang.fr/mathml-at-the-web-engines-hackfest-2015.html">previous report</a>, the main design issues in the
flexbox-based implementation released in 2013
can essentially be summarized in three points:</p>
<ol>
<li>WebKit’s code to stretch operator was not efficient at all and was limited to some basic fences buildable via Unicode characters.</li>
<li>WebKit’s MathML code violated many layout invariants, making the code unreliable.</li>
<li>WebKit’s MathML code relied heavily on the C++ renderer classes for flexboxes and had to manage too many anonymous renderers.</li>
</ol>
<p>For the first point, the performance issue had been fixed by Igalia developers
right after the initial feedback from WebKit developers and
<a href="https://bugs.webkit.org/show_bug.cgi?id=155434">we improved that again</a>
during our refactoring.
Partial support for the OpenType MATH table was added during
<a href="http://www.ulule.com/mathematics-ebooks/">my crowdfunding project</a> and allowed
to stretch more operators with the help of math fonts. For the second point,
<a href="https://bugs.webkit.org/show_bug.cgi?id=107613">the main issue</a> was also
fixed right after the initial feedback. However one could still have some
doubts regarding the layout steps, given the complexity implied by the third
point. That last issue was still problematic so far and addressing it was the
main achievement of our refactoring.</p>
<p>Technically, the dependence on flexbox is unnecessary and the implementation
actually only used a limited set of flexbox features. Thus executing the whole
flexbox code was overkill. It can also be a burden for
people working on other places of the layout code. For example
<a href="https://www.igalia.com/nc/igalia-247/igalian/item/jfernandez/">Javi Fernández</a> has worked on improving the
<a href="http://blogs.igalia.com/jfernandez/2015/01/12/box-alignment-and-grid-layout-ii/">box alignments</a> in the past
and he had hard time fixing the <a href="https://bugs.webkit.org/show_bug.cgi?id=138095">MathML code impacted by his changes</a>. This is probably the cause of the
<a href="https://webkit.org/b/151916">bad position of the summation symbol</a>
that can be seen in the <a href="#safari9VSrefactoring">screenshot above</a>.</p>
<p>From the layout perspective, most of the rendering logic was
implemented in the flexbox classes and the MathML “renderer” classes were really
just managing the creation and update of anonymous nodes and style.
Although this sounds good code
reuse it actually made impossible to understand how and when layout steps
happen or to add more advanced features.
The new implementation replaced this manipulation of the render tree with
simple arithmetic calculations on box metrics which is safer and more reliable.
Also, complex renderers such as
<code class="highlighter-rouge">RenderMathMLScripts</code> or <code class="highlighter-rouge">RenderMathMLRoot</code>
actually achieve better rendering quality with less code!</p>
<p>As an example of the complexity, <code class="highlighter-rouge">RenderMathMLUnderOver</code> can behave as a
<code class="highlighter-rouge">RenderMathMLScripts</code>
in some situation so we really want the former class to reuse the
latter class. As we will see below the old implementation of the two renderers
were quite different: <code class="highlighter-rouge">RenderMathMLUnderOver</code> only relied on setting column
direction in the user agent stylesheet while <code class="highlighter-rouge">RenderMathMLScripts</code> created
a complex render tree structures with anonymous style. Hence it seemed difficult
to share the two cases or to handle DOM changes causing to move from one
case to the other one. With our new implementation, this is simply reduced to
<a href="https://bugs.webkit.org/show_bug.cgi?id=155542">simple C++ inheritance</a>.</p>
<p>When I started to work on WebKit some years ago,
I made the mistake of continuing with
the existing approach. The implementation of <a href="https://bugs.webkit.org/show_bug.cgi?id=99618">multiscripts</a> or <a href="https://bugs.webkit.org/show_bug.cgi?id=44208">automatic italic mathvariant</a> added more anonymous objects
and made the situation even worse. After the end of my crowdfunding project,
<a href="https://www.igalia.com/nc/igalia-247/igalian/item/alex/">Alex Castro</a>
did more cleanup and tried to implement important features such as
<a href="https://bugs.webkit.org/show_bug.cgi?id=133845">displaystyle</a>
but he also soon realized that it was too hard to work with the current code
base…</p>
<h2 id="layout-refactoring">Layout Refactoring</h2>
<p>In order to solve the issues discussed in the previous section,
Javi and Alex worked on a new MathML branch where the first step was to remove
the inheritance on the flexbox layout classes.
During the Web Engines Hackfest 2015, I collaborated with the Igalia’s
web platform team
team to continue the work on this branch. In a second step, we rewrote many
MathML renderer functions so that they stop creating anonymous nodes or style.
We obtained very encouraging results: The implementation looked much
simpler and much more understandable!</p>
<p><a href="https://lists.webkit.org/pipermail/webkit-dev/2015-December/027840.html">Alex announced the initial plan on the webkit-dev mailing list</a>. He started opening bugs
and attaching patches to merge the first step.
However, that step still required many of the flexbox logic and so made code
hard to understand for reviewers. Hence when I joined Igalia four months ago
Alex asked me to try and see how to reorganize patches so that the two initial
steps can be submitted in one go.
This corresponds to the first phase mentioned in the introduction. As indicated
on the <a href="https://trac.webkit.org/wiki/MathML/Early_2016_Refactoring">wiki page</a>,
the layout refactoring consisted in rewriting the following member functions
of each renderer class:</p>
<ul>
<li>computePreferredLogicalWidths: calculate preferred widths, based on the
preferred widths of child renderers.</li>
<li>layoutBlock: set final position and size of child renderers.</li>
<li>firstLineBaseLine: calculate the ascent of the renderer.</li>
<li>paint (optional): perform special painting such as fraction bars.</li>
</ul>
<p>Refactored renderers no longer rely on any flexbox code nor anonymous
renderers and the functions mentioned above essentially perform arithmetic
computations. By reading the code, one can be sure that we follow
standard layout rules and that we do not perform unnecessary reflow.
Moreover, the <a href="http://www.mathml-association.org/MathMLinHTML5/S3.html">rules specific to math rendering</a> are only located in the MathML renderers and can be
better understood. Details for each class are provided in the next subsections.
After all the layout functions were rewritten and the code managing the
render tree structure removed, we were able to make the
<code class="highlighter-rouge">RenderMathMLBlock</code> class
inherit from <code class="highlighter-rouge">RenderBlock</code> instead of <code class="highlighter-rouge">RenderFlexibleBox</code>.
Many of the bugs could
then be immediately closed or otherwise fixed with small follow-up patches.</p>
<h3 id="spacing">Spacing</h3>
<p><code class="highlighter-rouge">RenderMathMLSpace</code> is a simple class inserting blank boxes for adjusting
spacing of math formulas.
Obviously, we do not need any of the complexity of flexbox so it was
straightforward to write the layout functions.</p>
<div style="text-align: center">
<math display="block"><mn>3</mn><mspace width="3em" /><mi>x</mi></math>
<small>Large space between 3 and x.</small>
</div>
<h3 id="grouping">Grouping</h3>
<p><code class="highlighter-rouge">RenderMathMLRow</code> performs rendering of a row of math items.
Since WebKit does not support linebreaking in MathML at the
moment, this is just putting child boxes on a same baseline.
One specificity is that some operators can be stretched vertically and so
<a href="https://wikimedia.org/api/rest_v1/media/math/render/svg/5d166c08a30ad598edc84d0bbe0223be8bcc5a6d">their width may depend on their height</a>.</p>
<div style="text-align: center">
<math display="block">
<mrow>
<mo>{</mo>
<mfrac><mn>2</mn><mi>x</mi></mfrac>
<msup>
<mi>x</mi>
<mn>3</mn>
</msup>
</mrow>
</math>
<small>Row containing a stretched brace, a fraction and a scripted element.</small>
</div>
<p>Again, flexbox features are useless here. With the old code, it was
not clear whether we were violating the CSS invariant with preferred and
logical widths and which kind of relayout or render tree changes would happen
when doing the stretch call. By properly implementing the layout functions
previously mentioned all of this became much more trustable.</p>
<h3 id="fractions">Fractions</h3>
<p><code class="highlighter-rouge">RenderMathMLFraction</code> draws a fraction with numerator and denominator.</p>
<div style="text-align: center">
<math display="block"><mfrac><mrow><mi>x</mi><mo>+</mo><mn>1</mn></mrow><mrow><mi>y</mi><mo>+</mo><mn>2</mn></mrow></mfrac></math>
<small>Simple fraction.</small>
</div>
<p>This used to be implemented using a column direction for the fraction element.
Numerator and denominator were wrapped into anonymous nodes with additional
style to leave space for the fraction bar and to adjust the horizontal
alignments.</p>
<pre>RenderMathMLFraction (flexbox with column direction)
RenderMathMLBlock (anonymous flexbox)
RenderMathMLRow (numerator)
...
RenderMathMLBlock (anonymous flexbox)
RenderMathMLRow (denominator)
...
</pre>
<p>It was relatively easy to implement this without any anonymous nodes and again
the use of flexbox did not sound justified.
For example, to calculate the preferred width we just take the maximum
of the preferred widths of the numerator and denominator.
For the layout, the calculation of the logical width is similar and we
calculate the horizontal coordinates of numerator and denominator so that
their centers are aligned. Vertical metrics are similarly calculated
from the vertical metrics of the numerator and denominator.
During that step, we also fixed some bugs with the <code class="highlighter-rouge">linethickness</code> attribute and
added support for some OpenType MATH table constants.</p>
<h3 id="scripts-above-and-below">Scripts above and below</h3>
<p><code class="highlighter-rouge">RenderMathMLUnderOver</code> is used to attach some scripts above and below a base.
Each child can itself be a horizontal stretchy operator.</p>
<div style="text-align: center">
<math display="block"><mover><mtext>base</mtext><mo>→</mo></mover></math>
<small>Base with stretchy arrow over it.</small>
</div>
<p>This was implemented in the user agent stylesheet by using
flexboxes with column direction for the corresponding MathML elements and
the C++ class had
additional rules to fire the stretching. So the problems and solutions for
this class were essentially a mixed of the cases of
<code class="highlighter-rouge">RenderMathMLFraction</code> and <code class="highlighter-rouge">RenderMathMLRow</code> we just discussed.</p>
<h3 id="subscripts-and-superscripts">Subscripts and Superscripts</h3>
<p><code class="highlighter-rouge">RenderMathMLScripts</code> is used for a base with some arbitrary number of scripts.
All the scripts can have different positions (pre, post, sub, super) and
metrics (width, ascent and descent). We must avoid collisions and take care
of horizontal and vertical alignements.</p>
<div style="text-align: center">
<math display="block">
<mmultiscripts>
<mtext>base</mtext>
<mi>a</mi>
<mi>b</mi>
<mi>c</mi>
<none />
<mprescripts />
<none />
<mi>d</mi>
<mi>e</mi>
<mi>f</mi>
</mmultiscripts>
</math>
<small>Base with pre and post scripts.</small>
</div>
<p>The old code used a complex render tree with additional style to achieve the
best possible result.
However, the quality was still bad as you can see for the script
attached to
the integral in the <a href="#safari9VSrefactoring">screenshot above</a>.
Managing the render tree was a nightmare: Just to give the idea, additional
anonymous node and style were used to allow horizontal and vertical
adjustments (similar to <code class="highlighter-rouge">RenderMathMLFraction</code> above) and prescripts had
negative order property so that they were positioned before the base.</p>
<pre>RenderMathMLScripts
Base Wrapper (anonymous flexbox)
RenderMathMLRow (base)
...
SubSupPair Wrapper (anonymous flexbox with column direction)
RenderMathMLRow (post-subscript)
...
RenderMathMLRow (subperscript)
...
SubSupPair Wrapper (anonymous flexbox with column direction)
RenderMathMLRow (post-subscript)
...
RenderMathMLRow (post-superscript)
...
... (more postscripts)
RenderMathMLBlock (prescripts separator)
SubSupPair Wrapper (anonymous flexbox with column direction and order -1)
RenderMathMLRow (pre-subscript)
...
RenderMathMLRow (pre-subperscript)
...
SubSupPair Wrapper (anonymous flexbox with column direction and order -1)
RenderMathMLRow (pre-subscript)
...
RenderMathMLRow (pre-superscript)
...
... (more prescripts)
</pre>
<p>Rules from TeX and the OpenType MATH table are
relatively complex and we decided to implement them directly in the new
refactoring as otherwise it was impossible to get decent quality. The code is
still complex but we now have clear rules, we only perform simple
calculations and the render tree structure matches the DOM tree.</p>
<h3 id="enclosing-notations">“Enclosing” Notations</h3>
<p><code class="highlighter-rouge">RenderMathMLMenclose</code> is a row of math items with some additional notations.
<a href="https://bugs.webkit.org/show_bug.cgi?id=85729">Gurpreet Kaur implemented this element two years ago</a>
but she followed the same approch, combining anonymous nodes and style for
some simple notations and special painting for others.</p>
<div style="text-align: center">
<math display="block">
<menclose notation="circle updiagonalstrike">
<mi>x</mi><mo>+</mo><mn>1</mn>
</menclose>
</math>
<small>circle and strike notations</small>
</div>
<p>During the refactoring, the code has been completely
rewritten so that <code class="highlighter-rouge">RenderMathMLMenclose</code> is now essentially a derived class
of <code class="highlighter-rouge">RenderMathMLRow</code>
with the measuring and painting functions adjusted to take into account the
additional notations. During that refactoring, we also
<a href="https://lists.webkit.org/pipermail/webkit-dev/2016-March/028064.html">removed support for unused radical notation</a>, which was implemented using an anonymous
<code class="highlighter-rouge">RenderMathMLSquareRoot</code> (see Radicals section below).</p>
<h3 id="helper-classes-for-operators">Helper Classes for Operators</h3>
<p>The <code class="highlighter-rouge">RenderMathMLOperator</code> class is used for math operators.
It was quite complex class and we decided to extract from it two features that
are unrelated to layout:</p>
<ul>
<li>
<p>The <a href="https://www.w3.org/TR/MathML3/appendixc.html">MathML operator dictionary</a> and corresponding search functions have been moved into a
<code class="highlighter-rouge">MathOperatorDictionary</code> class.</p>
</li>
<li>
<p>Selection, measuring and drawing of stretchy operators have been moved into a
<code class="highlighter-rouge">MathOperator</code> class. This is essentially
the <a href="http://frederic-wang.fr/opentype-math-in-harfbuzz.html">low-level text shaping being implemented in HarfBuzz</a>.</p>
</li>
</ul>
<p>The remaining code was indeed the real layout part but the mess with
anonymous node and style was only removed later (see Text Classes below).
Although it seems we just needed to move the code out of <code class="highlighter-rouge">RenderMathMLOperator</code>
into those new classes, the case of <code class="highlighter-rouge">MathOperator</code> was particularly difficult.
We had to split the effort into several small steps to make review possible
and also fixed many issues due to the entanglement and confusion of these
three different features of the <code class="highlighter-rouge">RenderMathMLOperator</code> class…
The work done for <code class="highlighter-rouge">MathOperator</code> actually improved the rendering
of stretchy operators as you can see for the horizontal braces in the
<a href="#safari9VSrefactoring">screenshot above</a>.</p>
<h3 id="radicals">Radicals</h3>
<p><code class="highlighter-rouge">RenderMathMLRoot</code> is used for square root or arbitrary N-th root.
Many of the TeX and OpenType MATH table rules
were already used by the old implementation with anonymous
nodes and style. However, there were bugs difficult to fix related to
<a href="https://bugs.webkit.org/show_bug.cgi?id=143397">zooming</a>,
<a href="https://bugs.webkit.org/show_bug.cgi?id=134018">child removal</a> or
<a href="https://bugs.webkit.org/show_bug.cgi?id=134020">style change</a> due to the
management of the anonymous <code class="highlighter-rouge">RenderMathMLOperator</code> to draw the radical sign.</p>
<div style="text-align: center">
<math display="block">
<msqrt>
<mi>x</mi><mo>+</mo><mn>1</mn>
</msqrt>
<mo>+</mo>
<mroot>
<mrow><mi>x</mi><mo>+</mo><mn>1</mn></mrow>
<mn>3</mn>
</mroot>
</math>
<small>square and cube roots</small>
</div>
<p>The old implementation actually had two classes for the square and general
cases (<code class="highlighter-rouge">RenderMathMLSquareRoot</code> and <code class="highlighter-rouge">RenderMathMLRoot</code>). The usual
technique
with various anonymous wrappers and style was used.
After the refactoring, we were able to merge everything in a single
<code class="highlighter-rouge">RenderMathMLRoot</code>
class. Because the square root behaves as an <code class="highlighter-rouge">mrow</code>, we also made that class
derive from <code class="highlighter-rouge">RenderMathMLRow</code> to reuse as much code as possible.
Here is are how the render trees used to look like:</p>
<pre>RenderMathMLSquareRoot
RenderMathMLBlock (anonymous used for metric adjustements)
RenderMathMLRadicalOperator (anonymous used for the radical symbol)
...
RenderMathMLRootWrapper (anonymous used for the children)
RenderMathMLRow (child 1)
...
RenderMathMLRow (child 2)
...
...
RenderMathMLRow (child N)
...
RenderMathMLRoot
RenderMathMLRootWrapper (anonymous for the index)
...
RenderMathMLBlock (anonymous used for metric adjustements)
RenderMathMLRadicalOperator (anonymous used for the radical symbol)
...
RenderMathMLRootWrapper (anonymous for the base)
...
</pre>
<p>Again, we rewrote the implementation using only simple box positioning.
The difficult part was to get rid of the anonymous
<code class="highlighter-rouge">RenderMathMLRadicalOperator</code> to draw the radical symbol. This class was
derived from <code class="highlighter-rouge">RenderMathMLOperator</code> and extended it with some fallback drawing
when math fonts were not available. After having extracted stretchy operator
shaping from <code class="highlighter-rouge">RenderMathMLOperator</code> it became possible to use the
<code class="highlighter-rouge">MathOperator</code>
helper class to draw the radical symbol. We implemented the fallback for
missing math fonts the same as Gecko: Use a scale transform to stretch
the base glyph for U+221A SQUARE ROOT. As a bonus, we used such transform to
implement glyph mirroring, as required to draw right-to-left radicals in
some Arabic mathematical notations.</p>
<h3 id="text-classes">Text Classes</h3>
<p>These classes are containers for math text such as variables or
operators. There is a generic <code class="highlighter-rouge">RenderMathMLToken</code> class and
a derived class <code class="highlighter-rouge">RenderMathMLOperator</code> adding features
specific to operators such as spacing, dictionary property, stretching…
Anonymous wrappers and style were used to implement
<a href="https://bugs.webkit.org/show_bug.cgi?id=44208">automatic italic mathvariant</a>
or <a href="https://bugs.webkit.org/show_bug.cgi?id=115787">operator spacing</a>. The <code class="highlighter-rouge">RenderText</code> child
of <code class="highlighter-rouge">RenderMathMLOperator</code> was (re)built as an anonymous text node
so that is was possible to
convert U+002D HYPHEN-MINUS into U+2212 MINUS SIGN
or to provide some text for anonymous operators created by
<code class="highlighter-rouge">RenderMathMLFenced</code> (see Unchanged Classes section).</p>
<pre>RenderMathMLToken (e.g. mi element)
RenderMathMLBlock (anonymous flexbox used to apply CSS italic)
RenderBlock (anonymous created elsewhere to honor CSS rules)
RenderText
text run "x"
RenderMathMLOperator (mo element)
RenderMathMLBlock (anonymous flexbox used for spacing)
RenderBlock (anonymous created elsewhere to honor CSS rules)
RenderText (anonymous destroyed and built again)
text run "−"
</pre>
<p>We did a big refactoring to remove all the anonymous nodes
created by the MathML renderer classes.
Just like for <code class="highlighter-rouge">MathOperator</code>, we had to be careful and submit
various small pieces as the text rendering was quite sensible to code change.</p>
<p>The simplified operator spacing that was supported by WebKit was easy to
implement with the new approach.
To do automatic italic mathvariant, we modified the paint function to use
<a href="https://en.wikipedia.org/wiki/Mathematical_Alphanumeric_Symbols">Mathematical Alphanumeric Symbols</a> instead of CSS italic as you can notice for the
variables displayed in the
<a href="#safari9VSrefactoring">screenshot above</a>. Hence we could
remove the <code class="highlighter-rouge">RenderMathMLBlock</code> anonymous wrapper.</p>
<p>The use of an anonymous node for the text prevented it to appear in the dumped
render tree of layout tests and also
required some hacks in the accessibility code
to expose that text. In order to address the cases of
the minus sign and of mfenced operators,
we decided to use our new <code class="highlighter-rouge">MathOperator</code> class again.
Indeed <code class="highlighter-rouge">MathOperator</code> is actually also able to draw unstretched operators
made of a single character and this works for the minus sign and for mfenced
operators used in practice.</p>
<h3 id="unchanged-classes">Unchanged Classes</h3>
<p>Two classes have not been modified but such modifications were not needed to
remove the dependency on <code class="highlighter-rouge">RenderFlexibleBox</code>:</p>
<ul>
<li>
<p><code class="highlighter-rouge">RenderMathMLFenced</code> is used for an mrow-like element that is
<a href="https://www.w3.org/TR/MathML/chapter3.html#presm.mfenced">defined in the MathML specification</a> as strictly equivalent
to constructions with rows and operators.
It is implemented as a derived class
of <code class="highlighter-rouge">RenderMathMLRow</code> and creates anonymous <code class="highlighter-rouge">RenderMathMLOperators</code>. This is the
only remaining class that modifies the render tree structure. Note that
prominent MathML websites and generators do not use the mfenced element,
so it is not a big concern.</p>
</li>
<li>
<p><code class="highlighter-rouge">RenderMathMLTable</code> is used for table layout. It is just derived from
<code class="highlighter-rouge">RenderTable</code>, not <code class="highlighter-rouge">RenderFlexibleBox</code>. We did not change anything for now
but we considered creating our own
implementation in order to make our code independent from HTML table,
to support MathML-specific table features and to make it better integrated
with the rest of the MathML code.</p>
</li>
</ul>
<h2 id="accessibility">Accessibility</h2>
<p>Even if our main focus was on rendering, the changes we made also had impact on
the MathML accessibility code. Indeed, the accessibility tree is generated
from the MathML renderer classes: Since we changed the latter during the
refactoring, we also had to adjust the accessibility code. Fortunately,
we are lucky to have <a href="https://www.igalia.com/nc/igalia-247/igalian/item/jdiggs/">Joanmarie Diggs</a> in our team and she was able to provide some help here.</p>
<p>First, the accessibility code exposes the <code class="highlighter-rouge">linethickness</code> of fractions to
implement Apple’s <code class="highlighter-rouge">AXMathLineThickness</code> attribute. In practice, this is really
necessary to know whether the <code class="highlighter-rouge">linethickness</code> is null or not
(e.g.
<a href="https://en.wikipedia.org/wiki/Binomial_coefficient">binomial coefficient</a> VS
the <a href="https://en.wikipedia.org/wiki/Legendre_symbol">Legendre symbol</a>).
Apple’s unit test seemed to expose the ratio between the actual thickness and
the default thickness but the accessibility code really just reads the actual
thickness calculated by <code class="highlighter-rouge">RenderMathMLFraction</code>.
Our fix and improvement for <code class="highlighter-rouge">linethickness</code> made the Apple’s unit test fail so we
had to adjust <code class="highlighter-rouge">RenderMathMLFraction</code> to expose the value expected by that test.</p>
<p>In general, the accessibility code does not care about anonymous nodes
created for layout purpose and there was some code to avoid exposing
them in the accessibility tree. So removing all the anonymous during the
layout refactoring was actually a good and safe thing to do. There were some
helper functions to implement Apple’s <code class="highlighter-rouge">AXMathRootRadicand</code> and
<code class="highlighter-rouge">AXMathRootIndex</code> attributes that had to be adjusted, though. These functions
used to do some work to skip the anonymous wrappers and we were actually able
to simplify them.</p>
<p>There was also some specific code for the <code class="highlighter-rouge">RenderMathMLOperators</code> and their
anonymous <code class="highlighter-rouge">RenderText</code> that were necessary to expose the text content.
Actually, there was an <a href="https://bugs.webkit.org/show_bug.cgi?id=139582">old bug</a>
in the accessibility code and the anonymous
<code class="highlighter-rouge">RenderMathMLOperators</code> created by mfenced were not correctly exposed. The
unit test
we had for mfenced operators was only checking the text content but it was still
passing and so the regression had never been detected before. After
the layout refactoring we removed the anonymous <code class="highlighter-rouge">RenderText</code> of mfenced
operators and so broke that test…
We thus spent some time to fix the <code class="highlighter-rouge">RenderMathMLOperator</code> code. Essentially,
we removed all the old hacks and only left a specific handling for mfenced
operators. We also used this opportunity to improve and extend our MathML
accessibility tests.</p>
<p>Finally, the MathML accessibility code was directly implemented into a generic
<code class="highlighter-rouge">AccessibilityRenderObject</code> class. There was some functions to access
math nodes and properties but also specific cases scattered all over the code
(anonymous boxes, mfenced operators, math roles etc). In order to
facilitate future work and maintenance we decided to move all the
MathML code into a new <code class="highlighter-rouge">AccessibilityMathMLElement</code> class. Hence the work implied
by the layout refactoring actually encouraged us to improve the organization and
testing of our accessibility code!</p>
<h2 id="conclusion">Conclusion</h2>
<p>In the past four months, Igalia’s web platform team has successfully upstreamed
the refactoring of WebKit’s MathML renderer classes and we are now very confident
about the quality of the layout code.
In addition to the people mentioned above I would personally like to thank
everybody who helped with this work.
More specifically, I am very grateful to other people from Igalia
(<a href="https://www.igalia.com/nc/igalia-247/igalian/item/mrobinson/">Martin Robinson</a>,
<a href="https://www.igalia.com/nc/igalia-247/igalian/item/svillar/">Sergio Villar</a> and
<a href="https://www.igalia.com/nc/igalia-247/igalian/item/mrego/">Manuel Rego</a>)
or Apple (<a href="http://whtconstruct.blogspot.fr/">Brent Fulgham</a>
and <a href="https://en.wikipedia.org/wiki/Darin_Adler">Darin Adler</a>) who have
spent some time to review patches.
As a nice side effect of this work,
mathematical formulas look better and the accessibility code has been improved.
More is happenning in the
<a href="https://trac.webkit.org/wiki/MathML/Early_2016_Refactoring">next two phases</a>.
We are looking forward to continuing
implementation of Web standards and collaboration with browser vendors
at the next <a href="http://www.webengineshackfest.org/">Web Engines Hackfest</a>!</p>
OpenType MATH in HarfBuzz2016-04-16T00:00:00+02:00http://frederic-wang.fr/opentype-math-in-harfbuzz
<div id="p1" class="ltx_para">
<p class="ltx_p"><span class="ltx_text">TL;DR:</span></p>
<ul id="I1" class="ltx_itemize">
<li id="I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_itemize"><span class="ltx_text">•</span></span>
<div id="I1.i1.p1" class="ltx_para">
<p class="ltx_p"><span class="ltx_text">Work is in progress to add OpenType MATH support in
HarfBuzz and will be instrumental for many math rendering engines relying
on that library, including browsers.</span></p>
</div>
</li>
<li id="I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_itemize"><span class="ltx_text">•</span></span>
<div id="I1.i2.p1" class="ltx_para">
<p class="ltx_p"><span class="ltx_text">For stretchy operators, an efficient way to determine the required number
of glyphs and their overlaps has been implemented and is described here.</span></p>
</div>
</li>
</ul>
</div>
<div id="p2" class="ltx_para">
<p class="ltx_p">In the context of <a href="http://www.igalia.com/webkit/" title="" class="ltx_ref">Igalia browser team</a>
effort to implement
<a href="http://www.mathml-association.org/MathMLinHTML5/" title="" class="ltx_ref">MathML support using TeX rules and OpenType features</a>,
I have started implementation of
<a href="https://github.com/behdad/HarfBuzz/issues/235" title="" class="ltx_ref">OpenType MATH support in HarfBuzz</a>. This table <a href="https://blogs.msdn.microsoft.com/murrays/2014/04/27/opentype-math-tables/" title="" class="ltx_ref">from the OpenType standard</a> is made of three subtables:</p>
</div>
<div id="p3" class="ltx_para">
<ul id="I2" class="ltx_itemize">
<li id="I2.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_itemize">•</span>
<div id="I2.i1.p1" class="ltx_para">
<p class="ltx_p">The <span class="ltx_text ltx_font_typewriter">MathConstants</span> table, which contains layout constants. For example, the thickness of the fraction bar of <math id="I2.i1.p1.m1" class="ltx_Math" alttext="\frac{a}{b}" display="inline"><semantics><mfrac><mi>a</mi><mi>b</mi></mfrac><annotation encoding="application/x-tex">\frac{a}{b}</annotation></semantics></math>.</p>
</div>
</li>
<li id="I2.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_itemize">•</span>
<div id="I2.i2.p1" class="ltx_para">
<p class="ltx_p">The <span class="ltx_text ltx_font_typewriter">MathGlyphInfo</span> table, which contains glyph properties. For instance, the italic correction indicating how slanted an integral is e.g. to properly place the subscript in <math id="I2.i2.p1.m1" class="ltx_Math" alttext="\displaystyle\displaystyle\int_{D}" display="inline"><semantics><mstyle displaystyle="true"><msub><mo largeop="true" symmetric="true">∫</mo><mi>D</mi></msub></mstyle><annotation encoding="application/x-tex">\displaystyle\displaystyle\int_{D}</annotation></semantics></math>.</p>
</div>
</li>
<li id="I2.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_itemize">•</span>
<div id="I2.i3.p1" class="ltx_para">
<p class="ltx_p">The <span class="ltx_text ltx_font_typewriter">MathVariants</span> table, which provides larger size variants for a base glyph or data to build a glyph assembly. For example, either a larger parenthesis or a assembly of U+239B, U+239C, U+239D to write something like:</p>
<table id="S0.Ex1" class="ltx_equation ltx_eqn_table">
<tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S0.Ex1.m1" class="ltx_Math" alttext="\left(\frac{\frac{\frac{a}{b}}{\frac{c}{d}}}{\frac{\frac{e}{f}}{\frac{g}{h}}}\right." display="block"><semantics><mrow><mo>(</mo><mfrac><mfrac><mfrac><mi>a</mi><mi>b</mi></mfrac><mfrac><mi>c</mi><mi>d</mi></mfrac></mfrac><mfrac><mfrac><mi>e</mi><mi>f</mi></mfrac><mfrac><mi>g</mi><mi>h</mi></mfrac></mfrac></mfrac></mrow><annotation encoding="application/x-tex">\left(\frac{\frac{\frac{a}{b}}{\frac{c}{d}}}{\frac{\frac{e}{f}}{\frac{g}{h}}}\right.</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr>
</table>
</div>
</li>
</ul>
</div>
<div id="p4" class="ltx_para">
<p class="ltx_p">Code to parse this table was added to Gecko and WebKit two years ago. The
existing code to build glyph assembly in these Web engines was adapted
to use the <span class="ltx_text ltx_font_typewriter">MathVariants</span> data instead of only private tables. However, as
we will see below the <span class="ltx_text ltx_font_typewriter">MathVariants</span> data to build glyph assembly is more
general, with arbitrary number of glyphs or with additional constraints on
glyph overlaps. Also
there are various fallback mechanisms
for old fonts and other bugs that I think we could get rid of
when we move to OpenType MATH fonts only.</p>
</div>
<div id="p5" class="ltx_para">
<p class="ltx_p">In order to add MathML support in Blink, it is very easy to
<a href="https://github.com/fred-wang/chromium.src/compare/master...fred-wang:MATH" title="" class="ltx_ref">import the OpenType MATH parsing code from WebKit</a>. However, after discussions
with some Google developers,
it seems that the best option is to directly add support
for this table in HarfBuzz. Since this library is used by Gecko, by WebKit
(at least the GTK port) and by many other applications such as Servo, XeTeX or
LibreOffice it make senses to share the implementation to improve math rendering
everywhere.</p>
</div>
<div id="p6" class="ltx_para">
<p class="ltx_p">The idea for HarfBuzz is to add an API to</p>
<ol id="I3" class="ltx_enumerate">
<li id="I3.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_enumerate">1.</span>
<div id="I3.i1.p1" class="ltx_para">
<p class="ltx_p">Expose data from the <span class="ltx_text ltx_font_typewriter">MathConstants</span> and <span class="ltx_text ltx_font_typewriter">MathGlyphInfo</span>.</p>
</div>
</li>
<li id="I3.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_enumerate">2.</span>
<div id="I3.i2.p1" class="ltx_para">
<p class="ltx_p">Shape stretchy operators to some target size with the help of the
<span class="ltx_text ltx_font_typewriter">MathVariants</span>.</p>
</div>
</li>
</ol>
<p class="ltx_p">It is then up to a higher-level math rendering engine
(e.g. TeX or MathML rendering engines) to beautifully display mathematical
formulas using this API. The design choice for exposing <span class="ltx_text ltx_font_typewriter">MathConstants</span> and
<span class="ltx_text ltx_font_typewriter">MathGlyphInfo</span> is almost obvious from the reading of the MATH table
specification. The choice for the shaping API is a bit more complex and
discussions is still in progress. For example because we want to accept
stretching after glyph-level mirroring (e.g. to draw RTL clockwise integrals) we
should accept any glyph and not just an input Unicode strings as it is the case
for other HarfBuzz shaping functions. This shaping also depends on a stretching
direction (horizontal/vertical) or on a target size (and Gecko even currently
has various ways to approximate that target size). Finally,
we should also have a way to expose italic correction for a glyph assembly
or to approximate preferred width for Web rendering engines.</p>
</div>
<div id="p7" class="ltx_para">
<p class="ltx_p">As I mentioned at the beginning, the data and algorithm to build glyph assembly
is the most complex part of the OpenType MATH and deserves a special interest.
The idea is that you have a list of <math id="p7.m1" class="ltx_Math" alttext="n\geq 1" display="inline"><semantics><mrow><mi>n</mi><mo>≥</mo><mn>1</mn></mrow><annotation encoding="application/x-tex">n\geq 1</annotation></semantics></math> glyphs available to build
the assembly. For each
<math id="p7.m2" class="ltx_Math" alttext="0\leq i\leq n-1" display="inline"><semantics><mrow><mn>0</mn><mo>≤</mo><mi>i</mi><mo>≤</mo><mrow><mi>n</mi><mo>-</mo><mn>1</mn></mrow></mrow><annotation encoding="application/x-tex">0\leq i\leq n-1</annotation></semantics></math>,
the glyph <math id="p7.m3" class="ltx_Math" alttext="g_{i}" display="inline"><semantics><msub><mi>g</mi><mi>i</mi></msub><annotation encoding="application/x-tex">g_{i}</annotation></semantics></math> has advance <math id="p7.m4" class="ltx_Math" alttext="a_{i}" display="inline"><semantics><msub><mi>a</mi><mi>i</mi></msub><annotation encoding="application/x-tex">a_{i}</annotation></semantics></math> in the stretch direction. Each <math id="p7.m5" class="ltx_Math" alttext="g_{i}" display="inline"><semantics><msub><mi>g</mi><mi>i</mi></msub><annotation encoding="application/x-tex">g_{i}</annotation></semantics></math> has
straight connector part at its start (of length <math id="p7.m6" class="ltx_Math" alttext="s_{i}" display="inline"><semantics><msub><mi>s</mi><mi>i</mi></msub><annotation encoding="application/x-tex">s_{i}</annotation></semantics></math>) and at its end
(of length <math id="p7.m7" class="ltx_Math" alttext="e_{i}" display="inline"><semantics><msub><mi>e</mi><mi>i</mi></msub><annotation encoding="application/x-tex">e_{i}</annotation></semantics></math>) so that we can align the glyphs on the stretch axis and glue
them together. Also, some of the glyphs are “extenders” which means that
they can be repeated 0, 1 or more times to make the assembly as large as
possible. Finally,
the end/start connectors of consecutive glyphs must overlap by at least a fixed
value <math id="p7.m8" class="ltx_Math" alttext="o_{\mathrm{min}}" display="inline"><semantics><msub><mi>o</mi><mi>min</mi></msub><annotation encoding="application/x-tex">o_{\mathrm{min}}</annotation></semantics></math> to avoid gaps at some resolutions but of course without
exceeding the length of the corresponding connectors. This gives some
flexibility to adjust the size of the assembly and get closer to the target
size <math id="p7.m9" class="ltx_Math" alttext="t" display="inline"><semantics><mi>t</mi><annotation encoding="application/x-tex">t</annotation></semantics></math>.</p>
</div>
<figure id="S0.F1" class="ltx_figure"><svg id="S0.F1.pic1" class="ltx_picture ltx_centering" height="294.73" overflow="visible" version="1.1" viewBox="-4.43 -232.91 450.39 294.73" width="450.39"><g transform="matrix(1 0 0 -1 0 -171.09)"><g stroke="#000000"><g fill="#000000"><g stroke-width="0.4pt" color="#000000"><g stroke="#0000FF"><g fill="#0000FF"><g stroke="#E6E6FF" color="#0000FF"><g fill="#E6E6FF"><path d="M 59.06 59.06 L 177.17 59.06 L 177.17 -59.06 L 59.06 -59.06 Z M 0 2.95 L 59.06 2.95 L 59.06 -2.95 L 0 -2.95 Z M 177.17 2.95 L 236.22 2.95 L 236.22 -2.95 L 177.17 -2.95 Z" style="stroke:none"></path></g></g><path d="M 118.11 0" style="fill:none"></path><g transform="matrix(1 0 0 1 110.02 -2.71)" color="#0000FF"><g class="ltx_svg_fog" transform="matrix(1 0 0 -1 0 12.45)"><switch><foreignObject color="#0000FF" height="100%" overflow="visible" width="16.19">
<p class="ltx_p"><math id="S0.F1.pic1.m1" class="ltx_Math" alttext="g_{i}" display="inline"><semantics><msub><mi>g</mi><mi>i</mi></msub><annotation encoding="application/x-tex">g_{i}</annotation></semantics></math></p></foreignObject></switch></g></g><g stroke-width="0.32pt" color="#0000FF"><g stroke-dasharray="none" stroke-dashoffset="0.0pt"><g stroke-linecap="round"><g stroke-linejoin="round"></g></g></g></g><path d="M 0.64 -88.58 L 29.53 -88.58 L 58.42 -88.58" style="fill:none"></path><g transform="matrix(-1 0 0 -1 0.64 -88.58)" color="#0000FF"><path d="M -1.66 2.21 C -1.52 1.38 0 0.14 0.42 0 C 0 -0.14 -1.52 -1.38 -1.66 -2.21" style="fill:none"></path></g><g transform="matrix(1 0 0 1 58.42 -88.58)" color="#0000FF"><path d="M -1.66 2.21 C -1.52 1.38 0 0.14 0.42 0 C 0 -0.14 -1.52 -1.38 -1.66 -2.21" style="fill:none"></path></g><g transform="matrix(1 0 0 1 21.43 -103.16)" color="#0000FF"><g class="ltx_svg_fog" transform="matrix(1 0 0 -1 0 12.45)"><switch><foreignObject color="#0000FF" height="100%" overflow="visible" width="16.19">
<p class="ltx_p"><math id="S0.F1.pic1.m2" class="ltx_Math" alttext="s_{i}" display="inline"><semantics><msub><mi>s</mi><mi>i</mi></msub><annotation encoding="application/x-tex">s_{i}</annotation></semantics></math></p></foreignObject></switch></g></g><path d="M 177.8 -88.58 L 206.69 -88.58 L 235.58 -88.58" style="fill:none"></path><g transform="matrix(-1 0 0 -1 177.8 -88.58)" color="#0000FF"><path d="M -1.66 2.21 C -1.52 1.38 0 0.14 0.42 0 C 0 -0.14 -1.52 -1.38 -1.66 -2.21" style="fill:none"></path></g><g transform="matrix(1 0 0 1 235.58 -88.58)" color="#0000FF"><path d="M -1.66 2.21 C -1.52 1.38 0 0.14 0.42 0 C 0 -0.14 -1.52 -1.38 -1.66 -2.21" style="fill:none"></path></g><g transform="matrix(1 0 0 1 198.6 -103.16)" color="#0000FF"><g class="ltx_svg_fog" transform="matrix(1 0 0 -1 0 12.45)"><switch><foreignObject color="#0000FF" height="100%" overflow="visible" width="16.19">
<p class="ltx_p"><math id="S0.F1.pic1.m3" class="ltx_Math" alttext="e_{i}" display="inline"><semantics><msub><mi>e</mi><mi>i</mi></msub><annotation encoding="application/x-tex">e_{i}</annotation></semantics></math></p></foreignObject></switch></g></g><path d="M 0.64 -177.17 L 118.11 -177.17 L 235.58 -177.17" style="fill:none"></path><g transform="matrix(-1 0 0 -1 0.64 -177.17)" color="#0000FF"><path d="M -1.66 2.21 C -1.52 1.38 0 0.14 0.42 0 C 0 -0.14 -1.52 -1.38 -1.66 -2.21" style="fill:none"></path></g><g transform="matrix(1 0 0 1 235.58 -177.17)" color="#0000FF"><path d="M -1.66 2.21 C -1.52 1.38 0 0.14 0.42 0 C 0 -0.14 -1.52 -1.38 -1.66 -2.21" style="fill:none"></path></g><g transform="matrix(1 0 0 1 110.02 -191.74)" color="#0000FF"><g class="ltx_svg_fog" transform="matrix(1 0 0 -1 0 12.45)"><switch><foreignObject color="#0000FF" height="100%" overflow="visible" width="16.19">
<p class="ltx_p"><math id="S0.F1.pic1.m4" class="ltx_Math" alttext="a_{i}" display="inline"><semantics><msub><mi>a</mi><mi>i</mi></msub><annotation encoding="application/x-tex">a_{i}</annotation></semantics></math></p></foreignObject></switch></g></g></g></g><g stroke="#FF0000"><g fill="#FF0000"><g stroke="#FFE6E6" color="#FF0000"><g fill="#FFE6E6"><path d="M 265.75 29.53 L 383.86 29.53 L 383.86 -88.58 L 265.75 -88.58 Z M 206.69 -26.57 L 265.75 -26.57 L 265.75 -32.48 L 206.69 -32.48 Z M 383.86 -26.57 L 442.91 -26.57 L 442.91 -32.48 L 383.86 -32.48 Z" style="stroke:none"></path></g></g><path d="M 324.8 -29.53" style="fill:none"></path><g transform="matrix(1 0 0 1 310.9 -32.24)" color="#FF0000"><g class="ltx_svg_fog" transform="matrix(1 0 0 -1 0 12.45)"><switch><foreignObject color="#FF0000" height="100%" overflow="visible" width="27.81">
<p class="ltx_p"><math id="S0.F1.pic1.m5" class="ltx_Math" alttext="g_{i+1}" display="inline"><semantics><msub><mi>g</mi><mrow><mi>i</mi><mo>+</mo><mn>1</mn></mrow></msub><annotation encoding="application/x-tex">g_{i+1}</annotation></semantics></math></p></foreignObject></switch></g></g><path d="M 207.33 -118.11 L 236.22 -118.11 L 265.11 -118.11" style="fill:none"></path><g transform="matrix(-1 0 0 -1 207.33 -118.11)" color="#FF0000"><path d="M -1.66 2.21 C -1.52 1.38 0 0.14 0.42 0 C 0 -0.14 -1.52 -1.38 -1.66 -2.21" style="fill:none"></path></g><g transform="matrix(1 0 0 1 265.11 -118.11)" color="#FF0000"><path d="M -1.66 2.21 C -1.52 1.38 0 0.14 0.42 0 C 0 -0.14 -1.52 -1.38 -1.66 -2.21" style="fill:none"></path></g><g transform="matrix(1 0 0 1 222.31 -132.68)" color="#FF0000"><g class="ltx_svg_fog" transform="matrix(1 0 0 -1 0 12.45)"><switch><foreignObject color="#FF0000" height="100%" overflow="visible" width="27.81">
<p class="ltx_p"><math id="S0.F1.pic1.m6" class="ltx_Math" alttext="s_{i+1}" display="inline"><semantics><msub><mi>s</mi><mrow><mi>i</mi><mo>+</mo><mn>1</mn></mrow></msub><annotation encoding="application/x-tex">s_{i+1}</annotation></semantics></math></p></foreignObject></switch></g></g><path d="M 384.49 -118.11 L 413.39 -118.11 L 442.28 -118.11" style="fill:none"></path><g transform="matrix(-1 0 0 -1 384.49 -118.11)" color="#FF0000"><path d="M -1.66 2.21 C -1.52 1.38 0 0.14 0.42 0 C 0 -0.14 -1.52 -1.38 -1.66 -2.21" style="fill:none"></path></g><g transform="matrix(1 0 0 1 442.28 -118.11)" color="#FF0000"><path d="M -1.66 2.21 C -1.52 1.38 0 0.14 0.42 0 C 0 -0.14 -1.52 -1.38 -1.66 -2.21" style="fill:none"></path></g><g transform="matrix(1 0 0 1 399.48 -132.68)" color="#FF0000"><g class="ltx_svg_fog" transform="matrix(1 0 0 -1 0 12.45)"><switch><foreignObject color="#FF0000" height="100%" overflow="visible" width="27.81">
<p class="ltx_p"><math id="S0.F1.pic1.m7" class="ltx_Math" alttext="e_{i+1}" display="inline"><semantics><msub><mi>e</mi><mrow><mi>i</mi><mo>+</mo><mn>1</mn></mrow></msub><annotation encoding="application/x-tex">e_{i+1}</annotation></semantics></math></p></foreignObject></switch></g></g><path d="M 207.33 -206.69 L 324.8 -206.69 L 442.28 -206.69" style="fill:none"></path><g transform="matrix(-1 0 0 -1 207.33 -206.69)" color="#FF0000"><path d="M -1.66 2.21 C -1.52 1.38 0 0.14 0.42 0 C 0 -0.14 -1.52 -1.38 -1.66 -2.21" style="fill:none"></path></g><g transform="matrix(1 0 0 1 442.28 -206.69)" color="#FF0000"><path d="M -1.66 2.21 C -1.52 1.38 0 0.14 0.42 0 C 0 -0.14 -1.52 -1.38 -1.66 -2.21" style="fill:none"></path></g><g transform="matrix(1 0 0 1 310.9 -221.27)" color="#FF0000"><g class="ltx_svg_fog" transform="matrix(1 0 0 -1 0 12.45)"><switch><foreignObject color="#FF0000" height="100%" overflow="visible" width="27.81">
<p class="ltx_p"><math id="S0.F1.pic1.m8" class="ltx_Math" alttext="a_{i+1}" display="inline"><semantics><msub><mi>a</mi><mrow><mi>i</mi><mo>+</mo><mn>1</mn></mrow></msub><annotation encoding="application/x-tex">a_{i+1}</annotation></semantics></math></p></foreignObject></switch></g></g></g></g><path d="M 207.33 -14.76 L 221.46 -14.76 L 235.58 -14.76" style="fill:none"></path><g transform="matrix(-1 0 0 -1 207.33 -14.76)"><path d="M -1.66 2.21 C -1.52 1.38 0 0.14 0.42 0 C 0 -0.14 -1.52 -1.38 -1.66 -2.21" style="fill:none"></path></g><g transform="matrix(1 0 0 1 235.58 -14.76)"><path d="M -1.66 2.21 C -1.52 1.38 0 0.14 0.42 0 C 0 -0.14 -1.52 -1.38 -1.66 -2.21" style="fill:none"></path></g><g transform="matrix(1 0 0 1 201.74 -29.34)"><g class="ltx_svg_fog" transform="matrix(1 0 0 -1 0 12.45)"><switch><foreignObject color="#000000" height="100%" overflow="visible" width="39.43">
<p class="ltx_p"><math id="S0.F1.pic1.m9" class="ltx_Math" alttext="o_{i,i+1}" display="inline"><semantics><msub><mi>o</mi><mrow><mi>i</mi><mo>,</mo><mrow><mi>i</mi><mo>+</mo><mn>1</mn></mrow></mrow></msub><annotation encoding="application/x-tex">o_{i,i+1}</annotation></semantics></math></p></foreignObject></switch></g></g></g></g></g></g></svg>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 1: </span>Two adjacent glyphs in an assembly</figcaption>
</figure>
<div id="p8" class="ltx_para">
<p class="ltx_p">To ensure that the width/height is distributed equally and the symmetry of the
shape is preserved, the MATH table specification suggests the following iterative
algorithm to determine the number of extenders and the connector overlaps
to reach a minimal target size <math id="p8.m1" class="ltx_Math" alttext="t" display="inline"><semantics><mi>t</mi><annotation encoding="application/x-tex">t</annotation></semantics></math>:</p>
</div>
<div id="p9" class="ltx_para">
<ol id="I4" class="ltx_enumerate">
<li id="I4.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_enumerate"><span class="ltx_text ltx_font_italic">1.</span></span>
<div id="I4.i1.p1" class="ltx_para">
<p class="ltx_p"><span class="ltx_text ltx_font_italic">Assemble all parts by overlapping connectors by maximum amount, and
removing all extenders. This gives the smallest possible result.</span></p>
</div>
</li>
<li id="I4.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_enumerate"><span class="ltx_text ltx_font_italic">2.</span></span>
<div id="I4.i2.p1" class="ltx_para">
<p class="ltx_p"><span class="ltx_text ltx_font_italic">Determine how much extra width/height can be distributed into all
connections between neighboring parts. If that is enough to achieve the size
goal, extend each connection equally by changing overlaps of connectors to
finish the job.</span></p>
</div>
</li>
<li id="I4.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_enumerate"><span class="ltx_text ltx_font_italic">3.</span></span>
<div id="I4.i3.p1" class="ltx_para">
<p class="ltx_p"><span class="ltx_text ltx_font_italic">If all connections have been extended to minimum overlap and further
growth is needed, add one of each extender, and repeat the process from the
first step.</span></p>
</div>
</li>
</ol>
</div>
<div id="p10" class="ltx_para">
<p class="ltx_p">We note that at each step, each extender is repeated the same number of times
<math id="p10.m1" class="ltx_Math" alttext="r\geq 0" display="inline"><semantics><mrow><mi>r</mi><mo>≥</mo><mn>0</mn></mrow><annotation encoding="application/x-tex">r\geq 0</annotation></semantics></math>. So if <math id="p10.m2" class="ltx_Math" alttext="I_{\mathrm{Ext}}" display="inline"><semantics><msub><mi>I</mi><mi>Ext</mi></msub><annotation encoding="application/x-tex">I_{\mathrm{Ext}}</annotation></semantics></math> (respectively <math id="p10.m3" class="ltx_Math" alttext="I_{\mathrm{NonExt}}" display="inline"><semantics><msub><mi>I</mi><mi>NonExt</mi></msub><annotation encoding="application/x-tex">I_{\mathrm{NonExt}}</annotation></semantics></math>) is the set of
indices <math id="p10.m4" class="ltx_Math" alttext="0\leq i\leq n-1" display="inline"><semantics><mrow><mn>0</mn><mo>≤</mo><mi>i</mi><mo>≤</mo><mrow><mi>n</mi><mo>-</mo><mn>1</mn></mrow></mrow><annotation encoding="application/x-tex">0\leq i\leq n-1</annotation></semantics></math> such that <math id="p10.m5" class="ltx_Math" alttext="g_{i}" display="inline"><semantics><msub><mi>g</mi><mi>i</mi></msub><annotation encoding="application/x-tex">g_{i}</annotation></semantics></math> is an extender
(respectively is not
an extender) we have <math id="p10.m6" class="ltx_Math" alttext="r_{i}=r" display="inline"><semantics><mrow><msub><mi>r</mi><mi>i</mi></msub><mo>=</mo><mi>r</mi></mrow><annotation encoding="application/x-tex">r_{i}=r</annotation></semantics></math> (respectively <math id="p10.m7" class="ltx_Math" alttext="r_{i}=1" display="inline"><semantics><mrow><msub><mi>r</mi><mi>i</mi></msub><mo>=</mo><mn>1</mn></mrow><annotation encoding="application/x-tex">r_{i}=1</annotation></semantics></math>). The size we can reach
at step <math id="p10.m8" class="ltx_Math" alttext="r" display="inline"><semantics><mi>r</mi><annotation encoding="application/x-tex">r</annotation></semantics></math> is at most the one obtained with the minimal connector overlap
<math id="p10.m9" class="ltx_Math" alttext="o_{\mathrm{min}}" display="inline"><semantics><msub><mi>o</mi><mi>min</mi></msub><annotation encoding="application/x-tex">o_{\mathrm{min}}</annotation></semantics></math> that is
</p>
<table id="S0.Ex2" class="ltx_equation ltx_eqn_table">
<tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S0.Ex2.m1" class="ltx_Math" alttext="\sum_{i=0}^{N-1}\left(\sum_{j=1}^{r_{i}}{a_{i}-o_{\mathrm{min}}}\right)+o_{
\mathrm{min}}=\left(\sum_{i\in I_{\mathrm{NonExt}}}{a_{i}-o_{\mathrm{min}}}
\right)+\left(\sum_{i\in I_{\mathrm{Ext}}}r{(a_{i}-o_{\mathrm{min}})}\right)+o%
_{\mathrm{min}}" display="block"><semantics><mrow><mrow><mrow><munderover><mo largeop="true" movablelimits="false" symmetric="true">∑</mo><mrow><mi>i</mi><mo>=</mo><mn>0</mn></mrow><mrow><mi>N</mi><mo>-</mo><mn>1</mn></mrow></munderover><mrow><mo>(</mo><mrow><mrow><munderover><mo largeop="true" movablelimits="false" symmetric="true">∑</mo><mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow><msub><mi>r</mi><mi>i</mi></msub></munderover><msub><mi>a</mi><mi>i</mi></msub></mrow><mo>-</mo><msub><mi>o</mi><mi>min</mi></msub></mrow><mo>)</mo></mrow></mrow><mo>+</mo><msub><mi>o</mi><mi>min</mi></msub></mrow><mo>=</mo><mrow><mrow><mo>(</mo><mrow><mrow><munder><mo largeop="true" movablelimits="false" symmetric="true">∑</mo><mrow><mi>i</mi><mo>∈</mo><msub><mi>I</mi><mi>NonExt</mi></msub></mrow></munder><msub><mi>a</mi><mi>i</mi></msub></mrow><mo>-</mo><msub><mi>o</mi><mi>min</mi></msub></mrow><mo>)</mo></mrow><mo>+</mo><mrow><mo>(</mo><mrow><munder><mo largeop="true" movablelimits="false" symmetric="true">∑</mo><mrow><mi>i</mi><mo>∈</mo><msub><mi>I</mi><mi>Ext</mi></msub></mrow></munder><mrow><mi>r</mi><mo></mo><mrow><mo stretchy="false">(</mo><mrow><msub><mi>a</mi><mi>i</mi></msub><mo>-</mo><msub><mi>o</mi><mi>min</mi></msub></mrow><mo stretchy="false">)</mo></mrow></mrow></mrow><mo>)</mo></mrow><mo>+</mo><msub><mi>o</mi><mi>min</mi></msub></mrow></mrow><annotation encoding="application/x-tex">\sum_{i=0}^{N-1}\left(\sum_{j=1}^{r_{i}}{a_{i}-o_{\mathrm{min}}}\right)+o_{
\mathrm{min}}=\left(\sum_{i\in I_{\mathrm{NonExt}}}{a_{i}-o_{\mathrm{min}}}
\right)+\left(\sum_{i\in I_{\mathrm{Ext}}}r{(a_{i}-o_{\mathrm{min}})}\right)+o%
_{\mathrm{min}}</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr>
</table>
</div>
<div id="p11" class="ltx_para">
<p class="ltx_p">We let <math id="p11.m1" class="ltx_Math" alttext="N_{\mathrm{Ext}}={|I_{\mathrm{Ext}}|}" display="inline"><semantics><mrow><msub><mi>N</mi><mi>Ext</mi></msub><mo>=</mo><mrow><mo stretchy="false">|</mo><msub><mi>I</mi><mi>Ext</mi></msub><mo stretchy="false">|</mo></mrow></mrow><annotation encoding="application/x-tex">N_{\mathrm{Ext}}={|I_{\mathrm{Ext}}|}</annotation></semantics></math> and
<math id="p11.m2" class="ltx_Math" alttext="N_{\mathrm{NonExt}}={|I_{\mathrm{NonExt}}|}" display="inline"><semantics><mrow><msub><mi>N</mi><mi>NonExt</mi></msub><mo>=</mo><mrow><mo stretchy="false">|</mo><msub><mi>I</mi><mi>NonExt</mi></msub><mo stretchy="false">|</mo></mrow></mrow><annotation encoding="application/x-tex">N_{\mathrm{NonExt}}={|I_{\mathrm{NonExt}}|}</annotation></semantics></math> be the number of extenders and
non-extenders. We also let
<math id="p11.m3" class="ltx_Math" alttext="S_{\mathrm{Ext}}=\sum_{i\in I_{\mathrm{Ext}}}a_{i}" display="inline"><semantics><mrow><msub><mi>S</mi><mi>Ext</mi></msub><mo>=</mo><mrow><msub><mo largeop="true" symmetric="true">∑</mo><mrow><mi>i</mi><mo>∈</mo><msub><mi>I</mi><mi>Ext</mi></msub></mrow></msub><msub><mi>a</mi><mi>i</mi></msub></mrow></mrow><annotation encoding="application/x-tex">S_{\mathrm{Ext}}=\sum_{i\in I_{\mathrm{Ext}}}a_{i}</annotation></semantics></math> and
<math id="p11.m4" class="ltx_Math" alttext="S_{\mathrm{NonExt}}=\sum_{i\in I_{\mathrm{NonExt}}}a_{i}" display="inline"><semantics><mrow><msub><mi>S</mi><mi>NonExt</mi></msub><mo>=</mo><mrow><msub><mo largeop="true" symmetric="true">∑</mo><mrow><mi>i</mi><mo>∈</mo><msub><mi>I</mi><mi>NonExt</mi></msub></mrow></msub><msub><mi>a</mi><mi>i</mi></msub></mrow></mrow><annotation encoding="application/x-tex">S_{\mathrm{NonExt}}=\sum_{i\in I_{\mathrm{NonExt}}}a_{i}</annotation></semantics></math> be the sum of advances
for extenders and non-extenders. If we want the advance of the glyph assembly
to reach the minimal size <math id="p11.m5" class="ltx_Math" alttext="t" display="inline"><semantics><mi>t</mi><annotation encoding="application/x-tex">t</annotation></semantics></math> then
</p>
<table id="S0.Ex3" class="ltx_equation ltx_eqn_table">
<tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S0.Ex3.m1" class="ltx_Math" alttext="{S_{\mathrm{NonExt}}-o_{\mathrm{min}}\left(N_{\mathrm{NonExt}}-1\right)}+{r%
\left(S_{\mathrm{Ext}}-o_{\mathrm{min}}N_{\mathrm{Ext}}\right)}\geq t" display="block"><semantics><mrow><mrow><mrow><msub><mi>S</mi><mi>NonExt</mi></msub><mo>-</mo><mrow><msub><mi>o</mi><mi>min</mi></msub><mo></mo><mrow><mo>(</mo><mrow><msub><mi>N</mi><mi>NonExt</mi></msub><mo>-</mo><mn>1</mn></mrow><mo>)</mo></mrow></mrow></mrow><mo>+</mo><mrow><mi>r</mi><mo></mo><mrow><mo>(</mo><mrow><msub><mi>S</mi><mi>Ext</mi></msub><mo>-</mo><mrow><msub><mi>o</mi><mi>min</mi></msub><mo></mo><msub><mi>N</mi><mi>Ext</mi></msub></mrow></mrow><mo>)</mo></mrow></mrow></mrow><mo>≥</mo><mi>t</mi></mrow><annotation encoding="application/x-tex">{S_{\mathrm{NonExt}}-o_{\mathrm{min}}\left(N_{\mathrm{NonExt}}-1\right)}+{r%
\left(S_{\mathrm{Ext}}-o_{\mathrm{min}}N_{\mathrm{Ext}}\right)}\geq t</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr>
</table>
</div>
<div id="p12" class="ltx_para">
<p class="ltx_p">We can assume <math id="p12.m1" class="ltx_Math" alttext="S_{\mathrm{Ext}}-o_{\mathrm{min}}N_{\mathrm{Ext}}>0" display="inline"><semantics><mrow><mrow><msub><mi>S</mi><mi>Ext</mi></msub><mo>-</mo><mrow><msub><mi>o</mi><mi>min</mi></msub><mo></mo><msub><mi>N</mi><mi>Ext</mi></msub></mrow></mrow><mo>></mo><mn>0</mn></mrow><annotation encoding="application/x-tex">S_{\mathrm{Ext}}-o_{\mathrm{min}}N_{\mathrm{Ext}}>0</annotation></semantics></math> or otherwise we
would have the extreme case where the overlap takes at least the full
advance of each extender. Then we obtain
</p>
<table id="S0.Ex4" class="ltx_equation ltx_eqn_table">
<tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S0.Ex4.m1" class="ltx_Math" alttext="r\geq r_{\mathrm{min}}=\max\left(0,\left\lceil\frac{t-{S_{\mathrm{NonExt}}+o_{
\mathrm{min}}\left(N_{\mathrm{NonExt}}-1\right)}}{S_{\mathrm{Ext}}-o_{\mathrm{
min}}N_{\mathrm{Ext}}}\right\rceil\right)" display="block"><semantics><mrow><mi>r</mi><mo>≥</mo><msub><mi>r</mi><mi>min</mi></msub><mo>=</mo><mrow><mi>max</mi><mo></mo><mrow><mo>(</mo><mn>0</mn><mo>,</mo><mrow><mo>⌈</mo><mfrac><mrow><mrow><mi>t</mi><mo>-</mo><msub><mi>S</mi><mi>NonExt</mi></msub></mrow><mo>+</mo><mrow><msub><mi>o</mi><mi>min</mi></msub><mo></mo><mrow><mo>(</mo><mrow><msub><mi>N</mi><mi>NonExt</mi></msub><mo>-</mo><mn>1</mn></mrow><mo>)</mo></mrow></mrow></mrow><mrow><msub><mi>S</mi><mi>Ext</mi></msub><mo>-</mo><mrow><msub><mi>o</mi><mi>min</mi></msub><mo></mo><msub><mi>N</mi><mi>Ext</mi></msub></mrow></mrow></mfrac><mo>⌉</mo></mrow><mo>)</mo></mrow></mrow></mrow><annotation encoding="application/x-tex">r\geq r_{\mathrm{min}}=\max\left(0,\left\lceil\frac{t-{S_{\mathrm{NonExt}}+o_{
\mathrm{min}}\left(N_{\mathrm{NonExt}}-1\right)}}{S_{\mathrm{Ext}}-o_{\mathrm{
min}}N_{\mathrm{Ext}}}\right\rceil\right)</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr>
</table>
</div>
<div id="p13" class="ltx_para">
<p class="ltx_p">This provides a first simplification of the algorithm sketched in the
MATH table specification:
Directly start iteration at step <math id="p13.m1" class="ltx_Math" alttext="r_{\mathrm{min}}" display="inline"><semantics><msub><mi>r</mi><mi>min</mi></msub><annotation encoding="application/x-tex">r_{\mathrm{min}}</annotation></semantics></math>. Note that at each step we
start at <em class="ltx_emph">possibly different</em> maximum overlaps and decrease all of them
by a same value. It is not clear what to do when one of the overlap reaches
<math id="p13.m2" class="ltx_Math" alttext="o_{\mathrm{min}}" display="inline"><semantics><msub><mi>o</mi><mi>min</mi></msub><annotation encoding="application/x-tex">o_{\mathrm{min}}</annotation></semantics></math> while others can still be decreased. However, the sketched
algorithm says all the connectors should reach minimum overlap before
the next increment of <math id="p13.m3" class="ltx_Math" alttext="r" display="inline"><semantics><mi>r</mi><annotation encoding="application/x-tex">r</annotation></semantics></math>,
which means the target size will indeed be reached at step <math id="p13.m4" class="ltx_Math" alttext="r_{\mathrm{min}}" display="inline"><semantics><msub><mi>r</mi><mi>min</mi></msub><annotation encoding="application/x-tex">r_{\mathrm{min}}</annotation></semantics></math>.</p>
</div>
<div id="p14" class="ltx_para">
<p class="ltx_p">One possible interpretation is to stop overlap decreasing for the adjacent
connectors that reached minimum overlap
and to continue uniform decreasing for the others until
all the connectors reach minimum overlap. In that case we may lose equal
distribution or symmetry. In practice, this should probably not matter much.
So we propose instead the dual option which should behave more or less the
same in most cases: Start with all overlaps set to <math id="p14.m1" class="ltx_Math" alttext="o_{\mathrm{min}}" display="inline"><semantics><msub><mi>o</mi><mi>min</mi></msub><annotation encoding="application/x-tex">o_{\mathrm{min}}</annotation></semantics></math> and increase
them evenly to reach a same value <math id="p14.m2" class="ltx_Math" alttext="o" display="inline"><semantics><mi>o</mi><annotation encoding="application/x-tex">o</annotation></semantics></math>. By the same reasoning as above we want
the inequality
</p>
<table id="S0.Ex5" class="ltx_equation ltx_eqn_table">
<tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S0.Ex5.m1" class="ltx_Math" alttext="{S_{\mathrm{NonExt}}-o\left(N_{\mathrm{NonExt}}-1\right)}+{r_{\mathrm{min}}
\left(S_{\mathrm{Ext}}-oN_{\mathrm{Ext}}\right)}\geq t" display="block"><semantics><mrow><mrow><mrow><msub><mi>S</mi><mi>NonExt</mi></msub><mo>-</mo><mrow><mi>o</mi><mo></mo><mrow><mo>(</mo><mrow><msub><mi>N</mi><mi>NonExt</mi></msub><mo>-</mo><mn>1</mn></mrow><mo>)</mo></mrow></mrow></mrow><mo>+</mo><mrow><msub><mi>r</mi><mi>min</mi></msub><mo></mo><mrow><mo>(</mo><mrow><msub><mi>S</mi><mi>Ext</mi></msub><mo>-</mo><mrow><mi>o</mi><mo></mo><msub><mi>N</mi><mi>Ext</mi></msub></mrow></mrow><mo>)</mo></mrow></mrow></mrow><mo>≥</mo><mi>t</mi></mrow><annotation encoding="application/x-tex">{S_{\mathrm{NonExt}}-o\left(N_{\mathrm{NonExt}}-1\right)}+{r_{\mathrm{min}}
\left(S_{\mathrm{Ext}}-oN_{\mathrm{Ext}}\right)}\geq t</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr>
</table>
<p class="ltx_p">which can be rewritten
</p>
<table id="S0.Ex6" class="ltx_equation ltx_eqn_table">
<tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S0.Ex6.m1" class="ltx_Math" alttext="S_{\mathrm{NonExt}}+r_{\mathrm{min}}S_{\mathrm{Ext}}-{o\left(N_{\mathrm{NonExt%
}}+{r_{\mathrm{min}}N_{\mathrm{Ext}}}-1\right)}\geq t" display="block"><semantics><mrow><mrow><mrow><msub><mi>S</mi><mi>NonExt</mi></msub><mo>+</mo><mrow><msub><mi>r</mi><mi>min</mi></msub><mo></mo><msub><mi>S</mi><mi>Ext</mi></msub></mrow></mrow><mo>-</mo><mrow><mi>o</mi><mo></mo><mrow><mo>(</mo><mrow><mrow><msub><mi>N</mi><mi>NonExt</mi></msub><mo>+</mo><mrow><msub><mi>r</mi><mi>min</mi></msub><mo></mo><msub><mi>N</mi><mi>Ext</mi></msub></mrow></mrow><mo>-</mo><mn>1</mn></mrow><mo>)</mo></mrow></mrow></mrow><mo>≥</mo><mi>t</mi></mrow><annotation encoding="application/x-tex">S_{\mathrm{NonExt}}+r_{\mathrm{min}}S_{\mathrm{Ext}}-{o\left(N_{\mathrm{NonExt%
}}+{r_{\mathrm{min}}N_{\mathrm{Ext}}}-1\right)}\geq t</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr>
</table>
</div>
<div id="p15" class="ltx_para">
<p class="ltx_p">We note that <math id="p15.m1" class="ltx_Math" alttext="N=N_{\mathrm{NonExt}}+{r_{\mathrm{min}}N_{\mathrm{Ext}}}" display="inline"><semantics><mrow><mi>N</mi><mo>=</mo><mrow><msub><mi>N</mi><mi>NonExt</mi></msub><mo>+</mo><mrow><msub><mi>r</mi><mi>min</mi></msub><mo></mo><msub><mi>N</mi><mi>Ext</mi></msub></mrow></mrow></mrow><annotation encoding="application/x-tex">N=N_{\mathrm{NonExt}}+{r_{\mathrm{min}}N_{\mathrm{Ext}}}</annotation></semantics></math> is just the exact
number of glyphs used in the assembly. If there is only a single glyph, then the
overlap value is irrelevant so we can assume
<math id="p15.m2" class="ltx_Math" alttext="N_{\mathrm{NonExt}}+{rN_{\mathrm{Ext}}}-1=N-1\geq 1" display="inline"><semantics><mrow><mrow><mrow><msub><mi>N</mi><mi>NonExt</mi></msub><mo>+</mo><mrow><mi>r</mi><mo></mo><msub><mi>N</mi><mi>Ext</mi></msub></mrow></mrow><mo>-</mo><mn>1</mn></mrow><mo>=</mo><mrow><mi>N</mi><mo>-</mo><mn>1</mn></mrow><mo>≥</mo><mn>1</mn></mrow><annotation encoding="application/x-tex">N_{\mathrm{NonExt}}+{rN_{\mathrm{Ext}}}-1=N-1\geq 1</annotation></semantics></math>. This provides the
greatest theorical value for the overlap <math id="p15.m3" class="ltx_Math" alttext="o" display="inline"><semantics><mi>o</mi><annotation encoding="application/x-tex">o</annotation></semantics></math>:
</p>
<table id="S0.Ex7" class="ltx_equation ltx_eqn_table">
<tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S0.Ex7.m1" class="ltx_Math" alttext="o_{\mathrm{min}}\leq o\leq o_{\mathrm{max}}^{\mathrm{theorical}}=\frac{S_{
\mathrm{NonExt}}+r_{\mathrm{min}}S_{\mathrm{Ext}}-t}{N_{\mathrm{NonExt}}+{r_{
\mathrm{min}}N_{\mathrm{Ext}}}-1}" display="block"><semantics><mrow><msub><mi>o</mi><mi>min</mi></msub><mo>≤</mo><mi>o</mi><mo>≤</mo><msubsup><mi>o</mi><mi>max</mi><mi>theorical</mi></msubsup><mo>=</mo><mfrac><mrow><mrow><msub><mi>S</mi><mi>NonExt</mi></msub><mo>+</mo><mrow><msub><mi>r</mi><mi>min</mi></msub><mo></mo><msub><mi>S</mi><mi>Ext</mi></msub></mrow></mrow><mo>-</mo><mi>t</mi></mrow><mrow><mrow><msub><mi>N</mi><mi>NonExt</mi></msub><mo>+</mo><mrow><msub><mi>r</mi><mi>min</mi></msub><mo></mo><msub><mi>N</mi><mi>Ext</mi></msub></mrow></mrow><mo>-</mo><mn>1</mn></mrow></mfrac></mrow><annotation encoding="application/x-tex">o_{\mathrm{min}}\leq o\leq o_{\mathrm{max}}^{\mathrm{theorical}}=\frac{S_{
\mathrm{NonExt}}+r_{\mathrm{min}}S_{\mathrm{Ext}}-t}{N_{\mathrm{NonExt}}+{r_{
\mathrm{min}}N_{\mathrm{Ext}}}-1}</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr>
</table>
</div>
<div id="p16" class="ltx_para">
<p class="ltx_p">Of course, we also have to take into account the limit imposed by the start and
end connector lengths. So <math id="p16.m1" class="ltx_Math" alttext="o_{\mathrm{max}}" display="inline"><semantics><msub><mi>o</mi><mi>max</mi></msub><annotation encoding="application/x-tex">o_{\mathrm{max}}</annotation></semantics></math> must also be at most
<math id="p16.m2" class="ltx_Math" alttext="\min{(e_{i},s_{i+1})}" display="inline"><semantics><mrow><mi>min</mi><mo></mo><mrow><mo stretchy="false">(</mo><msub><mi>e</mi><mi>i</mi></msub><mo>,</mo><msub><mi>s</mi><mrow><mi>i</mi><mo>+</mo><mn>1</mn></mrow></msub><mo stretchy="false">)</mo></mrow></mrow><annotation encoding="application/x-tex">\min{(e_{i},s_{i+1})}</annotation></semantics></math> for <math id="p16.m3" class="ltx_Math" alttext="0\leq i\leq n-2" display="inline"><semantics><mrow><mn>0</mn><mo>≤</mo><mi>i</mi><mo>≤</mo><mrow><mi>n</mi><mo>-</mo><mn>2</mn></mrow></mrow><annotation encoding="application/x-tex">0\leq i\leq n-2</annotation></semantics></math>. But if <math id="p16.m4" class="ltx_Math" alttext="r_{\mathrm{min}}\geq 2" display="inline"><semantics><mrow><msub><mi>r</mi><mi>min</mi></msub><mo>≥</mo><mn>2</mn></mrow><annotation encoding="application/x-tex">r_{\mathrm{min}}\geq 2</annotation></semantics></math>
then extender copies are connected and so <math id="p16.m5" class="ltx_Math" alttext="o_{\mathrm{max}}" display="inline"><semantics><msub><mi>o</mi><mi>max</mi></msub><annotation encoding="application/x-tex">o_{\mathrm{max}}</annotation></semantics></math> must also be at most
<math id="p16.m6" class="ltx_Math" alttext="\min{(e_{i},s_{i})}" display="inline"><semantics><mrow><mi>min</mi><mo></mo><mrow><mo stretchy="false">(</mo><msub><mi>e</mi><mi>i</mi></msub><mo>,</mo><msub><mi>s</mi><mi>i</mi></msub><mo stretchy="false">)</mo></mrow></mrow><annotation encoding="application/x-tex">\min{(e_{i},s_{i})}</annotation></semantics></math> for <math id="p16.m7" class="ltx_Math" alttext="i\in I_{\mathrm{Ext}}" display="inline"><semantics><mrow><mi>i</mi><mo>∈</mo><msub><mi>I</mi><mi>Ext</mi></msub></mrow><annotation encoding="application/x-tex">i\in I_{\mathrm{Ext}}</annotation></semantics></math>. To summarize, <math id="p16.m8" class="ltx_Math" alttext="o_{\mathrm{max}}" display="inline"><semantics><msub><mi>o</mi><mi>max</mi></msub><annotation encoding="application/x-tex">o_{\mathrm{max}}</annotation></semantics></math> is
the minimum of <math id="p16.m9" class="ltx_Math" alttext="o_{\mathrm{max}}^{\mathrm{theorical}}" display="inline"><semantics><msubsup><mi>o</mi><mi>max</mi><mi>theorical</mi></msubsup><annotation encoding="application/x-tex">o_{\mathrm{max}}^{\mathrm{theorical}}</annotation></semantics></math>, of <math id="p16.m10" class="ltx_Math" alttext="e_{i}" display="inline"><semantics><msub><mi>e</mi><mi>i</mi></msub><annotation encoding="application/x-tex">e_{i}</annotation></semantics></math> for <math id="p16.m11" class="ltx_Math" alttext="0\leq i\leq n-2" display="inline"><semantics><mrow><mn>0</mn><mo>≤</mo><mi>i</mi><mo>≤</mo><mrow><mi>n</mi><mo>-</mo><mn>2</mn></mrow></mrow><annotation encoding="application/x-tex">0\leq i\leq n-2</annotation></semantics></math>,
of <math id="p16.m12" class="ltx_Math" alttext="s_{i}" display="inline"><semantics><msub><mi>s</mi><mi>i</mi></msub><annotation encoding="application/x-tex">s_{i}</annotation></semantics></math> <math id="p16.m13" class="ltx_Math" alttext="1\leq i\leq n-1" display="inline"><semantics><mrow><mn>1</mn><mo>≤</mo><mi>i</mi><mo>≤</mo><mrow><mi>n</mi><mo>-</mo><mn>1</mn></mrow></mrow><annotation encoding="application/x-tex">1\leq i\leq n-1</annotation></semantics></math> and possibly of <math id="p16.m14" class="ltx_Math" alttext="e_{0}" display="inline"><semantics><msub><mi>e</mi><mn>0</mn></msub><annotation encoding="application/x-tex">e_{0}</annotation></semantics></math> (if <math id="p16.m15" class="ltx_Math" alttext="0\in I_{\mathrm{Ext}}" display="inline"><semantics><mrow><mn>0</mn><mo>∈</mo><msub><mi>I</mi><mi>Ext</mi></msub></mrow><annotation encoding="application/x-tex">0\in I_{\mathrm{Ext}}</annotation></semantics></math>)
and of of <math id="p16.m16" class="ltx_Math" alttext="s_{n-1}" display="inline"><semantics><msub><mi>s</mi><mrow><mi>n</mi><mo>-</mo><mn>1</mn></mrow></msub><annotation encoding="application/x-tex">s_{n-1}</annotation></semantics></math> (if <math id="p16.m17" class="ltx_Math" alttext="{n-1}\in I_{\mathrm{Ext}}" display="inline"><semantics><mrow><mrow><mi>n</mi><mo>-</mo><mn>1</mn></mrow><mo>∈</mo><msub><mi>I</mi><mi>Ext</mi></msub></mrow><annotation encoding="application/x-tex">{n-1}\in I_{\mathrm{Ext}}</annotation></semantics></math>).</p>
</div>
<div id="p17" class="ltx_para">
<p class="ltx_p">With the algorithm described above
<math id="p17.m1" class="ltx_Math" alttext="N_{\mathrm{Ext}}" display="inline"><semantics><msub><mi>N</mi><mi>Ext</mi></msub><annotation encoding="application/x-tex">N_{\mathrm{Ext}}</annotation></semantics></math>, <math id="p17.m2" class="ltx_Math" alttext="N_{\mathrm{NonExt}}" display="inline"><semantics><msub><mi>N</mi><mi>NonExt</mi></msub><annotation encoding="application/x-tex">N_{\mathrm{NonExt}}</annotation></semantics></math>, <math id="p17.m3" class="ltx_Math" alttext="S_{\mathrm{Ext}}" display="inline"><semantics><msub><mi>S</mi><mi>Ext</mi></msub><annotation encoding="application/x-tex">S_{\mathrm{Ext}}</annotation></semantics></math>, <math id="p17.m4" class="ltx_Math" alttext="S_{\mathrm{NonExt}}" display="inline"><semantics><msub><mi>S</mi><mi>NonExt</mi></msub><annotation encoding="application/x-tex">S_{\mathrm{NonExt}}</annotation></semantics></math>
and <math id="p17.m5" class="ltx_Math" alttext="r_{\mathrm{min}}" display="inline"><semantics><msub><mi>r</mi><mi>min</mi></msub><annotation encoding="application/x-tex">r_{\mathrm{min}}</annotation></semantics></math> and <math id="p17.m6" class="ltx_Math" alttext="o_{\mathrm{max}}" display="inline"><semantics><msub><mi>o</mi><mi>max</mi></msub><annotation encoding="application/x-tex">o_{\mathrm{max}}</annotation></semantics></math> can all be obtained using simple loops
on the glyphs <math id="p17.m7" class="ltx_Math" alttext="g_{i}" display="inline"><semantics><msub><mi>g</mi><mi>i</mi></msub><annotation encoding="application/x-tex">g_{i}</annotation></semantics></math>
and so the complexity is <math id="p17.m8" class="ltx_Math" alttext="O(n)" display="inline"><semantics><mrow><mi>O</mi><mo></mo><mrow><mo stretchy="false">(</mo><mi>n</mi><mo stretchy="false">)</mo></mrow></mrow><annotation encoding="application/x-tex">O(n)</annotation></semantics></math>. In practice <math id="p17.m9" class="ltx_Math" alttext="n" display="inline"><semantics><mi>n</mi><annotation encoding="application/x-tex">n</annotation></semantics></math> is small: For
existing fonts, assemblies are made of at most three non-extenders and two extenders that is
<math id="p17.m10" class="ltx_Math" alttext="n\leq 5" display="inline"><semantics><mrow><mi>n</mi><mo>≤</mo><mn>5</mn></mrow><annotation encoding="application/x-tex">n\leq 5</annotation></semantics></math> (incidentally, Gecko and WebKit do not currently support larger values of <math id="p17.m11" class="ltx_Math" alttext="n" display="inline"><semantics><mi>n</mi><annotation encoding="application/x-tex">n</annotation></semantics></math>).
This means that all the operations described above can be considered to have
constant complexity. This is much better than a naive implementation of the
iterative algorithm sketched in the OpenType MATH table specification which
seems to require at worst
</p>
<table id="S0.Ex8" class="ltx_equation ltx_eqn_table">
<tr class="ltx_equation ltx_eqn_row ltx_align_baseline">
<td class="ltx_eqn_cell ltx_eqn_center_padleft"></td>
<td class="ltx_eqn_cell ltx_align_center"><math id="S0.Ex8.m1" class="ltx_Math" alttext="\sum_{r=0}^{r_{\mathrm{min}}-1}{N_{\mathrm{NonExt}}+rN_{\mathrm{Ext}}}=N_{
\mathrm{NonExt}}r_{\mathrm{min}}+\frac{r_{\mathrm{min}}\left(r_{\mathrm{min}}-%
1\right)}{2}N_{\mathrm{Ext}}={O(n\times r_{\mathrm{min}}^{2})}" display="block"><semantics><mrow><mrow><mrow><munderover><mo largeop="true" movablelimits="false" symmetric="true">∑</mo><mrow><mi>r</mi><mo>=</mo><mn>0</mn></mrow><mrow><msub><mi>r</mi><mi>min</mi></msub><mo>-</mo><mn>1</mn></mrow></munderover><msub><mi>N</mi><mi>NonExt</mi></msub></mrow><mo>+</mo><mrow><mi>r</mi><mo></mo><msub><mi>N</mi><mi>Ext</mi></msub></mrow></mrow><mo>=</mo><mrow><mrow><msub><mi>N</mi><mi>NonExt</mi></msub><mo></mo><msub><mi>r</mi><mi>min</mi></msub></mrow><mo>+</mo><mrow><mfrac><mrow><msub><mi>r</mi><mi>min</mi></msub><mo></mo><mrow><mo>(</mo><mrow><msub><mi>r</mi><mi>min</mi></msub><mo>-</mo><mn>1</mn></mrow><mo>)</mo></mrow></mrow><mn>2</mn></mfrac><mo></mo><msub><mi>N</mi><mi>Ext</mi></msub></mrow></mrow><mo>=</mo><mrow><mi>O</mi><mo></mo><mrow><mo stretchy="false">(</mo><mrow><mi>n</mi><mo>×</mo><msubsup><mi>r</mi><mi>min</mi><mn>2</mn></msubsup></mrow><mo stretchy="false">)</mo></mrow></mrow></mrow><annotation encoding="application/x-tex">\sum_{r=0}^{r_{\mathrm{min}}-1}{N_{\mathrm{NonExt}}+rN_{\mathrm{Ext}}}=N_{
\mathrm{NonExt}}r_{\mathrm{min}}+\frac{r_{\mathrm{min}}\left(r_{\mathrm{min}}-%
1\right)}{2}N_{\mathrm{Ext}}={O(n\times r_{\mathrm{min}}^{2})}</annotation></semantics></math></td>
<td class="ltx_eqn_cell ltx_eqn_center_padright"></td>
</tr>
</table>
<p class="ltx_p">and at least <math id="p17.m12" class="ltx_Math" alttext="\Omega(r_{\mathrm{min}})" display="inline"><semantics><mrow><mi mathvariant="normal">Ω</mi><mo></mo><mrow><mo stretchy="false">(</mo><msub><mi>r</mi><mi>min</mi></msub><mo stretchy="false">)</mo></mrow></mrow><annotation encoding="application/x-tex">\Omega(r_{\mathrm{min}})</annotation></semantics></math>.</p>
</div>
<div id="p18" class="ltx_para">
<p class="ltx_p">One of issue is that the number of extender repetitions <math id="p18.m1" class="ltx_Math" alttext="r_{\mathrm{min}}" display="inline"><semantics><msub><mi>r</mi><mi>min</mi></msub><annotation encoding="application/x-tex">r_{\mathrm{min}}</annotation></semantics></math> and
the number of glyphs in the assembly <math id="p18.m2" class="ltx_Math" alttext="N" display="inline"><semantics><mi>N</mi><annotation encoding="application/x-tex">N</annotation></semantics></math> can
become arbitrary large since the target size <math id="p18.m3" class="ltx_Math" alttext="t" display="inline"><semantics><mi>t</mi><annotation encoding="application/x-tex">t</annotation></semantics></math> can take large values
e.g. if one writes
<span class="ltx_text ltx_font_typewriter">\underbrace{\hspace{65535em}}</span>
in LaTeX. The improvement proposed here does not solve that issue since setting
the coordinates of each glyph in the assembly and painting them
require <math id="p18.m4" class="ltx_Math" alttext="\Theta(N)" display="inline"><semantics><mrow><mi mathvariant="normal">Θ</mi><mo></mo><mrow><mo stretchy="false">(</mo><mi>N</mi><mo stretchy="false">)</mo></mrow></mrow><annotation encoding="application/x-tex">\Theta(N)</annotation></semantics></math> operations as well as
(in the case of HarfBuzz) a glyph buffer of size
<math id="p18.m5" class="ltx_Math" alttext="N" display="inline"><semantics><mi>N</mi><annotation encoding="application/x-tex">N</annotation></semantics></math>. However, such large stretchy operators do not
happen in real-life mathematical formulas. Hence to avoid possible hangs in
Web engines a solution is to impose a maximum limit <math id="p18.m6" class="ltx_Math" alttext="N_{\mathrm{max}}" display="inline"><semantics><msub><mi>N</mi><mi>max</mi></msub><annotation encoding="application/x-tex">N_{\mathrm{max}}</annotation></semantics></math> for the
number of glyph in the assembly so that the complexity is limited by the
size of the DOM tree. Currently, the proposal for HarfBuzz is
<math id="p18.m7" class="ltx_Math" alttext="N_{\mathrm{max}}=128" display="inline"><semantics><mrow><msub><mi>N</mi><mi>max</mi></msub><mo>=</mo><mn>128</mn></mrow><annotation encoding="application/x-tex">N_{\mathrm{max}}=128</annotation></semantics></math>. This means that if each assembly glyph is 1em large
you won’t be able to draw stretchy operators of size more than 128em, which
sounds a quite reasonable bound.
With the above proposal, <math id="p18.m8" class="ltx_Math" alttext="r_{\mathrm{min}}" display="inline"><semantics><msub><mi>r</mi><mi>min</mi></msub><annotation encoding="application/x-tex">r_{\mathrm{min}}</annotation></semantics></math> and so
<math id="p18.m9" class="ltx_Math" alttext="N" display="inline"><semantics><mi>N</mi><annotation encoding="application/x-tex">N</annotation></semantics></math> can be determined very quickly and the cases <math id="p18.m10" class="ltx_Math" alttext="N\geq N_{\mathrm{max}}" display="inline"><semantics><mrow><mi>N</mi><mo>≥</mo><msub><mi>N</mi><mi>max</mi></msub></mrow><annotation encoding="application/x-tex">N\geq N_{\mathrm{max}}</annotation></semantics></math> rejected, so that we avoid losing time with such edge cases…</p>
</div>
<div id="p19" class="ltx_para">
<p class="ltx_p">Finally, because in our proposal we use the same overlap <math id="p19.m1" class="ltx_Math" alttext="o" display="inline"><semantics><mi>o</mi><annotation encoding="application/x-tex">o</annotation></semantics></math> everywhere
an alternative for HarfBuzz would be to set the output buffer size to
<math id="p19.m2" class="ltx_Math" alttext="n" display="inline"><semantics><mi>n</mi><annotation encoding="application/x-tex">n</annotation></semantics></math> (i.e. ignore <math id="p19.m3" class="ltx_Math" alttext="r-1" display="inline"><semantics><mrow><mi>r</mi><mo>-</mo><mn>1</mn></mrow><annotation encoding="application/x-tex">r-1</annotation></semantics></math> copies of each extender and only keep the first one).
This will leave gaps that the client can fix by repeating extenders
as long as <math id="p19.m4" class="ltx_Math" alttext="o" display="inline"><semantics><mi>o</mi><annotation encoding="application/x-tex">o</annotation></semantics></math> is also provided. Then HarfBuzz math shaping can be done
with a complexity in time and space of just <math id="p19.m5" class="ltx_Math" alttext="O(n)" display="inline"><semantics><mrow><mi>O</mi><mo></mo><mrow><mo stretchy="false">(</mo><mi>n</mi><mo stretchy="false">)</mo></mrow></mrow><annotation encoding="application/x-tex">O(n)</annotation></semantics></math> and it will be up to the
client to optimize or limit the painting of extenders for large values of <math id="p19.m6" class="ltx_Math" alttext="N" display="inline"><semantics><mi>N</mi><annotation encoding="application/x-tex">N</annotation></semantics></math>…</p>
</div>
MathML at the Web Engines Hackfest 20152015-12-20T00:00:00+01:00http://frederic-wang.fr/mathml-at-the-web-engines-hackfest-2015<h2 id="hackfest">Hackfest</h2>
<p>Two weeks ago, I travelled to Spain to participate to the second <a href="http://www.webengineshackfest.org/">Web Engines
Hackfest</a> which was sponsored by
<a href="http://www.igalia.com/">Igalia</a> and <a href="https://www.collabora.com/">Collabora</a>.
Such an event has been organized by Igalia since 2009 and used to be focused on
WebkitGTK+. It is great to see that it has now been extended to any Web engines &
platforms and that a large percentage of non-igalian developers has been
involved this year. If you did not get the opportunity to attend this event
or if you are curious about what happened there, take a look at the
<a href="https://github.com/Igalia/webengineshackfest/wiki">wiki page</a> or
<a href="https://www.flickr.com/photos/webhackfest/sets/72157660108679713/">flickr album</a>.</p>
<div style="width: 500px; margin-left: auto; margin-right: auto; text-align: center;">
<a href="https://www.flickr.com/photos/webhackfest/23662778162/in/album-72157660108679713/" title="Last day of the hackfest"><img src="https://farm6.staticflickr.com/5659/23662778162_34eff30b58.jpg" width="500" height="375" alt="Last day of the hackfest" /></a><br />
<small>Photo from <a href="https://www.flickr.com/photos/webhackfest/">@webengineshackfest</a> licensed under
<a href="http://creativecommons.org/licenses/by-sa/2.0/">
<img alt="Creative Commons Attribution-ShareAlike" src="https://i.creativecommons.org/l/by-sa/2.0/80x15.png" style="border: 0;" /></a></small></div>
<p>I really like this kind of hacking-oriented and participant-driven event where developers can meet face to face, organize themselves in small collaboration
groups to efficiently make progress on a task or give passionate talk about
what they have recently been working on.
The only small bemol I have is that it is still mainly focused on WebKit/Blink
developments. Probably, the lack of Mozilla/Microsoft participants is
probably due to
<a href="https://wiki.mozilla.org/Coincidental_work_weeks/2014_Portland">Mozilla Coincidental</a>
<a href="https://wiki.mozilla.org/Coincidental_work_weeks/2015_Orlando">Work Weeks</a>
happening at the same period and to the proprietary nature of
<a href="https://blogs.windows.com/msedgedev/tag/edgehtml/">EdgeHTML</a>
(although <a href="https://blogs.windows.com/msedgedev/2015/12/05/open-source-chakra-core/">this is changing</a>?). However, I am
confident that Igalia will try and address this issue and I am looking
forward to coming back next year!</p>
<h2 id="mathml-developments">MathML developments</h2>
<p>This year, Igalia developer <a href="http://www.igalia.com/nc/igalia-247/igalian/item/alex/">Alejandro G. Castro</a> wanted to work with me on WebKit’s MathML layout code and more specifically on his <a href="https://github.com/alexgcastro/webkit/tree/MathMLLayout">MathML refactoring branch</a>. Indeed, as many people (including Google developers) who have tried to work on WebKit’s code in the past, he arrived to the conclusion that the WebKit’s MathML layout code has many design issues that make it a burden for the rest of the layout team and too complex to allow future improvements. I was quite excited about the work he has done with <a href="http://www.igalia.com/nc/igalia-247/igalian/item/jfernandez/">Javier Fernández</a> to try and move to a simple box model closer to what exists in Gecko and thus I actually extended my stay to work one whole week with them. We already submitted
<a href="https://lists.webkit.org/pipermail/webkit-dev/2015-December/027840.html">our proposal to the webkit-dev mailing list</a> and received positive feedback, so we
will now start merging what is ready.
At the end of the week, we were quite satisfied about the new approach and confident it will facilitate future maintenance and developements :-)</p>
<div style="width: 500px; margin-left: auto; margin-right: auto; text-align: center;">
<a href="https://www.flickr.com/photos/webhackfest/23403316889/" title="Main room"><img src="https://farm1.staticflickr.com/709/23403316889_a32cffd144.jpg" width="500" height="375" alt="Main room" /></a><br />
<small>Photo from <a href="https://www.flickr.com/photos/webhackfest/">@webengineshackfest</a> licensed under
<a href="http://creativecommons.org/licenses/by-sa/2.0/">
<img alt="Creative Commons Attribution-ShareAlike" src="https://i.creativecommons.org/l/by-sa/2.0/80x15.png" style="border: 0;" /></a></small></div>
<p>While reading a <a href="https://lists.w3.org/Archives/Public/www-math/2015Dec/0000.html">recent thread on the Math WG mailing list</a>, I realized that many MathML people have only vague understanding of why Google (or to be more accurate, the 3 or 4 engineers who really spent some time reviewing and testing the WebKit code) considered the implementation to be unsafe and not ready for release. Even worse,
<a href="https://kwarc.info/people/mkohlhase">Michael Kholhase</a> pointed out that for someone totally ignorant of the technical implementation details, the communication made some years ago around the “flexbox-based approach” gave the impression that it was “the right way” (indeed, it somewhat did improve the initial implementation) and the rationale to change that approach was not obvious. So let’s just try and give a quick overview of the main problems, even if I doubt someone can have good understanding of the situation without diving into the C++ code:</p>
<ol>
<li>WebKit’s code to stretch operator was not efficient at all and was limited to
some basic fences buildable via Unicode characters.</li>
<li>WebKit’s MathML code violated many layout invariants, making the code unreliable.</li>
<li>WebKit’s MathML code relied heavily on the C++ renderer classes for flexboxes
and has to manage too many anonymous renderers.</li>
</ol>
<p>The main security concerns were addressed a long time ago by <a href="http://www.igalia.com/nc/igalia-247/igalian/item/mrobinson/">Martin Robinson</a> and me. Glyph assembly for stretchy operators are now drawn using low-level font painting primitive instead of creating one renderer object for each piece and the preferred width for them no longer depends on vertical metrics (although we still need some work to obtain Gecko’s good operator spacing). Also, during my <a href="http://www.ulule.com/mathematics-ebooks/">crowdfunding project</a>, I implemented partial support for the <a href="https://www.microsoft.com/typography/otspec/math.htm">OpenType MATH table</a> in WebKit and more specifically the MathVariant subtable, which allows to directly use construction of stretchy operators specified by the font designer and not only the few Unicode constructions.</p>
<p>However, the MathML layout code still modifies the renderer tree to force the presence of anonymous renderers and still applies specific CSS rules to them. It is also spending too much time trying to adapt the parent flexbox renderer class which has at the same time too much features for what is needed for MathML (essentially automatic box alignment) and not enough to get exact placement and measuring needed for high-quality rendering (the TeXBook rules are more complex, taking into account various parameters for box shifts, drops, gaps etc).</p>
<p>During the hackfest, we started to rewrite a clean implementation of some MathML renderer classes similar to Gecko’s one and based on the <a href="http://www.mathml-association.org/MathMLinHTML5/">MathML in HTML5 implementation note</a>.
The code now becomes very simple and understandable. It can be summarized into four main functions. For instance, to draw a fraction we need:</p>
<ul>
<li><code class="highlighter-rouge">computePreferredLogicalWidths</code> which sets the preferred width of the
fraction during the first layout pass, by considering the widest between numerator and denominator.</li>
<li><code class="highlighter-rouge">layoutBlock</code> and <code class="highlighter-rouge">firstLineBaseline</code> which calculate the final
width/height/ascent of the fraction element and position the numerator and
denominator.</li>
<li><code class="highlighter-rouge">paint</code> which draws the fraction bar.</li>
</ul>
<p>Perhaps, the best example to illustrate how the complexity has been reduced
is the case of the renderer of <code class="highlighter-rouge">mmultiscripts</code>/<code class="highlighter-rouge">msub</code>/<code class="highlighter-rouge">msup</code>/<code class="highlighter-rouge">msubsup</code> elements
(attaching an arbitrary number of subscripts and superscripts before or after a
base). In the current WebKit implementation, we have to create three anonymous wrappers
(a first one for the base, a second one for prescripts and a third one for postscripts) and an anonymous wrapper
for each subscript/superscript pair, add alignment styles for these wrappers
and spend a lot of time
maintaining the integrity of the renderer tree when dynamic changes happen.
With the new code, we just need to do arithmetic calculations to position the
base and script boxes. This is somewhat more complex than the fraction example above but still, it remains arithmetic calculations and we can not reduce any further if we wish quality comparable to TeXBook / MATH rules.
We actually take into account many parameters
from the OpenType MATH table to get much better placement of scripts. We were able to fix <a href="https://bugs.webkit.org/show_bug.cgi?id=130325">bug 130325</a> in about twenty minutes instead of fighting with a CSS “negative margin” hack on anonymous renderers.</p>
<h2 id="mathml-dicussions">MathML dicussions</h2>
<p>The first day of the hackfest we also had an interesting “breakout session” to define the tasks to work on during the hackfest. Alejandro briefly presented the status of his refactoring branch and his plan for the hackfest. As said in the previous section, we have been quite successful in following this plan: Although it is not fully complete yet, we expect to merge the current changes soon. <a href="https://twitter.com/abrax5">Dominik Röttsches</a> who works on Blink’s font and layout modules was present at the MathML session and it was a good opportunity to discuss the future of MathML in Chrome. I gave him some references regarding the OpenType MATH table, math fonts and the <a href="http://www.mathml-association.org/MathMLinHTML5/">MathML in HTML5 implementation note</a>. Dominik said he will forward the references to his colleagues working on layout so that we can have deeper technical dicussion about MathML in Blink in the long term. Also, I mentioned <a href="https://github.com/googlei18n/noto-fonts/issues/330">noto-fonts issue 330</a>, which will be important for math rendering in Android and actually does not depend on the MathML issue, so that’s certainly something we could focus on in the short term.</p>
<div style="width: 500px; margin-left: auto; margin-right: auto; text-align: center;">
<a href="https://www.flickr.com/photos/webhackfest/23144428143/" title="Álex and Fred during the MathML breakout session"><img src="https://farm1.staticflickr.com/775/23144428143_1e0fa79fcc.jpg" width="500" height="375" alt="Álex and Fred during the MathML breakout session" /></a><br />
<small>Photo from <a href="https://www.flickr.com/photos/webhackfest/">@webengineshackfest</a> licensed under
<a href="http://creativecommons.org/licenses/by-sa/2.0/">
<img alt="Creative Commons Attribution-ShareAlike" src="https://i.creativecommons.org/l/by-sa/2.0/80x15.png" style="border: 0;" /></a></small></div>
<p>Alejandro also proposed to me to prepare a talk about <a href="http://www.webengineshackfest.org/slides/mathml-in-web-engines-by-frederic-wang/mathml/">MathML in Web Engines</a> and exciting stuff happening with the <a href="http://mathml-association.org/">MathML Association</a>. I thus gave a brief overview of MathML and presented some demos of native support in Gecko. I also explained how we are trying to start a constructive approach to solve miscommunication between users, spec authors and implementers ; and gather technical and financial resources to obtain a proper solution. In a slightly more technical part, I presented Microsoft’s OpenType MATH table and how it can be used for math rendering (and MathML in particular). Finally, I proposed my personal roadmap for MathML in Web engines. Although I do not believe I am a really great speaker, I received positive feedback from attendees. One of the thing I realized is that I do not know anything about the status and plan in EdgeHTML and so overlooked to mention it in my presentation. Its proprietary nature makes hard for external people to contribute to a MathML implementation and the fact that Microsoft is moving away from ActiveX <i>de facto</i> excludes <a href="http://www.dessci.com/en/products/mathplayer/">third-party plugin for good and fast math rendering</a> in the future. After I went back to Paris, I thus wrote to Microsoft employee <a href="https://www.christianheilmann.com/">Christian Heilmann</a> (previously working for Mozilla), mentioning at least the
<a href="http://www.mathml-association.org/MathMLinHTML5/">MathML in HTML5 Implementation Note</a> and its <a href="http://tests.mathml-association.org/">test suite</a> as a starting point. MathML is currently on the <a href="https://wpdev.uservoice.com/forums/257854-microsoft-edge-developer/filters/top">first page of the most voted feature requested for Microsoft Edge</a> and given the new direction taken with Microsoft Edge,
I hope we could start a discussion on MathML in EdgeHTML…</p>
<h2 id="conclusion">Conclusion</h2>
<p>This was a great hackfest and I’d like to thank again all the participants and sponsors
for making it possible! As Alejandro wrote to me, “I think we have started a very interesting work regarding MathML and web engines in the future.”. The short term plan is now to land the WebKit MathML refactoring started during the hackfest and to finish the work. I hope people now understand the importance of fonts with an OpenType MATH table for good mathematical rendering and we will continue to encourage browser vendors and font designers to make such fonts wide spread.</p>
<p>The new approach for WebKit MathML support gives good reason to be optmimistic
in the long term and we hope we will be able to get high-quality rendering.
The fact that the new approach addresses all the issues formulated by Google and that we started a dialogue on math rendering, gives several options for MathML in Blink. It remains to get Microsoft involved in implementing its own OpenType table in EdgeHTML. Overall, I believe that we can now have a very constructive discussion with the individuals/companies who really want native math support, with the Math WG members willing to have their specification implemented in browsers and with the browser vendors who want a math implementation which is good, clean and compatible with the rest of their code base. Hopefully, the <a href="http://mathml-association.org/">MathML Association</a> will be instrumental in achieving that. If everybody get involved,
2016 will definitely be an exciting year for native MathML in Web engines!</p>