Frédéric Wang Nélar

Stage d'implémentation des normes Web (session 2026)

2026-02-25T00:00:00+01:00

Les candidatures pour les « stages de programmation informatique » d’Igalia sont officiellement ouvertes jusqu’à début avril. Ils offrent aux étudiant·e·s l’occasion de participer au développement de logiciels libres tout en étant rémunéré·e·s 7 000 € brut pour 450 heures, réparties de juin à décembre 2026.

Comme chaque année, j’encadrerai un·e étudiant·e sur l’« Implémentation des normes Web » (Web Standards en anglais). L’objectif étant de modifier les navigateurs (Chromium, Firefox ou Safari…) afin d’améliorer le support de technologies Web (HTML, CSS, DOM…). Il faudra notamment étudier les spécifications correspondantes et écrire des tests de conformité. Notez bien que ce n’est pas un stage de développement Web mais de développement C++.

Un des objectifs de ce programme étant de lutter contre les discriminations professionnelles, tout le monde (y compris celles et ceux qui se sentent sous-représenté·e·s dans le secteur informatique) sont invité·e·s à candidater. Depuis 2016, mon équipe « Web Platform » a ainsi encadré 13 étudiant·e·s de différents pays dans le monde (Espagne, Inde, Italie, Australie, Cameroun, Chine, Vietnam, Angleterre et États-Unis) dont 7 femmes. L’année dernière, nous avions sélectionné Charlotte McCleary, une Américaine non-voyante qui a travaillé sur l’accessibilité dans Firefox au cours de son stage et a depuis rejoint Fizz Studio. J’aimerais encourager les étudiant·e·s Sourd·e·s à postuler et donne dans la vidéo ci-dessous une brève présentation du programme en LSF (en espérant que ce soit compréhensible et que vous serez indulgents avec mon piètre niveau en langue des signes 😅):

Si vous êtes intéréssé·e·s, remplissez ce formulaire en cochant la case Web Standards et en précisant éventuellement que vous avez trouvé cette offre via mon site Web. Enfin, si vous connaissez des étudiant·e·s qui pourraient participer, n’hésitez pas à partager l’annonce !

Stage d'implémentation des normes Web

2025-02-22T00:00:00+01:00

Igalia recherche des étudiant·e·s pour ses « stages » de développement logiciel ¹

Brève description :

Contribution aux logiciels libres.
En télétravail.
450 heures réparties sur 3 ou 6 mois.
Rémunération de 7 000 €.
Encadrement par un·e ingénieur·e d’Igalia.
Lutter contre les discriminations professionnelles dans le secteur informatique.

Chaque année, je m’occupe du stage « Implémentation des normes Web » qui consiste à modifier les navigateurs (Chromium, Firefox, Safari…) afin d’améliorer le support de technologies Web (HTML, CSS, DOM…). Il faut notamment étudier les spécifications correspondantes et écrire des tests de conformité. Notez bien que ce n’est pas un stage de développement Web mais de développement C++.

Je vous invite à lire My internship with Igalia ¹ de ma collègue Delan Azabani pour un exemple concret. Ces dernières années, en plus de la communication par messagerie instantanée, je mets en place des visioconférences hebdomadaires qui se révèlent plutôt efficaces pour permettre aux stagiaires de progresser.

J’ai commencé des cours de LSF depuis quelques mois et assisté à plusieurs spectacles de l’IVT, notamment « Parle plus fort ! » qui décrit avec humour les difficultés des Sourds au travail. Cette année, j’envisage de prendre un·e stagiaire Sourd·e afin de contribuer à une meilleure intégration des Sourds en milieu professionnel. Je pense que ce sera aussi une expérience positive pour mon entreprise et pour moi-même.

Profil recherché :

Étudiant·e en informatique niveau licence/master.
Résidant en région parisienne (pour faciliter l’encadrement).
Pouvant lire/écrire en anglais (et communiquer en LSF).
Intéressé·e par les technologies Web.
Connaissance de développement C/C++.

Si vous êtes intéressé·e, les candidatures se font ici jusqu’au 4 avril 2025.

Le programme “Coding Experience” d’Igalia ne correspond pas forcément à un stage au sens français du terme. Si vous souhaiteriez en faire un stage conventionné, précisez-le lors de la candidature et nous pourrons trouver une solution. ↩ ↩²

Five testharness.js mistakes

2025-02-21T00:00:00+01:00

Introduction

I recently led a small training session at Igalia where I proposed to find mistakes in five small testharness.js tests I wrote. These mistakes are based on actual issues I found in official web platform tests, or on mistakes I made myself in the past while writing tests, so I believe they would be useful to know. The feedback from my teammates was quite positive, with very good participation and many ideas. They suggested I write a blog post about it, so here it is.

Please read the tests carefully and try to find the mistakes before looking at the proposed fixes…

1. Multiple tests in one loop

We often need to perform identical assertions for a set of similar objects. A good practice is to split such checks into multiple test() calls, so that it’s easier to figure out which of the objects are causing failures. Below, I’m testing the reflected autoplay attribute on the <audio> and <video> elements. What small mistake did I make?

<!DOCTYPE html>
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<script>
  ["audio", "video"].forEach(tagName => {
    test(function() {
      let element = document.createElement(tagName);
      assert_equals(element.autoplay, false, "inital value");
      element.setAttribute("autoplay", "autoplay");
      assert_equals(element.autoplay, true, "after setting attribute");
      element.removeAttribute("autoplay");
      assert_equals(element.autoplay, false, "after removing attribute");
    }, "Basic test for HTMLMediaElement.autoplay.");
  });
</script>

Proposed fix

Each loop iteration creates one test, but they all have have the name "Basic test for HTMLMediaElement.autoplay.". Because this name identifies the test in various places (e.g. failure expectations), it must be unique to be useful. These tests will even cause a “Harness status: Error” with the message “duplicate test name”.

One way to solve that is to move the loop iteration into the test(), which will fix the error but won’t help you with fine-grained failure reports. We can instead use a different description for each iteration:

       assert_equals(element.autoplay, true, "after setting attribute");
       element.removeAttribute("autoplay");
       assert_equals(element.autoplay, false, "after removing attribute");
-    }, "Basic test for HTMLMediaElement.autoplay.");
+    }, `Basic test for HTMLMediaElement.autoplay (${tagName} element).`);
   });
 </script>

2. Cleanup between tests

Sometimes, it is convenient to reuse objects (e.g. DOM elements) for several test() calls, and some cleanup may be necessary. For instance, in the following test, I’m checking that setting the class attribute via setAttribute() or setAttributeNS() is properly reflected on the className property. However, I must clear the className at the end of the first test(), so that we can really catch the failure in the second test() if, for example, setAttributeNS() does not modify the className because of an implementation bug. What’s wrong with this approach?

<!DOCTYPE html>
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<div id="element"></div>
<script>
  test(function() {
    element.setAttribute("class", "myClass");
    assert_equals(element.className, "myClass");
    element.className = "";
  }, "Setting the class attribute via setAttribute().");

  test(function() {
    element.setAttributeNS(null, "class", "myClass");
    assert_equals(element.className, "myClass");
    element.className = "";
  }, "Setting the class attribute via setAttributeNS().");
</script>

Proposed fix

In general, it is difficult to guarantee that a final cleanup is executed. In this particular case, for example, if the assert_equals() fails because of bad browser implementation, then an exception is thrown and the rest of the function is not executed.

Fortunately, testharness.js provides a better way to perform cleanup after a test execution:

-  test(function() {
+  function resetClassName() { element.className = ""; }
+
+  test(function(t) {
+    t.add_cleanup(resetClassName);
     element.setAttribute("class", "myClass");
     assert_equals(element.className, "myClass");
-    element.className = "";
   }, "Setting the class attribute via setAttribute().");

-  test(function() {
+  test(function(t) {
+    t.add_cleanup(resetClassName);
     element.setAttributeNS(null, "class", "myClass");
     assert_equals(element.className, "myClass");
-    element.className = "";
   }, "Setting the class attribute via setAttributeNS().");

3. Checking whether an exception is thrown

Another very frequent test pattern involves checking whether a Web API throws an exception. Here, I’m trying to use DOMParser.parseFromString() to parse a small MathML document. The HTML spec says that it should throw a TypeError if one specifies the MathML MIME type. The second test() asserts that the rest of the try branch is not executed and that the correct exception type is found in the catch branch. Is this approach correct? Can the test be rewritten in a better way?

<!DOCTYPE html>
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<script>
  const parser = new DOMParser();
  const mathmlSource = `<?xml version="1.0"?>
<math xmlns="http://www.w3.org/1998/Math/MathML">
  <mfrac>
    <mn>1</mn>
    <mi>x</mi>
  </mfrac>
</math>`;

  test(function() {
    let doc = parser.parseFromString(mathmlSource, "application/xml");
    assert_equals(doc.documentElement.tagName, "math");
  }, "DOMParser's parseFromString() accepts application/xml");

  test(function() {
    try {
      parser.parseFromString(mathmlSource, "application/mathml+xml");
      assert_unreached();
    } catch(e) {
      assert_true(e instanceof TypeError);
    }
  }, "DOMParser's parseFromString() rejects application/mathml+xml");
</script>

Proposed fix

If the assert_unreached() is executed because of an implementation bug with parseFromString(), then the assertion will actually throw an exception. That exception won’t be a TypeError, so the test will still fail because of the assert_true(), but the failure report will look a bit confusing.

In this situation, it’s better to use the assert_throws_* APIs:

   test(function() {
-    try {
-      parser.parseFromString(mathmlSource, "application/mathml+xml");
-      assert_unreached();
-    } catch(e) {
-      assert_true(e instanceof TypeError);
-    }
+    assert_throws_js(TypeError,
+        _ => parser.parseFromString(mathmlSource, "application/mathml+xml"));
   }, "DOMParser's parseFromString() rejects application/mathml+xml");

4. Waiting for an event listener to be called

The following test verifies a very basic feature: clicking a button triggers the registered event listener. We use the (asynchronous) testdriver API test_driver.click() to emulate that user click, and a promise_test() call to wait for the click event listener to be called. The test may time out if there is something wrong in the browser implementation, but do you see a risk for flaky failures?

Note: the testdriver API only works when running tests automatically. If you run the test manually, you need to click the button yourself.

<!DOCTYPE html>
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<script src="/resources/testdriver.js"></script>
<script src="/resources/testdriver-vendor.js"></script>
<button id="button">Click me to run manually</button>
<script>
  promise_test(function() {
    test_driver.click(button);
    return new Promise(resolve => {
      button.addEventListener("click", resolve);
    });
  }, "Clicking the button triggers registered click event handler.");
</script>

Proposed fix

The problem I wanted to show here is that we are sending the click event before actually registering the listener. The test would likely still work, because test_driver.click() is asynchronous and communication to the test automation scripts is slow, whereas registering the event is synchronous.

But rather than making this kind of assumption, which poses a risk of flaky failures as well as making the test hard to read, I prefer to just move the statement that triggers the event into the Promise, after the listener registration:

 <button id="button">Click me to run manually</button>
 <script>
   promise_test(function() {
-    test_driver.click(button);
     return new Promise(resolve => {
       button.addEventListener("click", resolve);
+      test_driver.click(button);
     });
   }, "Clicking the button triggers registered click event handler.");
 </script>

My colleagues also pointed out that if the promise returned by test_driver.click() fails, then a “Harness status: Error” could actually be reported with “Unhandled rejection”. We can add a catch to handle this case:

 <button id="button">Click me to run manually</button>
 <script>
   promise_test(function() {
-    return new Promise(resolve => {
+    return new Promise((resolve, reject) => {
       button.addEventListener("click", resolve);
-      test_driver.click(button);
+      test_driver.click(button).catch(reject);
     });
   }, "Clicking the button triggers registered click event handler.");
 </script>

5. Dealing with asynchronous resources

It’s very common to deal with asynchronous resources in web platform tests. The following test case verifies the behavior of a frame with lazy loading: it is initially outside the viewport (so not loaded) and then scrolled into the viewport (which should trigger its load). The actual loading of the frame is tested via the window name of /common/window-name-setter.html (should be “spices”). Again, this test may time out if there is something wrong in the browser implementation, but can you see a way to make the test a bit more robust?

Side question: the <div id="log"></div> and add_cleanup() are not really necessary for this test to work, so what’s the point of using them? Can you think of one?

<!DOCTYPE html>
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<style>
  #lazyframe { margin-top: 10000px; }
</style>
<div id="log"></div>
<iframe id="lazyframe" loading="lazy"
        src="/common/window-name-setter.html"></iframe>
<script>
  promise_test(function() {
    return new Promise(resolve => {
      window.addEventListener("load", () => {
        assert_not_equals(lazyframe.contentWindow.name, "spices");
        resolve();
      });
    });
  }, "lazy frame not loaded after page load");
  promise_test(t => {
    t.add_cleanup(_ => window.scrollTo(0, 0));
    return new Promise(resolve => {
      lazyframe.addEventListener('load', () => {
        assert_equals(lazyframe.contentWindow.name, "spices");
        resolve();
      });
      lazyframe.scrollIntoView();
    });
  }, "lazy frame loaded after appearing in the viewport");
</script>

Proposed fix

This is similar to what we discussed in the previous tests. If the assert_equals() in the listener fails, then an exception is thrown, but it won’t be caught by the testharness.js framework. A “Harness status: Error” is reported, but the test will only complete after the timeout. This can slow down test execution, especially if this pattern is repeated for several tests.

To make sure we report the failure immediately in that case, we can instead reject the promise if the equality does not hold, or even better, place the assert_equals() check after the promise resolution:

 <script>
   promise_test(function() {
     return new Promise(resolve => {
-      window.addEventListener("load", () => {
-        assert_not_equals(lazyframe.contentWindow.name, "spices");
-        resolve();
-      });
-    });
+      window.addEventListener("load", resolve);
+    }).then(_ => {
+      assert_not_equals(lazyframe.contentWindow.name, "spices");
+    });;
   }, "lazy frame not loaded after page load");
   promise_test(t => {
     t.add_cleanup(_ => window.scrollTo(0, 0));
     return new Promise(resolve => {
-      lazyframe.addEventListener('load', () => {
-        assert_equals(lazyframe.contentWindow.name, "spices");
-        resolve();
-      });
+      lazyframe.addEventListener('load', resolve);
       lazyframe.scrollIntoView();
+    }).then(_ => {
+      assert_equals(lazyframe.contentWindow.name, "spices");
     });
   }, "lazy frame loaded after appearing in the viewport");
 </script>

Regarding the side question, if you run the test by opening the page in the browser, then the report will be appended at the bottom of the page by default. But lazyframe has very large height, and the page may be scrolled to some other location. An explicit <div id="log"> ensures the report is inserted inside that div at top of the page, while the add_cleanup() ensures that we scroll to that location after test execution.

My recent contributions to Gecko (3/3)

2024-09-05T00:00:00+02:00

Note: This blog post was written on June 2024. As of September 2024, final work to ship the feature is still in progress. Please follow bug 1797715 for the latest updates.

Introduction

This is the final blog post in a series about new web platform features implemented in Gecko, as part as an effort at Igalia to increase browser interoperability.

Let’s take a look at fetch priority attributes, which enable web developers to optimize resource loading by specifying the relative priority of resources to be fetched by the browser.

Fetch priority

The web.dev article on fetch priority explains in more detail how web developers can use fetch priority to optimize resource loading, but here’s a quick overview.

fetchpriority is a new attribute with the value auto (default behavior), high, or low. Setting the attribute on a script, link or img element indicates whether the corresponding resource should be loaded with normal, higher, or lower priority ¹:

<head>
  <script src="high.js" fetchpriority="high"></script>
  <link rel="stylesheet" href="auto.css" fetchpriority="auto">
</head>
<body>
  <img src="low.png" alt="low" fetchpriority="low">
</body>

The priority can also be set in the RequestInit parameter of the fetch() method:

await fetch("high.txt", {priority: "high"});

The <link> element has some interesting features. One of them is combining rel=preload and as to fetch a resource with a particular destination ²:

<link rel="preload" as="font" href="high.woff2" fetchpriority="high">

You can even use Link in HTTP response headers and in particular early hints sent before the final response:

103 Early Hint
Link: <high.js>; rel=preload; as=script; fetchpriority=high

These are basically all the places where a fetch priority attribute can be used.

Note that other parameters are also taken into account when deciding the priority to use for resources, such as the position of the element in the page (e.g. blocking resources in <head>), other attributes on the element (<script async>, <script defer>, <link media>, <link rel>…) or the resource’s destination.

Finally, some browsers implement speculative HTML parsing, allowing them to continue fetching resources declared in the HTML markup while the parser is blocked. As far as I understand, Firefox has its own separate HTML parsing code for that purpose, which also has to take fetch priority attributes into account.

Implementation-defined prioritization

If you have not run away after reading the complexity described in the previous section, let’s talk a bit more about how fetch priority attributes are interpreted. The spec contains the following step when fetching a resource (emphasis mine):

If request’s internal priority is null, then use request’s priority, initiator, destination, and render-blocking in an implementation-defined manner to set request’s internal priority to an implementation-defined object.

So browsers would use the high/low/auto hints as well as the destination in order to calculate an internal priority value ³, but the details of this value are not provided in the specification, and it’s up to the browser to decide what to do. This is a bit unfortunate for our interoperability goal, but that’s probably the best we can do, given that each browser already has its own stategies to optimize resource loading. I think this also gives browsers some flexibility to experiment with optimizations… which can be hard to predict when you realize that web devs also try to adapt their content to the behavior of (the most popular) browsers!

In any case, the spec authors were kind enough to provide a note with more suggestions (emphasis mine):

The implementation-defined object could encompass stream weight and dependency for HTTP/2, priorities used in Extensible Prioritization Scheme for HTTP for transports where it applies (including HTTP/3), and equivalent information used to prioritize dispatch and processing of HTTP/1 fetches. [RFC9218]

OK, so what does that mean? I’m not a networking expert, but this is what I could gather after discussing with the Necko team and reading some HTTP specs:

HTTP/1 does not have a dedicated prioritization mechanism, but Firefox uses its internal priority to order requests.
HTTP/2 has a “stream priority” mechanism and Firefox uses its internal priority to implement that part of the spec. However, it was considered too complex and inefficient, and is likely poorly supported by existing web servers…
In upcoming releases, Firefox will use its internal priority to implement the Extensible Prioritization Scheme used by HTTP/2 and HTTP/3. See bug 1865040 and bug 1864392. Essentially, this means using its internal priority to adjust the urgency parameter.

Note that various parts of Firefox rely on NS_NewChannel to load resources, including the fetching algorithm above, which Firefox uses to implement the fetch() method. However, other cases mentioned in the first section have their own code paths with their own calls to NS_NewChannel, so these places must also be adjusted to take the fetch priority and destination into account.

Finishing the implementation work

Summarizing a bit, implementing fetch priority is a matter of:

Adding fetchpriority to DOM objects for HTMLImageElement, HTMLLinkElement, HTMLScriptElement, and RequestInit.
Parsing the fetch priority attribute into an auto/low/high enum.
Passing the information to the callers of NS_NewChannel.
Using that information to set the internal priority.
Using that internal priority for HTTP requests.

Mirko Brodesser started this work in June 2023, and had already implemented almost all of the features discussed above. fetch(), <img>, and <link rel=preload as=image> were handled by Ziran Sun and I, while Valentin Gosu from Mozilla made HTTP requests use the internal priority.

The main blocker was due to that “implementation-defined” use of fetch priority. Mirko’s approach was to align Firefox with the behavior described in the web.dev article, which reflects Chromium’s implementation. But doing so would mean changing Firefox’s default behavior when fetchpriority is not specified (or explicitly set to auto), and it was not clear whether Chromium’s prioritization choices were the best fit for Firefox’s own implementation of resource loading.

After meeting with Mozilla, we agreed on a safer approach:

Introduce runtime preferences to control how Firefox adjusts internal priorities when low, high, or auto is specified. By default, auto does not affect the internal priority so current behavior is preserved.
Ask Mozilla’s performance team to run an experiment, so we can decide the best values for these preferences.
Ship fetch priority with the chosen values, probably cleaning things up a bit. Any other ideas, including the ones described in the web.dev article, could be handled in future enhancements.

We recently entered phase 2 of this plan, so fingers crossed it works as expected!

Internal WPT tests

This project is part of the interoperability effort, but again, the “implementation-defined” part meant that we had very few WPT tests for that feature, really only those checking fetchpriority attributes for the DOM part.

Fortunately Mirko, who is a proponent of Test-driven development, had written quite a lot of internal WPT tests that use internal APIs to retrieve the internal priority. To test Link headers, he used the handy wptserve pipes. The only thing he missed was checking support in Early hints, but some WPT tests for early hints using WPT Python Handlers were available, so integrating them into Mirko’s tests was not too difficult.

It was also straightforward for Ziran and I to extend Mirko’s tests to cover fetch, img, and <link rel=preload as=image>, with one exception: when the fetch() method uses a non-default destination. In most of these code paths, we call NS_NewChannel to perform a fetch. But fetch() is tricky, because if the fetch event is intercepted, the event handler might call the fetch() method again using the same destination (e.g. image).

Handling this correctly involves multiple processes and IPC communication, which ended up not working well with the internal APIs used by Mirko’s tests. It took me a while to understand what was happening in bug 1881040, and in the end I came up with a new approach.

Upstreamable WPT tests

First, let’s pause for a moment: all the tests we have so far use an internal API to verify the internal priority, but they don’t actually check how that internal priority is used by Firefox when it sends HTTP requests. Valentin mentioned we should probably have some tests covering that, and not only would it solve the problem with fetch() calls in fetch event handlers, it would also remove the use of an internal API, making the tests potentially reusable by other browsers.

To make this kind of test possible, I added a WPT Python Handler that parses the urgency from a HTTP request and responds with an urgency-dependent resource, such as a stylesheet with different property values, an image of a different size, or an audio or video file of a different duration.

When a test uses resources with different fetch priorities, this influences the urgency values of their HTTP requests, which in turn influences the response in a way that the test can check for in JavaScript. This is a bit complicated, but it works!

Conclusion

Fetch priority has been enabled in Firefox Nightly for a while, and experiments started recently to determine the optimal priority adjustments. If everything goes well, we will be able to push this feature to the finish line after the (northern) summer.

Helping implement this feature also gave me the opportunity to work a bit on the Firefox networking code, which I had not touched since the collaboration with IPFS, and I learned a lot about resource loading and WPT features for HTTP requests.

To me, the “implementation-defined” part was still a bit awkward for the web platform. We had to write our own internal WPT tests and do extra effort to prepare the feature for shipping. But in the end, I believe things went relatively smoothly.

Acknowledgments

To conclude this series of blog posts, I’d also like to thank Alexander Surkov, Cathie Chen, Jihye Hong, Martin Robinson, Mirko Brodesser, Oriol Brufau, Ziran Sun, and others at Igalia who helped on implementing these features in Firefox. Thank you to Emilio Cobos, Olli Pettay, Valentin Gosu, Zach Hoffman, and others from the Mozilla community who helped with the implementation, reviews, tests and discussions. Finally, our spelling and grammar expert Delan Azabani deserves special thanks for reviewing this series of blog post and providing useful feedback.

Other elements have been or are being considered (e.g. <iframe>, SVG <image> or SVG <script>), but these are the only ones listed in the HTML spec at the time of writing. ↩
As mentioned below, the browser needs to know about the actual destination in order to properly calculate the priority. ↩
As far as I know, Firefox does not take initiator into account, nor does it support render-blocking yet. ↩

My recent contributions to Gecko (2/3)

2024-07-09T00:00:00+02:00

Introduction

This is the second in a series of blog posts describing new web platform features Igalia has implemented in Gecko, as part of an effort to improve browser interoperability. I’ll talk about the task of implementing ‘content-visibility’, to which several Igalians have contributed since early 2022, and I’ll focus on two main roadblocks I had to overcome.

The ‘content-visibility’ property

In the past, Igalia worked on CSS containment, a feature allowing authors to isolate a subtree from the rest of the document to improve rendering performance. This is done using the ‘contain’ property, which accepts four kinds of containment: size, layout, style and paint.

‘content-visibility’ is a new property allowing authors to “hide” some content from the page, and save the browser unnecessary work by applying containment. The most interesting one is probably content-visibility: auto, which hides content that is not relevant to the user. This is essentially native “virtual scrolling”, allowing you to build virtualized or “recycled” lists without breaking accessibility and find-in-page.

To explain this, consider the typical example of a page with a series of posts, as shown below. By default, each post would have the four types of containment applied, plus it won’t be painted, won’t respond to hit-testing, and would use the dimensions specified in the ‘contain-intrinsic-size’ property. It’s only once a post becomes relevant to the user (e.g. when scrolled close enough to the viewport, or when focus is moved into the post) that the actual effort to properly render the content, and calculate its actual size, is performed:

div.post {
  content-visibility: auto;
  contain-intrinsic-size: 500px 1000px;
}

<div class="post">
...
</div>
<div class="post">
...
</div>
<div class="post">
...
</div>
<div class="post">
...
</div>

If a post later loses its relevance (e.g. when scrolled away, or when focus is lost) then it would use the dimensions specified by ‘contain-intrinsic-size’ again, discarding the content size that was obtained after layout. One can also avoid that and use the last remembered size instead:

div.post {
  contain-intrinsic-size: auto 500px auto 1000px;
}

Finally, there is also a content-visibility: hidden value, which is the same as content-visibility: auto but never reveals the content, enhancing other methods to hide content such as display: none or visibility: hidden.

This is just a quick overview of the feature, but I invite you to read the web.dev article on content-visibility for further details and thoughts.

Viewport distance for `content-visibility: auto`

As is often the case, the feature looks straightforward to implement, but issues appear when you get into the details.

In bug 1807253, my colleague Oriol Brufau raised an interoperability bug with a very simple test case, reproduced below for convenience. Chromium would report 0 and 42, whereas Firefox would sometimes report 0 twice, meaning that the post did not become relevant after a rendering update:

<!DOCTYPE html>
<div id="post" style="content-visibility: auto">
  <div style="height: 42px"></div>
</div>
<script>
console.log(post.clientHeight);
requestAnimationFrame(() => requestAnimationFrame(() => {
  console.log(post.clientHeight);
}));
</script>

It turned out that an early version of the specification relied too heavily on an modified version of IntersectionObserver to synchronously detect when an element is close to the viewport, as this was how it was implemented in Chromium. However, the initial implementation in Firefox relied on a standard IntersectionObserver (with asynchronous notifications of observers) and so failed to produce the behavior described in the specification. This issue was showing up in several WPT failures.

To solve that problem, the moment when we determine an element’s proximity to the viewport was moved into the HTML5 specification, at the step when the rendering is updated, more precisely when the ResizeObserver notifications are broadcast. My colleague Alexander Surkov had started rewriting Firefox’s implementation to align with this new behavior in early 2023, and I took over his work in November.

Since this touches the “update the rendering” step which is executed on every page, it was quite likely to break things… and indeed many regressions were caused by my patch, for example:

One regression was about white flickering of pages on every reload/navigation.
One more regression was about content-visibility: auto nodes not being rendered at all.
Another regression was about new resize loop errors appearing in tests.
Some test cases were also found where the “update the rendering step” would repeat indefinitely, causing performance regressions.
Last but not least, crashes were reported.

Some of these issues were due to the fact that support for the last remembered size in Firefox relied on an internal ResizeObserver. However, the CSS Box Sizing spec only says that the last remembered size is updated when ResizeObserver events are delivered, not that such an internal ResizeObserver object is actually needed. I removed this internal observer and ensured the last remembered size is computed directly in the “update the rendering” phase, making the whole thing simpler and more robust.

Dynamic changes to CSS ‘contain’ and ‘content-visibility’

Before sending the intent-to-ship, we reviewed remaining issues and stumbled on bug 1765615, which had been opened during the initial 2022 work. Mozilla indicated this performance bug was important enough to consider an optimization, so I started tackling the issue.

Elaborating a bit about what was mentioned above, a non-visible ‘content-visibility’ implies layout, style and paint containment, and when the element is not relevant to the user, it also implies size containment ¹. This has certain side effects, for example paint and layout containment establish an independent formatting context and affect how the contained box interacts with floats and how margin collapsing applies. Style containment can even have more drastic consequences, since they make counter-* and *-quote properties scoped to the subtree.

When we dynamically modify the ‘contain’ or ‘content-visibility’ properties, or when the relevance of a content-visibility: auto element changes, browsers must make sure that the rendering is properly updated. It turned out that there were almost no tests for that, and unsurprisingly, Chromium and WebKit had various invalidation bugs. Firefox was always forcing a rebuild of the tree used for rendering, which avoided such bugs but is not optimal.

I wrote a couple of web platform tests for ‘contain’ and ‘content-visibility’ ², and made sure that Firefox does the minimal invalidation effort needed, being careful not to cause any regressions. As a result, except for style containment changes, we’re now able to avoid the cost a rebuild of the tree used for rendering!

Conclusion

Almost two years after the initial work on ‘content-visibility’, I was able to send the intent-to-ship, and the feature finally became available in Firefox 125. Finishing the implementation work on this feature was challenging, but quite interesting to me.

I believe ‘content-visibility’ is a good example of why implementing a feature in different browsers is important to ensure that both the specification and tests are good enough. The lack of details in the spec regarding when we determine viewport proximity, and the absence for WPT tests for invalidation, definitely made the Firefox work take longer than expected. But finishing that implementation work was also useful for improving the spec, tests, and other implementations ³.

I’ll conclude this series of blog posts with fetch priority, which also has its own interesting story…

In both cases, “implies” means the used value of ‘contain’ is modified accordingly. ↩
One of the thing I had to handle with care was the update of the accessibility tree, since content that is not relevant to the user must not be exposed. Unfortunately it’s not possible to write WPT tests for accessibility yet so for now I had to write internal Firefox-specific non-regression tests. ↩
Another interesting report happened after the release and is related to content-visibility: auto on elements drawn in a canvas. ↩

My recent contributions to Gecko (1/3)

2024-06-21T00:00:00+02:00

Introduction

Igalia has been contributing to the web platform implementations of different web engines for a long time. One of our goals is ensuring that these implementations are interoperable, by relying on various web standards and web platform tests. In July 2023, I happily joined a project that focuses on this goal, and I worked more specifically on the Gecko web engine. One year later, three new features I contributed to are being shipped in Firefox. In this series of blog posts, I’ll give an overview of those features (namely registered custom properties, content visibility, and fetch priority) and my journey to make them “ride the train” as Mozilla people say.

Let’s start with registered custom properties, an enhancement of traditional CSS variables.

Registered custom properties

You may already be familiar with CSS variables, these “dash dash” names that facilitate the maintenance of a large web site by allowing author-defined CSS properties. In the example below, the :root selector defines a variable --main-theme-color with value “blue”, which is used for the style applied to other elements via the var() CSS function. As you can see, this makes the usage of the main theme color in different places more readable and makes customizing that color much easier.

:root { --main-theme-color: blue; }
p { color: var(--main-theme-color); }
section {
  padding: 1em;
  border: 1px solid var(--main-theme-color);
}
.progress-bar {
  height: 10px;
  width: 100%;
  background: linear-gradient(white, var(--main-theme-color));
}

<section>
  <p>Loading...</p>
  <div class="progress-bar"></div>
</section>

In browsers supporting CSS variables, you should see a frame containing the text “Loading” and a progress bar, all of these components being blue:

Having such CSS variables available is already nice, but they are lacking some features available to native CSS properties… For example, there is (almost) no syntax checking on specified values, they are always inherited, and their initial value is always the guaranteed invalid value. In order to improve on that situation, the CSS Properties and Values specification provides some APIs to register custom properties with further characteristics:

An accepted syntax for the property; for example, igalia | <url> | <integer>+ means either the custom identifier “igalia”, or a URL, or a space-separated list of integers.
Whether the property is inherited or non-inherited.
An initial value.

Custom properties can be registered via CSS or via a JS API, and these ways are equivalent. For example, to register --main-theme-color as a non-inherited color with initial value blue:

@property --main-theme-color {
  syntax: "<color>";
  inherits: false;
  initial-value: blue;
}

window.CSS.registerProperty({
  name: "--main-theme-color",
  syntax: "<color>",
  inherits: false,
  initialValue: blue,
});

Interpolation of registered custom properties

By having custom properties registered with a specific syntax, we open up the possibility of interpolating between two values of the properties when performing an animation. Consider the following example, where the width of the animated div depends on the custom property --my-length. Defining this property as a length allows browsers to interpolate it continuously between 10px and 200px when it is animated:

 @property --my-length {
   syntax: "<length>";
   inherits: false;
   initialValue: '0px';
 }
 @keyframes test {
   from {
     --my-length: 10px;
   }
   to {
     --my-length: 200px;
   }
 }
 div#animated {
   animation: test 2s linear both;
   width: var(--my-length, 10px);
   height: 200px;
   background: lightblue;
 }

With non-registered custom properties, we can instead only animate discretely; --my-length would suddenly jump from 10px to 200px halfway through the duration of the animation, which is generally not what is desired for lengths.

Custom properties in the cascade

If you check the Interop 2023 Dashboard for custom properties, you may notice that interoperability was really bad at the beginning of the year, and this was mainly due to Firefox’s low score. Consequently, when I joined the project, I was asked to help with improving that situation.

While the two registration methods previously mentioned had already been implemented, the main issue was that the CSS cascade was always treating custom properties as inherited and initialized with the guaranteed invalid value. This is indeed correct for unregistered custom properties, but it’s generally incorrect for registered custom properties!

In bug 1840478, bug 1855887, and others, I made registered custom properties work properly in the cascade, including non-inherited properties and registered initial values. But in the past, with the previous assumptions around inheritance and initial values, it was possible to store the computed values of custom properties on an element as a “cheap” map, considering only the properties actually specified on the element or an ancestor and (in most cases) only taking shallow copies of the parent’s map. As a result, when generalizing the cascade for registered custom properties, I had to be careful to avoid introducing performance regressions for existing content.

Custom properties in animations

Another area where the situation was pretty bad was animations. Not only was Firefox unable to interpolate registered custom properties between two values — one of the main motivations for the new spec — but it was actually unable to animate custom properties at all!

The main problem was that the existing animation code referred to CSS properties using an enum nsCSSPropertyID, with all custom properties represented by the single value nsCSSPropertyID::eCSSPropertyExtra_variable. To make this work for custom properties, I had to essentially replace that value with a structure containing the nsCSSPropertyID and the name of the custom properties.

I uploaded patches to bug 1846516 to perform that change throughout the whole codebase, and with a few more tweaks, I was able to make registered custom properties animate discretely, but my patches still needed some polish before they could be reviewed. I had to move onto other tasks, but fortunately, some Mozilla folks were kind enough to take over this task, and more generally, complete the work on registered custom properties!

Conclusion

This was an interesting task to work on, and because a lot of the work happened in Stylo, the CSS engine shared by Servo and Gecko, I also had the opportunity to train more on the Rust programming language. Thanks to help from folks at Mozilla, we were able to get excellent progress on registered custom properties in Firefox in 2023, and this feature is expected to ship in Firefox 128!

As I said, I’ve since moved onto other tasks, which I’ll describe in subsequent blog posts in this series. Stay tuned for content-visibility, enabling interesting layout optimizations for web pages.

Flygskam and o caminho da Corunha

2024-05-23T00:00:00+02:00

Prolegomenon

Early next June, I’m traveling from Paris to A Coruña for the Web Engines Hackfest and other internal Igalia events. In recent years I’ve done it by train, as previously mentioned. Some colleagues at Igalia were curious about it so I decided to write this blog post, investigating possible ways to do it from various European places. I wish this can also motivate more people at Igalia and beyond, who are still hesistant to give up alternatives with heavier carbon footprint.

In addition to various trip planners, I’ve also used this nice map from Wikipedia, which gives a good overview of high-speed rail in Europe:

I also sought advice from Nicolò Ribaudo who is quite familiar with train traveling and provided useful recommendations. In particular, he mentioned the ÖBB trip planner, which seems quite efficient combining trains from multiple operators.

I’ve focused on big european cities (with airports) that are close to Spain but this is definitely not exhaustive. There is probably a lot more to discuss beyong trip planning, but hopefully that can be a good starting point.

Paris as a departure or connection

Based on my experience traveling from Paris, I found these direct trains between big cities:

Renfe offers several AVE and Alvia trains traveling every days between A Coruña and Madrid, with a duration 3h30-4h ¹. There are two important thing to note:
1. These local trains are only avalailble for sell maybe 2 months in advance at best.
2. These trains are connecting to Madrid Chamartín which is maybe 30 minutes away from Madrid Atocha by the Madrid metro.
Renfe also proposes even more options (say one each half hour during the day) between Barcelona and Madrid Atocha, with a duration of 2h30-3h ².
The SNCF proposes two or three direct trains during the day between Paris Gare de Lyon and Barcelona, with a duration of 6h30-7h. If you are coming from Paris and want to to take a train to Madrid, you will likely have to cross the station and pass some x-ray baggage scanner, so be sure to keep enough time for the connection.
The Eurostar offers several options during the day to connect Paris with cities below. They connect to Gare du Nord or Gare de l’Est, which are very close to each other but ~30 minutes away from Gare de Lyon by public transport.
- Brussels (duration ~1h30)
- Amsterdam (duration ~3h30)
- London (duration ~2h30)
- Cologne (duration ~3h30)

Personally, I’m doing this one-day-and-half trip (inbound trip is similar):

Take the train from Paris (9:42 AM) to Barcelona (4:33 PM).
Keep enough time for the Barcelona Sants connection.
Take a train from Barcelona to Madrid in the evening.
Stay one night in Madrid.
Take a train from Madrid to A Coruña in the morning.

From London, Amsterdam, Brussels, Berlin one could instead do a two-days trip:

Travel to Paris in the morning ³.
Keep enough time for the Paris connection.
Take the train from Paris (2:42PM) to Barcelona (9:27PM).
Stay one night in Barcelona.
Travel from Barcelona to Madrid Atocha.
Keep enough time for the Madrid Metro connection.
Travel from Madrid Chamartín to A Coruña.

I also looked at the trip with the minimum number of connections to go to Barcelona from big cities in Switzerland and a a similar traject is possible. See later for alternatives.

Finally, Nicolò mentioned that ÖBB recently started running a night train from Berlin to Paris, which you can probably use to do a similar trip as mine.

Estimate of CO₂ emissions

In order to estimate CO₂ emission for the trips suggested in the previous section, I compiled information from different sources:

Eurostar sustainability 2023, independent study carried out by EcoRes SCRL in July 2023.
The own data provided by SNCF.
Ecopassenger, developed in cooperation with the International Railways Union, the Sustainable Development Foundation and the German Institute for Environment and Energy.
Ecotree a company for tree planating providing a CO₂ emission calculator.

There would be a lot to discuss about the methodology but the goal here is only to give a rough idea. Anyway, below is an estimate of kilograms of CO₂ per passenger for some of the train trips previously mentioned:

	Eurostar	SNCF	Ecopassenger ⁴	Ecotree
Berlin ↔ Cologne	-	-	17-19	1-3
Cologne ↔ Paris	5.2	5.2	7.4	1-3
London ↔ Paris	2.4	2.4	1.7	1-3
Brussels ↔ Paris	1.6	1.8	1.8	1-2
Amsterdam ↔ Paris	2.6	2.9	9.3-9.5	1-3
Paris ↔ Barcelona	-	3.8	-	1-6
Barcelona ↔ Madrid	-	-	17-20	1-6
Madrid ↔ A Coruña	-	-	-	1-3

The best/worst cases are compiled into the following table and compared with a flight to A Coruña (with a connection by Madrid) as calculated by Ecotree. If we follow these data, a train from Berlin, London, Paris, Brussels and Amsterdam won’t at worst be around 50kg of CO₂ per passenger and represent at least a reduction of around 90% compared to using a plane:

Departure	Flight by Madrid (Ecotree)	Train ⁵	CO₂ reduction
Berlin	448	6-55.4	≥87%
London	388	4-32	≥91%
Brussels	396	4-30.8	≥92%
Amsterdam	396	4-38.5	≥90%
Paris	368	3-29	≥92%
Madrid	244	1-3	≥98%

More cities and trains

Disclaimer: This section is essentially based on information provided by Nicolò and what I found on internet.

In addition to the SNCF train previously mentioned, Renfe proposes two trains that are quite important for traveling from France to Spain:

One train between Lyon and Barcelona per day (duration ~5h)
One train between Marseille and Madrid per day (duration ~8h)

From Switzerland, you can pass by the south of France using more trains/connections to arrive faster in Barcelone than what was previously proposed. For example, taking a local train to go from Genève to Lyon followed by the Lyon to Barcelona train mentioned above can be done in around 7h. From Zurich, with connection at Genève and Lyon, it takes around 10h as opposed to 12h for a single connection in Paris.

From the Netherlands, Belgium or Germany it makes sense to consider the night trains from Paris to the border with Spain. Those trains do not have a fixed schedule, but vary depending on the weekday. Most of them arrive in Portbou and from there you can take a regional train to Barcelona. Some of them arrive to Latour-de-Carol, and from there it’s three hours on a regional train to Barcelona. In any case, you’ll be early enough at the border so that it’s possible to then arrive to A Coruña in the afternoon or evening. Rarely the night train arrives at the border on the west coast, and continuing from there to A Coruña with the regional trains that go on the northern coast might be a good experience.

From Belgium it’s also possible to take a TGV from Brussels to Lyon, and then from there take the train to Barcelona. This avoids a stressful connection in Paris, where you need to move between the two stations Gare du Nord and Gare de Lyon.

From Italy, the main trouble is to deal with connections. The Turin–Lyon high-speed railway contruction may help in the future but it’s not running yet. The alternatives are either to go on the coast through Genova-Ventimiglia-Nice, or with the Eurocity from Milan to Switzerland finally get to France.

From Portugal, which is so close geographically and culturally to Galicia, we could think there should be an easy way to travel to A Coruña. But apparently neither Comboios de Portugal nor Renfe provides any direct train:

From Porto, the ÖBB trip planner suggests connection at Vigo-Guixar for a total duration of 6h30. As a comparison, the website of the Web Engines Hackfest indicates a 3-hours trip by car and a 5-hours trip by bus.
Between Libon and Porto, Comboios de Portugal seems to propose trains taking 2h30-3h. You can combine that with the other trains to do the trip to A Coruña with one night at Porto or Vigo.

Last but not least, A Coruña railway station is a one-quarter walk away from the Igalia office and a three-quarter walks away from Palexco (Web Engines Hackfest’s venue). This is more convenient than the A Coruña airport which is around 10 km away from A Coruña.

Notes and references

Flygskam: anti-flying social movement started in Sweden, with the aim of reducing the environmental impact of aviation.
O Caminho de Santiago: The Way of St. James, a network of pilgrims’ ways leading to Santiago de Compostela (whose railway station you will likely stop at if you take a train to A Coruña).

Incidentally, people travelling from far away are unlikely to find a direct flight to the A Coruña airport. In that case, using local trains from bigger airports like Madrid or Porto may be a better option. ↩
Direct train between A Coruña and Barcelona seems to be rarer, slower and no longer available as night train. So a connection or night stay in Madrid seems the best option. ↩
Deutsche Bahn offers a lot of Berlin to Cologne trains per day with a duration of 4h-4h30, including early/late trains or night trains, that you can combine with the Cologne-Paris Eurostar. Deutsche Bahn also offers (non-direct) ICE trains to go from Paris to Berlin. ↩
Ecopassenger gives information per train, so I’m provided some lower/upper bounds based on different trains. ↩
Based on the previous data from Eurostar, SNCF, Ecopassenger, Ecotree, trying to find the lowest/highest sum for each individual segment. ↩

Time travel debugging of WebKit with rr

2024-05-21T00:00:00+02:00

Introduction

rr is a debugging tool for Linux that was originally developed by Mozilla for Firefox. It has long been adopted by Igalia and other web platform developers for Chromium and WebKit too. Back in 2019, there were breakout sessions on this topic at the Web Engines Hackfest and BlinkOn.

For WebKitGTK, the Flatpak SDK provides a copy of rr, but recently I was unable to use the instructions on trac.webkit.org. Fortunately, my colleague Adrián Pérez suggested using a direct build without flatpak or the bubblewrap sandbox, and that indeed solved my problem. I thought it might be interesting to share this information with others, so I decided to write this blog post.

Disclaimer: The build instructions below may be imperfect, will likely become outdated, and are in any case not a replacement for the official ones for WebKitGTK development. Use them at your own risk!

CMake configuration

The approach that worked for me was thus to perform a direct build from my system. I came up with the following configuration step:

cmake -S. -BWebKitBuild/Release \
   -DCMAKE_BUILD_TYPE=Release \
   -DCMAKE_INSTALL_PREFIX=$HOME/WebKit/WebKitBuild/install/Release \
   -GNinja -DPORT=GTK -DENABLE_BUBBLEWRAP_SANDBOX=OFF \
   -DDEVELOPER_MODE=ON -DDEVELOPER_MODE_FATAL_WARNINGS=OFF \
   -DENABLE_TOOLS=ON -DENABLE_LAYOUT_TESTS=ON

where:

The -B option specifies the build directory, which is traditionnaly called WebKitBuild/ for the WebKit project.
CMAKE_BUILD_TYPE specifies the build type, e.g. optimized release builds (Release, corresponding to --release for the offical script) or debug builds with assertions (Debug, corresponding to --debug) ¹.
CMAKE_INSTALL_PREFIX specifies the installation directory, which I place inside WebKitBuild/install/ ².
The -G option specifies the build system generator. I used Ninja, which is the default for the offical script too.
-DPORT=GTK is for building WebKitGTK. I haven’t tested rr with other Linux ports.
-DENABLE_BUBBLEWRAP_SANDBOX=OFF was suggested by Adrián. The bubblewrap sandbox probably does not make sense without flatpak, so it should be safe to disable it anyway.
I extracted the other -D flags from the official script, trying to stay as close as possible to what it provides for WebKit development (being able to run layout tests, building the Tools/, ignoring fatal warnings, etc).

Needless to say, the advantage of using flatpak is that it automatically downloads and install all the required dependencies. But if you do your own build, you need to figure out what they are and perform the setup manually. Generally, this is straightforward using your distribution’s package manager, but there can be some tricky exceptions ³.

While we are still at the configuration step, I believe it’s worth sharing two more tricks for WebKit developers:

You can use -DENABLE_SANITIZERS=address to produce Asan builds or builds with other sanitizers.
You can use -DCMAKE_CXX_FLAGS="-DENABLE_TREE_DEBUGGING" in release builds if you want to get access to the tree debugging functions (ShowRenderTree and the like). This flag is turned on by default for debug builds.

Building and running WebKit

Once the configure step is successful, you can build and install WebKit using the following CMake command ².

cmake --build WebKitBuild/Release --target install

When that operation completes, you should be able to run MiniBrowser with the following command:

LD_LIBRARY_PATH=WebKitBuild/install/Release/lib ./WebKitBuild/Release/bin/MiniBrowser

For WebKitTestRunner, some extra environment variables are necessary ²:

TEST_RUNNER_INJECTED_BUNDLE_FILENAME=$HOME/WebKit/WebKitBuild/Release/lib/libTestRunnerInjectedBundle.so LD_LIBRARY_PATH=WebKitBuild/install/Release/lib ./WebKitBuild/Release/bin/WebKitTestRunner filename.html

You can also use the official scripts, Tools/Script/run-minibrowser and Tools/Script/run-webkit-tests. They expect some particular paths, but a quick workaround is to use a symbolic link:

ln -s $HOME/WebKit/WebKitBuild $HOME/WebKit/WebKitBuild/GTK

Using rr for WebKit debugging

rr is generally easily installable from your distribution’s package manager. However, as stated on the project wiki page:

Support for the latest hardware and kernel features may require building rr from Github master.

Indeed, using the source has always worked best for me to avoid mysterious execution failures when starting the recording ⁴.

If you are not familiar with rr, I strongly invite you to take a look at the overview on the project home page or at some of the references I mentioned in the introduction. In any case, the first step is to record a trace by passing the program and arguments to rr. For example, to record a trace for MiniBrowser:

LD_LIBRARY_PATH=WebKitBuild/install/Debug/lib rr ./WebKitBuild/Debug/bin/MiniBrowser https://www.igalia.com/

After the program exits, you can replay the recorded trace as many times as you want. For hard-to-reproduce bugs (e.g. non-deterministic issues or involving a lot of manual steps), that means you only need to be able to record and reproduce the bug once and then can just focus on debugging. You can even turn off your machine after hours of exhausting debugging, then continue the effort later when you have more time and energy! The trace is played in a deterministic way, always using the same timing and pointer addresses. You can use most gdb commands (to run the program, interrupt it, and inspect data), but the real power comes from new commands to perform reverse execution!

Before coming to that, let’s explain how to handle programs with multiple processes, which is the case for WebKit and modern browsers in general. After you recorded a trace, you can display the pids of all recorded processes using the rr ps command. For example, we can see in the following output that the MiniBrowser process (pid 24103) actually forked three child processes, including the Network Process (pid 24113) and the Web Process (24116):

PID     PPID    EXIT    CMD
 --      0       ./WebKitBuild/Debug/bin/MiniBrowser https://www.igalia.com/
 24103   -9      ./WebKitBuild/Debug/bin/WebKitNetworkProcess 7 12
 24103   1       (forked without exec)
 24103   -9      ./WebKitBuild/Debug/bin/WebKitWebProcess 15 15

Here is a small debugging session similar to the single-process example from Chromium Chronicle #13 ⁵. We use the option -p 24116 to attach the debugger to the Web Process and -e to start debugging from where it exited:

rr replay -p 24116 -e
(rr) break RenderFlexibleBox::layoutBlock
(rr) rc # Run back to the last layout call
Thread 2 hit Breakpoint 1, WebCore::RenderFlexibleBox::layoutBlock (this=0x7f66699cc400, relayoutChildren=false) at /home/fred/src-obj/WebKit/Source/WebCore/rendering/RenderFlexibleBox.cpp:420
(rr) # Inspect anything you want here. To find the previous Layout call on this object:
(rr) cond 1 this == 0x7f66699cc400
(rr) rc
Thread 2 hit Breakpoint 1, WebCore::RenderFlexibleBox::layoutBlock (this=0x7f66699cc400, relayoutChildren=false) at /home/fred/src-obj/WebKit/Source/WebCore/rendering/RenderFlexibleBox.cpp:420
420     {
(rr) delete 1
(rr) watch -l m_style.m_nonInheritedFlags.effectiveDisplay # Or find the last time the effective display was changed
Thread 4 hit Hardware watchpoint 2: -location m_style.m_nonInheritedFlags.effectiveDisplay

Old value = 16
New value = 0
0x00007f6685234f39 in WebCore::RenderStyle::RenderStyle (this=0x7f66699cc4a8) at /home/fred/src-obj/WebKit/Source/WebCore/rendering/style/RenderStyle.cpp:176
176     RenderStyle::RenderStyle(RenderStyle&&) = default;

rc is an abbreviation for reverse-continue and continues execution backward. Similarly, you can use reverse-next, reverse-step and reverse-finish commands, or their abbreviations. Notice that the watchpoint change is naturally reversed compared to normal execution: the old value (sixteen) is the one after intialization, while the new value (zero) is the one before initialization!

Restarting playback from a known point in time

rr also has a concept of “event” and associates a number to each event it records. They can be obtained by the when command, or printed to the standard output using the -M option. To elaborate a bit more, suppose you add the following printf in RenderFlexibleBox::layoutBlock:

@@ -423,6 +423,8 @@ void RenderFlexibleBox::layoutBlock(bool relayoutChildren, LayoutUnit)
     if (!relayoutChildren && simplifiedLayout())
         return;

+    printf("this=%p\n", this);
+

After building, recording and replaying again, the output should look like this:

$ rr -M replay -p 70285 -e # replay with the new PID of the web process.
...
[rr 70285 57408]this=0x7f742203fa00
[rr 70285 57423]this=0x7f742203fc80
[rr 70285 57425]this=0x7f7422040200
...

Each printed output is now annotated with two numbers in bracket: a PID and an event number. So in order to restart from when an interesting output happened (let’s say [rr 70285 57425]this=0x7f7422040200), you can now execute run 57425 from the debugging session, or equivalently:

rr replay -p 70285 -g 57425

Older traces and parallel debugging

Another interesting thing to know is that traces are stored in ~/.local/share/rr/ and you can always specify an older trace to the rr command e.g. rr ps ~/.local/share/rr/MiniBrowser-0. Be aware that the executable image must not change, but you can use rr pack to be able to run old traces after a rebuild, or even to copy traces to another machine.

To be honest, most the time I’m just using the latest trace. However, one thing I’ve sometimes found useful is what I would call the “parallel debugging” technique. Basically, I’m recording one trace for a testcase that exhibits the bug and another one for a very similar testcase (e.g. with one CSS property difference) that behaves correctly. Then I replay the two traces side by side, comparing them to understand where the issue comes from and what can be done to fix it.

The usage documentation also provides further tips, but this should be enough to get you started with time travel debugging in WebKit!

RelWithDebInfo build type (which yields an optimized release build with debug symbols) might also be interesting to consider in some situations, e.g. debugging bugs that reproduce in release builds but not in debug builds. ↩
Using an installation directory might not be necessary, but without that, I had trouble making the whole thing work properly (wrong libraries loaded or libraries not found). ↩ ↩² ↩³
In my case, I chose the easiest path to disable some features, namely -DUSE_JPEGXL=OFF, -DUSE_LIBBACKTRACE=OFF, and -DUSE_GSTREAMER_TRANSCODER=OFF. ↩
Incidentally, you are likely to get an error saying that perf_event_paranoid is required to be at most 1, which you can force using sudo sysctl kernel.perf_event_paranoid=1. ↩
The equivalent example would probably have been to watch for the previous style change with watch -l m_style, but this was exceeding my hardware watchpoint limit, so I narrowed it down to a smaller observation scope. ↩

Infinite version of the Set card game

2023-06-14T00:00:00+02:00

edit 2023/06/17: I elaborated a bit more in the conclusion about the open problem of finding a minimal $\kappa$ .

The Set Game

I visited A Coruña last week for the Web Engines Hackfest and to participate to internal events with my fellow Igalians. One of our tradition being to play board games, my colleague Ioanna presented a card game called Set. To be honest I was not very good at it, but it made me think of a potential generalization for infinite sets that is worth a blog post…

Basically, we have a deck of $\lambda^\mu$ cards with $\mu = 4$ features (number of shapes, shape, shading and color), each of them taking $\lambda = 3$ possible values (e.g. red, green or purple for the color). Given $\kappa$ cards on the table, players must extract $\lambda$ cards forming what is called a Set, which is defined as follows: for each of the $\mu$ features, either the cards use the same value or they use pairwise distinct values.

Formally, this can be generalized for any cardinal $\lambda$ as follows:

A card is a function from $\mu$ (the features) to $\lambda$ (the values).
A set of cards $S$ is a Set iff for any feature $\alpha < \mu$ , the mapping $\Phi_\alpha^S : S \rightarrow \lambda$
that maps a card $c$ to the value $c(\alpha)$ is either constant or one-to-one.

Given a value $\kappa$ such that $\lambda \leq \kappa \leq \lambda^\mu$ , can we always extract a Set when $\kappa$ cards are put on the table? Or said otherwise, is there a set of $\kappa$ cards from which we cannot extract any Set?

Trivial cases ( $\lambda \leq 2$ or $\mu \leq 1$ )

Given $\kappa \geq \lambda$ cards, we can always extract a Set $S$ in the following trivial cases:

If $\mu = 0$ then the deck contains only one card $c = \emptyset$ . If $\lambda \geq 2$ such a set of $\kappa$ cards does not exist. Otherwise we can just take $S = \emptyset$ or $S ={\{c\}}$ : these are Sets since the definition is trivial for $\mu = 0$ .
If $\mu \geq 1$ and $\lambda = 0$ then the deck is empty. We take $S = \emptyset$ and for any $\alpha < \mu$ , $\Phi_\alpha^\emptyset = \emptyset$ is both constant and one-to-one.
If $\mu = 1$ and $\lambda \geq 1$ , a card $c$ is fully determined by its value $c(0)$ , so distinct cards give distinct values. So we can pick any $S$ of size $\lambda$ : it is a Set since $\Phi_0$ is one-to-one.
If $\lambda = 1$ then we can pick any singleton $S$ : it is a Set since for any feature $\alpha < \mu$ the mapping $\Phi_\alpha^S$ is both constant and one-to-one.
If $\lambda = 2$ then we can pick any pair of cards $S$ : it is a Set since for any feature $\alpha < \mu$ the mapping $\Phi_\alpha^S$ is either constant or one-to-one (depending on whether the two cards display the same value or not).

👉🏼 For the rest of this blog post, I’ll assume $\mu \geq 2$ and $\lambda \geq 3$ .

Not enough cards on the table ( $\kappa \leq \mu$ )

If $\mu \geq \kappa \geq \lambda \geq 3$ then we consider cards $c_\alpha$ for each $\alpha < \kappa$ defined for each $\beta < \mu$ as $c_\alpha{(\beta)} = \delta_{\alpha, \beta}$ (using Kronecker delta). If we extract a subset $S$ from these cards and $\alpha_1, \alpha_2, \alpha_3 < \kappa \leq \mu$ are indices for elements of $S$ then $\Phi_{\alpha_1}^S$ respectively evaluates to 1, 0 and 0 for $\alpha_1, \alpha_2, \alpha_3$ so $S$ is not a Set.

👉🏼 For the rest of this blog post, we’ll assume $\mu < \kappa$ and will even focus on the minimal case $\kappa = \lambda$ .

Finite number of values ( $\lambda < \aleph_0$ )

Let’s consider a finite number of values $\lambda \geq 3$ and define the card $c_\alpha$ for each $\alpha < \lambda$ as follows: $c_\alpha(\beta) = \delta_{\alpha, 0}$ for $\beta=0$ (again Kronecker delta) and $c_\alpha(\beta) = \alpha$ for $0 < \beta < \mu$ . Since $\mu \geq 2$ , the latter case shows that $S = \{ c_\alpha : \alpha < \lambda \}$ contains exactly $\lambda$ cards. Since $\kappa = \lambda < \aleph_0$ , the only way to extract a subset of size $\lambda$ would be to take all the cards. But they don’t form a Set since by construction ${\Phi_0{(c_0)}} = 1 \neq 0 = {\Phi_0{(c_1)}} = {\Phi_0{(c_2)}}$ .

👉🏼 For the rest of the blog post, I’ll assume $\lambda$ is infinite.

Singular number of values ( $\mi{cf}(\lambda) < \lambda$ )

If $\lambda$ is a singular cardinal, then we consider a cofinal sequence ${\{ \alpha_{\gamma}, \gamma < \nu \}} \subseteq \lambda$ of length $\nu < \lambda$ and define the card $c_\alpha$ for $\alpha < \lambda$ as follows:

For $\beta = 0$ , we consider the smallest ordinal $\gamma < \nu \leq \lambda$ such that $\alpha < \alpha_\gamma$ and define $c_\alpha{(0)} = \gamma$ .
For any $1 \leq \beta < \mu$ , ${c_\alpha{(\beta)}} = \alpha$ .

Since $\mu \geq 2$ , the latter case shows that these are $\lambda$ distinct cards. Consider $S \subseteq {\{c_\alpha, \alpha < \lambda \}}$ . If $\Phi_0^S$ evaluates to a constant value $\gamma < \nu$ then $S = {\left(\Phi_0^S\right)}^{-1}(\{\gamma\})$ has size at most ${|\alpha_\gamma|} < \lambda$ . If instead $\Phi_0^S$ is one-to-one then it takes at most $\nu$ distinct values so again ${|S|} \leq \nu < \lambda$ . Hence $S$ is not a Set.

👉🏼 For the rest of the blog post, I’ll assume $\lambda$ is an infinite regular cardinal.

Finite number of features ( $\mu < \aleph_0$ )

In this section, we assume that the number of features $\mu$ is finite. Let’s consider $\lambda$ cards $c_\alpha$ and extract a Set $S$ by induction as follows:

$S_0 = \{ c_\alpha : \alpha < \lambda \}$
For any β<μ\beta < \mu, we construct Sβ+1⊆SβS_{\beta+1} \subseteq S_{\beta} of cardinality λ\lambda. We note that λ\lambda is regular and Sβ=⋃α∊ΦβSβ(Sβ)(ΦβSβ)−1({α}) S_\beta = {\bigcup_{\alpha \in \Phi_\beta^{S_\beta}{(S_\beta)}} {\left(\Phi_\beta^{S_\beta}\right)}^{-1}{(\{\alpha\})} }
so there are only two possible cases:
- If $\Phi_\beta^{S_\beta}{(S_\beta)}$ is of cardinality $\lambda$ then pick $\lambda$ elements from $S_\beta$ with pairwise distinct image by $\Phi_\beta^{S_\beta}$ .
- Otherwise, if there is $\alpha < \lambda$ such that ${\left(\Phi_\beta^{S_\beta}\right)}^{-1}{(\{\alpha\})}$ is of cardinality $\lambda$ , then let it be our $S_{\beta+1}$ .
$S = S_{\mu}$

Then by construction, $S$ is of size $\lambda$ and for any $\beta < \mu$ , $S \subseteq S_{\beta+1}$ which means that $\Phi_\beta^S = { {(\Phi_\beta^{S_\beta})}_{| S}}$ is either constant or one-to-one.

Incidentally, although I said I would focus on the case $\kappa = \lambda$ the result of this session shows that we can extract a Set if more than $\lambda$ cards are put on the table!

Summary and open questions

Above are the results I found from a preliminary investigation, which can be summarized as follows:

If $\lambda \leq 2$ or $\mu \leq 1$ then we can always find a Set from $\kappa \geq \lambda$ cards.
If $3 \leq \lambda \leq \mu$ then for any $\kappa$ such that $\lambda \leq \kappa \leq \mu$ there is a set of $\kappa$ cards from which we cannot extract any Set.
If $2 \leq \mu < \lambda < \aleph_0$ there is a set of $\lambda$ cards from which we cannot extract any Set.
If $2 \leq \mu$ and $\lambda$ is singular then there is a set of $\lambda$ cards from which we cannot extract any Set.
If $2 \leq \mu < \aleph_0 \leq {\mi{cf}(\lambda)} = \lambda$ , then we can always find a Set from $\kappa \geq \lambda$ cards.

Note that for the standard game $3 = \lambda < \mu = 4$ the only of the results above that applies is (2). Indeed, having only three or four cards on the table is generally not enough to extract a Set!

So far, I was not able to find an answer for the case $\aleph_0 \leq \mu < {\mi{cf}(\lambda)} = \lambda \leq \kappa$ . It looks like the inductive construction from the previous paragraph could work, but it’s not clear what guarantees that taking intersection at limit step would preserve size $\kappa$ (an idea would be to use closed unbounded $S_\beta$ instead but I didn’t find a satisfying proof). I also failed to build a counter-example set of $\lambda$ cards without any Set subset, despite several attempts.

More generally, an open problem is to determine the minimal number of cards $\kappa$ (with $\lambda \leq \kappa \leq \lambda^\mu$ ) to put on the table to ensure players can always extract a Set subset… or even if such a number actually exists! If it does, then in cases (2) (3) (4) we only know $\kappa > \lambda$ . In cases (1) and (5) the minimum value $\kappa = \lambda$ works ; and when $\mu \geq 2$ and $\lambda \geq 3$ are finite, the maximum value $\kappa = \lambda^\mu$ means taking the full deck, which works too (e.g. it always contains the Set given by $\forall \alpha < \lambda, \forall \beta < \mu, c_\alpha(\beta) = \alpha$ ). Incidentally, note that the latter case is consistent with (2) and (3) since we have $\lambda^\mu > \mu, \lambda$ . But in general for infinite parameters putting $\kappa = \lambda^\mu$ cards on the table does not mean putting the full deck, so it’s less obvious whether we can extract a Set…

MathML in Chrome 109

2023-01-31T00:00:00+01:00

2023/06/15: Most of this article was written in January 2023, but I only finished and published it in June. Hopefully, information is still up-to-date 😃

TL;DR

Igalia announced early this year that MathML is back to Chromium. This is an excellent news, for those like me who read and write mathematics on the web. Native support for such a universal human language sounds uncontroversial but it has taken a quarter century to get MathML implemented in all web engines… I invite you to listen to the corresponding episode of Igalia Chats where you can find some rationale about why it took so long. In this blog post, I’ll try to elaborate a bit about technical challenges we were able to overcome, thanks to the introduction of MathML Core. To be honest, things are still not perfect, but at least we can now rely on a clean foundation and continue to iterate improvements and fix bugs!

Timeline

As a reference, I’m providing below a list of important events (based on this timeline and this slide) for native MathML support in browsers:

1993-1995: Early HTML specifications contained a <MATH> tag, which was experimented in Dave Raggett’s Arena browser.
1998-2003: Initial specifications of MathML were published. Volunteer contributor Roger Sidje added support in Gecko for XHTML documents and that was released in Mozilla 1.0.
2006-2013:
- MathML was integrated in HTML5. George Chavchanidze created a CSS-based implementation and shipped it in Opera. Support was implemented in all Open Source browsers thanks to the effort of volunteer contributors (including Roger Sidje, François Sausset, Alex Miłowski, Dave Barton and myself). However, MathML was removed from Chrome shortly after.
- Apple engineers, notably Chris Fleizach and James Craig, implemented NSAccessibility support for WebKit and VoiceOver.
2014-2016:
- I published an implementation note with extra rendering rules based on Donald Knuth’s T_EXBook and Microsoft’s OpenType MATH table. I implemented these rules in Gecko and WebKit via a crowdfunding project.
- During a few codecamps and hackfests, Joanmarie Diggs (Igalia), Alexander Surkov (at that time at Mozilla) and I (who just joined the AcceSciTech project) worked on ATK and NSAccessibility support for Gecko, WebKit and Orca.
- Jamie Teh and Neil Soiffer added MathML accessibility support for Gecko and Chromium by exposing the raw source code via a dedicated MSAA API.
- Igalia performed a big refactoring of WebKit’s implementation. Initially done by Alejandro García in 2015, I took over the work when I joined the company the following year.
2019-2023:
- Igalia launched the “MathML in Chromium” project with the support of various sponsors. Rob Buis and I worked on a brand new implementation in Chromium. Brian Kardell also joined Igalia at the beginning of the project and has been instrumental in communication, advocacy and standardization efforts.
- A new MathML Core specification became the reference for browser implementations and was shipped in Chromium 109.
- Alexander Surkov joined Igalia and helped with the MathML AAM specification, which we used to implement ATK and NSAccessibility support in Chromium.
- I’ve regularly presented our results in conferences. For completeness, here are the talks I gave:
  - November 2022: Shipping MathML in Chrome 109 at BlinkOn 17 - video.
  - June 2022: Shipping MathML in Chrome at the Web Engines Hackfest.
  - May 2022: Remainder estimate for MathML integration - video.
  - November 2021: MathML, onde estamos? - video.
  - November 2019: MathML Core at BlinkOn 11.
  - October 2019: MathML in Browsers at the Web Engines Hackfest - video.
  - April 2019: MathML in LayoutNG at BlinkOn 10.

A simple fraction

One of the first thing you can notice is that a significant portion of the work was done by volunteer contributors at the time when the set of requirements to ship a feature or integrate it in the specification was very small. Let’s consider some basic MathML formula to illustrate that:

<math style="font: 32pt STIX Two Math;">
  <mfrac>
    <mspace width="2em" height="1em" depth=".5em" style="background: red"/>
    <mspace width="2em" height=".5em" depth="1em" style="background: blue"/>
  </mfrac>
</math>

The <math> element is the root container for a new MathML formula. The <mfrac> element describes a fraction with two <mspace/> children corresponding respectively to its numerator and denominator. These <mspace/> elements are boxes of specified width, ascent and descent, as described by their width, height and depth attributes (using T_EX terminology). Finally, the style attribute attaches inline style to MathML elements.

MathML 3 describes how to interpret the attributes of the <mspace> element. In general, it does not really define interaction with other web technologies and the behavior of the style attribute “is not specified”. However, browser implementers can safely assume it has the same syntax and semantic as HTML and that’s basically what MathML Core says. So we know the <math> element uses the font family STIX Two Math and font size of 32pt, while the <mspace> elements have red and blue background.

But how should we layout the <mfrac> element?

T_EX-based layout

MathML 3 does not really say much more than the semantic “used for fractions” and the syntax <mfrac> numerator denominator </mfrac>. For the fraction bar, “the exact thickness of these is left up to the rendering agent”. MathML Core has a more detailed description involving vertical position and thickness of the fraction bar, minimal gaps between numerator/denominator and fraction bar as well as minimal shifts of the numerator/denominator’s baseline with respect to the baseline, horizontal centering of the numerator/denominator and more.

One of the essential part is that many of these parameters are taken from STIX Two Math’s Open Type MATH table such as fractionNumeratorShiftUp, fractionDenominatorGapMin etc and are generalizing the rules from the T_EXBook. You may notice parameters like fractionNumeratorDisplayStyleShiftUp, fractionDenomDisplayStyleGapMin etc which are variants based on the math-style we will explain below.

In any case, as long as they follow the layout algorithm described in MathML Core, all browsers should now render the fraction example the same, right?

Exposing MathML magic

Assuming STIX Two Math is available or provided as a web font, Chrome and Firefox renders the previous example that way:

You can notice that the size of the numerator/denominator is different from what you get in Safari. Indeed, the web inspector shows that the <mspace> elements has a smaller font-size than the one of the <math> or <mfrac> elements. This magic is suggested by the remaining part of the description of MathML 3 we have ignored so far:

The mfrac element sets displaystyle to “false”, or if it was already false increments scriptlevel by 1, within numerator and denominator.

Let’s try to explain this a bit more. The displaystyle and scriptlevel properties are concepts from MathML 3 which are themselves generalizing T_EX. Imagine many nested superscripts: the font size is scaled down each time you enter a superscript, and becomes smaller and smaller. Here, the scriptlevel corresponds to the nesting level and it affects the font size. Regarding displaystyle=false, this is typically used when you want to render the formula with more compact height e.g. for formulas rendered inline within a paragraph of text.

So the MathML 3 spec essentially says that the subformulas numerator/denominator should be rendered with compact height, and if we are already in such a mode we should even scale down the font size within these subformulas. The only problem is that this behavior is not specified at all by CSS!

In order to solve that, we introduced the math-style and math-depth CSS properties, explaining how they affect font-size in the cascade. That way the behavior can be completely handled by the CSS engines. Then MathML Core describes how displaystyle/scriptlevel attributes are treated as presentational hints together with appropriate rules in the User Agent stylesheet. At the end everything is well-specified and Safari’s behavior is a bug.

Integration with the web platform

The previous paragraphs described how we integrated T_EX-based math layout inside the web platform. MathML 3 already started that by defined a SGML tree and HTML5 / MathML Core went further by defining the parsing rules, a WebIDL or integration with HTML and SVG.

The integration problem can also be seen on the other side: how do we interpret CSS features for math layout? For example what happens if one specifies a CSS width on the <mfrac> element? Or a padding? What would be the CSS layout if there are less or more children than just the numerator/denominator? What happen if float: left or position: absolute are specified on a child? Probably math authors don’t care too much but that’s still something that needs to be defined in the spec and implemented in a cross-compatible way.

Answering these questions were also useful for many reasons. Just to list a few examples off the top of my head:

In order to make things manageable, MathML Core focuses on a susbet of MathML 3 and some people are interested in writing polyfills for legacy or future features, which is something that can often be done by CSS.
Using a terminology that takes into account direction/writing-mode facilitates the implementation of right-to-left math and could allow vertical math in the future.
To avoid visual confusion between the fraction bar and another adjacent items (e.g. minus sign or another fraction’s bar), a default 1-pixel space is added around the element. This can be easily done by setting the padding in the User Agent stylesheet… as long as it’s supported!
Bypassing MathML layout in invalid subtrees (e.g. <mfrac> with three children) as was done in old implementations are making things harder for fuzzers and testcase reducers. Additionally these fuzzers often find issues happening with exotic CSS features, so better if the spec says how to handle them.

Web Platform tests

We’ve seen some examples on how to improve the MathML specification. This leads to the topic of Web Platform tests, which are very important to verify conformance with that specification and ensure browser interoperability. That’s a good tool and motivation for implementers and they may also prevent regressions since they are run regularly, at least each time a change is made in the browser repository.

However, MathML 3 was using manual tests where one has to visually compare the rendering of the 1675 testcases in a browser against a sample rendering and decide whether it passes or not. That’s a quite tedious and not very robust way of testing!

Fortunately, this has changed with MathML Core with around 3000 tests following the format of Web Platform tests! Trying to analyze that a bit, Chrome has an excellent pass rate except for things that need to be clarified in the spec. Firefox and Safari are also doing good job but failing on tests for new behavior or features that got clarified by MathML Core.

A complementary metrics for interoperability is MDN’s browser-compat-data. You can find these compatibility data at the bottom of MDN pages, for example for the MathML article. Again, a quick analysis confirms that browsers are quite interoperable for MathML Core, possibly mixed with other web technologies, but not for new CSS features (implemented in Chromium) or legacy MathML features (remaining in Firefox and Safari).

Accessibility

Abraham Nemeth who developed Braille code for mathematical notations, presented his view on math accessibility in the book Braille into the next millenium (2000):

The Principle of Meaning Versus Notation: In my view, it is the transcriber’s function to supply only notation, not meaning in an accessible form (speech or braille). It is the reader’s function to extract the meaning from the notation the transcriber supplies. Consider the common notation (x,y). That notation can mean many things: the ordered pair whose first component is x and second component is y ; the point in the cartesian coordinate with abscissa x and ordinate y; the open interval on the real line with left endpoint x and right endpoint y; or the greatest common divisor of x and y. The transcriber’s function, however, is only to convey this five-symbol expression to the reader. It is the reader’s function to extract whatever meaning his experience and the context of the text permit.

However, for people with different need, or for various uses cases, it is often important to provide more semantic. This is something various members from the MathML WG have worked on and many discussions are currently happening.

In any case, for native MathML implementation in browsers we are limited to what was called “presentation MathML” in legacy specifications, which gives a very small amount of information: an <mfrac> can be fraction, binomial coefficient etc ; an <msup> can be used to denote a power, a derivative, an index etc ; an <mtable> can be used for matrices, to perform table layout etc

As explained above, some effort was done to implement native accessibility support for MathML in WebKit/Gecko for various assistive technologies in Linux, macOS or Windows. For the present project, we didn’t try to do more than what currently exists. We only documented existing support in the MathML AAM specification and implemented it in Chromium. This is the bare minimum we can do to ensure the reader can follow Nemeth’s approach.

Conclusion

As the main person who has led this effort, this is great personal achievement and I’d like to thank all the people who supported it, including my colleagues at Igalia, members of the MathML community, standardization groups, browser developers, web developers, sponsors and contributors to the Open Collective. Finally we made it! 🎉

Update on OpenType MATH fonts

2022-06-20T00:00:00+02:00

I mentioned in a previous post that Igalia organized the Web Engines Hackfest 2022 last week. As usual, fonts were one of the topic discussed. Dominik Röttsches presented COLRv1 color vector fonts in Chrome and OSS (transcript) and we also settled a breakout session on Tuesday morning. Because one issue raised was the availability of OpenType MATH fonts on operating systems, I believe it’s worth giving an update on the latest status…

There are only a few fonts with an OpenType MATH table. Such fonts can be used for math layout e.g. modern TeX engines to render LaTeX, Microsoft Office to render OMML or Web engines to render MathML. Three of such fonts are interesting to consider, so I’m providing a quick overview together with screenshots generated by XeTeX from the LaTeX formula $${\sqrt{\sum_{n=1}^\infty {\frac{10}{n^4}}}} = {\int_0^\infty \frac{2x dx}{e^x-1}} = \frac{\pi^2}{3} \in {\mathbb R}$$:

Cambria Math is a proprietary font installed by default on Microsoft Windows.
STIX Two Math is a font under the OFL license with a large unicode coverage for scientific characters.
Latin Modern Math is a font under the GUST license with the Computer Modern style that TeX uses by default.

Recently, Igalia has been in touch with Myles C. Maxfield who has helped with internal discussion at Apple regarding inclusion of STIX Two Math in the list of fonts on macOS. Last week he came back to us announcing it’s now the case on all the betas of macOS 13 Ventura 🎉 ! I just tested it this morning and indeed STIX Two Math is now showing up as expected in the Font Book. Here is the rendering of the last slide of my hackfest presentation in Safari 16.0:

Obviously, this is a good news for Chromium and Firefox too. For the former, we are preparing our MathML intent-to-ship and having decent rendering on macOS by default is important. As for the latter, we could in the future finally get rid of hardcoded tables to support the deprecated STIXGeneral set.

Another interesting platform to consider for Chromium is Android. Last week, there has been new activity on the Noto fonts bug and answers seem more encouraging now… So let’s hope we can get a proper math font on Android soon!

Finally, I’m not exactly sure about the situation on Linux and it may be different for the various distributions. STIX and Latin Modern should generally be available as system packages that can be easily installed. It would be nicer if they were pre-installed by default though…

Short blog post from Madrid's hotel room

2022-06-17T00:00:00+02:00

This week, I finally went back to A Coruña for the Web Engines Hackfest and internal company meetings. These were my first on-site events since the COVID-19 pandemic. After two years of non-super-exciting virtual conferences I was so glad to finally be able to meet with colleagues and other people from the Web.

Igalia has grown considerably and I finally get to know many new hires in person. Obviously, some people were still not able to travel despite the effort we put to settle strong sanitary measures. Nevertheless, our infrastructure has also improved a lot and we were able to provide remote communication during these events, in order to give people a chance to attend and participate !

Work on the Madrid–Galicia high-speed rail line finally completed last December, meaning one can now travel with fast trains between Paris - Barcelona - Madrid - A Coruña. This takes about one day and a half though and, because I’m voting for the Legislative elections in France, I had to shorten a bit my stay and miss nice social activities 😥… That’s a pity, but I’m looking forward to participating more next time!

Finally on the technical side, my main contribution was to present our upcoming plan to ship MathML in Chromium. The summary is that we are happy with this first implementation and will send the intent-to-ship next week. There are minor issues to address, but the consensus from the conversations we had with other attendees (including folks from Google and Mozilla) is that they should not be a blocker and can be refined depending on the feedback from API owners. So let’s do it and see what happens…

There is definitely a lot more to write and nice pictures to share, but it’s starting to be late here and I have a train back to Paris tomorrow. 😉

Igalia's contribution to the Mozilla project and Open Prioritization

2020-07-13T00:00:00+02:00

As many web platform developer and Firefox users, I believe Mozilla’s mission is instrumental for a better Internet. In a recent Igalia’s chat about the Web Ecosystem Health, participants made the usual observation regarding this important role played by Mozilla on the one hand and the limited development resources and small Firefox’s usage share on the other hand. In this blog post, I’d like to explain an experimental idea we are launching at Igalia to try and make browser development better match the interest of the web developer and user community.

Igalia’s contribution to browser repositories

As mentioned in the past in this blog, Igalia has contributed to different part of Firefox such as multimedia (e.g. <video> support), layout (e.g. Stylo, WebRender, CSS, MathML), scripts (e.g. BigInt, WebAssembly) or accessibility (e.g. ARIA). But is it enough?

Although commit count is an imperfect metric it is also one of the easiest to obtain. Let’s take a look at how Igalia’s commits repositories of the Chromium (chromium, v8), Mozilla (mozilla-central, servo, servo-web-render) and WebKit projects were distributed last year:

Diagram showing, the distribution of Igalia's contributions to browser repositories in 2019 (~5200 commits). Chromium (~73%), Mozilla (~4%) and WebKit (~23%).

As you can see, in absolute value Igalia contributed roughly 3/4 to Chromium, 1/4 to WebKit, with a small remaining amount to Mozilla. This is not surprising since Igalia is a consulting company and our work depends on the importance of browsers in the market where Chromium dominates and WebKit is also quite good for iOS devices and embedded systems.

This suggests a different way to measure our contribution by considering, for each project, the percentage relative to the total amount of commits:

Diagram showing, for each project, the percentage of Igalia's commits in 2019 relative to the total amount of the project. From left to right: Chromium (~3.96%), Mozilla (~0.43%) and WebKit (~10.92%).

In the WebKit project, where ~80% of the contributions were made by Apple, Igalia was second with ~10% of the total. In the Chromium project, the huge Google team made more than 90% of the contributions and many more companies are involved, but Igalia was second with about 4% of the total. In the Mozilla project, Mozilla is also doing ~90% of the contributions but Igalia only had ~0.5% of the total. Interestingly, the second contributing organization was… the community of unindentified gmail.com addresses! Of course, this shows the importance of volunteers in the Mozilla project where a great effort is done to encourage participation.

Open Prioritization

From the commit count, it’s clear Igalia is not contributing as much to the Mozilla project as to Chromium or WebKit projects. But this is expected and is just reflecting the priority set by large companies. The solid base of Firefox users as well as the large amount of volunteer contributors show that the Mozilla project is nevertheless still attractive for many people. Could we turn this into browser development that is not funded by advertising or selling devices?

Another related question is whether the internet can really be shaped by the global community as defended by the Mozilla’s mission? Is the web doomed to be controlled by big corporations doing technology’s “evangelism” or lobbying at standardization committees? Are there prioritization issues that can be addressed by moving to a more collective decision process?

At Igalia, we internally try and follow a more democratic organization and, at our level, intend to make the world a better place. Today, we are launching a new Open Prioritization experiment to verify whether crowdfunding could be a way to influence how browser development is prioritized. Below is a short (5 min) introductory video:

I strongly recommend you to take a look at the proposed projects and read the FAQ to understand how this is going to work. But remember this is an experiment so we are starting with a few ideas that we selected and tasks that are relatively small. We know there are tons of user reports in bug trackers and suggestions of standards, but we are not going to solve everything in one day !

If the process is successful, we can consider generalizing this approach, but we need to test it first, check what works and what doesn’t, consider whether it is worth pursuing, analyze how it can be improved, etc

Two Crowdfunding Tasks for Firefox

Representation of the CIELAB color space (top view) by Holger Everding, under CC-SA 4.0.

As explained in the previous paragraph, we are starting with small tasks. For Firefox, we selected the following ones:

CSS lab() colors. This is about giving web developers a way to express colors using the CIELAB color space which approximates better the human perception. My colleague Brian Kardell wrote a blog with more details. Some investigations have been made by Apple and Google. Let’s see what we can do for Firefox !
SVG path d attribute. This is about expressing SVG path using the corresponding CSS syntax for example <path style="d: path('M0,0 L10,10,...')">. This will likely involve a refactoring to use the same parser for both SVG and CSS paths. It’s a small feature but part of a more general convergence effort between SVG and CSS that Igalia has been involved in.

Conclusion

Is this crowd-funded experiment going to work? Can this approach solve the prioritization problems or at least help a bit? How can we improve that idea in the future?…

There are many open questions but we will only be able to answer them if we have enough people participating. I’ll personally pledge for the two Firefox projects and I invite you to at least take a look and decide whether there is something there that is interesting for you. Let’s try and see!

Contributions to Web Platform Interoperability (First Half of 2020)

2020-07-06T00:00:00+02:00

Note: This blog post was co-authored by AMP and Igalia teams.

Web developers continue to face challenges with web interoperability issues and a lack of implementation of important features. As an open-source project, the AMP Project can help represent developers and aid in addressing these challenges. In the last few years, we have partnered with Igalia to collaborate on helping advance predictability and interoperability among browsers. Standards and the degree of interoperability that we want can be a long process. New features frequently require experimentation to get things rolling, course corrections along the way and then, ultimately as more implementations and users begin exploring the space, doing really interesting things and finding issues at the edges we continue to advance interoperability.

Both AMP and Igalia are very pleased to have been able to play important roles at all stages of this process and help drive things forward. During the first half of this year, here’s what we’ve been up to…

Default Aspect Ratio of Images

In our previous blog post we mentioned our experiment to implement the intrinsic size attribute in WebKit. Although this was a useful prototype for standardization discussions, at the end there was a consensus to switch to an alternative approach. This new approach addresses the same use case without the need of a new attribute. The idea is pretty simple: use specified width and height attributes of an image to determine the default aspect ratio. If additional CSS is used e.g. “width: 100%; height: auto;”, browsers can then compute the final size of the image, without waiting for it to be downloaded. This avoids any relayout that could cause bad user experience. This was implemented in Firefox and Chromium and we did the same in WebKit. We implemented this under a flag which is currently on by default in Safari Tech Preview and the latest iOS 14 beta.

Scrolling

We continued our efforts to enhance scroll features. In WebKit, we began with scroll-behavior, which provides the ability to do smooth scrolling. Based on our previous patch, it has landed and is guarded by an experimental flag “CSSOM View Smooth Scrolling” which is disabled by default. Smooth scrolling currently has a generic platform-independent implementation controlled by a timer in the web process, and we continue working on a more efficient alternative relying on the native iOS UI interfaces to perform scrolling.

We have also started to work on overscroll and overscroll customization, especially for the scrollend event. The scrollend event, as you might expect, is fired when the scroll is finished, but it lacked interoperability and required some additional tests. We added web platform tests for programmatic scroll and user scroll including scrollbar, dragging selection and keyboard scrolling. With these in place, we are now working on a patch in WebKit which supports scrollend for programmatic scroll and Mac user scroll.

On the Chrome side, we continue working on the standard scroll values in non-default writing modes. This is an interesting set of challenges surrounding the scroll API and how it works with writing modes which was previously not entirely interoperable or well defined. Gaining interoperability requires changes, and we have to be sure that those changes are safe. Our current changes are implemented and guarded by a runtime flag “CSSOM View Scroll Coordinates”. With the help of Google engineers, we are trying to collect user data to decide whether it is safe to enable it by default.

Another minor interoperability fix that we were involved in was to ensure that the scrolling attribute of frames recognizes values “noscroll” or “off”. That was already the case in Firefox and this is now the case in Chromium and WebKit too.

Intersection and Resize Observers

As mentioned in our previous blog post, we drove the implementation of IntersectionObserver (enabled in iOS 12.2) and ResizeObserver (enabled in iOS 14 beta) in WebKit. We have made a few enhancements to these useful developer APIs this year.

Users reported difficulties with observe root of inner iframe and the specification was modified to accept an explicit document as a root parameter. This was implemented in Chromium and we implemented the same change in WebKit and Firefox. It is currently available Safari Tech Preview, iOS 14 beta and Firefox 75.

A bug was also reported with ResizeObserver incorrectly computing size for non-default zoom levels, which was in particular causing a bug on twitter feeds. We landed a patch last April and the fix is available in the latest Safari Tech Preview and iOS 14 beta.

Resource Loading

Another thing that we have been concerned with is how we can give more control and power to authors to more effectively tell the browser how to manage the loading of resources and improve performance.

The work that we started in 2019 on lazy loading has matured a lot along with the specification.

The lazy image loading implementation in WebKit therefore passes the related WPT tests and is functional and comparable to the Firefox and Chrome implementations. However, as you might expect, as we compare uses and implementation notes it becomes apparent that determining the moment when the lazy image load should start is not defined well enough. Before this can be enabled in releases some more work has to be done on improving that. The related frame lazy loading work has not started yet since the specification is not in place.

We also added an implementation for stale-while-revalidate. The stale-while-revalidate Cache-Control directive allows a grace period in which the browser is permitted to serve a stale asset while the browser is checking for a newer version. This is useful for non-critical resources where some degree of staleness is acceptable, like fonts. The feature has been enabled recently in WebKit trunk, but it is still disabled in the latest iOS 14 beta.

Contributions were made to improve prefetching in WebKit taking into account its cache partitioning mechanism. Before this work can be enabled some more patches have to be landed and possibly specified (for example, prenavigate) in more detail. Finally, various general Fetch improvements have been done, improving the fetch WPT score. Examples are:

What’s next

There is still a lot to do in scrolling and resource loading improvements and we will continue to focus on the features mentioned such as scrollend event, overscroll behavior and scroll behavior, lazy loading, stale-while-revalidate and prefetching.

As a continuation of the work done for aspect ratio calculation of images, we will consider the more general CSS aspect-ratio property. Performance metrics such as the ones provided by the Web Vitals project is also critical for web developers to ensure that their websites provide a good user experience and we are willing to investigate support for these in Safari.

We love doing this work to improve the platform and we’re happy to be able to collaborate in ways that contribute to bettering the web commons for all of us.

Transcendental number from the oscillating zeno's paradox

2020-04-06T00:00:00+02:00

Oscillating Zeno’s paradox

In a previous blog post, I described an oscillating Zeno’s paradox, which can be formalized as follows. Atalanta moves on the real line. She leaves from 0 and at step $n$ she decides to move forward or backward by $\frac{1}{2^{n+1}}$ . If $\epsilon_n \in {\{-1, 1\}}$ corresponds to the chosen direction then writing $S_0 = 0$ and

S_{n+1} = S_n + \frac{\epsilon_n}{2^{n+1}}

we obtain sequence of Atalatan’s position on the real line.

In the non-oscillating case ( $\epsilon_n = 1$ for all $n$ ) then this just corresponds to a geometric series of common ration $\frac{1}{2}$ . $S_n$ converges to 1 by lower values, with remaining distance being $\frac{1}{2^n}$ . More generally, $S_n$ is just the partial sum of an absolutely convergent series and so is convergent to a destination $S_\infty = {\sum_{n=0}^{+\infty} \frac{\epsilon_n}{2^{n+1}}}$ . It is easy to see that $\left|S_n\right| < 1$ , $\left|S_\infty\right| \leq 1$ and ${\left| S_\infty - S_n \right| \leq \frac{1}{2^n}}$ . If additionally $\epsilon_n$ is chosen so that it is not ultimately constant (i.e. Atalatan’s position keeps oscillating) then this becomes a strict inequality ${\left| S_\infty - S_n \right| < \frac{1}{2^n}}$ .

In theory, it is possible to reach any destination $x \in {[-1,1]}$ using this approach. Just define $\epsilon_n = 1$ if and only if $S_n \leq x$ (i.e. “move toward $x$ ”). It is easy to prove by recurrence that ${\left| x - S_n \right| \leq \frac{1}{2^n}}$ for all $n$ and so $S_\infty = x$ . However, this assumes we already know $x$ in order to decide the sequence of directions $\epsilon_n$ . Let’s see how to build sequences without knowing the destination.

Avoiding an at most countable subset

Suppose that $X \subseteq \mathbb R$ is at most countable and let ${(x_n)}_{n \in \mathbb N}$ be a sequence of real numbers such that $X = \left\{ x_n : n \in \mathbb N \right\}$ . We want to find a sequence such that $S_\infty \notin X$ .

Assuming additionally that it has infinitely many terms above $1$ and infinitely many terms below $-1$ , then we obtain a non-ultimately-constant sequence by defining $\epsilon_n = 1$ if and only if $S_n \geq x_n$ (i.e. “move away from $x$ ”). This is because $-1 < S_n < 1$ so there are always infinitely many terms in front of and behind Atalanta.

Incidentally, $X = \mathbb N$ with trivial enumeration $x_n = n$ shows a simple counter-example with no term below $-1$ and ultimately constant $\epsilon_n$ . In that case, $S_0 = 0$ and $S_{1} = \frac{1}{2}$ and $S_{n+1} = S_{n} - \frac{1}{2^{n+1}}$ for $n \geq 1$ . But $S_\infty = 0 \in \mathbb N$ .

To workaround that issue, we can alternatively define the sequence $\epsilon_{2n} = 1$ if and only if $S_n \geq x$ and $\epsilon_{2n+1} = -\epsilon_{2n}$ . This still moves away from all $x_n$ once, keeps oscillating and does not require additional assumption on $X$ .

Whatever the decision chosen, we can show that $S_{\infty} \notin X$ . Otherwise, consider one $N$ such that $\epsilon_{N}$ is defined by moving away from $S_{\infty}$ . Then either ${S_N - S_\infty} \geq 0$ and $S_{N+1} = {S_N + \frac{1}{2^{N+1}}} \geq {S_\infty + \frac{1}{2^{N+1}}}$ or $S_{N+1} = {S_N - \frac{1}{2^{N+1}}} < {S_\infty - \frac{1}{2^{N+1}}}$ . But in any case, we would have ${\left| S_\infty - S_{N+1} \right| \geq \frac{1}{2^{N+1}}}$ which contradicts our strict inequality for not ultimately constant $\epsilon_n$ .

As explained in the previous blog, this shows that $\mathbb R$ (or any complete subspace containing $\mathbb Q$ ) is uncountable. Otherwise, we could use a sequence enumerating it to arrive to a contradiction.

Similarly, because $\mathbb Q$ is countable then we can find a sequence such that $\left\{ x_n : n \in \mathbb N \right\} = \mathbb Q$ , which gives us an irrational number $S_\infty$ . If instead $X$ is the set of algebraic numbers (which is countable and contains $\mathbb Q$ ) then we obtain a transcendental $S_\infty$ .

An explicit computation

The big difference between the case of $\mathbb Q$ and the set of algebraic numbers is that it’s much easier to actually compute a list of ${(x_n)}_{n \in \mathbb N}$ that lists all the $\mathbb Q$ . For example, a simple way to enumerate the fractions $\frac{p}{q}$ without trying to avoid duplicate values is, for each $M=0, 1, 2, ...$ to enumerate the integers $p$ and $q \neq 0$ bounded by $M$ in the lexicographical order. One can similarly enumerate for each $M$ the non-constant polynomial of degree at most $M$ and integer coefficient bounded by $M$ but it’s not obvious how to actually compute their roots.

One can rely on root finding algorithm to estimate the roots $\alpha$ in order to be able to calculate the sign $S_n - \alpha$ and so determine $\epsilon_n$ . But in theory, $\alpha$ can be arbitrary close to $S_n$ so it’s not clear how much precision to request.

Instead, let’s consider ${(P_n)}_{n \in \mathbb N}$ an enumeration of the non-constant polynomial with integer cofficients and define $\epsilon_n = 1$ if and only if ${(P_n\prime P_n)}{(S_n)} \geq 0$ . With that new formula, $\epsilon_n$ can be calculated very easily in $O(\deg P_n)$ from the cofficients of $P_n$ using Horner’s method.

For any integer $M \geq 1$ , let’s consider $Q_M^\pm = {(X \pm 2)}^M$ . Then its product with its derivative is $M {(X \pm 2)}^{2M -1}$ . Its value at any $x$ has the same sign as $x \pm 2$ and if $\left|x\right| < 1$ it just $\pm$ . For any $n$ , we can find $M$ large enough such that $Q_M^+$ and $Q_M^-$ have not been enumerated yet. So $\epsilon_n$ is not ultimately constant. Alternatively, we could have used the trick from the previous section to force that property.

Now let’s prove that $S_\infty$ is still algebraic. If that’s not the case, let $n$ such that ${P_n(S_\infty)} = 0$ . We can choose $n$ such that $P_n$ is of minimal degree and so ${P_n\prime(S_\infty)} \neq 0$ . Since $P_n$ has only finitely many roots, there is $k \geq n$ such that at maximum distance $\frac{1}{2^k}$ around $S_\infty$ , $S_\infty$ is the only root of $P_n$ and $P_n\prime$ does not vanish. Let’s $M$ large enough such that the polynomial $R_M = P_n^{2M+1}$ has not been enumerated before $P_k$ . Let’s $l$ such that $P_l = R_M$ .

In the considered neighbourhood of $S_\infty$ , $P_l$ has only one root $S_\infty$ , has the same sign as $P_n$ and its derivative ${(2M+1)}P_n^{2M} P_n\prime$ has the same nonzero constant sign as $P_n\prime$ . Analyzing each combination $P_l\prime$ positive/negative and $S_l$ greater/smaller than $S_\infty$ , we verify that $\epsilon_l$ is chosen such that ${\left| S_\infty - S_{l+1} \right| \geq \frac{1}{2^{l+1}}}$ which is obviously also true if $S_l = S_\infty$ . This agains contradict our strict inequality. Q.E.D.

A Zeno paradox to prove the reals are uncountable

2020-04-03T00:00:00+02:00

A simple Zeno’s paradox

A simple variant of Zeno’s paradoxes can be described as follows. Atalanta wishes to walk to the end of a path. When she gets halfway, she still have to walk the remaining half of the path ; When she gets halfway of that remaining half, she still needs to walk the remaining quarter of the path ; When she gets halfway of that remaining quarter, she still needs to walk the remaining eight of the path ; and so on. Hence it seems she will never reach the end of the path, which is paradoxical.

Obviously, this is not true: Atalanta is going to reach the end of the path after a certain amount of time. The issue in the previous reasoning is that (assuming she walks at constant velocity) walking these subpaths also takes her respectively half, a quarter, a eight, etc of the total time she actually needs to arrive to her destination… and the observations are only done for a total time that is less than the one needed.

A modern variant

Now suppose the ground is an infinite real line, itself covered by an infinite treadmill on which Atalanta is walking. This treadmill brings Atalanta in the opposite direction by twice her velocity. She can decide to turn the treadmill on or off, but its state must remain the same during the whole walk of a subpath. In the following schema, the treadmill was enabled when she was on the second subpath:

After the total duration considered in the previous problem, Atalanta has walked exactly the same total distance on the treadmill. The conveyor belt also moves by summing up distances that are twice the distances on subpaths. Because the treadmill can be on or off, the conveyor’s belt position is not following a simple linear function of the time. Let’s admit for a while it still converges to a certain distance when getting closer to the total duration, I will come back to this at the end of the blog post.

With respect to the ground, Atalanta’s position is basically given by her departure position, moved backward by the distance of the conveyor belt with respect to the ground and moved forward by the distance she walked on the treadmill. As a consequence of the previous paragraph, Atalanta is also converging to a destination with respect to the ground.

Modern Zeno’s paradox (with a flaw)

Now consider an enumeration of all the points of the real line. At the beginning the treadmill is off. Before walking the first half of the path, Atalanta picks the first point in that enumeration and enables the treadmill if she can see that point in front of her. Before walking the next quarter of the path, she picks the second point and enables or disables the treadmill according to whether she can see that second point in front of her. She continues that way for each subpath and each point of the real line.

When a given point is picked there are two options. Either it is not in front of her and so she will just walk forward to a certain distance ; or otherwise the the treamill will additionnaly take her backward by twice that distance. In any case, with respect to the ground, she is moving away from the picked point by the distance she walks during that subpath.

At some step, the point of the real line corresponding to Atalanta’s destination will be picked and she will move away from that destination by a certain distance. But the remaining subpaths are half, a quarter, a eight, etc that distance. To have a chance to converge to the destination again she must now always keep the same direction. But so far, only finitely many points have been picked and so there is at least one point beyond the destination that has not been chosen yet (actually infinitely many). This means she will have to change her direction at least once in the future and so will never be able to reach her destination!

Uncountability of the real line

How to solve the paradox in this modern variant?

First, let’s go back to why the distance of the conveyor belt with respect to the ground is convergent. Consider the binary number whose digits describe the sequences of states of the treadmill: 0 if turned off and 1 if turned on. Place the binary point after the first digit to obtain a real number. Then one can check that the distance of the conveyor belt is given by that real number multiplied by Atalanta’s total distance on the treadmill. Of course, one must admit that such a binary number with infinitely many digits after the binary point is well defined!

For a rigorous proof, note that the conveyor belt is always going in the same direction and at most twice Atalanta’s total walking distance on the treadmill. So the convergence of the conveyor belt just comes from the monotone convergence theorem. This assumption was correct.

However, reading more carefully the argumentation, we actually assumed that there is a countable enumeration of all the points of the real line. But Cantor proved at the end of the 19th century that this is wrong, the real line is uncountable!

It is interesting to actually turn this modern Zeno’s paradox into an apparently “elementary” alternative of Cantor’s diagonal argument, in a way that involves only concepts understandable by the Ancient Greeks and no advanced mathematical formalisms. In particular, this is not mentioning explicitly the positional notation invented by Indian mathematicians. As a comparison, Cantor’s proof assumes that a decimal notation with infinitely many digits after the decimal point is well defined (similar to what we used for our proof with binary numbers) ; or it must deal with the edge case of a real number with two different decimal notations (e.g. 0.4999999… = 0.5).

For completeness

Nevertheless, we still had to use some kind of completeness property in order to prove that the moving distance of the conveyor belt is convergent. Actually, Cauchy completeness is enough.

It is also interesting to note that all the moves as well as Atalanta’s total distance on the treadmill are rational numbers. Since the rational numbers are countable, this modern Zeno’s paradox can also be performed by considering only rational points on the ground. In that case, the flaw is then in the assumption that the conveyor belt converges to a rational point: the set of rational numbers is not complete.

More generally, we can also applie this modern Zeno’s paradox to any ordered field since they include a subset isomorphic to the rational numbers. The conclusion becomes that if the field is Cauchy-complete then it is uncountable.

S[α] for strings of ordinals

2020-03-08T00:00:00+01:00

Update 2020/03/10: Added complexity analysis for S.substr(α, N)

In a previous blog post, I defined for each ordinal $\alpha$ a string $S_\alpha$ (made of the characters for the empty set, comma, opening brace and closing brace) that enumerates the element of $\alpha$ . I gave a simple formulas to calculate the length $L_\alpha$ of this string.

My colleague Ioanna was a bit disappointed that I didn’t provide a script for calculating the infinite $S_\alpha$ strings. Obviously, the complexity would be “ $\Omega(\omega)$ ” but it is still possible to evaluate the string at a given position: Given $\beta < L_\alpha$ , what is the character of $S_\alpha$ at position $\beta$ ?

Since the initial segments of the strings are compatible, another way to express this is by introducing the class $S = \bigcup_{\alpha \in {\mathrm{Ord}}} S_\alpha$ corresponding to a giant string enumerating the class $\mathrm{Ord}$ of all ordinals. Given an ordinal $\alpha$ , what is $S[\alpha]$ ?

A small generalization of this S.charAt(α) operation is S.substr(α, N) calculating the substring of length $N$ starting at $\alpha$ .

Example

$S_{2}$ is "∅,{∅}", so $S[0]$ is the character ∅, $S[1]$ is a comma, $S[2]$ is an opening brace and $S[4]$ is a closing brace.
$S_{\omega+1}$ is made of of an $\omega$ -concatenation of finite strings (the character ∅, a comma, $S_1$ surrounded by braces, a comma, $S_2$ surrounded by braces, a comma, $S_4$ surrounded by braces, a comma, etc), followed by a comma, an opening brace, the same $\omega$ -concatenation of finite strings and finally a closing brace. So $S[\omega]$ is a comma, $S[\omega+1]$ is an opening brace, $S[\omega+2]$ is the character "∅" and $S[\omega2]$ is a closing brace.

In the previous example, we have basically analyzed the string $S_{\alpha+1}$ at a given successor ordinal, splitting it into two copies of $S_\alpha$ , comma and braces. This suggests some easy values of $S$ :

Lemma

For any $n < \omega$ and $\beta \geq 1$ , $S[\alpha]$ is:

A comma if $\alpha$ can be written $2^n - 3$ or $\omega^\beta 2^n + n$
An opening brace if $\alpha$ can be written $2^n - 2$ or $\omega^\beta 2^n + n + 1$
A closing brace if $\alpha$ can be written $2^{n+1} - 4$ or $\omega^\beta 2^{n+1} + n$

Proof: For any ordinal $\alpha$ , by viewing the string $S_{\alpha+1}$ as a concatenation of $S_\alpha$ , a comma, an opening fence, $S_\alpha$ and a closing fence, we deduce that:

${S\left[L_{\alpha}\right]}$ is a comma.
${S\left[L_{\alpha} + 1 \right]}$ is an opening brace.
$L_{\alpha+1}$ is a successor ordinal and ${S\left[L_{\alpha+1} - 1 \right]}$ is a closing brace.

The lemma follows immediately from the calculation of string lengths performed in the previous blog post. □

Warning: The rest of the blog post gives the solution to this puzzle, so you might want to have fun solving it yourself first and then go back checking my proposed solution later 😉...

More generally, the proof of the lemma can be extended by saying that if we find $n$ such that ${2^n - 1} \leq \alpha \leq {2^{n+1} - 5}$ then $\delta = {\alpha - {(2^n - 1)}} \leq {2^n - 4}$ is the index of ${S[\alpha]}$ in the second substring $S_\alpha$ of $S_{\alpha+1}$ and so ${S[\alpha]} = {S[\delta]}$ .

Details will be provided in the theorem below but one can already write a simple JavaScript recursive program to evalute $S$ at finite ordinals:

Script for $\alpha < \omega$

The character of

S

at position

\alpha =

The following intermediary step will be helpful to evaluate $S$ at infinite ordinal $\alpha$ :

Proposition

Let $\alpha$ is infinite and $\beta = \log_{\omega}(\alpha) \geq 1$ . Let $1 \leq q < \omega$ and $0 \leq \rho < \omega^\beta$ be the quotient and remainder of the euclidean division of $\alpha$ by $\omega^\beta$ . Let’s define:

$n = \begin{cases} \left\lfloor {\log_2(q)} \right\rfloor & \text{ if } q \text{ is not a power of 2 or this value is } \leq \rho \\ \left\lfloor {\log_2(q)} \right\rfloor - 1 & \text{ otherwise.} \end{cases}$

Then $S[\alpha]$ is:

A comma if $\alpha = {\omega^{\beta} 2^n + n}$
An opening brace if $\alpha = {\omega^{\beta} 2^n + n + 1}$
An closing brace if $\alpha = {\omega^{\beta} 2^{n+1} + n}$
$S[\delta]$ otherwise where $\delta$ is the unique ordinal such that $\alpha = {\omega^\beta 2^n + n + 2 + \delta}$ and $\delta < {\omega^{\beta} 2^n + n}$ .

Proof: First by construction we have $\alpha = { {\omega^\beta q } + \rho}$ .

If the first case of the definition of $n$ , $2^n \leq q < 2^{n+1}$ so we always have $\alpha < {\omega^\beta {(q+1)}} \leq \omega^\beta 2^{n+1} \leq L_{\omega{\beta} + n + 1}$ . If additionnaly $q$ is not a power then $2^n < q$ and so $\alpha \geq {\omega^{\beta}q} \geq {\omega^{\beta}{(2^n+1)}} \geq L_{\omega{\beta} + n}$ . Otherwise $q = 2^n$ and $n \leq \rho$ and so again $\alpha \geq L_{\omega{\beta} + n}$ .

In the second case of the definition of $n$ , $\left\lfloor {\log_2(q)} \right\rfloor \geq \rho + 1$ so $n \geq \rho \geq 0$ . $q$ is a power of 2 and more precisely $q = 2^{n+1} > 2^n$ so we deduce the same way as in the previous case that $\alpha \geq L_{\omega{\beta} + n}$ . Moreover, $\rho < n + 1$ so again $\alpha = {\omega^\beta 2^{n+1} + \rho} < L_{\omega{\beta} + n + 1}$ .

We can thus view $S_{\omega \beta + n + 1}$ as a concatenation of $S_{\omega \beta+n}$ , a comma, an opening brace, $S_{\omega \beta+n}$ and a closing brace. We assume that ${ {\omega^{\beta} {2^n}} + n + 2} \leq \alpha < {\omega^{\beta} 2^{n+1} + n}$ as the three other cases are handled by the lemma. Then $\delta$ is well-defined and is actually the index of $S[\alpha]$ in the second copy of $S_{\omega \beta + n}$ so ${S[\alpha]} = S{[\delta]}$ . □

We are now ready to give a nice way to evaluate $S$ at any ordinal $\alpha$ :

Theorem

$S[\alpha]$ can be calculated inductively as follows:

If $\alpha = 0$ , $S[\alpha]$ is the character "∅".
If 0<α≤ω0 < \alpha \leq \omega then 2⌊log2(α+3)⌋−3≤α≤2⌊log2(α+3)⌋+1−4 \right\rfloor}} - 3} \leq \alpha \leq \right\rfloor + 1}} - 4} and S[α]S[\alpha] is equal to:
- A comma if $\alpha = {2^{\left\lfloor {\log_2{(\alpha+3)}} \right\rfloor} - 3}$
- An opening brace if $\alpha = {2^{\left\lfloor {\log_2{(\alpha+3)}} \right\rfloor} - 2}$
- A closing brace if $\alpha = {2^{\left\lfloor {\log_2{(\alpha+3)}} \right\rfloor + 1}} - 4$
- $S[\delta]$ otherwise where $\delta = {\alpha - \left( 2^{\left\lfloor {\log_2{(\alpha+3)}} \right\rfloor} - 1 \right)} \leq {2^{\left\lfloor {\log_2{(\alpha+3)}} \right\rfloor} - 4} \leq \alpha - 3$ .
  Moreover $\delta$ compares against $\alpha$ as follows: $\frac{\delta}{\alpha} \leq \frac{1}{2} + \frac{1}{2^{\left\lfloor {\log_2{(\alpha+3)}} \right\rfloor + 1}} \leq \frac{3}{4}$ and ${\left\lfloor {\log_2{(\delta+3)}} \right\rfloor} < {\left\lfloor {\log_2{(\alpha+3)}} \right\rfloor}$
If α\alpha is infinite, let 0<c1<ω0 < c_1 < \omega be the coefficient in its Cantor normal form corresponding to the smallest infinite term and c0<ωc_0 < \omega be finite term if there is one, or zero otherwise. Let kk be the exponent corresponding to the smallest nonzero term in the binary decomposition of c1c_1. Then S[α]S[\alpha] is equal to:
- A closing brace if $c_0 \leq k - 1$ .
- A comma if $c_0 = k$ .
- An opening brace if $c_0 = k + 1$ .
- $S[{c_0 - {(k+2)}}]$ if $c_0 \geq k + 2$ .

Proof:

The case $\alpha = 0$ is clear. Suppose $0 < \alpha < \omega$ and let $l = {\left\lfloor {\log_2{(\alpha+3)}} \right\rfloor}$ . By definition $2^l \leq \alpha + 3 < 2^{l+1}$ so $2^l - 3 \leq \alpha \leq 2^{l+1} - 4$ . Let’s consider the case $2^l - 1 \leq \alpha \leq 2^{l+1} - 5$ as the three other cases are already known from the Lemma. $S_{l+1}$ is made of the concatenation of $S_l$ , a comma and $S_l$ surrounded by braces. $\delta$ is actually the index of $S[\alpha]$ in the second copy of $S_l$ so ${S[\alpha]} = S{[\delta]}$ . The first equality is straighforward:

$\delta = {\alpha - {(2^l -1)}} \leq {2^{l+1} - 5 - {(2^l -1)}} = {2^l - 4} = {2^l - 1 - 3}\leq \alpha - 3$

Morever we have:

$\frac{\delta}{\alpha} \leq {1 - \frac{2^l - 1}{2^{l+1}+5}} \leq {1 - \frac{2^l - 1}{2^{l+1}}} \leq {\frac{1}{2} + \frac{1}{2^{l+1}}}$

and so the second inequality follows from the fact that $l \geq 1$ . Finally, the third inequality comes from:

$2^{\left\lfloor {\log_2{(\delta+3)}} \right\rfloor} \leq {\delta + 3} \leq {2^l - 4 + 3} < 2^l$

Let’s consider the case of an infinite $\alpha$ . For some $N \geq 1$ , $\beta_N > \beta_{N-1} > \dots > \beta_1 \geq 1$ and $c_0 < \omega$ and $0 < c_1, c_2, c_N < \omega$ , we can write Cantor’s Normal form as:

$\alpha = { \omega^{\beta_N} c_N} + { \omega^{\beta_{N-1}} c_{N-1} } + \dots + { \omega^{\beta_{1}} c_{1} } + c_0$

With the notation of the proposition, we have $\beta = \beta_N$ , $q = c_N$ and

$\rho = { \omega^{\beta_{N-1}} c_{N-1} } + \dots + { \omega^{\beta_{1}} c_{1} } + c_0$

If $N \geq 2$ , then $\rho$ is infinite and so $n = \left\lfloor {\log_2(q)} \right\rfloor$ and we are in the fouth bullet of the proposition. Moreover $q \geq 2^n$ and we can write:

$\alpha = { \omega^{\beta} 2^n} + {\omega^{\beta} \left(q-2^n\right)} + \rho = { \omega^{\beta} 2^n} + n + 2 + \delta$

where $\delta = {\omega^{\beta} \left(q-2^n\right)} + \rho$ is infinite and so cancels out the $n + 2$ term. It follows that $S{[\alpha]} = S\left[ {\omega^{\beta} \left(q-2^n\right)} + \rho \right]$ . Essentially, we have just removed from $c_{N}$ its term of highest exponent in its binary decomposition!

By repeated application of the theorem, we can remove each binary digit of the $c_i$ for $i$ going from $N$ to $2$ . When then arrive at $i = 1$ :

${S[\alpha]} = S\left[ \omega^{\beta_1} c_1 + c_0\right]$

With the notation of the proposition, we now have $\beta = \beta_1$ , $q = c_1$ and $\rho = c_0$ . If the binary decomposition of $c_1$ has more than one nonzero digit then so $q$ is not a power of 2. So although $\rho$ is now finite, we are still in the first case of the proposition and $\delta$ remains infinite. So we can remove all but the last digit of $c_1$ by repeated application of the proposition:

${S[\alpha]} = S\left[ \omega^{\beta_1} 2^k + c_0\right]$

where $k$ is the exponent corresponding to the smallest nonzero term in the binary decomposition of $c_1$ .

Using the lemma, $S[\alpha]$ is a comma if $k = c_0$ , an opening brace if $k = c_0 - 1$ and a closing brace if $k = c_0 + 1$ .

If $k \leq c_0 - 2$ then $k = \left\lfloor {\log_2(2^k)} \right\rfloor \leq c_0$ and writing

${\omega^{\beta_1} 2^k + c_0} = \omega^{\beta_1} 2^k + k + 2 + \left(c_0 - {(k + 2)}\right)$

we deduce from the proposition that ${S[\alpha]} = {S[{c_0 - {(k+2)}}]}$ .

Finally, if $k \geq c_0 + 2$ then $k = \left\lfloor {\log_2(2^k)} \right\rfloor > c_0$ and writing

${\omega^{\beta_1} 2^k + c_0} = {\omega^{\beta_1} 2^{k-1}} + k - 1 + 2 + {\omega^{\beta_1} 2^{k-1}} + c_0$

we deduce from the proposition that

${S[\alpha]} = S\left[ \omega^{\beta_1} 2^{k-1} + c_0\right]$

We have essentially decremented $k$ and we can repeat this until we reach the case $k = c_0 + 1$ for which we already said that the character is a closing brace. □

As an application of this theorem, here is a few simple exercises:

Exercise 1

$S\left[2^72 + 2\right]$ is an empty set.
$S\left[\omega^72\right]$ is a comma.
$S\left[\omega^72 72 + 72\right]$ is a closing brace.
$S\left[\omega^72 72 + \omega^42 42 + 12\right]$ is an opening brace.

Exercise 2

The evaluation of $S$ at

Limit ordinals is either a comma or a closing brace.
Epsilon numbers is a comma.
Indecomposable ordinals is a comma.

Corollary 1: Time complexity

Let $\alpha$ be an ordinal. Let $c_1 < \omega$ be the coefficient in its Cantor normal form corresponding to the smallest infinite term if there is one, or zero otherwise. Let $c_0 < \omega$ be its finite term if there is one, or zero otherwise. Then $S[\alpha]$ can be evaluated in:

$O\left( \log_2{(c_0+2)}^2 + \log_2{(c_1+2)} \right)$ elementary arithmetic operations and comparisons on integers.
$O\left( \log_2{(c_0+2)}\right)$ elementary operations if integers are represented in binary and leading/trailing zero counting and bit shifts are elementary operations.

Proof: First note that the “+ 2” is just to workaround for the edge cases $c_0 = 0$ or $c_1 = 0$ .

For the infinite case $c_1 > 0$ we need to calculate $k$ that is performing the find first set. A naive implementation can be done in $O(\log_2{(c_1+2)})$ steps by browsing the digits of $c_1$ to find the first nonzero for example by calculating the remainder modulo increasing power of 2. Each iteration requires only $O(1)$ elementary integer operations %, * and comparisons. Then returning the result of moving to the finite case requires $O(1)$ integer operations +, − and comparisons.

For the finite case $c_1 = 0$ and $\alpha = c_0$ , first notice that we only require $O\left( \log_2{(c_0+2)}\right)$ recursive calls given the inequality:

${\left\lfloor {\log_2{(\delta+3)}} \right\rfloor} < {\left\lfloor {\log_2{(\alpha+3)}} \right\rfloor}$

The case $\alpha = 0$ only requires one comparison. For the case $\alpha > 0$ , we need to calculate $l = {\left\lfloor {\log_2{(\alpha+3)}} \right\rfloor}$ which is integer rounding of the binary logarithm or even just $2^l$ . As above, we can provide a naive implementation by calculating the quotient modulo increasing power of 2 in $O(l)$ comparisons and elementary integer operations /, *. Then returning the result of moving to a $\delta < \alpha$ requires only $O(1)$ integer operations and comparisons. In total, complexity is $O\left( \log_2{(c_0+2)}^2\right)$ .

Finally, this can be simplified if one calculates $k$ and $l$ by a simple leading/trailing zero counting (or similar) and $2^l$ by a bit shift. □

Script based on Cantor normal form

If the coefficients of Cantor normal form of

\alpha

corresponding the smallest infinite term and finite term are

c_1

c_0

=
Then the character of

S

at position

\alpha

Once we have an algorithm for S.charAt(α), it is easy to get an algorithm of $N$ times that complexity for S.substr(α, N) calculating the substring of length $N$ starting at position $\alpha$ by repeated calls to S.charAt(α). Let’s analyze a bit more carefully how we can make this recursive and more efficient:

Corollary 2: Algorithm for S.substr(α, N)

Let $\alpha$ be an ordinal and $N < \omega$ . Then S.substr(α, N) can be calculated as follows:

If $N = 0$ then it is the empty string.
If $\alpha = 0$ and $N = 1$ then it is the character "∅".
If 0<α<ω0 < \alpha < \omega, let l=⌊log2(α+N+2)⌋l = \left\lfloor {\log_2{(\alpha+N+2)}} \right\rfloor. We have 2l−3≤α+N−1≤2l+1−42^l-3 \leq {\alpha + N - 1} \leq 2^{l+1} - 4 and the result is obtained by concatenating the following strings:
1. If $\alpha \leq 2^l - 4$ , the substring at offset $\alpha$ and length $2^l - 3 - \alpha$ .
2. If $\alpha \leq {2^l - 3}$ , a comma.
3. If $\alpha \leq {2^l - 2} \leq {\alpha + N - 1}$ , an opening brace.
4. If ${\alpha + N - 1} \geq {2^l - 1}$ , the substring at offset $\delta = \mathrm{max}{(0, \alpha - {(2^l - 1)})}$ and length $1 + \mathrm{min}{({\alpha + N - 1}, {2^{l+1} - 5})} - \mathrm{max}{(\alpha, {(2^l - 1)})}$ .
5. If ${\alpha + N - 1} = {2^{l+1} - 4}$ , a closing brace.
If α\alpha is infinite, let 0<c1<ω0 < c_1 < \omega be the coefficient in its Cantor normal form corresponding to the smallest infinite term and c0<ωc_0 < \omega be finite term if there is one, or zero otherwise. Let kk be the exponent corresponding to the smallest nonzero term in the binary decomposition of c1c_1. Then the result is obtained by a concatenating the following strings:
1. If $c_0 \leq k - 1$ , ${\mathrm{min}{(N, k - c_0)}}$ closing braces.
2. If $c_0 \leq k \leq c_0 + N - 1$ , a comma.
3. If $c_0 \leq k + 1 \leq c_0 + N - 1$ , an opening brace.
4. If $c_0 + N - 1 \geq k + 2$ , the substring at offset $\delta = \mathrm{max}{({0, {c_0 - {(k + 2)}}})}$ and length $c_0 + N - {\mathrm{max}{(c_0, k + 2)}}$ .

Moreover, this only adds $O(N)$ compared to the complexity of evaluating to a single offset.

Proof: The algorithm is just direct application of the Theorem. For the case where $\alpha \geq \omega$ , the only change is that we add at most $N$ characters before moving to the finite case. The case $\alpha < \omega$ is essentially a divide-and-conquer algorithm and we have a relation of the form:

{T(L)} = 2{T\left(\frac{L}{2}\right)} + {f(L)}

where $L = 2^l = {\Theta{(\alpha + N)}}$ and $f{(L)}$ is $O(1)$ or $O(\log_2{L})$ depending on available operations, but in any case $O{(\sqrt{L})}$ . So from the master theorem, ${T{(L)}} = {O(L)} = {O(\alpha+N)}$ . In general, this bound is not as good as repeating $N$ calls to S.charAt!

However, we note that if we assume that $\alpha > N - 4$ then ${\alpha + 2 + N} < {2{(\alpha+3)}}$ and so

$\log_2\left(\alpha + 2 + N\right) < {1 + \log_2{(\alpha+3)}}$

We can easily discard the edge case where these start/end offsets point to a brace surrounding the right substring of the iterative step and so we get:

$\left\lfloor {\log_2{(\alpha+N+2)}} \right\rfloor = \left\lfloor {\log_2{(\alpha+3)}} \right\rfloor$

which means that the left substring is just empty and so the complexity is not changed compared to S.charAt. Finally, when $\alpha \leq N - 4$ the previous bound tells us that the steps are done in $O(N)$ . □

Script for S.substr(α, N)

If the coefficients of Cantor normal form of

\alpha

corresponding the smallest infinite term and finite term are

c_1

c_0

=
Then the character of

S

substring of length

N

=
and offset

\alpha

Ordinals as strings of ∅, commas and braces

2020-03-05T00:00:00+01:00

This week, my colleagues at Igalia were talking about cutting text at a 72-characters limit for word-wrapping emails. One might say that the real question is whether 72 is a cut or a limit. Or wonder how one has decided to set that particular value?

The latter link explains the recursive definition of 72 as a finite set, which can be written with only braces, commas and the empty set $\empty$ :

$0 = {\{\}} = \empty$ $1 = 0 \union {\{0\}} = {\{0\}} = {\{\empty\}}$ $2 = 1 \union {\{1\}} = {\{0,1\}} = {\{ \empty, {\{\empty\}} \}}$ $3 = 2 \union {\{2\}} = {\{0,1,2\}} = {\{ \empty, {\{\empty\}}, {\{ \empty, {\{\empty\}} \}} \}}$

and so forth until

$72 = 71 \union {\{71\}} = {\{0,1,2,\dots,71\}} = \dots$

You can notice that $1$ has only one element $∅$ and for any $n \geq 1$ , in order to enumerate the elements of $n+1$ using only the four characters "∅", ",", "{" and "}", you first write the elements of $n$ , followed by a comma, followed by an opening brace, followed by the elements of $n$ and finally followed by a closing brace. One can write a simple JavaScript loop that initializes a variable string = "∅" and performs the concatenation string += `,{${string}}` at each step:

The elements of are

Length of the string

You notice that the length of the string obtained for $n=6$ already exceeds the 72-character limit. From the previous recursive construction of the string one can deduce a recursive definition of its length:

$L_1 = 1$ $L_{n+1} = L_n + 2 + L_n + 1 = 2L_n +3$

It is easy to see that $L_n = {\Omega{(2^n)}}$ so although the simple JavaScript loop above uses a time complexity of ${O{(n)}}$ string concatenations, the space complexity is at least exponential. That’s why in the definition of 72 above I used dots instead of showing the whole string!

One can however easily modify the previous JavaScript loop to calculate the length of such a string. At Igalia, we have been working on a new native type called BigInt which is particularly useful to accurately calculate large integers. The case $n = 72$ demonstrates why it is more appropriate than JavaScript Number:

The length of the strings enumerating the elements of has the following length (first field uses Number, second field uses BigInt):

One can do better and try to calculate an explicit formula for $L_n$ for any $n \geq 1$ . Since for all $i \geq 1$ , $L_{i+2} - L_{i+1} = 2\left(L_{i+1} - L_i\right)$ the sequence $\left(L_{i+1} - L_i\right)_{i \geq 1}$ is a geometric sequence with common ratio 2 and for all $i \geq 1$ we can write

${L_{i+1} - L_{i}} = {2^{i-1} \left( L_2 - L_1 \right)}$

Summing this up for $1 \leq i \leq n - 1$ :

$L_n - L_1 = {\sum_{i=1}^{n-1} L_{i+1} - L_{i}} = {\left( L_2 - L_1 \right) \left(\sum_{i=0}^{n-2} 2^i \right)}= { {\left( L_2 - L_1 \right)} {\left(2^{n-1}-1\right)} }$

Finally, given that $L_1 = 1$ and $L_2 = 2L_1 + 3 = 5$ , one can write for all $n \geq 1$ :

$L_n = (2^{n-1}-1) (5-1) + 1 = 2^{n+1} - 3$

This formula can be used to calculate $L_n$ using built-in JavaScript exponentation instead of a loop, as done below.

The length of the strings enumerating the elements of has the following length (first field uses Number, second field uses BigInt):

Order-type of infinite strings

It is possible to generalize this problem to infinite ordinal numbers. The string $S_{\alpha}$ and its length $L_\alpha$ are defined by induction for any ordinal $\alpha$ . $S_0$ is the empty string $L_0 = 0$ . $S_1$ is the string with one character "∅" and $L_1 = 1$ . For $\alpha \geq 1$ , $S_{\alpha+1}$ is the string obtained by concatenating the string $S_\alpha$ , a comma character ",", an opening brace character "{", the string $S_\alpha$ and finally a closing brace character "}". It is of length $L_{\alpha+1} = L_\alpha + 2 + L_\alpha + 1$ . Beware that ordinal operations are not commutative so this cannot be simplified a priori!

For any limit ordinal $\lambda$ , the string $S_\lambda$ is obtained by taking the union of the $L_\alpha$ -sequence $S_\alpha$ for $\alpha < \lambda$ . This is still a well-defined sequence since these strings always agree on the character at a given index. $S_\lambda$ is of length $L_\lambda = \sup_{\alpha < \lambda} L_\alpha$ .

By construction, $L_\alpha$ is an increasing sequence of ordinals and so $L_\alpha \geq \alpha$ . Can we calculate it more explicitly?

Warning: The rest of the blog post gives the solution to this puzzle, so you might want to have fun solving it yourself first and then go back checking my proposed solution later 😉...

Let’s consider some examples. In the case $\omega = \mathbb N$ , the string $S_\omega$ can be more easily seen as the $\omega$ -concatenation of finite strings: the empty set, a comma, $S_1$ surrounded by braces, a comma, $S_2$ surrounded by braces, a comma, $S_3$ surrounded by braces, a comma, etc So clearly $L_{\omega} = \omega$ and this aligns with the definition for limit ordinal.

In order to write $S_{\omega+1}$ , you first write the elements of $\omega$ , followed by a comma, followed by an opening brace, followed by the elements of $\omega$ and finally followed by a closing brace. So really this is an $\omega + \omega + 1$ sequence. This aligns with the definition in the successor ordinal case, using the fact that $k + \omega = \omega$ for any $k < \omega$ :

$L_{\omega+1} = L_\omega + 2 + L_\omega + 1 = \omega + 2 + \omega + 1 = \omega2 + 1$

Continuing for other successor ordinals, one can write

$L_{\omega+2} = L_{\omega+1} + 2 + L_{\omega+1} + 1 = \omega2 + 1 + 2 + \omega2 + 1 + 1 = \omega4 + 2$ $L_{\omega+3} = L_{\omega+2} + 2 + L_{\omega+2} + 1 = \omega4 + 1 + 2 + \omega4 + 2 + 1 = \omega8 + 3$

and more generally for all $n < \omega$ :

$L_{\omega+n} = \omega{2^n} + n$

To calculate $S_{\omega2}$ , one takes the union for $n < \omega$ of the $L_{\omega+n}$ -sequence of characters describing $\omega+n$ . Its length is

$L_{\omega2} = {\sup_{n < \omega} \left(\omega{2^n} + n\right)} = \omega^2$

Then the exact same calculation as above leads for all $n < \omega$ to

$L_{\omega2+n} = \omega^2{2^n} + n$

Continuing this kind of calculations for larger values $\omega3, \omega4, \dots, \omega^2$ suggests the following conjecture: For every infinite ordinal $\alpha$ , the order-type of the sequence describing its element is:

$L_{\alpha} = L_{\omega \beta + n} = \omega^\beta 2^n + n$

where $\beta \geq 1$ and $n < \omega$ are the quotient and remainder of the euclidean division of $\alpha$ by $\omega$ .

Let’s sketch the proof by induction on $\beta$ . We already explained the case $\beta = 1$ in the example above. Similarly, for a given $\beta \geq 1$ , it is enough to prove the case $n = 0$ , the formula for $n > 0$ follows, using the property $\forall k < \omega, k + \omega^\beta = \omega^\beta$ and the calculations we gave for the case $\beta = 1$ .

If the formula holds for some $\beta \geq 1$ then because $\omega{(\beta + 1)}$ is limit and the lengths are increasing, the formula holds for $\beta + 1$ :

$L_{\omega {(\beta + 1)}} = L_{\omega\beta + \omega} = {\sup_{n < \omega} L_{\omega \beta + n}} = {\sup_{n < \omega} {(\omega^\beta 2^n + n)} } = \omega^{\beta+1}$

Finally, if the formula holds for all $\beta$ below a limit ordinal $\lambda$ then because the lengths are increasing the formula holds for $\lambda$ too:

$L_{\omega \lambda} = {\sup_{\beta < \lambda} L_{\omega \beta}} = {\sup_{\beta < \lambda} \omega^{\beta}} = \omega^{\lambda}$

Q.E.D.

Sorry, no JavaScript program to calculate these infinite strings 😇...

Review of my year 2019 at Igalia

2019-12-10T00:00:00+01:00

Co-owner and mentorship

In 2016, I was among the new software engineers who joined Igalia. Three years later I applied to become co-owner of the company and the legal paperwork was completed in April. As my colleague Andy explained on his blog, this does not change a lot of things in practice because most of the decisions are taken within the assembly. However, I’m still very happy and proud of having achieved this step 😄

One of my new duty has been to be the “mentor” of Miyoung Shin since February and to help with her integration at Igalia. Shin has been instrumental in Igalia’s project to improve Chromium’s Code Health with an impressive number of ~500 commits. You can watch the video of her BlinkOn lightning talk on YouTube. In addition to her excellent technical contribution she has also invested her energy in company’s life, helping with the organization of social activities during our summits, something which has really been appreciated by her colleagues. I’m really glad that she has recently entered the assembly and will be able to take a more active role in the company’s decisions!

Julie (left) and Shin (right) at BlinkOn 11.

Working with new colleagues

Igalia also hired Cathie Chen last December and I’ve gotten the chance to work with her as part of our Web Platform collaboration with AMP. Cathie has been very productive and we have been able to push forward several features, mostly related to scrolling. Additionally, she implemented ResizeObserver in WebKit, which is a quite exciting new feature for web developers!

Cathie executing traditional Chinese sign language.

The other project I’ve been contributed to this year is MathML. We are introducing new math and font concepts into Chromium’s new layout engine, so the project is both technically challenging and fascinating. For this project, I was able to rely on Rob’s excellent development skills to complete this year’s implementation roadmap on our Chromium branch and start upstreaming patches.

In addition, a lot of extra non-implementation effort has been done which led to consensus between browser implementers, MathML enthusiasts and other web developer groups. Brian Kardell became a developer advocate for Igalia in March and has been very helpful talking to different people, explaining our project and introducing ideas from the Extensible Web, HTML or CSS Communities. I strongly recommend his recent Igalia chat and lightning talk for a good explanation of what we have been doing this year.

Brian presenting his lightning talk at BlinkOn11.

Conferences

These are the developer conferences I attended this year and gave talks about our MathML project:

April (Toronto): BlinkOn 10 - talk
October (A Coruña): Web Engines Hackfest 2019 - talk&video
November (Sunnyvalle): BlinkOn 11 - talk

Me, giving my BlinkOn 11 talk on "MathML Core".

As usual it was nice to meet the web platform and chromium communities during these events and to hear about the great projects happening. I’m also super excited that the assembly decided to try something new for my favorite event. Indeed, next year we will organize the Web Engines Hackfest in May to avoid conflicts with other browser events but more importantly, we will move to a bigger venue so that we can welcome more people. I’m really looking forward to seeing how things go!

Paris - A Coruña by train

Environmental protection is an important topic discussed in our cooperative. This year, I’ve tried to reduce my carbon footprint when traveling to Igalia’s headquarters by using train instead of plane. Obviously the latter is faster but the former is much more confortable and has less security constraints. It is possible to use high-speed train from Paris to Barcelona and then a Trenhotel to avoid a hotel night. For a quantitative comparison, let’s consider this table, based on my personal travel experience:

Traject	Transport	Duration	Price	CO2/passenger
Paris - Barcelona	TGV	~6h30	60-90€	6kg
Barcelona - A Coruña	Trenhotel	~15h	40-60€	Unspecified
Paris - A Coruña (via Madrid)	Iberia	4h30-5h	100-150€	192kg

The price depends on how early the tickets are booked but it is more or less the same for train and plane. Renfe does not seem to indicate estimated CO2 emission and trains to Galicia are probably not the most eco-friendly. However, using this online estimator, the 1200 kilometers between Barcelona and A Coruña would emit 50kg by national train. This would mean a reduction of 71% in the emission of CO2, which is quite significant. Hopefully things will get better when one day AVE is available between Madrid and A Coruña… 😉

Review of Igalia's Web Platform activities (H2 2018)

2019-02-27T00:00:00+01:00

This blog post reviews Igalia’s activity around the Web Platform, focusing on the second semester of 2018.

Projects

MathML

During 2018 we have continued discussions to implement MathML in Chromium with Google and people interested in math layout. The project was finally launched early this year and we have encouraging progress. Stay tuned for more details!

Javascript

As mentioned in the previous report, Igalia has proposed and developed the specification for BigInt, enabling math on arbitrary-sized integers in JavaScript. We’ve continued to land patches for BigInt support in SpiderMonkey and JSC. For the latter, you can watch this video demonstrating the current support. Currently, these two support are under a preference flag but we hope to make it enable by default after we are done polishing the implementations. We also added support for BigInt to several Node.js APIs (e.g. fs.Stat or process.hrtime.bigint).

Regarding “object-oriented” features, we submitted patches private and public instance fields support to JSC and they are pending review. At the same time, we are working on private methods for V8

We contributed other nice features to V8 such as a spec change for template strings and iterator protocol, support for Object.fromEntries, Symbol.prototype.description, miscellaneous optimizations.

At TC39, we maintained or developed many proposals (BigInt, class fields, private methods, decorators, …) and led the ECMAScript Internationalization effort. Additionally, at the WebAssembly Working Group we edited the WebAssembly JS and Web API and early version of WebAssembly/ES Module integration specifications.

Last but not least, we contributed various conformance tests to test262 and Web Platform Tests to ensure interoperability between the various features mentioned above (BigInt, Class fields, Private methods…). In Node.js, we worked on the new Web Platform Tests driver with update automation and continued porting and fixing more Web Platform Tests in Node.js core.

We also worked on the new Web Platform Tests driver with update automation, and continued porting and fixing more Web Platform Tests in Node.js core. Outside of core, we implemented the initial JavaScript API for llnode, a Node.js/V8 plugin for the LLDB debugger.

Accessibility

Igalia has continued its involvement at the W3C. We have achieved the following:

The ARIA Working Group Charter has been renewed
AccName 1.1, Graphics ARIA 1.0 and Graphics AAM 1.0 became recommendations.
A new version of the 1.2 suite (ARIA, Core-AAM, AccName and the Authoring Practices) is available. In particular, we published a first working draft for ARIA 1.2 which will focus on role parity with HTML 5 elements. A working draft for Core-AAM 1.2 is also available.

We are also collaborating with Google to implement ATK support in Chromium. This work will make it possible for users of the Orca screen reader to use Chrome/Chromium as their browser. During H2 we began implementing the foundational accessibility support. During H1 2019 we will continue this work. It is our hope that sufficient progress will be made during H2 2019 for users to begin using Chrome with Orca.

Web Platform Predictability

On Web Platform Predictability, we’ve continued our collaboration with AMP to do bug fixes and implement new features in WebKit. You can read a review of the work done in 2018 on the AMP blog post.

We have worked on a lot of interoperability issues related to editing and selection thanks to financial support from Bloomberg. For example when deleting the last cell of a table some browsers keep an empty table while others delete the whole table. The latter can be problematic, for example if users press backspace continuously to delete a long line, they can accidentally end up deleting the whole table. This was fixed in Chromium and WebKit.

Another issue is that style is lost when transforming some text into list items. When running execCommand() with insertOrderedList/insertUnorderedList on some styled paragraph, the new list item loses the original text’s style. This behavior is not interoperable and we have proposed a fix so that Firefox, Edge, Safari and Chrome behave the same for this operation. We landed a patch for Chromium. After discussion with Apple, it was decided not to implement this change in Safari as it would break some iOS rich text editor apps, mismatching the required platform behavior.

We have also been working on CSS Grid interoperability. We imported Web Platform Tests into WebKit (cf bugs 191515 and 191369 and at the same time completing the missing features and bug fixes so that browsers using WebKit are interoperable, passing 100% of the Grid test suite. For details, see 191358, 189582, 189698, 191881, 191938, 170175, 191473 and 191963. Last but not least, we are exporting more than 100 internal browser tests to the Web Platform test suite.

CSS

Bloomberg is supporting our work to develop new CSS features. One of the new exciting features we’ve been working on is CSS Containment. The goal is to improve the rendering performance of web pages by isolating a subtree from the rest of the document. You can read details on Manuel Rego’s blog post.

Regarding CSS Grid Layout we’ve continued our maintenance duties, bug triage of the Chromium and WebKit bug trackers, and fixed the most severe bugs. One change with impact on end users was related to how percentages row tracks and gaps work in grid containers with indefinite size, the last spec resolution was implemented in both Chromium and WebKit. We are finishing the level 1 of the specification with some missing/incomplete features. First we’ve been working on the new Baseline Alignment algorithm (cf. CSS WG issues 1039, 1365 and 1409). We fixed related issues in Chromium and WebKit. Similarly, we’ve worked on Content Alignment logic (see CSS WG issue 2557) and resolved a bug in Chromium. The new algorithm for baseline alignment caused an important performance regression for certain resizing use cases so we’ve fixed them with some performance optimization and that landed in Chromium.

We have also worked on various topics related to CSS Text 3. We’ve fixed several bugs to increase the pass rate for the Web Platform test suite in Chromium such as bugs 854624, 900727 and 768363. We are also working on a new CSS value ‘break-spaces’ for the ‘white-space’ property. For details, see the CSS WG discussions: issue 2465 and pull request. We implemented this new property in Chromium under a CSSText3BreakSpaces flag. Additionally, we are currently porting this implementation to Chromium’s new layout engine ‘LayoutNG’. We have plans to implement this feature in WebKit during the second semester.

Multimedia

WebRTC: The libwebrtc branch is now upstreamed in WebKit and has been tested with popular servers.
Media Source Extensions: WebM MSE support is upstreamed in WebKit.
We implemented basic support for <video> and <audio> elements in Servo.

Other activities

Web Engines Hackfest 2018

Last October, we organized the Web Engines Hackfest at our A Coruña office. It was a great event with about 70 attendees from all the web engines, thank you to all the participants! As usual, you can find more information on the event wiki including link to slides and videos of speakers.

TPAC 2018

Again in October, but this time in Lyon (France), 12 people from Igalia attended TPAC and participated in several discussions on the different meetings. Igalia had a booth there showcasing several demos of our last developments running on top of WPE (a WebKit port for embedded devices). Last, Manuel Rego gave a talk on the W3C Developers Meetup about how to contribute to CSS.

This.Javascript: State of Browsers

In December, we also participated with other browser developers to the online This.Javascript: State of Browsers event organized by ThisDot. We talked more specifically about the current work in WebKit.

New Igalians

We are excited to announce that new Igalians are joining us to continue our Web platform effort:

Cathie Chen, a Chinese engineer with about 10 years of experience working on browsers. Among other contributions to Chromium, she worked on the new LayoutNG code and added support for list markers.
Caio Lima a Brazilian developer who recently graduated from the Federal University of Bahia. He participated to our coding experience program and notably worked on BigInt support in JSC.
Oriol Brufau a recent graduate in math from Barcelona who is also involved in the CSSWG and the development of various browser engines. He participated to our coding experience program and implemented the CSS Logical Properties and Values in WebKit and Chromium.

Coding Experience Programs

Last fall, Sven Sauleau joined our coding experience program and started to work on various BigInt/WebAssembly improvements in V8.

Conclusion

We are thrilled with the web platform achievements we made last semester and we look forward to more work on the web platform in 2019!

Review of Igalia's Web Platform activities (H1 2018)

2018-07-10T00:00:00+02:00

This is the semiyearly report to let people know a bit more about Igalia’s activities around the Web Platform, focusing on the activity of the first semester of year 2018.

Projects

Javascript

Igalia has proposed and developed the specification for BigInt, enabling math on arbitrary-sized integers in JavaScript. Igalia has been developing implementations in SpiderMonkey and JSC, where core patches have landed. Chrome and Node.js shipped implementations of BigInt, and the proposal is at Stage 3 in TC39.

Igalia is also continuing to develop several features for JavaScript classes, including class fields. We developed a prototype implementation of class fields in JSC. We have maintained Stage 3 in TC39 for our specification of class features, including static variants.

We also participated to WebAssembly (now at First Public Working Draft) and internationalization features for new features such as Intl.RelativeTimeFormat (currently at Stage 3).

Finally, we have written more tests for JS language features, performed maintenance and optimization and participated to other spec discussions at TC39. Among performance optimizations, we have contributed a significant optimization to Promise performance to V8.

Accessibility

Igalia has continued the standardization effort at the W3C. We are pleased to announce that the following milestones have been reached:

WAI-ARIA Graphics Module and Graphics Accessibility API Mappings moved to proposed recommendation.
Accessible Name and Description Computation moved to Candidate Recommendation.

A new charter for the ARIA WG as well as drafts for ARIA 1.2 and Core Accessibility API Mappings 1.2 are in preparation and are expected to be published this summer.

On the development side, we implemented new ARIA features and fixed several bugs in WebKit and Gecko. We have refined platform-specific tools that are needed to automate accessibility Web Platform Tests (examine the accessibility tree, obtain information about accessible objects, listen for accessibility events, etc) and hope we will be able to integrate them in Web Platform Tests. Finally we continued maintenance of the Orca screen reader, in particular fixing some accessibility-event-flood issues in Caja and Nautilus that had significant impact on Orca users.

Web Platform Predictability

Thanks to support from Bloomberg, we were able to improve interoperability for various Editing/Selection use cases. For example when using backspace to delete text content just after a table (W3C issue) or deleting a list item inside a content cell.

We were also pleased to continue our collaboration with the AMP project. They provide us a list of bugs and enhancement requests (mostly for the WebKit iOS port) with concrete use cases and repro cases. We check the status and plans in WebKit, do debugging/analysis and of course actually submit patches to address the issues. That’s not always easy (e.g. when it is touching proprietary code or requires to find some specific reviewers) but at least we make discussions move forward. The topics are very diverse, it can be about MessageChannel API, CSSOM View, CSS transitions, CSS animations, iOS frame scrolling custom elements or navigating special links and many others.

In general, our projects are always a good opportunity to write new Web Platform Tests import them in WebKit/Chromium/Mozilla or improve the testing infrastructure. We have been able to work on tests for several specifications we work on.

CSS

Thanks to support from Bloomberg we’ve been pursuing our activities around CSS:

CSS Grid Layout: we removed “grid-“ prefix from gutter properties and added support percentage to column-gap property in Multicolumn. We worked on baseline alignment. Finally we are now improving the memory structures of the implementation in order to make bigger grids possible.
Composition underline is no longer always black in Chromium, it uses now the current text color for by default
CSS Containment: We updated Chromium implementation to align with the spec and fixed several bugs.
CSS Text 3: We started to implement the new CSS value ‘break-spaces’

We also got more involved in the CSS Working Group, in particular participating to the face-to-face meeting in Berlin and will attend TPAC’s meeting in October.

WebKit

We have also continued improving the web platform implementation of some Linux ports of WebKit (namely GTK and WPE). A lot of this work was possible thanks to the financial support of Metrological.

We improved Web Driver support and made it work with WPE too.
Web Animations are now enabled.
Service Workers are now enabled behind an experimental features flag.
We implemented most of the missing functionality of ImageBitmap.
OffscreenCanvas is now mostly implemented, but pending review.
Web Platform Tests integration: We landed a patch to make possible to run tests against the official Web Platform Tests repository.
WebRTC: We made the raspberry pi work with Firefox and Chrome properly and started pushing upstream all the branch. We have already upstreamed all the changes required in the current WebKit classes, the libwebrtc compilation and the getUserMedia support. We have two main pending patches: PeerConnection support and EnumerateDevices support.
Media Source Extensions: We refined our implementation and fixed several bugs ; dealing with the corresponding issues on the gstreamer side.
WebVR: Most of the API (85%) is now implemented using OpenVR and we can already get the tracking information.

Other activities

Preparation of Web Engines Hackfest 2018

Igalia has been organizing and hosting the Web Engines Hackfest since 2009, a three days event where Web Platform developers can meet, discuss and work together. We are still working on the list of invited, sponsors and talks but you can already save the date: It will happen from 1st to 3rd of October in A Coruña!

New Igalians

This semester, new developers have joined Igalia to pursue the Web platform effort:

Rob Buis, a Dutch developer currently living in Germany. He is a well-known member of the Chromium community and is currently helping on the web platform implementation in WebKit.
Qiuyi Zhang (Joyee), based in China is a prominent member of the Node.js community who is now also assisting our compilers team on V8 developments.
Dominik Infuer, an Austrian specialist in compilers and programming language implementation who is currently helping on our JSC effort.

Coding Experience Programs

Two students have started a coding experience program some weeks ago:

Oriol Brufau, a recent graduate in math from Spain who has been an active collaborator of the CSS Working Group and a contributor to the Mozilla project. He is working on the CSS Logical Properties and Values specification, implementing it in Chromium implementation.
Darshan Kadu, a computer science student from India, who contributed to GIMP and Blender. He is working on Web Platform Tests with focus on WebKit’s infrastructure and the GTK & WPE ports in particular.

Additionally, Caio Lima is continuing his coding experience in Igalia and is among other things working on implementing BigInt in JSC.

Conclusion

Thank you for reading this blog post and we look forward to more work on the web platform this semester!

BlinkOn 9: Working on the Web Platform from a cooperative

2018-04-26T00:00:00+02:00

Last week, I attended BlinkOn 9. I was very happy to spend some time with my colleagues working on Chromium, including a new developer who will join my team next week (to be announced soon!).

This edition had the usual format with presentations, brainstorming, lightning talks and informal chats with Chromium developers. I attended several interesting presentations on web platform standardization, implementation and testing. It was also great to talk to Googlers in order to coordinate on some of Igalia’s projects such as the collaboration with AMP or MathML in Chromium.

In the previous edition, I realized that one can propose non-technical talks (e.g. the one about inclusion and diversity) and some people seemed curious about Igalia. Hence I proposed a presentation “Working on the Web Platform from a Cooperative” that gives:

An introduction to Igalia and its activities.
A description of our non-hierarchical model and benefits it brings.
An overview of Igalia’s contribution to the Web Platform.

From the feedback I got, people appreciated the presentation and liked to get more insight on Igalia. Unfortunately, I was not able to record the talk due to technical issues. Of course, thirty minutes is a bit short to develop all the ideas and reply properly to all the questions. But for those who are interested here are more pointers:

About “equal salary” VS “cost of living”, you might want to read Andy Wingo’s blog posts “time for money” and “a simple (local) solution to the pay gap”. Several years ago, Robert O’Callahan had already wondered whether it really made sense to take into account the cost of living to determine salaries. Personally, I believe that as long as our “target salary” is high enough for all places where we work, we don’t really need to worry about this issue and can instead spend time focusing on more interesting agreements to keep making Igalia a great working place.
About dependency on the customers, see the last paragraph of “work groups” in Andy’s blog post “but that would be anarchy!” especially treating customers as partners. As I said during the talk, as long as we have enough customers we have some freedom to accept contracts that are more interesting for our strategy and aligned with our values or negociate improvements to existing contracts ; without worrying about unstability and uncertainty.
About the meaning of “Igalia”, the simple answer is “it does not mean anything”. If you join Igalia and get the opportunity to learn about the company history, there is a more complete answer about how the name was found…
Regarding founders of Igalia in 2001: Dape (who attended BlinkOn), Alex, Juanjo, Xavi, Berto and Chema are indeed still working at Igalia and in general, very few people have left Igalia since its creation.

Finally we had two related tricky questions from Google employees:

How do you sync with the browser vendors’ own agenda?
What can Google (or any other browser vendor) could do to facilitate involvements of third-party contributors?

One could enumerate different situations but unfortunately there is not a generic answer. In some cases collaboration worked very well and was quite successful. In other cases, things were more complicated and we had to “fight” to convince browser vendors to keep some existing code or accept new features.

Communication is very important. We try to sync with browser vendors using video conferencing or by attending conferences, but some companies/teams are more or less inclined to reveal information (especially when strategic products are involved). In general, I have the impression that the more the teams work close to the Web Platform, the more they are used to the democratic and open-source culture and welcome third-party contributions.

Although the ideal is to work upstream, we have recently been developing skills to manage separate forks and rebase them regularly against the main branch. This is a good option to find a balance between the request of the customer to implement features and the need of the browser vendors to focus on their own tasks. Chromium for Wayland is a good example of that approach.

Hence probably one way to help third-party contributors is to improve communication. We had some issues with developers not even willing to talk to us or not taking time to review or comment on our patches/CLs. If browser vendors could indicate that they don’t like an approach as soon as possible or that they won’t accept patches until some refactoring is complete, that would help us a lot to discuss with clients, properly schedule our tasks and consider the option of an experimental branch.

Another way to help third-party contributors would be to advertize more that such contributions are actually possible. Indeed, many people think that “everything is implemented by browser vendors” which can make difficult to find clients for web platform development. When companies rant about Google not implementing feature X, fixing bug Y or participating to standard Z, instead of ignoring them or denying the importance of the request, it would probably be more constructive to mention that they can actually pay consulting companies to do that job. As I indicated in the talk, we recently had such successful collaborations with Bloomberg, Metrological or AMP and we would be happy to find more!

There are probably more to reply to these questions, but that’s my quick thought on the matter for now. I’ll try discussing with my colleagues and see if we have more ideas to share.

Review of Igalia's Web Platform activities (H2 2017)

2018-01-12T00:00:00+01:00

Last september, I published a first blog post to let people know a bit more about Igalia’s activities around the Web platform, with a plan to repeat such a review each semester. The present blog post focuses on the activity of the second semester of 2017.

Accessibility

As part of Igalia’s commitment to diversity and inclusion, we continue our effort to standardize and implement accessibility technologies. More specifically, Igalian Joanmarie Diggs continues to serve as chair of the W3C’s ARIA working group and as an editor of Accessible Rich Internet Applications (WAI-ARIA) 1.1, Core Accessibility API Mappings 1.1, Digital Publishing WAI-ARIA Module 1.0, Digital Publishing Accessibility API Mappings 1.0 all of which became W3C Recommandations in December! Work on versions 1.2 of ARIA and the Core AAM will begin in January. Stay tuned for the First Public Working Drafts.

We also contributed patches to fix several issues in the ARIA implementations of WebKit and Gecko and implemented support for the new DPub ARIA roles. We expect to continue this collaboration with Apple and Mozilla next year as well as to resume more active maintenance of Orca, the screen reader used to access graphical desktop environments in GNU/Linux.

Last but not least, progress continues on switching to Web Platform Tests for ARIA and “Accessibility API Mappings” tests. This task is challenging because, unlike other aspects of the Web Platform, testing accessibility mappings cannot be done by solely examining what is rendered by the user agent. Instead, an additional tool, an “Accessible Technology Test Adapter” (ATTA) must be also be run. ATTAs work in a similar fashion to assistive technologies such as screen readers, using the implemented platform accessibility API to query information about elements and reporting what it obtains back to WPT which in turn determines if a test passed or failed. As a result, the tests are currently officially manual while the platform ATTAs continue to be developed and refined. We hope to make sufficient progress during 2018 that ATTA integration into WPT can begin.

CSS

This semester, we were glad to receive Bloomberg’s support again to pursue our activities around CSS. After a long commitment to CSS and a lot of feedback to Editors, several of our members finally joined the Working Group! Incidentally and as mentioned in a previous blog post, during the CSS Working Group face-to-face meeting in Paris we got the opportunity to answer Microsoft’s questions regarding The Story of CSS Grid, from Its Creators (see also the video). You might want to take a look at our own videos for CSS Grid Layout, regarding alignment and placement and easy design.

On the development side, we maintained and fixed bugs in Grid Layout implementation for Blink and WebKit. We also implemented alignment of positioned items in Blink and WebKit. We have several improvements and bug fixes for editing/selection from Bloomberg’s downstream branch that we’ve already upstreamed or plan to upstream. Finally, it’s worth mentioning that the work done on display: contents by our former coding experience student Emilio Cobos was taken over and completed by antiik (for WebKit) and rune (for Blink) and is now enabled by default! We plan to pursue these developments next year and have various ideas. One of them is improving the way grids are stored in memory to allow huge grids (e.g. spreadsheet).

Web Platform Predictability

One of the area where we would like to increase our activity is Web Platform Predictability. This is obviously essential for our users but is also instrumental for a company like Igalia making developments on all the open source Javascript and Web engines, to ensure that our work is implemented consistently across all platforms. This semester, we were able to put more effort on this thanks to financial support from Bloomberg and Google AMP.

We have implemented more frame sandboxing attributes WebKit to improve user safety and make control of sandboxed documents more flexible. We improved the sandboxed navigation browser context flag and implemented the new allow-popup-to-escape-sandbox, allow-top-navigation-without-user-activation, and allow-modals values for the sandbox attribute.

Currently, HTML frame scrolling is not implemented in WebKit/iOS. As a consequence, one has to use the non-standard -webkit-overflow-scrolling: touch property on overflow nodes to emulate scrollable elements. In parallel to the progresses toward using more standard HTML frame scrolling we have also worked on annoying issues related to overflow nodes, including flickering/jittering of “position: fixed” nodes or broken Find UI or a regression causing content to disappear.

Another important task as part of our CSS effort was to address compatibility issues between the different browsers. For example we fixed editing bugs related to HTML List items: WebKit’s Bug 174593/Chromium’s Issue 744936 and WebKit’s Bug 173148/Chromium’s Issue 731621. Inconsistencies in web engines regarding selection with floats have also been detected and we submitted the first patches for WebKit and Blink. Finally, we are currently improving line-breaking behavior in Blink and WebKit, which implies the implementation of new CSS values and properties defined in the last draft of the CSS Text 3 specification.

We expect to continue this effort on Web Platform Predictability next year and we are discussing more ideas e.g. WebPackage or flexbox compatibility issues. For sure, Web Platform Tests are an important aspect to ensure cross-platform inter-operability and we would like to help improving synchronization with the conformance tests of browser repositories. This includes the accessibility tests mentioned above.

MathML

Last November, we launched a fundraising Campaign to implement MathML in Chromium and presented it during Frankfurt Book Fair and TPAC. We have gotten very positive feedback so far with encouragement from people excited about this project. We strongly believe the native MathML implementation in the browsers will bring about a huge impact to STEM education across the globe and all the incumbent industries will benefit from the technology. As pointed out by Rick Byers, the web platform is a commons and we believe that a more collective commitment and contribution are essential for making this world a better place!

While waiting for progress on Chromium’s side, we have provided minimal maintenance for MathML in WebKit:

We fixed all the debug ASSERTs reported on Bugzilla.
We did follow-up code clean up and refactoring.
We imported Web Platform tests in WebKit.
We performed review of MathML patches.

Regarding the last point, we would like to thank Minsheng Liu, a new volunteer who has started to contribute patches to WebKit to fix issues with MathML operators. He is willing to continue to work on MathML development in 2018 as well so stay tuned for more improvements!

Javascript

During the second semester of 2017, we worked on the design, standardization and implementation of several JavaScript features thanks to sponsorship from Bloomberg and Mozilla.

One of the new features we focused on recently is BigInt. We are working on an implementation of BigInt in SpiderMonkey, which is currently feature-complete but requires more optimization and cleanup. We wrote corresponding test262 conformance tests, which are mostly complete and upstreamed. Next semester, we intend to finish that work while our coding experience student Caio Lima continues work on a BigInt implementation on JSC, which has already started to land. Google also decided to implement that feature in V8 based on the specification we wrote. The BigInt specification that we wrote reached Stage 3 of TC39 standardization. We plan to keep working on these two BigInt implementations, making specification tweaks as needed, with an aim towards reaching Stage 4 at TC39 for the BigInt proposal in 2018.

Igalia is also proposing class fields and private methods for JavaScript. Similarly to BigInt, we were able to move them to Stage 3 and we are working to move them to stage 4. Our plan is to write test262 tests for private methods and work on an implementation in a JavaScript engine early next year.

Igalia implemented and shipped async iterators and generators in Chrome 63, providing a convenient syntax for exposing and using asynchronous data streams, e.g., HTML streams. Additionally, we shipped a major performance optimization for Promises and async functions in V8.

We implemented and shipped two internationalization features in Chrome, Intl.PluralRules and Intl.NumberFormat.prototype.formatToParts. To push the specifications of internationalization features forwards, we have been editing various other internationalization-related specifications such as Intl.RelativeTimeFormat, Intl.Locale and Intl.ListFormat; we also convened and led the first of what will be a monthly meeting of internationalization experts to propose and refine further API details.

Finally, Igalia has also been formalizing WebAssembly’s JavaScript API specification, which reached the W3C first public working draft stage, and plans to go on to improve testing of that specification as the next step once further editorial issues are fixed.

Miscellaneous

Thanks to sponsorship from Mozilla we have continued our involvement in the Quantum Render project with the goal of using Servo’s WebRender in Firefox.

Support from Metrological has also given us the opportunity to implement more web standards from some Linux ports of WebKit (GTK and WPE, including:

WebRTC
WebM
WebVR
Web Crypto
Web Driver
WebP animations support
HTML interactive form validation
MSE

Conclusion

Thanks for reading and we look forward to more work on the web platform in 2018. Onwards and upwards!

Recent Browser Events

2017-10-23T00:00:00+02:00

TL;DR

At Igalia, we attend many browser events. This is a quick summary of some recents conferences I participated to… or that gave me the opportunity to meet Igalians in Paris 😉.

Week 31: Paris - CSS WG F2F - W3C

My teammate Sergio attended the CSS WG F2F meeting as an observer. On Tuesday morning, I also made an appearance (but it was so brief that ceux que j’ai rencontrés ne m’ont peut-être pas vu). Together with other browser vendors and WG members, Sergio gave an interview regarding the successful story of CSS Grid Layout. By the way, given our implementation work in WebKit and Blink, Igalia finally decided to join the CSS Working Group 😊. Of course, during that week I had dinner with Sergio and it was nice to chat with my colleague in a French restaurant of Montmartre.

Week 38: Tokyo - BlinkOn 8 - Google

Jacobo, Gyuyoung and I attended BlinkOn 8. I had nice discussions and listened to interesting talks about a wide range of topics (Layout NG, Accessibility, CSS, Fonts, Web Predictability & Standards, etc). It was a pleasure to finally meet in persons some developers I had been in touch with during my projects on Ozone/Wayland and WebKit/iOS. For the lightning talks, we presented our activities on embedded linux platforms and the Web Platform. Incidentally, it was great to see Igalia’s work mentioned during the Next Generation Rendering Engine session. Obviously, I had the opportunity to visit places and taste Japanese food in Asakusa, Ueno and Roppongi 😋.

Week 40: A Coruña - Web Engines Hackfest - Igalia

I attended one of my favorite events, that gathers the whole browser community during three days for technical presentations, breakout sessions, hacking and galician food. This year, we had many sponsors and attendees. It is good to see that the event is becoming more and more popular! It was long overdue, but I was finally able to make Brotli and WOFF2 installable as system libraries on Linux and usable by WebKitGTK+ 😊. I opened similar bugs in Gecko and the same could be done in Chromium. Among the things I enjoyed, I met Jonathan Kew in person and heard more about Antonio and Maksim’s progress on Ozone/Wayland. As usual, it was nice to share time with colleagues, attend the assembly meeting, play football matches, have meals, visit Asturias… and tell one’s story 😉.

Week 41: San Jose - WebKit Contributors Meeting - Apple

In the past months, I have mostly been working on WebKit at Igalia and I would have been happy to see my fellow WebKit developers. However, given the events in Japan and Spain, I was not willing to make another trip to the USA just after. Hence I had to miss the WebKit Contributors Meeting again this year 😞. Fortunately, my colleagues Alex, Michael and Žan were present. Igalia is an important contributor to WebKit and we will continue to send people and propose some talks next year.

Week 42: Paris - Monthly Speaker Series - Mozilla

This Wednesday, I attended a conference on Privacy as a Competitive Advantage in Mozilla’s office. It was nice to hear about the increasing interest on privacy and to see the regulation made by the European Union in that direction. My colleague Philippe was visiting the office to work with some Mozilla developers on one of our project, so I was also able to meet him in the conference room. Actually, Mozilla employees were kind enough to let me stay at the office after the conference… Hence I was able to work on Apple’s Web Engine on a project sponsored by Google at the Mozilla office… probably something you can only do at Igalia 😉. Last but not least, Guillaume was also in holidays in Paris this week, so I let you imagine what happens when three French guys meet (hint: it involves food 😋).

Review of Igalia's Web Platform activities (H1 2017)

2017-09-07T00:00:00+02:00

Introduction

For many years Igalia has been committed to and dedicated efforts to the improvement of Web Platform in all open-source Web Engines (Chromium, WebKit, Servo, Gecko) and JavaScript implementations (V8, SpiderMonkey, ChakraCore, JSC). We have been working in the implementation and standardization of some important technologies (CSS Grid/Flexbox, ECMAScript, WebRTC, WebVR, ARIA, MathML, etc). This blog post contains a review of these activities performed during the first half (and a bit more) of 2017.

Projects

CSS

A few years ago Bloomberg and Igalia started a collaboration to implement a new layout model for the Web Platform. Bloomberg had complex layout requirements and what the Web provided was not enough and caused performance issues. CSS Grid Layout seemed to be the right choice, a feature that would provide such complex designs with more flexibility than the currently available methods.

We’ve been implementing CSS Grid Layout in Blink and WebKit, initially behind some flags as an experimental feature. This year, after some coordination effort to ensure interoperability (talking to the different parties involved like browser vendors, the CSS Working Group and the web authors community), it has been shipped by default in Chrome 58 and Safari 10.1. This is a huge step for the layout on the web, and modern websites will benefit from this new model and enjoy all the features provided by CSS Grid Layout spec.

Since the CSS Grid Layout shared the same alignment properties as the CSS Flexible Box feature, a new spec has been defined to generalize alignment for all the layout models. We started implementing this new spec as part of our work on Grid, being Grid the first layout model supporting it.

Finally, we worked on other minor CSS features in Blink such as caret-color or :focus-within and also several interoperability issues related to Editing and Selection.

MathML

MathML is a W3C recommendation to represent mathematical formulae that has been included in many other standards such as ISO/IEC, HTML5, ebook and office formats. There are many tools available to handle it, including various assistive technologies as well as generators from the popular LaTeX typesetting system.

After the improvements we performed in WebKit’s MathML implementation, we have regularly been in contact with Google to see how we can implement MathML in Chromium. Early this year, we had several meetings with Google’s layout team to discuss this in further details. We agreed that MathML is an important feature to consider for users and that the right approach would be to rely on the new LayoutNG model currently being implemented. We created a prototype for a small LayoutNG-based MathML implementation as a proof-of-concept and as a basis for future technical discussions. We are going to follow-up on this after the end of Q3, once Chromium’s layout team has made more progress on LayoutNG.

Servo

Servo is Mozilla’s next-generation web content engine based on Rust, a language that guarantees memory safety. Servo relies on a Rust project called WebRender which replaces the typical rasterizer and compositor duo in the web browser stack. WebRender makes extensive use of GPU batching to achieve very exciting performance improvements in common web pages. Mozilla has decided to make WebRender part of the Quantum Render project.

We’ve had the opportunity to collaborate with Mozilla for a few years now, focusing on the graphics stack. Our work has focused on bringing full support for CSS stacking and clipping to WebRender, so that it will be available in both Servo and Gecko. This has involved creating a data structure similar to what WebKit calls the “scroll tree” in WebRender. The scroll tree divides the scene into independently scrolled elements, clipped elements, and various transformation spaces defined by CSS transforms. The tree allows WebRender to handle page interaction independently of page layout, allowing maximum performance and responsiveness.

WebRTC

WebRTC is a collection of communications protocols and APIs that enable real-time communication over peer-to-peer connections. Typical use cases include video conferencing, file transfer, chat, or desktop sharing. Igalia has been working on the WebRTC implementation in WebKit and this development is currently sponsored by Metrological.

This year we have continued the implementation effort in WebKit for the WebKitGTK and WebKit WPE ports, as well as the maintenance of two test servers for WebRTC: Ericsson’s p2p and Google’s apprtc. Finally, a lot of progress has been done to add support for Jitsi using the existing OpenWebRTC backend.

Since OpenWebRTC development is not an active project anymore and given libwebrtc is gaining traction in both Blink and the WebRTC implementation of WebKit for Apple software, we are taking the first steps to replace the original WebRTC implementation in WebKitGTK based on OpenWebRTC, with a new one based on libwebrtc. Hopefully, this way we will share more code between platforms and get more robust support of WebRTC for the end users. GStreamer integration in this new implementation is an issue we will have to study, as it’s not built in libwebrtc. libwebrtc offers many services, but not every WebRTC implementation uses all of them. This seems to be the case for the Apple WebRTC implementation, and it may become our case too if we need tighter integration with GStreamer or hardware decoding.

WebVR

WebVR is an API that provides support for virtual reality devices in Web engines. Implementation and devices are currently actively developed by browser vendors and it looks like it is going to be a huge thing. Igalia has started to investigate on that topic to see how we can join that effort. This year, we have been in discussions with Mozilla, Google and Apple to see how we could help in the implementation of WebVR on Linux. We decided to start experimenting an implementation within WebKitGTK. We announced our intention on the webkit-dev mailing list and got encouraging feedback from Apple and the WebKit community.

ARIA

ARIA defines a way to make Web content and Web applications more accessible to people with disabilities. Igalia strengthened its ongoing committment to the W3C: Joanmarie Diggs joined Richard Schwerdtfeger as a co-Chair of the W3C’s ARIA working group, and became editor of the Core Accessibility API Mappings, [Digital Publishing Accessibility API Mappings] (https://w3c.github.io/aria/dpub-aam/dpub-aam.html), and Accessible Name and Description: Computation and API Mappings specifications. Her main focus over the past six months has been to get ARIA 1.1 transitioned to Proposed Recommendation through a combination of implementation and bugfixing in WebKit and Gecko, creation of automated testing tools to verify platform accessibility API exposure in GNU/Linux and macOS, and working with fellow Working Group members to ensure the platform mappings stated in the various “AAM” specs are complete and accurate. We will provide more information about these activities after ARIA 1.1 and the related AAM specs are further along on their respective REC tracks.

Web Platform Predictability for WebKit

The AMP Project has recently sponsored Igalia to improve WebKit’s implementation of the Web platform. We have worked on many issues, the main ones being:

Frame sandboxing: Implementing sandbox values to allow trusted third-party resources to open unsandboxed popups or restrict unsafe operations of malicious ones.
Frame scrolling on iOS: Addressing issues with scrollable nodes; trying to move to a more standard and interoperable approach with scrollable iframes.
Root scroller: Finding a solution to the old interoperability issue about how to scroll the main frame; considering a new rootScroller API.

This project aligns with Web Platform Predictability which aims at making the Web more predictable for developers by improving interoperability, ensuring version compatibility and reducing footguns. It has been a good opportunity to collaborate with Google and Apple on improving the Web. You can find further details in this blog post.

JavaScript

Igalia has been involved in design, standardization and implementation of several JavaScript features in collaboration with Bloomberg and Mozilla.

In implementation, Bloomberg has been sponsoring implementation of modern JavaScript features in V8, SpiderMonkey, JSC and ChakraCore, in collaboration with the open source community:

Implementation of many ES6 features in V8, such as generators, destructuring binding and arrow functions
Async/await and async iterators and generators in V8 and some work in JSC
Optimizing SpiderMonkey generators
Ongoing implementation of BigInt in SpiderMonkey and class field declarations in JSC

On the design/standardization side, Igalia is active in TC39 and with Bloomberg’s support

Arbitrary precision integers (BigInt)
Class field declarations, private methods and decorators
New RegExp features such as named capture groups

In partnership with Mozilla, Igalia has been involved in the specification of various JavaScript standard library features for internationalization, in specification, implementation in V8, code reviews in other JavaScript engines, as well as working with the underlying ICU library.

Other activities

Preparation of Web Engines Hackfest 2017

Igalia has been organizing and hosting the Web Engines Hackfest since 2009. This event under an unconference format has been a great opportunity for Web Engines developers to meet, discuss and work together on the web platform and on web engines in general. We announced the 2017 edition and many developers already confirmed their attendance. We would like to thank our sponsors for supporting this event and we are looking forward to seeing you in October!

Coding Experience

Emilio Cobos has completed his coding experience program on implementation of web standards. He has been working in the implementation of “display: contents” in Blink but some work is pending due to unresolved CSS WG issues. He also started the corresponding work in WebKit but implementation is still very partial. It has been a pleasure to mentor a skilled hacker like Emilio and we wish him the best for his future projects!

New Igalians

During this semester we have been glad to welcome new igalians who will help us to pursue Web platform developments:

Daniel Ehrenberg joined Igalia in January. He is an active contributor to the V8 JavaScript engine and has been representing Igalia at the ECMAScript TC39 meetings.
Alicia Boya joined Igalia in March. She has experience in many areas of computing, including web development, computer graphics, networks, security, and software design with performance which we believe will be valuable for our Web platform activities.
Ms2ger joined Igalia in July. He is a well-known hacker of the Mozilla community and has wide experience in both Gecko and Servo. He has noticeably worked in DOM implementation and web platform test automation.

Conclusion

Igalia has been involved in a wide range of Web Platform technologies going from Javascript and layout engines to accessibility or multimedia features. Efforts have been made in all parts of the process:

Participation to standardization bodies (W3C, TC39).
Elaboration of conformance tests (web-platform-tests test262).
Implementation and bug fixes in all open source web engines.
Discussion with users, browser vendors and other companies.

Although, some of this work has been sponsored by Google or Mozilla, it is important to highlight how external companies (other than browser vendors) can make good contributions to the Web Platform, playing an important role on its evolution. Alan Stearns already pointed out the responsibility of the Web Plaform users on the evolution of CSS while Rachel Andrew emphasized how any company or web author can effectively contribute to the W3C in many ways.

As mentioned in this blog post, Bloomberg is an important contributor of several open source projects and they’ve been a key player in the development of CSS Grid Layout or Javascript. Similarly, Metrological’s support has been instrumental for the implementation of WebRTC in WebKit. We believe others could follow their examples and we are looking forward to seeing more companies sponsoring Web Platform developments!

The AMP Project and Igalia working together to improve WebKit and the Web Platform

2017-08-30T00:00:00+02:00

TL;DR

The AMP Project and Igalia have recently been collaborating to improve WebKit’s implementation of the Web platform. Both teams are committed to make the Web better and we expect that all developers and users will benefit from this effort. In this blog post, we review some of the bug fixes and features currently being considered:

Frame sandboxing: Implementing sandbox values to allow trusted third-party resources to open unsandboxed popups or restrict unsafe operations of malicious ones.
Frame scrolling on iOS: Trying to move to a more standard and interoperable approach via iframe elements; addressing miscellaneous issues with scrollable nodes (e.g. visual artifacts while scrolling, view not scrolled when using “Find Text”…).
Root scroller: Finding a solution to the old interoperability issue about how to scroll the main frame; considering a new rootScroller API.

Some demo pages for frame sandboxing and scrolling are also available if you wish to test features discussed in this blog post.

Introduction

AMP is an open-source project to enable websites and ads that are consistently fast, beautiful and high-performing across devices and distribution platforms. Several interoperability bugs and missing features in WebKit have caused problems to AMP users and to Web developers in general. Although it is possible to add platform-specific workarounds to AMP, the best way to help the Web Platform community is to directly fix these issues in WebKit, so that everybody can benefit from these improvements.

Igalia is a consulting company with a team dedicated to Web Platform developments in all open-source Web Engines (Chromium, WebKit, Servo, Gecko) working in the implementation and standardization of miscellaneous technologies (CSS Grid/flexbox, ECMAScript, WebRTC, WebVR, ARIA, MathML, etc). Given this expertise, the AMP Project sponsored Igalia so that they can lead these developments in WebKit. It is worth noting that this project aligns with the Web Predictability effort supported by both Google and Igalia, which aims at making the Web more predictable for developers. In particular, the following aspects are considered:

Interoperability: Effort is made to write Web Platform Tests (WPT), to follow Web standards and ensure consistent behaviors between web engines or operating systems.
Compatibility: Changes are carefully analyzed using telemetry techniques or user feedback in order to avoid breaking compatibility with previous versions of WebKit.
Reducing footguns: Removals of non-standard features (e.g. CSS vendor prefixes) are attempted while new features are carefully introduced.

Below we provide further description of the WebKit improvements, showing concretely how the above principles are followed.

Frame sandboxing

A sandbox attribute can be specified on the iframe element in order to enable a set of restrictions on any content it hosts. These conditions can be relaxed by specifying a list of values such as allow-scripts (to allow javascript execution in the frame) or allow-popups (to allow the frame to open popups). By default, the same restrictions apply to a popup opened by a sandboxed frame.

Figure 1: Example of sandboxed frames (Can they navigate their top frame or open popups? Are such popups also sandboxed?)

However, sometimes this behavior is not wanted. Consider for example the case of an advertisement inside a sandboxed frame. If a popup is opened from this frame then it is likely that a non-sandboxed context is desired on the landing page. In order to handle this use case, a new allow-popups-to-escape-sandbox value has been introduced. The value is now supported in Safari Technology Preview 34.

While performing that work, it was noticed that some WPT tests for the sandbox attribute were still failing. It turns out that WebKit does not really follow the rules to allow navigation. More specifically, navigating a top context is never allowed when such context corresponds to an opened popup. We have made some changes to WebKit so that it behaves more closely to the specification. This is integrated into Safari Technology Preview 35 and you can for example try this W3C test. Note that this test requires to change preferences to allow popups.

It is worth noting that web engines may slightly depart from the specification regarding the previously mentioned rules. In particular, WebKit checks a same-origin condition to be sure that one frame is allowed to navigate another one. WebKit always has contained a special case to ignore this condition when a sandboxed frame with the allow-top-navigation flag tries and navigate its top frame. This feature, sometimes known as “frame busting,” has been used by third-party resources to perform malicious auto-redirecting. As a consequence, Chromium developers proposed to restrict frame busting to the case where the navigation is triggered by a user gesture.

According to Chromium’s telemetry frame busting without a user gesture is very rare. But when experimenting with the behavior change of allow-top-navigation several regressions were reported. Hence it was instead decided to introduce the allow-top-navigation-by-user-activation flag in order to provide this improved safety context while still preserving backward compatibility. We implemented this feature in WebKit and it is now available in Safari Technology Preview 37.

Finally, another proposed security improvement is to use an allow-modals flag to explicitly allow sandboxed frames to display modal dialogs (with alert, prompt, etc). That is, the default behavior for sandboxed frames will be to forbid such modal dialogs. Again, such a change of behavior must be done with care. Experiments in Chromium showed that the usage of modal dialogs in sandboxed frames is very low and no users complained. Hence we implemented that behavior in WebKit and the feature should arrive in Safari Technology Preview soon.

Check out the frame sandboxing demos if if you want to test the new allow-popup-to-escape-sandbox, allow-top-navigation-without-user-activation and allow-modals flags.

Frame scrolling on iOS

Apple’s UI choice was to (almost) always “flatten” (expand) frames so that users do not require to scroll them. The rationale for this is that it avoids to be trapped into hierarchy of nested frames. Changing that behavior is likely to cause a big backward compatibility issue on iOS so for now we proposed a less radical solution: Add a heuristic to support the case of “fullscreen” iframes used by the AMP Project. Note that such exceptions already exist in WebKit, e.g. to avoid making offscreen content visible.

We thus added the following heuristic into WebKit Nightly: do not flatten out-of-flow iframes (e.g. position: absolute) that have viewport units (e.g. vw and vh). This includes the case of the “fullscreen” iframe previously mentioned. For now it is still under a developer flag so that WebKit developers can control when they want to enable it. Of course, if this is successful we might consider more advanced heuristics.

The fact that frames are never scrollable in iOS is an obvious interoperability issue. As a workaround, it is possible to emulate such “scrollable nodes” behavior using overflow: scroll nodes with the -webkit-overflow-scrolling: touch property set. This is not really ideal for our Web Predictability goal as we would like to get rid of browser vendor prefixes. Also, in practice such workarounds lead to even more problems in AMP as explained in these blog posts. That’s why implementing scrolling of frames is one of the main goals of this project and significant steps have already been made in that direction.

Figure 2: C++ classes involved in frame scrolling

The (relatively complex) class hierarchy involved in frame scrolling is summarized in Figure 2. The frame flattening heuristic mentioned above is handled in the WebCore::RenderIFrame class (in purple). The WebCore::ScrollingTreeFrameScrollingNodeIOS and WebCore::ScrollingTreeOverflowScrollingNodeIOS classes from the scrolling tree (in blue) are used to scroll, respectively, the main frame and overflow nodes on iOS. Scrolling of non-main frames will obviously have some code to share with the former, but it will also have some parts in common with the latter. For example, passing an extra UIScrollView layer is needed instead of relying on the one contained in the WKWebView of the main frame. An important step is thus to introduce a special class for scrolling inner frames that would share some logic from the two other classes and some refactoring to ensure optimal code reuse. Similar refactoring has been done for scrolling node states (in red) to move the scrolling layer parameter into WebCore::ScrollingStateNode instead of having separate members for WebCore::ScrollingStateOverflowScrollingNode and WebCore::ScrollingStateFrameScrollingNode.

The scrolling coordinator classes (in green) are also important, for example to handle hit testing. At the moment, this is not really implemented for overflow nodes but it might be important to have it for scrollable frames. Again, one sees that some logic is shared for asynchronous scrolling on macOS (WebCore::ScrollingCoordinatorMac) and iOS (WebCore::ScrollingCoordinatorIOS) in ancestor classes. Indeed, our effort to make frames scrollable on iOS is also opening the possibility of asynchronous scrolling of frames on macOS, something that is currently not implemented.

Figure 4: Video of this demo page on WebKit iOS with experimental patches to make frame scrollables (2017/07/10)

Finally, some more work is necessary in the render classes (purple) to ensure that the layer hierarchies are correctly built. Patches have been uploaded and you can view the result on the video of Figure 4. Notice that this work has not been reviewed yet and there are known bugs, for example with overlapping elements (hit testing not implemented) or position: fixed elements.

Various other scrolling bugs were reported, analyzed and sometimes fixed by Apple. The switch from overflow nodes to scrollable iframes is unlikely to address them. For example, the “Find Text” operation in iOS has advanced features done by the UI process (highlight, smart magnification) but the scrolling operation needed only works for the main frame. It looks like this could be fixed by unifying a bit the scrolling code path with macOS. There are also several jump and flickering bugs with position: fixed nodes. Finally, Apple fixed inconsistent scrolling inertia used for the main frame and the one used for inner scrollable nodes by making the former the same as the latter.

Root Scroller

The CSSOM View specification extends the DOM element with some scrolling properties. That specification indicates that the element to consider to scroll the main view is document.body in quirks mode while it is document.documentElement in no-quirks mode. This is the behavior that has always been followed by browsers like Firefox or Interner Explorer. However, WebKit-based browsers always treat document.body as the root scroller. This interoperability issue has been a big problem for web developers. One convenient workaround was to introduce the document.scrollingElement which returns the element to use for scrolling the main view (document.body or document.documentElement) and was recently implemented in WebKit. Use this test page to verify whether your browser supports the document.scrollingElement property and which DOM element is used to scroll the main view in no-quirks mode.

Nevertheless, this does not solve the issue with existing web pages. Chromium’s Web Platform Predictability team has made a huge communication effort with Web authors and developers which has drastically reduced the use of document.body in no-quirks mode. For instance, Chromium’s telemetry on Figure 3 indicates that the percentage of document.body.scrollTop in no-quirks pages has gone from 18% down to 0.0003% during the past three years. Hence the Chromium team is now considering shipping the standard behavior.

Figure 3: Use of document.body.scrollTop in no-quirks mode over time (Chromium's UseCounter)

In WebKit, the issue has been known for a long time and an old attempt to fix it was reverted for causing regressions. For now, we imported the CSSOM View tests and just marked the one related to the scrolling element as failing. An analysis of the situation has been left on WebKit’s bug; Depending on how things evolve on Chromium’s side we could consider the discussion and implementation work in WebKit.

Related to that work, a new API is being proposed to set the root scroller to an arbitrary scrolling element, giving more flexibility to authors of Web applications. Today, this is unfortunately not possible without losing some of the special features of the main view (e.g. on iOS, Safari’s URL bar is hidden when scrolling the main view to maximize the screen space). Such API is currently being experimented in Chromium and we plan to investigate whether this can be implemented in WebKit too.

Conclusion

In the past months, The AMP Project and Igalia have worked on analyzing some interoperability issue and fixing them in WebKit. Many improvements for frame sandboxing are going to be available soon. Significant progress has also been made for frame scrolling on iOS and collaboration continues with Apple reviewers to ensure that the work will be integrated in future versions of WebKit. Improvements to “root scrolling” are also being considered although they are pending on the evolution of the issues on Chromium’s side. All these efforts are expected to be useful for WebKit users and the Web platform in general.

Last but not least, I would like to thank Apple engineers Simon Fraser, Chris Dumez, and Youenn Fablet for their reviews and help, as well as Google and the AMP team for supporting that project.

MathZilla collection ported to WebExtensions

2017-04-29T00:00:00+02:00

MathZilla is a collection of MathML-related add-ons for Mozilla applications. It provides nice features such as forcing native MathML rendering (e.g. on Wikipedia), using Web fonts to render MathML or providing a context menu item to copy math formulas into the clipboard.

Initially written as a single XUL overlay extension (with even binary code for the LaTeX-to-MathML converter) it grows up as a collection of restartless add-ons using bootstrapped or SDK-based extensions, following the evolution of Mozilla’s recommendations. Also, SDK-based extensions were first generated using a Python program called cfx before Mozilla recommended to switch to a JS-based replacement called jpm.

Mozilla announced some time ago that they will transition to the WebExtensions format. On the one hand this sounds bad because developers have to re-write their legacy add-ons again and actually be sure that the transition is even possible or does break anything. On the other hand it is good for long-term interoperability since e.g. Chromium browsers or Microsoft Edge support that format. My colleague Michael Catanzaro also mentioned in a recent blog post that WebExtensions are considered for Epiphany too. It is not clear what Mozilla’s plan is for Thunderbird or SeaMonkey but hopefully they will use that format too (in the past I was suggested to make the MathZilla add-ons compatible with SeaMonkey).

Recently, Mozilla announced their plans for Firefox 57 which is basically to allow only add-ons written as WebExtensions. This means I had to re-write the Mathzilla add-ons again or they will stop working at the end of the year. In general, I believe the features have been preserved although there might be some small behavior changes or minor bugs due to the WebExtensions format. Please check the GitHub bug trackers and release notes for known issues and report any other problems you find. Finally, I reorganized a bit the git repositories and add-on names. Here is the updated list (some add-ons are still being reviewed by Mozilla):

MathML Fonts (~2300 users) - Provide MathML fonts as Web fonts, which is useful when they can not be installed (e.g. Firefox for Android).
Native MathML (~1400 users) - Force MathJax/KaTeX/MediaWiki to use native MathML rendering.
MathML Copy (~500 users) - Add context menu items to copy a MathML formula or other annotations attached to it (e.g. LaTeX) into the clipboard.
TeXZilla (~500 users) - Add-on giving access to TeXZilla, a Unicode TeX-to-MathML converter.
MathML Font Settings (~300 users) - Add context menu items to configure MathML font settings. Note that in recent Mozilla versions the advanced font preferences menu allows to configure “Fonts for Mathematics”.
Presentation MathML Polyfill (~200 users) - Add support for some advanced presentation MathML features (currently using David Carlisle’s “mml3ff” XSLT stylesheet).
Content MathML Polyfill (~200 users) - Add support for some content MathML features (currently using David Carlisle’s “ctop” XSLT stylesheet).
MathML Zoom (~100 users) - Allow zooming of mathematical formulas.
MathML View Source (experimental) - This is a re-writing of Mozilla’s ‘view MathML source’ feature with better syntax highlighting and serialization. The idea originated from this thread.
Image To MathML (experimental) - Try and convert images of mathematical formulas into MathML. It has not been ported to WebExtensions yet and I do not plan to do it in the short term.

As a conclusion, I’d like to thank all the MathZilla users for their kind comments, bug reporting and financial support. The next step will probably be to ensure addons work in more browsers but that will be for another time ;-)

Mus Window System

2017-01-23T00:00:00+01:00

Update 2017/02/01: The day I published this blog post, the old mus client library was removed. Some stack traces were taken during the analysis Antonio & I made in fall 2016, but they are now obsolete. I’m updating the blog in order to try and clarify things and fix possible inaccuracies.

TL; DR

Igalia has recently been working on making Chromium Desktop run natively on Ozone/Wayland thanks to support from Renesas. This is however still experimental and there is a lot of UI work to do. In order to facilitate discussion at BlinkOn7 and more specifically how browser windows are managed, we started an analysis of the Chromium Window system. An overview is provided in this blog post.

Introduction

Antonio described how we were able to get upstream Chromium running on Linux/Ozone/Wayland last year. However for that purpose the browser windows were really embedded in the “ash” environment (with additional widgets) which was itself drawn in an Ozone/Wayland window. This configuration is obviously not ideal but it at least allowed us to do some initial experiments and to present demos of Chromium running on R-Car M3 at CES 2017. If you are interested you can check our ces-demos-2017 and meta-browser repositories on GitHub. Our next goal is to have all the browser windows handled as native Ozone/Wayland windows.

In a previous blog post, I gave an overview of the architecture of Ozone. In particular, I described classes used to handle Ozone windows for the X11 or Wayland platforms. However, this was only a small part of the picture and to understand things globally one really has to take into account the classes for the Aura window system and Mus window client & server.

Class Hierarchy

A large number of C++ classes and types are involved in the chromium window system. The following diagram provides an overview of the class hierarchy. It can merely be divided into four parts:

In blue, the native Ozone Windows and their host class in Aura.
In red and orange, the Aura Window Tree.
In purple, the Mus Window Tree (client-side). Update 2017/02/01: These classes have been removed.
In green and brown, the Mus Window Tree (server-side) and its associated factories.

I used the following convention which is more or less based on UML:

Rectangles represent classes/interfaces while ellipses represent enums, types, defines etc. You can click them to access the corresponding C++ source file.
Inheritance is indicated by a white arrow to the base class from the derived class. For implementation of Mojo interfaces, dashed white arrows are used.
Other associations are represented by an arrow with open or diamond head. They mean that the origin of the arrow has a member involving one or more instances of the pointed class, you can hover over the arrow head to see the name of that class member.

Native Ozone Windows (blue)

ui::PlatformWindow is an abstract class representing a single window in the underlying platform windowing system. Examples of implementations include ui::X11WindowOzone or ui::WaylandWindow.

ui::PlatformWindow can be stored in a ui::ws::PlatformDisplayDefault which implements the ui::PlatformWindowDelegate interface too. This in turn allows to use platform windows as ui::ws::Display. ui::ws::Display also has an associated delegate class.

aura::WindowTreeHost is a generic class that hosts an embedded aura:Window root and bridges between the native window and that embedded root window. One can register instance of aura::WindowTreeHostObserver as observers. There are various implementations of aura::WindowTreeHost but we only consider aura::WindowTreeHostPlatform here.

Native ui::PlatformWindow windows are stored in an instance of aura::WindowTreeHostPlatform which also holds the gfx::AcceleratedWidget to paint on. aura::WindowTreeHostPlatform implements the ui::PlatformWindowDelegate so that it can listen events from the underlying ui::PlatformWindow.

aura::WindowTreeHostMus is a derived class of aura::WindowTreeHostPlatform that has additional logic make it usable in mus. One can listen changes from aura::WindowTreeHostMus by implementing the aura::WindowTreeHostMusDelegate interface.

Aura Windows (orange)

aura::Window represents windows in Aura. One can communicate with instances of that class using a aura::WindowDelegate. It is also possible to listen for events by registering a list of aura::WindowObserver.

aura::WindowPort defines an interface to enable aura::Window to be used either with mus via aura::WindowMus or with the classical environment via aura::WindowPortLocal. Here, we only consider the former.

WindowPortMus holds a reference to a aura::WindowTreeClient. All the changes to the former are forwarded to the latter so that we are sure they are propagated to the server. Conversely, changes received by aura::WindowTreeClient are sent back to WindowPortMus. However, aura::WindowTreeClient uses an intermediary aura::WindowMus class for that purpose to avoid that the changes are submitted again to the server.

Aura Window Tree Client (red)

aura::WindowTreeClient is an implementation of the ui::mojom::WindowManager and ui::mojom::WindowTreeClient Mojo interfaces for Aura. As in previous classes, we can associate to it a aura::WindowTreeClientDelegate or register a list of aura::WindowTreeClientObserver.

aura::WindowTreeClient also implements the aura::WindowManagerClient interface and has as an associated aura::WindowManagerDelegate. It has a set of roots aura::WindowPortMus which serve as an intermediary to the actual aura:Window instances.

In order to use an Aura Window Tree Client, you first create it using a Mojo connector, aura::WindowTreeClientDelegate as well as optional aura::WindowManagerDelegate and Mojo interface request for ui::mojom::WindowTreeClient. After that you need to call ConnectViaWindowTreeFactory or ConnectAsWindowManager to connect to the Window server and create the corresponding ui::mojom::WindowTree connection. Obviously, AddObserver can be used to add observers.

Finally, the CreateWindowPortForTopLevel function can be used to request the server to create a new top level window and associate a WindowPortMus to it.

Mus Window Tree Client (purple)

Update 2017/02/01: These classes have been removed.

ui::WindowTreeClient is the equivalent of aura:WindowTreeClient in mus. It implements the ui::mojom::WindowManager and ui::mojom::WindowTreeClient Mojo interfaces. It derives from ui::WindowManagerClient, has a list of ui::WindowTreeClientObserver and is associated to ui::WindowManagerDelegate and ui::WindowTreeClientDelegate.

Regarding windows, they are simpler than in Aura since we only have one class ui::Window class. In a similar way as Aura, we can register ui::WindowObserver. We can also add ui::Window as root windows or embedded windows of ui::WindowTreeClient.

In order to use a Mus Window Tree Client, you first create it using a ui::WindowTreeClientDelegate as well as optional ui::WindowManagerDelegate and Mojo interface request for ui::mojom::WindowTreeClient. After that you need to call ConnectViaWindowTreeFactory or ConnectAsWindowManager to connect to the Window server and create the corresponding ui::mojom::WindowTree connection. Obviously, AddObserver can be used to add observers.

Finally, the NewTopLevelWindow and NewWindow functions can be used to create new windows. This will cause the corresponding functions to be called on the connected ui::mojom::WindowTree.

Mus Window Tree Server (green and brown)

We just saw the Aura and Mus window tree clients. The ui::ws::WindowTree Mojo interface is used to for the window tree server and is implemented in ui::ws::WindowTree. In order to create window trees, we can use either the ui::ws::WindowManagerWindowTreeFactory or the ui::ws::WindowTreeFactory which are available through the corresponding ui::mojom::WindowManagerWindowTreeFactory and ui::mojom::WindowTreeFactory Mojo interfaces.

ui::ws::WindowTreeHostFactory implements the ui::ws::WindowTreeHostFactory Mojo interface. It allows to create ui::mojom::WindowTreeHost, more precisely ui::ws::Display implementing that interface. Under the hood, ui::ws::Display created that way actually have an associated ws::DisplayBinding creating a ui::ws::WindowTree. Of course, the display also has a native window associated to it.

Mojo Window Tree APIs (pink)

Update 2017/02/01: The Mus client has been removed. This section only applies to the Aura client.

As we have just seen in previous sections the Aura and Mus window systems are very similar and in particular implement the same Mojo ui::mojom::WindowManager and ui::mojom::WindowTreeClient interfaces. In particular:

The ui::mojom::WindowManager::WmCreateTopLevelWindow function can be used to create a new top level window. This results in a new windows (aura::Window or ui::Window) being added to the set of embedded windows.
The ui::mojom::WindowManager::WmNewDisplayAdded function is invoked when a new display is added. This results in a new window being added to the set of window roots (aura::WindowMus or ui::Window).
The ui::mojom::WindowTreeClient::OnEmbed function is invoked when Embed is called on the connected ui::mojom::WindowTree. The window parameter will be set as the window root (aura:WindowMus or ui:Window). Note that we assume that the set of roots is empty when such an embedding happens.

Note that for Aura Window Tree Client, the creation of new aura::WindowMus window roots implies the creation of the associated aura::WindowTreeHostMus and aura::WindowPortMus. The creation of the new aura:Window window by aura::WmCreateTopLevelWindow is left to the corresponding implementation in aura::WindowManagerDelegate.

On the server side, the ui::ws::WindowTree implements the ui::mojom::WindowTree mojo interface. In particular:

The ui::mojom::WindowTree::Embed function can be used to embed a WindowTreeClient at a given window. This will result of OnEmbed being called on the window tree client.
The ui::mojom::WindowTree::NewWindow function can be used to create a new window. It will call OnChangeCompleted on the window tree client.
The ui::mojom::WindowTree::NewTopLevelWindow function can be used to request the window manager to create a new top level window. It will call OnTopLevelCreated (or OnChangeCompleted in case of failure) on the window tree client.

Analysis of Window Creation

Chrome/Mash

In our project, we are interested in how Chrome/Mash windows are created. When the chrome browser process is launched, it takes the regular content/browser startup path. That means in chrome/app/chrome_main.cc it does not take the ::MainMash path, but instead content::ContentMain. This is the call stack of Chrome window creation within the mash shell:

#1 0x7f7104e9fd02 ui::WindowTreeClient::NewTopLevelWindow()
#2 0x5647d0ed9067 BrowserFrameMus::BrowserFrameMus()
#3 0x5647d09c4a47 NativeBrowserFrameFactory::Create()
#4 0x5647d096bbc0 BrowserFrame::InitBrowserFrame()
#5 0x5647d0877b8a BrowserWindow::CreateBrowserWindow()
#6 0x5647d07d5a48 Browser::Browser()
...

Update 2017/02/01: This has changed since the time when the stack trace was taken. The BrowserFrameMus constructor now creates a views:DesktopNativeWidgetAura and thus an aura::Window.

The outtermost window created when one executes chrome --mash happens as follows. When the UI service starts (see Service::OnStart in services/ui/service.cc), an instance of WindowServer is created. This creations triggers the creation of a GpuServiceProxy instance. See related snippets below:

void Service::OnStart() {
  (...)
  // Gpu must be running before the PlatformScreen can be initialized.
  window_server_.reset(new ws::WindowServer(this));
  (..)
}

WindowServer::WindowServer(WindowServerDelegate* delegate)
: delegate_(delegate),
  (..)
  gpu_proxy_(new GpuServiceProxy(this)),
  (..)
{ }

GpuServiceProxy, then, starts off the GPU service initialization. By the time the GPU connection is established, the following callback is called: GpuServiceProxy::OnInternalGpuChannelEstablished. This calls back to WindowServer::OnGpuChannelEstablished, which calls Service::StartDisplayInit. The later schedules a call to PlatformScreenStub::FixedSizeScreenConfiguration. This finally triggers the creation of an Ozone top level window via an ui::ws::Display:

#0 base::debug::StackTrace::StackTrace()
#1 ui::(anonymous namespace)::OzonePlatformX11::CreatePlatformWindow()
#2 ui::ws::PlatformDisplayDefault::Init()
#3 ui::ws::Display::Init()
#4 ui::ws::DisplayManager::OnDisplayAdded()
#5 ui::ws::PlatformScreenStub::FixedSizeScreenConfiguration()
(..)

On the client-side, the addition of a new display will cause aura::WindowTreeClient::WmNewDisplayAdded to be called and will eventually lead to the creation of an aura::WindowTreeHostMus with the corresponding display id. As shown on the graph, this class derives from aura::WindowTreeHost and hence instantiates an aura::Window.

Mus demo

The Chromium source code contains a small mus demo that is also useful do experiments. The demo is implemented via a class with the following inheritance:

class MusDemo : public service_manager::Service,
                public aura::WindowTreeClientDelegate,
                public aura::WindowManagerDelegate

i.e. it can be started as Mus service and handles some events from aura:WindowTreeClient and aura::WindowManager.

When the demo service is launched, it creates a new Aura environment and an aura::WindowTreeClient, it does the Mojo request to create an ui::WindowTree and then performs the connection:

#0 aura::WindowTreeClient::ConnectAsWindowManager()
#1 ui::demo::MusDemo::OnStart()
#2 service_manager::ServiceContext::OnStart()

Similarly to Chrome/Mash, a new display and its associated native window is created by the UI:

#0 ui::ws::WindowTree::AddRootForWindowManager()
#1 ui::ws::Display::CreateWindowManagerDisplayRootFromFactory()
#2 ui::ws::Display::InitWindowManagerDisplayRoots()
#3 ui::ws::PlatformDisplayDefault::OnAcceleratedWidgetAvailable()
#4 ui::X11WindowBase::Create()
#5 ui::(anonymous namespace)::OzonePlatformX11::CreatePlatformWindow()
#6 ui::ws::PlatformDisplayDefault::Init()
#7 ui::ws::Display::Init()
#8 ui::ws::DisplayManager::OnDisplayAdded()

The Aura Window Tree Client listens the changes and eventually aura::WindowTreeClient::WmNewDisplayAdded is called. It then informs its MusDemo delegate so that the display is added to the screen display list…

#0 DisplayList::AddDisplay
#1 ui::demo::MusDemo::OnWmWillCreateDisplay
#2 aura::WindowTreeClient::WmNewDisplayAddedImpl
#3 aura::WindowTreeClient::WmNewDisplayAdded

…and the animated bitmap window is added to the Window Tree:

#0  ui::demo::MusDemo::OnWmNewDisplay
#1  aura::WindowTreeClient::WmNewDisplayAddedImpl
#2  aura::WindowTreeClient::WmNewDisplayAdded

Update 2017/02/01: Again, aura::WindowTreeClient::WmNewDisplayAdded will lead to the creation of an aura Window.

Conclusion

Currently, both Chrome/Mash or Mus demo create a ui::ws::Display containing a native Ozone window. The Mus demo class uses the Aura WindowTreeClient to track the creation of that display and pass it to the Mus Demo class. The actual chrome browser window is created inside the native Mash window using a Mus WindowTreeClient. Again, this is not what we want since each chrome browser should have its own native window. It will be interesting to first experiment multiple native windows with the Mus demo.

Update 2017/02/01: aura::WindowTreeClient is now always used to create Aura Windows for both external windows (corresponding to a display & a native window) and internal windows. However, we still want all chrome browser to be external windows.

It seems that the Ozone/Mash work has been mostly done with ChromeOS in mind and will probably need some adjustments to make things work on desktop Linux. We are very willing to discuss this and other steps for Desktop Chrome Wayland/Ozone in details with Google engineers. Antonio will attend BlinkOn7 next week and will be very happy to talk with you so do not hesitate to get in touch!

¡Igalia is hiring!

2016-12-22T00:00:00+01:00

If you read this blog, you probably know that I joined Igalia early this year, where I have been involved in projects related to free software and web engines. You may however not be aware that Igalia has a flat & cooperative structure where all decisions (projects, events, recruitments, company agreements etc) are voted by members of an assembly. In my opinion such an organization allows to take better decisions and to avoid frustrations, compared to more traditional hierarchical organizations.

After several months as a staff, I finally applied to become an assembly member and my application was approved in November! Hence I attended my first assembly last week where I got access to all the internal information and was also able to vote… In particular, we approved the opening of two new job positions. If you are interested in state-of-the-art free software projects and if you are willing to join a company with great human values, you should definitely consider applying!

STIX Two in Gecko and WebKit

2016-12-10T00:00:00+01:00

On the 1st of December, the STIX Fonts project announced the release of STIX 2. If you never heard about this project, it is described as follows:

The mission of the Scientific and Technical Information Exchange (STIX) font creation project is the preparation of a comprehensive set of fonts that serve the scientific and engineering community in the process from manuscript creation through final publication, both in electronic and print formats.

This sounds a very exciting goal but the way it has been achieved has made the STIX project infamous for its numerous delays, for its poor or confusing packaging, for delivering math fonts with too many bugs to be usable, for its lack of openness & communication, for its bad handling of third-party feedback & contribution…

Because of these laborious travels towards unsatisfactory releases, some snarky people claim that the project was actually named after Styx (Στύξ) the river from Greek mythology that one has to cross to enter the Underworld. Or that the story of the project is summarized by Baudelaire’s verses from L’Irrémédiable:

Une Idée, une Forme, un Être
Parti de l’azur et tombé
Dans un Styx bourbeux et plombé
Où nul œil du Ciel ne pénètre ;

More seriously, the good news is that the STIX Consortium finally released text fonts with a beautiful design and a companion math font that is usable in math rendering engines such as Word Processors, LaTeX and Web Engines. Indeed, WebKit and Gecko have supported OpenType-based MathML layout for more than three years (with recent improvements by Igalia) and STIX Two now has correct OpenType data and metrics!

Of course, the STIX Consortium did not address all the technical or organizational issues that have made its reputation but I count on Khaled Hosny to maintain his more open XITS fork with enhancements that have been ignored for STIX Two (e.g. Arabic and RTL features) or with fixes of already reported bugs.

As Jacques Distler wrote in a recent blog post, OS vendors should ideally bundle the STIX Two fonts in their default installation. For now, users can download and install the OTF fonts themselves. Note however that the STIX Two archive contains WOFF and WOFF2 fonts that page authors can use as web fonts.

I just landed patches in Gecko and WebKit so that future releases will try and find STIX Two on your system for MathML rendering. However, you can already do the proper font configuration via the preference menu of your browser:

For Gecko-based applications (e.g. Firefox, Seamonkey or Thunderbird), go to the font preference and select STIX Two Math as the “font for mathematics”.
For WebKit-based applications (e.g. Epiphany or Safari) add the following rule to your user stylesheet: math { font-family: "STIX Two Math"; }.

Finally, here is a screenshot of MathML formulas rendered by Firefox 49 using STIX Two:

And the same page rendered by Epiphany 3.22.3:

Chromium on R-Car M3 & AGL/Wayland

2016-12-03T00:00:00+01:00

As my fellow igalian Antonio Gomes explained on his blog, we have recenly been working on making the master branch of Chromium run on Linux Desktop using the upstream Ozone/Wayland backend. This effort is supported by Renesas and they were interested to rely on these recent developments to showcase Chromium running on the latest generation of their R-Car systems-on-chip for automotive applications. Ideally, they were willing to see this happening for the Automotive Grade Linux distribution.

Luckily, support for R-Car M3 was already available in the development branch of AGL and the AGL instructions for R-Car M2 actually worked pretty well for M3 too mutatis mutandis. There is also an OpenEmbedded/Yocto BSP layer for Chromium but unfortunately Wayland is only supported up to m48 (using the Ozone backend from Intel’s fork) and X11 is only supported up to m52. Hence we created a (for-now-private) fork of meta-browser that:

Uses the latest development version of Chromium, in particular the Linux/Ozone/Mash/Wayland support Igalia has been working on recently.
Relies on the new GN build system rather than the deprecated GYP one.
Supports the ARM64 architecture instead of just ARM32.
Installs more resources such as Mojo service manifests.
Applies more patches (e.g. build fixes or the workaround for issue 2485673002).

Renesas also provided us some R-Car M3 boards. I received my package this week and hence was able to start playing with it on Wednesday. Again, the AGL instructions for R-Car M2 apply well to M3 and the elinux page is very helpful to understand the various parts of the board. After having started AGL on the board and opened a Weston terminal (using mouse & keyboard as on the video or using ssh) I was able to run chromium via the following command:

/usr/bin/chromium/chrome --mash --ozone-plaform=wayland \
                         --user-data-dir=/tmp/user-data-dir \
                         --no-sandbox

The first line is the usual command currently used to execute Ozone/Mash/Wayland. The second line works around the fact that the AGL demo is running as root. The third line disables sandbox because the deprecated setuid sandbox does not work with Ozone and the newer user namespaces sandbox is not supported by the AGL kernel.

The following video a gives an overview of the hardware preparation as well as some basic tests with bouncyballs.org and webengineshackfest.org:

chromium-57-r-car-M3-2016-12-02 from Frédéric Wang on Vimeo.

You can see that chromium runs relatively smoothly but we will do more testing to confirm these initial experiments. Also, we find the general issues with Linux/Ozone/Mash (internal VS external windows, no fullscreen, keyboard restricted to US layout, unwanted ChromeOS widgets etc) but we expect to continue to collaborate with Google to fix these bugs!

Analysis of Ozone Wayland

2016-10-25T00:00:00+02:00

Introduction

In the past two months, I have been working with my colleague Antonio Gomes on a Chromium project supported by Renesas. The goal is to have Chrome running on Wayland with accelerated rendering happening in a separate GPU process. Intel has done a great job to develop a specific Wayland platform for Ozone but unfortunately the code only works for older releases of Chromium and is not fully upstreamed yet. In theory, easy out-of-tree platforms was one of the guiding principles of Ozone but in practice we had to fix many build and run time errors for Ozone, even though we were working upstream. More generally, it is very challenging to keep in sync with all the refactoring work happening upstream and some work is definitely required to align Intel’s code with Google’s goals.

To summarize the situation briefly, if we follow Intel’s approach then the currently upstreamed Wayland code only compiles for ChromeOS (i.e. target_os="chromeos" in your build config) not for standard Linux build of Chrome. You can only have accelerated rendering at the price of removing separation of UI/GPU processes (i.e. via the --in-process-gpu switch). If you keep the default behavior of having the UI and GPU components running in separate processes then accelerated rendering fails. A fallback to software rendering is then attempted but it in turn fails in gpu_process_transport_factory.cc.

When Antonio joined Igalia, he did some experiments on Ozone/Wayland and wrote a quick workaround to make the software rendering fallback work when UI and GPU are in separate processes. He was also able to run standard Linux build of Chrome on Wayland by upstreaming more code from Intel. He also noticed that in Google’s code, only GL drawing is happening in the GPU component (i.e. Wayland objects are owned by the UI, contrary to Intel’s approach) which is the reason why accelerated rendering fails in separate-process mode. After discussion with Google developers it however became clear that they would prefer a different approach that is consistent with two features they are working on:

Mus-ash, a project to separate the window management and shell functionality of ash from the chrome browser process.
Mojo, a new API for inter-process communication.

During the project, we were able to get chrome running on Wayland using the Mus code path, either on ChromeOS or on standard Linux build. For that code path, GPU and UI components are currently running in the same process so as discussed above accelerated rendering works for Wayland. However, the plan for mus is to move these two components into separate processes and hence we need to adapt the Wayland code in order to allow communication between the GPU (doing GL drawings) and the UI (owning Wayland objects).

I’ll let Antonio describe more precisely on his blog the work we have been doing to get chrome --mash running on Wayland. In this blog post, I’m aligning with Google’s goal and hence I’m focusing on this Mus code path. I’m going to give a quick overview of the structure of Ozone and more specifically what is used by the Wayland platform to perform accelerated rendering.

Ozone Architecture

Ozone Platform

OzonePlatform is the main Ozone interface used to instantiate and initialize an Ozone platform (X11, Wayland, DRM/GBM…). It provides factory getters for helper classes (SurfaceFactoryOzone, PlatformWindow…).

The goal is to have OzonePlatform::InitializeForUI and OzonePlatform::InitializeForGPU called from different processes as well as two different Mojo connectors for the UI and GPU services respectively. However at the time of writing, the two components are only in different threads in the same process. There is also only one Mojo connector available: The one for the service:ui service, passed to OzonePlatform::InitializeForUI.

Two important implementations to consider in this project are OzonePlatformWayland (the one we work on) and OzonePlatformGbm (the one that seems the best maintained and tested upstream).

Native Display Delegate

NativeDisplayDelegate is used to perform display configuration. The DRM/GBM platform has its own DrmNativeDisplayDelegate implementation. For the Wayland platform, we do not actually need real display devices, so we just do as for X11 and use the FakeDisplayDelegate class. See Issue 2389053003.

Window

AcceleratedWidget is just a platform-specific object representing a surface on which compositors can paint pixels.

PlatformWindow represents a single window in the underlying platform windowing system with the usual property: It can be minimized, maximized, closed, put in fullscreen, etc

PlatformWindowDelegate. This is just a delegate to which the PlatformWindow sends events like e.g. OnBoundsChanged.

The implementations of PlatformWindow used for the Wayland and DRM/GBM platforms are respectively WaylandWindow and DrmWindow. At the moment, several features in WaylandWindow are not fully implemented but the minimal code to paint content into the window is present.

Surface and OpenGL

SurfaceFactoryOzone provides surface to be used to paint on a window. The implementations for the Wayland and DRM/GBM platforms are respectively WaylandSurfaceFactory and GbmSurfaceFactory. There are two options:

Accelerated drawing (GL path). This is done via a GLOzone instance returned by SurfaceFactoryOzone::GetGLOzone
Software Drawing (Skia). This is done via a SurfaceOzoneCanvas instance returned by SurfaceFactoryOzone::CreateCanvasForWidget.

In this project, we focus on accelerated rendering and on EGL, so we consider the GLOzoneEGL class. The following virtual pure functions must be implemented in derived class:

GLOzoneEGL::LoadGLES2Bindings, performing the GL initalization. In general, the default works well.
GLOzoneEGL::CreateOffscreenGLSurface returning an offscreen GLSurface of the specified dimension. It seems that it is mostly needed to create a dummy zero-dimensional offscreen surface during initialization. Hence the generic SurfacelessEGL should work fine. See Issue 2387063002.
GLOzoneEGL::GetNativeDisplay returns the EGL display connection to use.
GLOzoneEGL::CreateViewGLSurface returns a new GLSurface associated to a gfx::AcceleratedWidget.

SurfaceFactoryOzone also provides functions to create NativePixmap, which represents a buffer that can be directly imported via GL for rendering, or exported via dma-buf fds. The DRM/GBM platform implements it and uses GbmPixmap. For now, such pixmap objects are not needed by the Wayland platform so the SurfaceFactoryOzone function members are not implemented.

The instances of GLSurface returned by CreateViewGLSurface for the Wayland and DRM/GBM platforms are respectively GLSurfaceWayland and GbmSurface. GLSurfaceWayland is just a gl::NativeViewGLSurfaceEGL associated to a window created by wl_egl_window_create. GbmSurface instead provides surface-like semantics by deriving from GbmSurfaceless itself deriving from gl::SurfacelessEGL. Internally, a framebuffer is bound automatically for GL drawing in the GPU and the result are exported to the UI via pixmaps.

Wayland Platform

Wayland Connection

WaylandConnection is a class specific to the Wayland platform that helps to instantiate all the objects necessary to communicate with the Wayland display server.

It also manages a map from gfx::AcceleratedWidget instances to WaylandWindow instances that you can modify with the public GetWindow, AddWindow and RemoveWindow function members.

Wayland Platform Initialization

OzonePlatformWayland::InitializeUI is called from the UI thread. It creates instances of WaylandConnection and WaylandSurfaceFactory as members of OzonePlatformWayland. The Mojo connection of the service:ui service is discarded.
OzonePlatformWayland::InitializeGPU in the same process but a different thread. The WaylandConnection and WaylandSurfaceFactory members previously created are hence accessible from the GPU thread too. Nothing is done by this initialization function.

Wayland Window

A WaylandWindow is tied to a WaylandConnection and takes care of registering and unregistering itself to that WaylandConnection. It is merely a wrapper to native Wayland surfaces (wl_surface and xdg_surface) with additional window bounds. It also maintains communication with the PlatformWindowDelegate.

Wayland Surface and OpenGL

Currently, the constructor of WaylandSurfaceFactory receives the WaylandConnection. Because we want the UI process to hold the WaylandConnection and the GPU process to use WaylandSurfaceFactory for the GL rendering, the current setup will not work after mus is split. Instead, we should pass it the Mojo connector of the service:gpu service in order to indirectly communicate with the service:ui service (for testing purpose, we can for now just use the Mojo connector of the service:ui service passed to OzonePlatformWayland::InitializeUI).

The WaylandSurfaceFactory::CreateCanvasForWidget function and the WaylandCanvasSurface instance created require the WaylandConnection. However, they are used for software rendering so we can ignore them for now.

GLOzoneEGL::LoadGLES2Bindings and GLOzoneEGL::CreateOffscreenGLSurface do not seem fundamental here and they do not need any Wayland-specific code. Hence we can probably just keep the current default implementations.

At the moment GLOzoneEGLWayland::GetNativeDisplay just returns a native display provided by the WaylandConnection. It seems that we will not be able to do so when the WaylandConnection is no longer available on the GPU side. Probably we should just do like GLOzoneEGLGbm and return EGL_DEFAULT_DISPLAY.

The remaining GLOzoneEGLWayland::CreateViewGLSurface uses WaylandConnection::GetWindow to retrieve the WaylandWindow associated to AcceleratedWidget to draw on. This window provides a wl_surface and sizes that can be passed to wl_egl_window_create to create an egl_window. Then as said in the previous paragraph, a GLSurfaceWayland instance is created which is merely a gl::NativeViewGLSurfaceEGL associated to the egl_window.

Conclusion

In this blog post, an overview of the Ozone Architecture was provided, with focus on the Wayland platform. It also contains an analysis of how we could get accelerated rendering in a separate GPU process aligned with Google’s goals (Mus+ash and Mojo) and hence have it well-integrated with upstream code.

The main problem is that the GLOzoneEGLWayland code is very tied to the Wayland native objects (connection, surface, window…). We should instead only provide a Mojo connector to the GLOzoneEGLWayland class and hence to the GLSurfaceWayland class it constructs. Instances of WaylandWindow and WaylandConnector will only live on the UI side while the GL classes will live on the GPU side.

The GLSurfaceWayland should be rewritten to derive from SurfacelessEGL instead of NativeViewGLSurfaceEGL. We would then follow what is done in GbmSurface to provide some surface-like semantics but without the need to have a real egl_window object. Under the hood, the GL drawings will be performed on a framebuffer.

We should then find a way to export those framebuffers to the UI component via the Mojo connector. Again, following the DRM/GBM code with this NativePixmap objects seems the right option. At the end, the UI would convert the buffers into wl_buffers that can finally be attach to the wl_surface of the WaylandWindow.

During the past two months we have been maintaining the upstream Ozone code and made the Wayland platform work again. Try bots now build and run tests for the Wayland platform and we expect that such continuous testing will make the whole thing more robust. We have also been working on making Linux desktop work with the Mus+ash code path and started encouraging experiments of Mojo communication between the UI and GPU components of the Wayland platform.

It is really great to work with Antonio on this project and we are looking forward to continuing the collaboration on this with Google and Ozone developers. Last but not least, I would like to thank Renesas for supporting Igalia in this work to add Wayland support to chromium.

Fonts at the Web Engines Hackfest 2016

2016-10-06T00:00:00+02:00

Last week I travelled to Galicia for one of the regular gatherings organized by Igalia. It was a great pleasure to meet again all the Igalians and friends. Moreover, this time was a bit special since we celebrated our 15th anniversary :-)

Photo by Alberto Garcia licensed under CC BY-SA 2.0.

I also attended the third edition of the Web Engines Hackfest, sponsored by Igalia, Collabora and Mozilla. This year, we had various participants from the Web Platform including folks from Apple, Collabora, Google, Igalia, Huawei, Mozilla or Red Hat. For my first hackfest as an Igalian, I invited some experts on fonts & math rendering to collaborate on OpenType MATH support HarfBuzz and its use in math rendering engines. In this blog post, I am going to focus on the work I have made with Behdad Esfahbod and Khaled Hosny. I think it was again a great and productive hackfest and I am looking forward to attending the next edition!

OpenType MATH in HarfBuzz

Photo by @webengineshackfest licensed under CC BY-SA 2.0.

Behdad gave a talk with a nice overview of the work accomplished in HarfBuzz during ten years. One thing appearing recently in HarfBuzz is the need for APIs to parse OpenType tables on all platforms. As part of my job at Igalia, I had started to experiment adding support for the MATH table some months ago and it was nice to have Behdad finally available to review, fix and improve commits.

When I talked to Mozilla employee Karl Tomlinson, it became apparent that the simple shaping API for stretchy operators proposed in my blog post would not cover all the special cases currently implemented in Gecko. Moreover, this shaping API is also very similar to another one existing in HarfBuzz for non-math script so we would have to decide the best way to share the logic.

As a consequence, we decided for now to focus on providing an API to access all the data of the MATH table. After the Web Engines Hackfest, such a math API is now integrated into the development repository of HarfBuzz and will available in version 1.3.3 :-)

MathML in Web Rendering Engines

Currently, several math rendering engines have their own code to parse the data of the OpenType MATH table. But many of them actually use HarfBuzz for normal text shaping and hence could just rely on the new math API the math rendering too. Before the hackfest, Khaled already had tested my work-in-progress branch with libmathview and I had done the same for Igalia’s Chromium MathML branch.

MathML test for OpenType MATH Fraction parameters in Gecko, Blink and WebKit.

Once the new API landed into HarfBuzz, Khaled was also able to use it for the XeTeX typesetting system. I also started to experiment this for Gecko and WebKit. This seems to work pretty well and we get consistent results for Gecko, Blink and WebKit! Some random thoughts:

The math data is exposed through a hb_font_t which contains the text size. This means that the various values are now directly resolved and returned as a fixed-point number which should allow to avoid rounding errors we may currently have in Gecko or WebKit when multiplying by float factors.
HarfBuzz has some magic to automatically handle invalid offsets and sizes that greatly simplifies the code, compared to what exist in Gecko and WebKit.
Contrary to Gecko’s implementation, HarfBuzz does not cache the latest result for glyph-specific data. Maybe we want to keep that?
The WebKit changes were tested on the GTK port, where HarfBuzz is enabled. Other ports may still need to use the existing parsing code from the WebKit tree. Perhaps Apple should consider adding support for the OpenType MATH table to CoreText?

Brotli/WOFF2/OTS libraries

Photo by @webengineshackfest licensed under CC BY-SA 2.0.

We also updated the copies of WOFF2 and OTS libraries in WebKit and Gecko respectively. This improves one requirement from the WOFF2 specification and allows to pass the corresponding conformance test.

Gecko, WebKit and Chromium bundle their own copy of the Brotli, WOFF2 or OTS libraries in their source repositories. However:

We have to use more or less automated mechanisms to keep these bundled copies up-to-date. This is especially annoying for Brotli and WOFF2 since they are still in development and we must be sure to always integrate the latest security fixes. Also, we get compiler warnings or coding style errors that do not exist upstream and that must be disabled or patched until they are fixed upstream and imported again.
This obviously is not an optimal sharing of system library and may increase the size of binaries. Using shared libraries is what maintainers of Linux (or other FLOSS systems) generally ask and this was raised during the WebKitGTK+ session. Similarly, we should really use the system Brotli/WOFF2 bundled in future releases of Apple’s operating systems.

There are several issues that make hard for package maintainers to provide these libraries: no released binaries or release tags, no proper build system to generate shared libraries, use of git submodule to include one library source code into another etc Things have gotten a bit better for Brotli and I was able to tweak the CMake script to produce shared libraries. For WOFF2, issue 40 and issue 49 have been inactive but hopefully these will be addressed in the future…

MathML Improvements in WebKit

2016-07-13T00:00:00+02:00

In a previous blog post, I explained the work made by Igalia’s web platform team to refactor WebKit’s MathML layout classes. I stated that although some rendering improvements were a nice side effect, the main goal of the first phase was really to clean the code up so that it is easier for developers to work on MathML in the future. Indeed this really made things easier to review: Quite unexpectedly to me, the second phase only took 4 days to be upstreamed… Kudos to Brent Fulgham for having reviewed so many patches in such a short period of time!

In this blog post, I am going to give an overview of the improvements made during these two first phases taking changeset r203109 as a reference. The changes will be available in WebKitGTK+ 2.14 in September and are likely to be included this month in the next Safari Technology Preview. It definitely remains more work to do such as the third phase or other rendering improvements, but I believe we have already made a big step forward!

Mathematical Fonts

Two years ago, basic support for operator stretching via the OpenType MATH table was added to WebKit. During the refactoring, we improved that support and also made use of more parameters to improve the math layout (see section about OpenType MATH parameters below). While Latin Modern Math will be used in most screenshots, the following one shows that you can use any math fonts. By default WebKit will try and use one of these fonts but if none are available or if you force a non-math font-family then the rendering quality may not be good.

The following screenshot gives the rendering for various fonts. For the last one we used the value sans-serif to illustrate issues with non-math fonts (displaystyle integral too small, mathvariant italic glyphs taken from another font, missing italic correction etc).

Integral obtained by contour integration
WebKitGTK+ r203109 with various font-family values.
From top to bottom: TeX Gyre Schola, Latin Modern, DejaVu Math, STIX, and sans-serif.

The `href` Attribute

This new feature is obvious: You can now create a hyperlink for any part of a mathematical formula! Here is a screenshot of the MathML Torture Test 21 with additional links, as displayed in WebKit r203109. Click the image to load the test case in your browser and test it.

The `mathvariant` Attribute

Unicode contains Mathematical Alphanumeric Symbols to convey special meaning such as double-struck or specific Arabic styles. Characters for these symbols are generally provided by math fonts. In MathML, mathematical variables are automatically rendered using the italic characters from this Unicode block. One can also access these characters via the mathvariant attribute and that attribute is actually used by many LaTeX-to-MathML converters.

In the following screenshot, you can see that the letters f, x and y are now drawn with this special mathematical italic glyphs and that WebKit uses the conventional fraktur style for the Lie algebra g. Note that the prime is still too small because WebKit does not make use of the ssty feature yet.

Homomorphism of Lie algebra
Top: Safari 9.1.1. Bottom: Safari r203109.

Operators and Spacing

As said in my previous blog post, the rendering of large and stretchy operators have been rewritten a lot and as a consequence the rendering has improved. Also, I mentioned that the width of operators may depend on their height. This may cause accumulated approximations during the computation of preferred widths. The old flexbox-based implementation incorrectly forced layout during preferred computation to avoid that but a quick workaround for that security concern caused the approximate preferred widths to be used for the logical widths. With our new implementation, the logical width is now correctly calculated. Finally, we added partial support for the mpadded element which is often used to tweak spacing in mathematical formulas.

The screenshot below illustrates the fix for a serious regression with large operator (summation symbol) as well as improvements in the rendering of stretchy operators (horizontal braces). Note that the formula has a hack with a zero-width mpadded element which used to cause improper spacing (large gap between the group of a’s and the group of b’s).

Tests 21 and 22 from the MathML torture test
Left: Safari 9.1.1. Right: Safari r203109.

The following screenshot shows how incorrect width computations used to cause excessive spacing after the stretchy radical and slash symbols:

Sine of 1 degree
Top: Safari 9.1.1. Bottom: Safari r203109.

The `displaystyle` Property

Mathematical formulas can be integrated inside a paragraph of text (inline math in TeX terminology) or displayed in its own horizontally centered paragraph (display math in TeX terminology). In the latter case, the formula is in displaystyle and does not have any restrictions on vertical spacing. In the former case, the layout of the mathematical formula is modified a bit to optimize this vertical spacing and to better integrate within the surrounding text. The displaystyle property can also be set using the corresponding attribute or can change automatically in subformulas (e.g. in fractions).

In the following screenshot the fix for the large operator regression is obvious but you can also notice that the summation is now slightly different for the definition of a Bézier curve (top) and for the one of a rational Bézier curve (bottom). For example, to save some vertical space in the fractions, the sigma symbol is drawn smaller and the scripts attached to it are moved on its right. However, the script size could still be improved when we implement the scriptlevel property.

Definitions of Bézier Curve
Left: Safari 9.1.1. Right: Safari r203109.

OpenType MATH Parameters

Many new OpenType MATH features have been implemented following the MathML in HTML5 Implementation Note:

Use of the AxisHeight parameter to set vertical position of fractions, tables and symmetric operators.
Use of layout constants for radicals (some of them were already used), scripts and fractions. This improves math spacing and positioning and allows to adjust them according to value of the displaystyle property discussed in the previous section.
Use of the italic correction of large operator glyph to set the position of subscripts.

The screenshots below illustrate some of these improvements. For the first one, the use of AxisHeight allows to better align the fraction bar with the plus sign. For the second one, the use of layout constants for scripts as well as the italic correction of the integral improves the placement of limits.

Generalized Continued Fraction
Top: Safari 9.1.1. Bottom: Safari r203109.

Definition of the Gamma Function
Top: Safari 9.1.1. Bottom: Safari r203109.

Right-to-left radicals

One of the advantage of the old flexbox-based implementation is that right-to-left layout was available for free. This support has of course been preserved in the new implementation. We also added a simple workaround to mirror radicals using a scale transform as shown in the screenshot below. However, general glyph-level mirroring is still missing.

Test 13 from the MathML torture test (Machrek Style)
Top: Safari 9.1.1. Bottom: Safari r203109.

Conclusion

Igalia’s web platform team has been able to follow the MathML in HTML5 Implementation Note in order to significantly improve the rendering of mathematical expressions in WebKit. More work remains to do but we will definitely appreciate any feedback that can help improving native MathML support in web engines. We are also excited to continue work and collaboration at the next Web Engines Hackfest!

MathML Refactoring in WebKit

2016-07-09T00:00:00+02:00

Introduction

If you follow WebKit developments, you are certainly aware that Igalia has been working on WebKit’s MathML implementation for some time. More recently, effort has been made to write a clean implementation addressing issues reported by WebKit reviewers in the past. After joining Igalia in March, I have been in charge of getting this work reviewed and merged into WebKit’s development branch. In the past four months, we have been successful in upstreaming the first phase of the refactoring and the work accomplished is described in this blog post.

MathML torture tests with the Latin Modern Math font
Left: Safari 9.1.1. Right: Current MathML branch.

Note that the focus was on code refactoring so the improvement may not be obvious for non-developers. Nevertheless many issues have already been fixed as a consequence of that work: math italic, position of scripts, stretchy and large operators, rendering update and more. More importantly, this preliminary step opens the way for beautiful math rendering based on TeX rules and the OpenType MATH table. Rendering improvements and implementation of new features have already started in the next phase of the refactoring, so stay tuned…

Design Issues

As explained in a previous report, the main design issues in the flexbox-based implementation released in 2013 can essentially be summarized in three points:

WebKit’s code to stretch operator was not efficient at all and was limited to some basic fences buildable via Unicode characters.
WebKit’s MathML code violated many layout invariants, making the code unreliable.
WebKit’s MathML code relied heavily on the C++ renderer classes for flexboxes and had to manage too many anonymous renderers.

For the first point, the performance issue had been fixed by Igalia developers right after the initial feedback from WebKit developers and we improved that again during our refactoring. Partial support for the OpenType MATH table was added during my crowdfunding project and allowed to stretch more operators with the help of math fonts. For the second point, the main issue was also fixed right after the initial feedback. However one could still have some doubts regarding the layout steps, given the complexity implied by the third point. That last issue was still problematic so far and addressing it was the main achievement of our refactoring.

Technically, the dependence on flexbox is unnecessary and the implementation actually only used a limited set of flexbox features. Thus executing the whole flexbox code was overkill. It can also be a burden for people working on other places of the layout code. For example Javi Fernández has worked on improving the box alignments in the past and he had hard time fixing the MathML code impacted by his changes. This is probably the cause of the bad position of the summation symbol that can be seen in the screenshot above.

From the layout perspective, most of the rendering logic was implemented in the flexbox classes and the MathML “renderer” classes were really just managing the creation and update of anonymous nodes and style. Although this sounds good code reuse it actually made impossible to understand how and when layout steps happen or to add more advanced features. The new implementation replaced this manipulation of the render tree with simple arithmetic calculations on box metrics which is safer and more reliable. Also, complex renderers such as RenderMathMLScripts or RenderMathMLRoot actually achieve better rendering quality with less code!

As an example of the complexity, RenderMathMLUnderOver can behave as a RenderMathMLScripts in some situation so we really want the former class to reuse the latter class. As we will see below the old implementation of the two renderers were quite different: RenderMathMLUnderOver only relied on setting column direction in the user agent stylesheet while RenderMathMLScripts created a complex render tree structures with anonymous style. Hence it seemed difficult to share the two cases or to handle DOM changes causing to move from one case to the other one. With our new implementation, this is simply reduced to simple C++ inheritance.

When I started to work on WebKit some years ago, I made the mistake of continuing with the existing approach. The implementation of multiscripts or automatic italic mathvariant added more anonymous objects and made the situation even worse. After the end of my crowdfunding project, Alex Castro did more cleanup and tried to implement important features such as displaystyle but he also soon realized that it was too hard to work with the current code base…

Layout Refactoring

In order to solve the issues discussed in the previous section, Javi and Alex worked on a new MathML branch where the first step was to remove the inheritance on the flexbox layout classes. During the Web Engines Hackfest 2015, I collaborated with the Igalia’s web platform team team to continue the work on this branch. In a second step, we rewrote many MathML renderer functions so that they stop creating anonymous nodes or style. We obtained very encouraging results: The implementation looked much simpler and much more understandable!

Alex announced the initial plan on the webkit-dev mailing list. He started opening bugs and attaching patches to merge the first step. However, that step still required many of the flexbox logic and so made code hard to understand for reviewers. Hence when I joined Igalia four months ago Alex asked me to try and see how to reorganize patches so that the two initial steps can be submitted in one go. This corresponds to the first phase mentioned in the introduction. As indicated on the wiki page, the layout refactoring consisted in rewriting the following member functions of each renderer class:

computePreferredLogicalWidths: calculate preferred widths, based on the preferred widths of child renderers.
layoutBlock: set final position and size of child renderers.
firstLineBaseLine: calculate the ascent of the renderer.
paint (optional): perform special painting such as fraction bars.

Refactored renderers no longer rely on any flexbox code nor anonymous renderers and the functions mentioned above essentially perform arithmetic computations. By reading the code, one can be sure that we follow standard layout rules and that we do not perform unnecessary reflow. Moreover, the rules specific to math rendering are only located in the MathML renderers and can be better understood. Details for each class are provided in the next subsections. After all the layout functions were rewritten and the code managing the render tree structure removed, we were able to make the RenderMathMLBlock class inherit from RenderBlock instead of RenderFlexibleBox. Many of the bugs could then be immediately closed or otherwise fixed with small follow-up patches.

Spacing

RenderMathMLSpace is a simple class inserting blank boxes for adjusting spacing of math formulas. Obviously, we do not need any of the complexity of flexbox so it was straightforward to write the layout functions.

3 x

Large space between 3 and x.

Grouping

RenderMathMLRow performs rendering of a row of math items. Since WebKit does not support linebreaking in MathML at the moment, this is just putting child boxes on a same baseline. One specificity is that some operators can be stretched vertically and so their width may depend on their height.

{\frac{2}{x} x^{3}

Row containing a stretched brace, a fraction and a scripted element.

Again, flexbox features are useless here. With the old code, it was not clear whether we were violating the CSS invariant with preferred and logical widths and which kind of relayout or render tree changes would happen when doing the stretch call. By properly implementing the layout functions previously mentioned all of this became much more trustable.

Fractions

RenderMathMLFraction draws a fraction with numerator and denominator.

\frac{x + 1}{y + 2}

Simple fraction.

This used to be implemented using a column direction for the fraction element. Numerator and denominator were wrapped into anonymous nodes with additional style to leave space for the fraction bar and to adjust the horizontal alignments.

RenderMathMLFraction (flexbox with column direction)
  RenderMathMLBlock (anonymous flexbox)
    RenderMathMLRow (numerator)
      ...
  RenderMathMLBlock (anonymous flexbox)
    RenderMathMLRow (denominator)
      ...

It was relatively easy to implement this without any anonymous nodes and again the use of flexbox did not sound justified. For example, to calculate the preferred width we just take the maximum of the preferred widths of the numerator and denominator. For the layout, the calculation of the logical width is similar and we calculate the horizontal coordinates of numerator and denominator so that their centers are aligned. Vertical metrics are similarly calculated from the vertical metrics of the numerator and denominator. During that step, we also fixed some bugs with the linethickness attribute and added support for some OpenType MATH table constants.

Scripts above and below

RenderMathMLUnderOver is used to attach some scripts above and below a base. Each child can itself be a horizontal stretchy operator.

\vec{base}

Base with stretchy arrow over it.

This was implemented in the user agent stylesheet by using flexboxes with column direction for the corresponding MathML elements and the C++ class had additional rules to fire the stretching. So the problems and solutions for this class were essentially a mixed of the cases of RenderMathMLFraction and RenderMathMLRow we just discussed.

Subscripts and Superscripts

RenderMathMLScripts is used for a base with some arbitrary number of scripts. All the scripts can have different positions (pre, post, sub, super) and metrics (width, ascent and descent). We must avoid collisions and take care of horizontal and vertical alignements.

{}^{d}_{e}^{f}{base}_{a}^{b}_{c}

Base with pre and post scripts.

The old code used a complex render tree with additional style to achieve the best possible result. However, the quality was still bad as you can see for the script attached to the integral in the screenshot above. Managing the render tree was a nightmare: Just to give the idea, additional anonymous node and style were used to allow horizontal and vertical adjustments (similar to RenderMathMLFraction above) and prescripts had negative order property so that they were positioned before the base.

RenderMathMLScripts
    Base Wrapper (anonymous flexbox)
      RenderMathMLRow (base)
        ...
    SubSupPair Wrapper (anonymous flexbox with column direction)
      RenderMathMLRow (post-subscript)
        ...
      RenderMathMLRow (subperscript)
        ...
    SubSupPair Wrapper (anonymous flexbox with column direction)
      RenderMathMLRow (post-subscript)
        ...
      RenderMathMLRow (post-superscript)
        ...
    ... (more postscripts)
    RenderMathMLBlock (prescripts separator)
    SubSupPair Wrapper (anonymous flexbox with column direction and order -1)
      RenderMathMLRow (pre-subscript)
        ...
      RenderMathMLRow (pre-subperscript)
        ...
    SubSupPair Wrapper (anonymous flexbox with column direction and order -1)
      RenderMathMLRow (pre-subscript)
        ...
      RenderMathMLRow (pre-superscript)
        ...
    ... (more prescripts)

Rules from TeX and the OpenType MATH table are relatively complex and we decided to implement them directly in the new refactoring as otherwise it was impossible to get decent quality. The code is still complex but we now have clear rules, we only perform simple calculations and the render tree structure matches the DOM tree.

“Enclosing” Notations

RenderMathMLMenclose is a row of math items with some additional notations. Gurpreet Kaur implemented this element two years ago but she followed the same approch, combining anonymous nodes and style for some simple notations and special painting for others.

x + 1

circle and strike notations

During the refactoring, the code has been completely rewritten so that RenderMathMLMenclose is now essentially a derived class of RenderMathMLRow with the measuring and painting functions adjusted to take into account the additional notations. During that refactoring, we also removed support for unused radical notation, which was implemented using an anonymous RenderMathMLSquareRoot (see Radicals section below).

Helper Classes for Operators

The RenderMathMLOperator class is used for math operators. It was quite complex class and we decided to extract from it two features that are unrelated to layout:

The MathML operator dictionary and corresponding search functions have been moved into a MathOperatorDictionary class.
Selection, measuring and drawing of stretchy operators have been moved into a MathOperator class. This is essentially the low-level text shaping being implemented in HarfBuzz.

The remaining code was indeed the real layout part but the mess with anonymous node and style was only removed later (see Text Classes below). Although it seems we just needed to move the code out of RenderMathMLOperator into those new classes, the case of MathOperator was particularly difficult. We had to split the effort into several small steps to make review possible and also fixed many issues due to the entanglement and confusion of these three different features of the RenderMathMLOperator class… The work done for MathOperator actually improved the rendering of stretchy operators as you can see for the horizontal braces in the screenshot above.

Radicals

RenderMathMLRoot is used for square root or arbitrary N-th root. Many of the TeX and OpenType MATH table rules were already used by the old implementation with anonymous nodes and style. However, there were bugs difficult to fix related to zooming, child removal or style change due to the management of the anonymous RenderMathMLOperator to draw the radical sign.

\sqrt{x + 1} + \sqrt[3]{x + 1}

square and cube roots

The old implementation actually had two classes for the square and general cases (RenderMathMLSquareRoot and RenderMathMLRoot). The usual technique with various anonymous wrappers and style was used. After the refactoring, we were able to merge everything in a single RenderMathMLRoot class. Because the square root behaves as an mrow, we also made that class derive from RenderMathMLRow to reuse as much code as possible. Here is are how the render trees used to look like:

RenderMathMLSquareRoot
  RenderMathMLBlock (anonymous used for metric adjustements)
    RenderMathMLRadicalOperator (anonymous used for the radical symbol)
      ...
  RenderMathMLRootWrapper (anonymous used for the children)
    RenderMathMLRow (child 1)
      ...
    RenderMathMLRow (child 2)
      ...
    ...
    RenderMathMLRow (child N)
      ...

RenderMathMLRoot
  RenderMathMLRootWrapper (anonymous for the index)
    ...
  RenderMathMLBlock (anonymous used for metric adjustements)
     RenderMathMLRadicalOperator (anonymous used for the radical symbol)
       ...
  RenderMathMLRootWrapper (anonymous for the base)
    ...

Again, we rewrote the implementation using only simple box positioning. The difficult part was to get rid of the anonymous RenderMathMLRadicalOperator to draw the radical symbol. This class was derived from RenderMathMLOperator and extended it with some fallback drawing when math fonts were not available. After having extracted stretchy operator shaping from RenderMathMLOperator it became possible to use the MathOperator helper class to draw the radical symbol. We implemented the fallback for missing math fonts the same as Gecko: Use a scale transform to stretch the base glyph for U+221A SQUARE ROOT. As a bonus, we used such transform to implement glyph mirroring, as required to draw right-to-left radicals in some Arabic mathematical notations.

Text Classes

These classes are containers for math text such as variables or operators. There is a generic RenderMathMLToken class and a derived class RenderMathMLOperator adding features specific to operators such as spacing, dictionary property, stretching… Anonymous wrappers and style were used to implement automatic italic mathvariant or operator spacing. The RenderText child of RenderMathMLOperator was (re)built as an anonymous text node so that is was possible to convert U+002D HYPHEN-MINUS into U+2212 MINUS SIGN or to provide some text for anonymous operators created by RenderMathMLFenced (see Unchanged Classes section).

RenderMathMLToken (e.g. mi element)
  RenderMathMLBlock (anonymous flexbox used to apply CSS italic)
    RenderBlock (anonymous created elsewhere to honor CSS rules)
      RenderText
        text run "x"

RenderMathMLOperator (mo element)
   RenderMathMLBlock (anonymous flexbox used for spacing)
    RenderBlock (anonymous created elsewhere to honor CSS rules)
      RenderText (anonymous destroyed and built again)
         text run "−"

We did a big refactoring to remove all the anonymous nodes created by the MathML renderer classes. Just like for MathOperator, we had to be careful and submit various small pieces as the text rendering was quite sensible to code change.

The simplified operator spacing that was supported by WebKit was easy to implement with the new approach. To do automatic italic mathvariant, we modified the paint function to use Mathematical Alphanumeric Symbols instead of CSS italic as you can notice for the variables displayed in the screenshot above. Hence we could remove the RenderMathMLBlock anonymous wrapper.

The use of an anonymous node for the text prevented it to appear in the dumped render tree of layout tests and also required some hacks in the accessibility code to expose that text. In order to address the cases of the minus sign and of mfenced operators, we decided to use our new MathOperator class again. Indeed MathOperator is actually also able to draw unstretched operators made of a single character and this works for the minus sign and for mfenced operators used in practice.

Unchanged Classes

Two classes have not been modified but such modifications were not needed to remove the dependency on RenderFlexibleBox:

RenderMathMLFenced is used for an mrow-like element that is defined in the MathML specification as strictly equivalent to constructions with rows and operators. It is implemented as a derived class of RenderMathMLRow and creates anonymous RenderMathMLOperators. This is the only remaining class that modifies the render tree structure. Note that prominent MathML websites and generators do not use the mfenced element, so it is not a big concern.
RenderMathMLTable is used for table layout. It is just derived from RenderTable, not RenderFlexibleBox. We did not change anything for now but we considered creating our own implementation in order to make our code independent from HTML table, to support MathML-specific table features and to make it better integrated with the rest of the MathML code.

Accessibility

Even if our main focus was on rendering, the changes we made also had impact on the MathML accessibility code. Indeed, the accessibility tree is generated from the MathML renderer classes: Since we changed the latter during the refactoring, we also had to adjust the accessibility code. Fortunately, we are lucky to have Joanmarie Diggs in our team and she was able to provide some help here.

First, the accessibility code exposes the linethickness of fractions to implement Apple’s AXMathLineThickness attribute. In practice, this is really necessary to know whether the linethickness is null or not (e.g. binomial coefficient VS the Legendre symbol). Apple’s unit test seemed to expose the ratio between the actual thickness and the default thickness but the accessibility code really just reads the actual thickness calculated by RenderMathMLFraction. Our fix and improvement for linethickness made the Apple’s unit test fail so we had to adjust RenderMathMLFraction to expose the value expected by that test.

In general, the accessibility code does not care about anonymous nodes created for layout purpose and there was some code to avoid exposing them in the accessibility tree. So removing all the anonymous during the layout refactoring was actually a good and safe thing to do. There were some helper functions to implement Apple’s AXMathRootRadicand and AXMathRootIndex attributes that had to be adjusted, though. These functions used to do some work to skip the anonymous wrappers and we were actually able to simplify them.

There was also some specific code for the RenderMathMLOperators and their anonymous RenderText that were necessary to expose the text content. Actually, there was an old bug in the accessibility code and the anonymous RenderMathMLOperators created by mfenced were not correctly exposed. The unit test we had for mfenced operators was only checking the text content but it was still passing and so the regression had never been detected before. After the layout refactoring we removed the anonymous RenderText of mfenced operators and so broke that test… We thus spent some time to fix the RenderMathMLOperator code. Essentially, we removed all the old hacks and only left a specific handling for mfenced operators. We also used this opportunity to improve and extend our MathML accessibility tests.

Finally, the MathML accessibility code was directly implemented into a generic AccessibilityRenderObject class. There was some functions to access math nodes and properties but also specific cases scattered all over the code (anonymous boxes, mfenced operators, math roles etc). In order to facilitate future work and maintenance we decided to move all the MathML code into a new AccessibilityMathMLElement class. Hence the work implied by the layout refactoring actually encouraged us to improve the organization and testing of our accessibility code!

Conclusion

In the past four months, Igalia’s web platform team has successfully upstreamed the refactoring of WebKit’s MathML renderer classes and we are now very confident about the quality of the layout code. In addition to the people mentioned above I would personally like to thank everybody who helped with this work. More specifically, I am very grateful to other people from Igalia (Martin Robinson, Sergio Villar and Manuel Rego) or Apple (Brent Fulgham and Darin Adler) who have spent some time to review patches. As a nice side effect of this work, mathematical formulas look better and the accessibility code has been improved. More is happenning in the next two phases. We are looking forward to continuing implementation of Web standards and collaboration with browser vendors at the next Web Engines Hackfest!

OpenType MATH in HarfBuzz

2016-04-16T00:00:00+02:00

TL;DR:

•

Work is in progress to add OpenType MATH support in HarfBuzz and will be instrumental for many math rendering engines relying on that library, including browsers.
•

For stretchy operators, an efficient way to determine the required number of glyphs and their overlaps has been implemented and is described here.

In the context of Igalia browser team effort to implement MathML support using TeX rules and OpenType features, I have started implementation of OpenType MATH support in HarfBuzz. This table from the OpenType standard is made of three subtables:

•

The MathConstants table, which contains layout constants. For example, the thickness of the fraction bar of $\frac{a}{b}$ .
•

The MathGlyphInfo table, which contains glyph properties. For instance, the italic correction indicating how slanted an integral is e.g. to properly place the subscript in $\displaystyle\displaystyle\int_{D}$ .
•

The MathVariants table, which provides larger size variants for a base glyph or data to build a glyph assembly. For example, either a larger parenthesis or a assembly of U+239B, U+239C, U+239D to write something like:

$\left(\frac{\frac{\frac{a}{b}}{\frac{c}{d}}}{\frac{\frac{e}{f}}{\frac{g}{h}}}\right.$

Code to parse this table was added to Gecko and WebKit two years ago. The existing code to build glyph assembly in these Web engines was adapted to use the MathVariants data instead of only private tables. However, as we will see below the MathVariants data to build glyph assembly is more general, with arbitrary number of glyphs or with additional constraints on glyph overlaps. Also there are various fallback mechanisms for old fonts and other bugs that I think we could get rid of when we move to OpenType MATH fonts only.

In order to add MathML support in Blink, it is very easy to import the OpenType MATH parsing code from WebKit. However, after discussions with some Google developers, it seems that the best option is to directly add support for this table in HarfBuzz. Since this library is used by Gecko, by WebKit (at least the GTK port) and by many other applications such as Servo, XeTeX or LibreOffice it make senses to share the implementation to improve math rendering everywhere.

The idea for HarfBuzz is to add an API to

1.

Expose data from the MathConstants and MathGlyphInfo.
2.

Shape stretchy operators to some target size with the help of the MathVariants.

It is then up to a higher-level math rendering engine (e.g. TeX or MathML rendering engines) to beautifully display mathematical formulas using this API. The design choice for exposing MathConstants and MathGlyphInfo is almost obvious from the reading of the MATH table specification. The choice for the shaping API is a bit more complex and discussions is still in progress. For example because we want to accept stretching after glyph-level mirroring (e.g. to draw RTL clockwise integrals) we should accept any glyph and not just an input Unicode strings as it is the case for other HarfBuzz shaping functions. This shaping also depends on a stretching direction (horizontal/vertical) or on a target size (and Gecko even currently has various ways to approximate that target size). Finally, we should also have a way to expose italic correction for a glyph assembly or to approximate preferred width for Web rendering engines.

As I mentioned at the beginning, the data and algorithm to build glyph assembly is the most complex part of the OpenType MATH and deserves a special interest. The idea is that you have a list of $n\geq 1$ glyphs available to build the assembly. For each $0\leq i\leq n-1$ , the glyph $g_{i}$ has advance $a_{i}$ in the stretch direction. Each $g_{i}$ has straight connector part at its start (of length $s_{i}$ ) and at its end (of length $e_{i}$ ) so that we can align the glyphs on the stretch axis and glue them together. Also, some of the glyphs are “extenders” which means that they can be repeated 0, 1 or more times to make the assembly as large as possible. Finally, the end/start connectors of consecutive glyphs must overlap by at least a fixed value $o_{\mathrm{min}}$ to avoid gaps at some resolutions but of course without exceeding the length of the corresponding connectors. This gives some flexibility to adjust the size of the assembly and get closer to the target size $t$ .

Figure 1: Two adjacent glyphs in an assembly

To ensure that the width/height is distributed equally and the symmetry of the shape is preserved, the MATH table specification suggests the following iterative algorithm to determine the number of extenders and the connector overlaps to reach a minimal target size $t$ :

1.

Assemble all parts by overlapping connectors by maximum amount, and removing all extenders. This gives the smallest possible result.
2.

Determine how much extra width/height can be distributed into all connections between neighboring parts. If that is enough to achieve the size goal, extend each connection equally by changing overlaps of connectors to finish the job.
3.

If all connections have been extended to minimum overlap and further growth is needed, add one of each extender, and repeat the process from the first step.

We note that at each step, each extender is repeated the same number of times $r\geq 0$ . So if $I_{\mathrm{Ext}}$ (respectively $I_{\mathrm{NonExt}}$ ) is the set of indices $0\leq i\leq n-1$ such that $g_{i}$ is an extender (respectively is not an extender) we have $r_{i}=r$ (respectively $r_{i}=1$ ). The size we can reach at step $r$ is at most the one obtained with the minimal connector overlap $o_{\mathrm{min}}$ that is

\sum_{i=0}^{N-1}\left(\sum_{j=1}^{r_{i}}{a_{i}-o_{\mathrm{min}}}\right)+o_{ \mathrm{min}}=\left(\sum_{i\in I_{\mathrm{NonExt}}}{a_{i}-o_{\mathrm{min}}} \right)+\left(\sum_{i\in I_{\mathrm{Ext}}}r{(a_{i}-o_{\mathrm{min}})}\right)+o% _{\mathrm{min}}

We let $N_{\mathrm{Ext}}={|I_{\mathrm{Ext}}|}$ and $N_{\mathrm{NonExt}}={|I_{\mathrm{NonExt}}|}$ be the number of extenders and non-extenders. We also let $S_{\mathrm{Ext}}=\sum_{i\in I_{\mathrm{Ext}}}a_{i}$ and $S_{\mathrm{NonExt}}=\sum_{i\in I_{\mathrm{NonExt}}}a_{i}$ be the sum of advances for extenders and non-extenders. If we want the advance of the glyph assembly to reach the minimal size $t$ then

{S_{\mathrm{NonExt}}-o_{\mathrm{min}}\left(N_{\mathrm{NonExt}}-1\right)}+{r% \left(S_{\mathrm{Ext}}-o_{\mathrm{min}}N_{\mathrm{Ext}}\right)}\geq t

We can assume $S_{\mathrm{Ext}}-o_{\mathrm{min}}N_{\mathrm{Ext}}>0$ or otherwise we would have the extreme case where the overlap takes at least the full advance of each extender. Then we obtain

r\geq r_{\mathrm{min}}=\max\left(0,\left\lceil\frac{t-{S_{\mathrm{NonExt}}+o_{ \mathrm{min}}\left(N_{\mathrm{NonExt}}-1\right)}}{S_{\mathrm{Ext}}-o_{\mathrm{ min}}N_{\mathrm{Ext}}}\right\rceil\right)

This provides a first simplification of the algorithm sketched in the MATH table specification: Directly start iteration at step $r_{\mathrm{min}}$ . Note that at each step we start at possibly different maximum overlaps and decrease all of them by a same value. It is not clear what to do when one of the overlap reaches $o_{\mathrm{min}}$ while others can still be decreased. However, the sketched algorithm says all the connectors should reach minimum overlap before the next increment of $r$ , which means the target size will indeed be reached at step $r_{\mathrm{min}}$ .

One possible interpretation is to stop overlap decreasing for the adjacent connectors that reached minimum overlap and to continue uniform decreasing for the others until all the connectors reach minimum overlap. In that case we may lose equal distribution or symmetry. In practice, this should probably not matter much. So we propose instead the dual option which should behave more or less the same in most cases: Start with all overlaps set to $o_{\mathrm{min}}$ and increase them evenly to reach a same value $o$ . By the same reasoning as above we want the inequality

{S_{\mathrm{NonExt}}-o\left(N_{\mathrm{NonExt}}-1\right)}+{r_{\mathrm{min}} \left(S_{\mathrm{Ext}}-oN_{\mathrm{Ext}}\right)}\geq t

which can be rewritten

S_{\mathrm{NonExt}}+r_{\mathrm{min}}S_{\mathrm{Ext}}-{o\left(N_{\mathrm{NonExt% }}+{r_{\mathrm{min}}N_{\mathrm{Ext}}}-1\right)}\geq t

We note that $N=N_{\mathrm{NonExt}}+{r_{\mathrm{min}}N_{\mathrm{Ext}}}$ is just the exact number of glyphs used in the assembly. If there is only a single glyph, then the overlap value is irrelevant so we can assume $N_{\mathrm{NonExt}}+{rN_{\mathrm{Ext}}}-1=N-1\geq 1$ . This provides the greatest theorical value for the overlap $o$ :

o_{\mathrm{min}}\leq o\leq o_{\mathrm{max}}^{\mathrm{theorical}}=\frac{S_{ \mathrm{NonExt}}+r_{\mathrm{min}}S_{\mathrm{Ext}}-t}{N_{\mathrm{NonExt}}+{r_{ \mathrm{min}}N_{\mathrm{Ext}}}-1}

Of course, we also have to take into account the limit imposed by the start and end connector lengths. So $o_{\mathrm{max}}$ must also be at most $\min{(e_{i},s_{i+1})}$ for $0\leq i\leq n-2$ . But if $r_{\mathrm{min}}\geq 2$ then extender copies are connected and so $o_{\mathrm{max}}$ must also be at most $\min{(e_{i},s_{i})}$ for $i\in I_{\mathrm{Ext}}$ . To summarize, $o_{\mathrm{max}}$ is the minimum of $o_{\mathrm{max}}^{\mathrm{theorical}}$ , of $e_{i}$ for $0\leq i\leq n-2$ , of $s_{i}$ $1\leq i\leq n-1$ and possibly of $e_{0}$ (if $0\in I_{\mathrm{Ext}}$ ) and of of $s_{n-1}$ (if ${n-1}\in I_{\mathrm{Ext}}$ ).

With the algorithm described above $N_{\mathrm{Ext}}$ , $N_{\mathrm{NonExt}}$ , $S_{\mathrm{Ext}}$ , $S_{\mathrm{NonExt}}$ and $r_{\mathrm{min}}$ and $o_{\mathrm{max}}$ can all be obtained using simple loops on the glyphs $g_{i}$ and so the complexity is $O(n)$ . In practice $n$ is small: For existing fonts, assemblies are made of at most three non-extenders and two extenders that is $n\leq 5$ (incidentally, Gecko and WebKit do not currently support larger values of $n$ ). This means that all the operations described above can be considered to have constant complexity. This is much better than a naive implementation of the iterative algorithm sketched in the OpenType MATH table specification which seems to require at worst

\sum_{r=0}^{r_{\mathrm{min}}-1}{N_{\mathrm{NonExt}}+rN_{\mathrm{Ext}}}=N_{ \mathrm{NonExt}}r_{\mathrm{min}}+\frac{r_{\mathrm{min}}\left(r_{\mathrm{min}}-% 1\right)}{2}N_{\mathrm{Ext}}={O(n\times r_{\mathrm{min}}^{2})}

and at least $\Omega(r_{\mathrm{min}})$ .

One of issue is that the number of extender repetitions $r_{\mathrm{min}}$ and the number of glyphs in the assembly $N$ can become arbitrary large since the target size $t$ can take large values e.g. if one writes \underbrace{\hspace{65535em}} in LaTeX. The improvement proposed here does not solve that issue since setting the coordinates of each glyph in the assembly and painting them require $\Theta(N)$ operations as well as (in the case of HarfBuzz) a glyph buffer of size $N$ . However, such large stretchy operators do not happen in real-life mathematical formulas. Hence to avoid possible hangs in Web engines a solution is to impose a maximum limit $N_{\mathrm{max}}$ for the number of glyph in the assembly so that the complexity is limited by the size of the DOM tree. Currently, the proposal for HarfBuzz is $N_{\mathrm{max}}=128$ . This means that if each assembly glyph is 1em large you won’t be able to draw stretchy operators of size more than 128em, which sounds a quite reasonable bound. With the above proposal, $r_{\mathrm{min}}$ and so $N$ can be determined very quickly and the cases $N\geq N_{\mathrm{max}}$ rejected, so that we avoid losing time with such edge cases…

Finally, because in our proposal we use the same overlap $o$ everywhere an alternative for HarfBuzz would be to set the output buffer size to $n$ (i.e. ignore $r-1$ copies of each extender and only keep the first one). This will leave gaps that the client can fix by repeating extenders as long as $o$ is also provided. Then HarfBuzz math shaping can be done with a complexity in time and space of just $O(n)$ and it will be up to the client to optimize or limit the painting of extenders for large values of $N$ …

MathML at the Web Engines Hackfest 2015

2015-12-20T00:00:00+01:00

Hackfest

Two weeks ago, I travelled to Spain to participate to the second Web Engines Hackfest which was sponsored by Igalia and Collabora. Such an event has been organized by Igalia since 2009 and used to be focused on WebkitGTK+. It is great to see that it has now been extended to any Web engines & platforms and that a large percentage of non-igalian developers has been involved this year. If you did not get the opportunity to attend this event or if you are curious about what happened there, take a look at the wiki page or flickr album.

Photo from @webengineshackfest licensed under

I really like this kind of hacking-oriented and participant-driven event where developers can meet face to face, organize themselves in small collaboration groups to efficiently make progress on a task or give passionate talk about what they have recently been working on. The only small bemol I have is that it is still mainly focused on WebKit/Blink developments. Probably, the lack of Mozilla/Microsoft participants is probably due to Mozilla Coincidental Work Weeks happening at the same period and to the proprietary nature of EdgeHTML (although this is changing?). However, I am confident that Igalia will try and address this issue and I am looking forward to coming back next year!

MathML developments

This year, Igalia developer Alejandro G. Castro wanted to work with me on WebKit’s MathML layout code and more specifically on his MathML refactoring branch. Indeed, as many people (including Google developers) who have tried to work on WebKit’s code in the past, he arrived to the conclusion that the WebKit’s MathML layout code has many design issues that make it a burden for the rest of the layout team and too complex to allow future improvements. I was quite excited about the work he has done with Javier Fernández to try and move to a simple box model closer to what exists in Gecko and thus I actually extended my stay to work one whole week with them. We already submitted our proposal to the webkit-dev mailing list and received positive feedback, so we will now start merging what is ready. At the end of the week, we were quite satisfied about the new approach and confident it will facilitate future maintenance and developements :-)

Photo from @webengineshackfest licensed under

While reading a recent thread on the Math WG mailing list, I realized that many MathML people have only vague understanding of why Google (or to be more accurate, the 3 or 4 engineers who really spent some time reviewing and testing the WebKit code) considered the implementation to be unsafe and not ready for release. Even worse, Michael Kholhase pointed out that for someone totally ignorant of the technical implementation details, the communication made some years ago around the “flexbox-based approach” gave the impression that it was “the right way” (indeed, it somewhat did improve the initial implementation) and the rationale to change that approach was not obvious. So let’s just try and give a quick overview of the main problems, even if I doubt someone can have good understanding of the situation without diving into the C++ code:

WebKit’s code to stretch operator was not efficient at all and was limited to some basic fences buildable via Unicode characters.
WebKit’s MathML code violated many layout invariants, making the code unreliable.
WebKit’s MathML code relied heavily on the C++ renderer classes for flexboxes and has to manage too many anonymous renderers.

The main security concerns were addressed a long time ago by Martin Robinson and me. Glyph assembly for stretchy operators are now drawn using low-level font painting primitive instead of creating one renderer object for each piece and the preferred width for them no longer depends on vertical metrics (although we still need some work to obtain Gecko’s good operator spacing). Also, during my crowdfunding project, I implemented partial support for the OpenType MATH table in WebKit and more specifically the MathVariant subtable, which allows to directly use construction of stretchy operators specified by the font designer and not only the few Unicode constructions.

However, the MathML layout code still modifies the renderer tree to force the presence of anonymous renderers and still applies specific CSS rules to them. It is also spending too much time trying to adapt the parent flexbox renderer class which has at the same time too much features for what is needed for MathML (essentially automatic box alignment) and not enough to get exact placement and measuring needed for high-quality rendering (the TeXBook rules are more complex, taking into account various parameters for box shifts, drops, gaps etc).

During the hackfest, we started to rewrite a clean implementation of some MathML renderer classes similar to Gecko’s one and based on the MathML in HTML5 implementation note. The code now becomes very simple and understandable. It can be summarized into four main functions. For instance, to draw a fraction we need:

computePreferredLogicalWidths which sets the preferred width of the fraction during the first layout pass, by considering the widest between numerator and denominator.
layoutBlock and firstLineBaseline which calculate the final width/height/ascent of the fraction element and position the numerator and denominator.
paint which draws the fraction bar.

Perhaps, the best example to illustrate how the complexity has been reduced is the case of the renderer of mmultiscripts/msub/msup/msubsup elements (attaching an arbitrary number of subscripts and superscripts before or after a base). In the current WebKit implementation, we have to create three anonymous wrappers (a first one for the base, a second one for prescripts and a third one for postscripts) and an anonymous wrapper for each subscript/superscript pair, add alignment styles for these wrappers and spend a lot of time maintaining the integrity of the renderer tree when dynamic changes happen. With the new code, we just need to do arithmetic calculations to position the base and script boxes. This is somewhat more complex than the fraction example above but still, it remains arithmetic calculations and we can not reduce any further if we wish quality comparable to TeXBook / MATH rules. We actually take into account many parameters from the OpenType MATH table to get much better placement of scripts. We were able to fix bug 130325 in about twenty minutes instead of fighting with a CSS “negative margin” hack on anonymous renderers.

MathML dicussions

The first day of the hackfest we also had an interesting “breakout session” to define the tasks to work on during the hackfest. Alejandro briefly presented the status of his refactoring branch and his plan for the hackfest. As said in the previous section, we have been quite successful in following this plan: Although it is not fully complete yet, we expect to merge the current changes soon. Dominik Röttsches who works on Blink’s font and layout modules was present at the MathML session and it was a good opportunity to discuss the future of MathML in Chrome. I gave him some references regarding the OpenType MATH table, math fonts and the MathML in HTML5 implementation note. Dominik said he will forward the references to his colleagues working on layout so that we can have deeper technical dicussion about MathML in Blink in the long term. Also, I mentioned noto-fonts issue 330, which will be important for math rendering in Android and actually does not depend on the MathML issue, so that’s certainly something we could focus on in the short term.

Photo from @webengineshackfest licensed under

Alejandro also proposed to me to prepare a talk about MathML in Web Engines and exciting stuff happening with the MathML Association. I thus gave a brief overview of MathML and presented some demos of native support in Gecko. I also explained how we are trying to start a constructive approach to solve miscommunication between users, spec authors and implementers ; and gather technical and financial resources to obtain a proper solution. In a slightly more technical part, I presented Microsoft’s OpenType MATH table and how it can be used for math rendering (and MathML in particular). Finally, I proposed my personal roadmap for MathML in Web engines. Although I do not believe I am a really great speaker, I received positive feedback from attendees. One of the thing I realized is that I do not know anything about the status and plan in EdgeHTML and so overlooked to mention it in my presentation. Its proprietary nature makes hard for external people to contribute to a MathML implementation and the fact that Microsoft is moving away from ActiveX de facto excludes third-party plugin for good and fast math rendering in the future. After I went back to Paris, I thus wrote to Microsoft employee Christian Heilmann (previously working for Mozilla), mentioning at least the MathML in HTML5 Implementation Note and its test suite as a starting point. MathML is currently on the first page of the most voted feature requested for Microsoft Edge and given the new direction taken with Microsoft Edge, I hope we could start a discussion on MathML in EdgeHTML…

Conclusion

This was a great hackfest and I’d like to thank again all the participants and sponsors for making it possible! As Alejandro wrote to me, “I think we have started a very interesting work regarding MathML and web engines in the future.”. The short term plan is now to land the WebKit MathML refactoring started during the hackfest and to finish the work. I hope people now understand the importance of fonts with an OpenType MATH table for good mathematical rendering and we will continue to encourage browser vendors and font designers to make such fonts wide spread.

The new approach for WebKit MathML support gives good reason to be optmimistic in the long term and we hope we will be able to get high-quality rendering. The fact that the new approach addresses all the issues formulated by Google and that we started a dialogue on math rendering, gives several options for MathML in Blink. It remains to get Microsoft involved in implementing its own OpenType table in EdgeHTML. Overall, I believe that we can now have a very constructive discussion with the individuals/companies who really want native math support, with the Math WG members willing to have their specification implemented in browsers and with the browser vendors who want a math implementation which is good, clean and compatible with the rest of their code base. Hopefully, the MathML Association will be instrumental in achieving that. If everybody get involved, 2016 will definitely be an exciting year for native MathML in Web engines!

The canonical well-ordering of α×α (part 2)

2015-01-31T00:00:00+01:00

In a my previous blog post, I discussed the canonical well-ordering on $\alpha\times\alpha$ and stated theorem 0.5 below to calculate its order-type $\gamma(\alpha)$ . Subsequent corollaries provided a bound for $\gamma(\alpha)$ , its fixed points and a proof that infinite cardinals were among these fixed points (and so that cardinal addition and multiplication is trivial). In this second part, I’m going to provide the proof of this theorem.

First, we note that for all $n<\omega$ , there is only one order-type for $n\times n$ , since the notion of cardinal and ordinal is the same for finite sets. So indeed

\forall n<\omega,\gamma(n)={|n\times n|}=n^{2}

and taking the limit, we get our first infinite fixed point:

\gamma(\omega)=\sup_{n<\omega}{\gamma(n)}=\omega

For all $\alpha$ the ordering on ${(\alpha+1)}\times{(\alpha+1)}$ is as follows: first the elements of $\alpha\times\alpha$ ordered as $\gamma(\alpha)$ , followed by the elements $(\xi,\alpha)$ for $0\leq\xi<\alpha$ , followed by the elements $(\alpha,\xi)$ for $0\leq\xi\leq\alpha$ . Hence

\forall\alpha,{\gamma(\alpha+1)}={\gamma(\alpha)}+\alpha\cdot 2+1

From this, we can try to calculate the next values of $\gamma$ after $\omega$ :

{\gamma(\omega+1)}=\omega+{\omega\cdot 2}+1={\omega\cdot 3}+1

{\gamma(\omega+2)}={\omega\cdot 3+1}+\omega+1+\omega+1+1={\omega\cdot 5}+2

The important point is $1+\omega=\omega$ ; in general we will use several times the property $\alpha+\omega^{\beta}=\omega^{\beta}$ if $\alpha<\omega^{\beta}$ . By a simple recurrence (see proposition 0.1), we can generalize the expression to arbitrary $n$ :

{\gamma(\omega+n)}=\omega\left(2n+1\right)+n

and so taking the limit

{\gamma(\omega\cdot 2)}=\omega^{2}

which is a limit ordinal not fixed by $\gamma$ . We note another point that will be used later: taking the limit “eliminates” the smallest terms. More generally, we can perform the same calculation, starting from any arbitrary $\alpha\geq\omega$ :

Proposition 0.1.

For any limit ordinal $\alpha$ and $1\leq n<\omega$ we have

\gamma(\alpha+n)=\gamma(\alpha)+{\alpha\cdot{2n}}+n

Proof.

For $n=1$ this is the relation for $\gamma(\alpha+1)$ explained above. If the relation is true for $n$ then

\gamma(\alpha+n+1)=\gamma(\alpha+n)+{\left(\alpha+n\right)\cdot 2}+1

\gamma(\alpha+n+1)=\gamma(\alpha)+{\alpha\cdot{2n}}+n+\alpha+n+\alpha+n+1

and since $\alpha\geq\omega$ we have $n+\alpha=\alpha$ and so

\gamma(\alpha+n+1)=\gamma(\alpha)+{\alpha\cdot{(2n+1)}}+n+1

Which shows the result at step $n+1$ . ∎

As above, we can take the limit and say ${\gamma(\alpha+\omega)}=\sup_{n<\omega}{\gamma(\alpha+n)}=\gamma(\alpha)+% \alpha\cdot\omega$ which is consistent with ${\gamma(\omega\cdot 2)}=\omega+\omega^{2}=\omega^{2}$ . However, if we consider the Cantor Normal Form $\alpha=\omega^{\beta_{1}}n_{1}+...+\omega^{\beta_{k}}n_{k}$ , then for all $n<\omega$ we have can use the fact that “ $\omega^{\beta_{1}}$ will eliminate smaller terms on its left” that is $\alpha\cdot n=\omega^{\beta_{1}}{(n_{1}n)}+\omega^{\beta_{2}}n_{2}+...+\omega^% {\beta_{k}}n_{k}$ . Then using the fact that “taking the limit eliminates the smallest terms” we get $\alpha\cdot\omega=\sup_{n<\omega}\alpha\cdot n=\omega^{\beta_{1}+1}$ . So actually, we have a nicer formula where $\alpha\cdot\omega$ is put in Cantor Normal Form:

\forall\alpha\geq\omega,{\gamma(\alpha+\omega)}=\gamma(\alpha)+\omega^{\log_{% \omega}(\alpha)+1}

This can be generalized by the following proposition:

Proposition 0.2.

For any $\alpha\geq\omega$ and $\beta\geq 1$ such that $\log_{\omega}(\alpha)+1\geq\beta$ we have

{\gamma(\alpha+\omega^{\beta})}=\gamma(\alpha)+\omega^{\log_{\omega}(\alpha)+\beta}

Proof.

We prove by induction on $\beta$ that for all such $\alpha$ the expression is true. We just verified $\beta=1$ and the limit case is obvious by continuity of $\gamma$ and of the sum/exponentiation in the second variable. For the successor step, if $\log_{\omega}(\alpha)+1\geq\beta+1$ then a fortiori $\forall 1\leq n<\omega,\log_{\omega}(\alpha+\omega^{\beta}\cdot n)+1\geq\beta+% 1\geq\beta$ . We can then use the induction hypothesis to prove by induction on $1\leq n<\omega$ that ${\gamma(\alpha+{\omega^{\beta}\cdot n})}=\gamma(\alpha)+\omega^{\log_{\omega}(% \alpha)+\beta}\cdot n$ . For $n=1$ , this is just the induction hypothesis of $\beta$ (for the same $\alpha$ !). For the successor step, we need to use the induction hypothesis of $\beta$ (for $\alpha+\omega^{\beta}\cdot n$ ) which is ${\gamma(\alpha+{\omega^{\beta}\cdot{(n+1)}})}={\gamma(\alpha+{\omega^{\beta}% \cdot n})}+\omega^{\log_{\omega}(\alpha)+\beta}$ . Finally, ${\gamma(\alpha+{\omega^{\beta+1}})}={\sup_{1\leq n<\omega}{\gamma(\alpha+{% \omega^{\beta}\cdot n})}}=\gamma(\alpha)+\omega^{\log_{\omega}(\alpha)+\beta+1}$ , as wanted. ∎

For all $\alpha\geq 1$ , $\log_{\omega}(\omega^{\alpha})+1=\alpha+1$ so the previous paragraph also gives ${\gamma(\omega^{\alpha+1})}=\gamma\left(\omega^{\alpha}+\omega^{\alpha+1}% \right)={\gamma(\omega^{\alpha})}+\omega^{\alpha\cdot 2+1}$ . Then, we find

\gamma(\omega^{2})=\omega+\omega^{3}=\omega^{3}

\gamma(\omega^{3})=\omega^{3}+\omega^{5}=\omega^{5}

And more generally by induction on $n<\omega$ , we can show that

\gamma(\omega^{n+1})=\omega^{2n+1}

Then we deduce another fixed point

\gamma(\omega^{\omega})={\sup_{1\leq n<\omega}\gamma\left({\omega^{n}}\right)}% =\omega^{\omega}

The following proposition tries to generalize the expression of ${\gamma(\omega^{\alpha+1})}$ .

Proposition 0.3.

For any $\alpha,\beta\geq 1$ such that $\log_{\omega}(\alpha)>\log_{\omega}(\beta)$ we have

{\gamma(\omega^{\alpha+\beta})}={\gamma(\omega^{\alpha})}+\omega^{\alpha\cdot 2% +\beta}

Proof.

This is done by induction on $\beta<\alpha$ for a fixed $\alpha$ . We already verified the case $\beta=1$ in the previous paragraph and the limit case is obvious by continuity of $\gamma$ and of the sum/exponentiation in the second variable. For the successor step, we have ${\gamma(\omega^{\alpha+\beta+1})}={\gamma(\omega^{\alpha+\beta})}+\omega^{{(% \alpha+\beta)}\cdot 2+1}$ and by induction hypothesis, ${\gamma(\omega^{\alpha+\beta})}={\gamma(\omega^{\alpha})}+\omega^{\alpha\cdot 2% +\beta}$ . Since $\log_{\omega}(\alpha)>\log_{\omega}(\beta+1)=\log_{\omega}(\beta)$ we have ${(\alpha+\beta)}\cdot 2+1=\alpha+\beta+\alpha+\beta+1=\alpha\cdot 2+\beta+1>% \alpha\cdot 2+\beta$ and so $\omega^{\alpha\cdot 2+\beta}+\omega^{\alpha\cdot 2+\beta+1}=\omega^{\alpha% \cdot 2+\beta+1}$ . Finally, ${\gamma(\omega^{\alpha+\beta+1})}={\gamma(\omega^{\alpha+\beta})}+\omega^{% \alpha\cdot 2+\beta+1}$ as wanted. ∎

For any $1\leq n<\omega$ and $\alpha\geq 1$ , if $1\leq\beta<\omega^{\alpha}$ then $\log_{\omega}(\beta)<\alpha=\log_{\omega}(\omega^{\alpha}\cdot n)$ . Hence proposition 0.3 gives ${\gamma(\omega^{{\omega^{\alpha}\cdot n}+\beta})}={\gamma(\omega^{{\omega^{% \alpha}\cdot n}})}+\omega^{{\omega^{\alpha}\cdot{2n}}+\beta}$ Then by continuity of $\gamma$ and of the sum/exponentiation in the second variable, we can consider the limit $\beta\rightarrow\omega^{\alpha}$ to get ${\gamma(\omega^{{\omega^{\alpha}\cdot(n+1)}})}={\gamma(\omega^{{\omega^{\alpha% }\cdot n}})}+\omega^{{\omega^{\alpha}\cdot{(2n+1)}}}$ . So continuing our calculation we have

{\gamma(\omega^{\omega\cdot 2})}=\omega^{\omega}+\omega^{\omega\cdot 3}=\omega% ^{\omega\cdot 3}

{\gamma(\omega^{\omega\cdot 3})}=\omega^{\omega\cdot 3}+\omega^{\omega\cdot 5}% =\omega^{\omega\cdot 5}

and taking the limit we find another fixed point

{\gamma(\omega^{\omega^{2}})}=\omega^{\omega^{2}}

then again

{\gamma(\omega^{\omega^{2}\cdot 2})}=\omega^{\omega^{2}}+\omega^{\omega^{2}% \cdot 3}=\omega^{\omega^{2}\cdot 3}

{\gamma(\omega^{\omega^{2}\cdot 3})}=\omega^{\omega^{2}\cdot 3}+\omega^{\omega% ^{2}\cdot 5}=\omega^{\omega^{2}\cdot 5}

and taking the limit we find another fixed point

{\gamma(\omega^{\omega^{3}})}=\omega^{\omega^{3}}

More generally, we have the following proposition:

Proposition 0.4.

For any ordinal $\alpha$ and $1\leq n<\omega$ we have

{\gamma(\omega^{{\omega^{\alpha}\cdot n}})}=\omega^{{\omega^{\alpha}\cdot{(2n-% 1)}}}

Proof.

From the relation ${\gamma(\omega^{{\omega^{\alpha}\cdot(n+1)}})}={\gamma(\omega^{{\omega^{\alpha% }\cdot n}})}+\omega^{{\omega^{\alpha}\cdot{(2n+1)}}}$ , we deduce by induction on $n$ that

\forall n\geq 2,{\gamma(\omega^{{\omega^{\alpha}\cdot n}})}={\gamma(\omega^{{% \omega^{\alpha}}})}+\omega^{{\omega^{\alpha}\cdot{(2n-1)}}}

Taking the limit $n\rightarrow\omega$ ,we obtain

{\gamma(\omega^{{\omega^{\alpha+1}}})}={\gamma(\omega^{{\omega^{\alpha}}})}+% \omega^{{\omega^{\alpha+1}}}

We can then show by induction that all the $\omega^{\omega^{\alpha}}$ are actually fixed points, using the previous relation at successor step, the continuity of $\gamma$ at limit step and the fact that $\gamma(\omega)=\omega$ . This means

{\gamma(\omega^{{\omega^{\alpha}}})}=\omega^{{\omega^{\alpha}}}=\omega^{{% \omega^{\alpha}\cdot{(2\times 1-1)}}}

Then for $n\geq 2$ , we get

{\gamma(\omega^{{\omega^{\alpha}\cdot n}})}={\gamma(\omega^{{\omega^{\alpha}}}% )}+\omega^{{\omega^{\alpha}\cdot{(2n-1)}}}={\omega^{{\omega^{\alpha}}}}+\omega% ^{{\omega^{\alpha}\cdot{(2n-1)}}}=\omega^{{\omega^{\alpha}\cdot{(2n-1)}}}

∎

Equipped with these four propositions, we have a way to recursively calculate $\gamma$ . We are ready to prove the main theorem:

Theorem 0.5.

For all ordinal $\alpha$ , we denote $\gamma(\alpha)$ the order-type of the canonical ordering of $\alpha\times\alpha$ . Then $\gamma$ can be calculated as follows:

1.

Finite Ordinals: For any $n<\omega$ we have

$\gamma(n)=n^{2}$

Limit Ordinals: For any limit ordinal $\alpha$ ,

(a)

If $\omega^{\log_{\omega}\left(\log_{\omega}\left(\alpha\right)\right)}$ does not divide $\log_{\omega}(\alpha)$ then

$\gamma(\alpha)=\omega^{\log_{\omega}(\alpha)}\cdot\alpha$
(b)

Otherwise, we write $\alpha={\omega^{\log_{\omega}(\alpha)}n}+\rho$ for some $n\geq 1$ . If $n\geq 2$ then

$\gamma(\alpha)=\omega^{\log_{\omega}(\alpha)}\cdot\left({\omega^{\log_{\omega}% (\alpha)}\cdot{(n-1)}}+\rho\right)$

(like the first case but we “decrement $n$ in the second factor”)

(c)

Otherwise, $\alpha={\omega^{\log_{\omega}(\alpha)}}+\rho$ and we write $\log_{\omega}(\alpha)=\omega^{\log_{\omega}\left(\log_{\omega}\left(\alpha% \right)\right)}m$ for some $m\geq 1$ . We have

\gamma(\alpha)=\omega^{\log_{\omega}(\alpha)}\cdot\left(\omega^{\omega^{\log_{% \omega}\left(\log_{\omega}\left(\alpha\right)\right)}\cdot\left(m-1\right)}+% \rho\right)

(like the first case but we “decrement $m$ in the second factor”)

3.

Infinite Successor Ordinals: For any limit ordinal $\alpha$ and $1\leq n<\omega$ we have

$\gamma(\alpha+n)=\gamma(\alpha)+{\alpha\cdot{2n}}+n$

where $\gamma(\alpha)$ is determined as in the previous point.

Proof.

The “Finite Ordinals” has been discussed at the beginning and the “Infinite Successor Ordinals” is proposition 0.1. Now let’s consider the Cantor Normal Form $\omega^{\beta_{1}}n+...+\omega^{\beta_{k}}n_{k}$ of a limit ordinal $\alpha\geq\omega$ (so $\beta_{k}\geq 1$ and $\beta_{1}=\log_{\omega}(\alpha)$ ). First, from proposition 0.2 we can make successively extract the $n_{k}$ terms $\omega^{\beta_{k}}$ (by left-multiplying them by $\omega^{\beta_{1}}$ ), then the $n_{k-1}$ terms $\omega^{\beta_{k-1}}$ , … then the $n_{2}$ terms $\omega^{\beta_{2}}$ and finally $n-1$ terms $\omega^{\beta_{1}}$ . We obtain:

{\gamma(\alpha)}={\gamma(\omega^{\beta_{1}})}+{\omega^{\beta_{1}}\left(\omega^% {\beta_{1}}{(n-1)}+\omega^{\beta_{2}}n_{2}+\dots+\omega^{\beta_{k}}n_{k}\right)}

We now write $\beta_{1}=\omega^{\delta}m+\sigma$ where $\delta=\log_{\omega}{\beta_{1}}$ and $m,\sigma$ are the quotient and remainder of the Euclidean division of $\beta_{1}$ by $\omega^{\delta}$ . We can then use proposition 0.3 to extract $\sigma$ :

\sigma=0\implies{\gamma(\omega^{\beta_{1}})}={\gamma(\omega^{\omega^{\delta}m})}

\sigma\neq 0\implies{\gamma(\omega^{\beta_{1}})}={\gamma(\omega^{\omega^{% \delta}m})}+\omega^{\omega^{\delta}\cdot{(2m)}+\sigma}

Finally, using proposition 0.4 we obtain

{\gamma(\omega^{\omega^{\delta}m})}=\omega^{{\omega^{\delta}\cdot{(2m-1)}}}

$\sigma\neq 0$ means that $\omega^{\delta}=\omega^{\log_{\omega}\left(\log_{\omega}\left(\alpha\right)% \right)}$ does not divide $\beta_{1}={\log_{\omega}(\alpha)}$ . In that case, $\omega^{\omega^{\delta}\cdot{(2m)}+\sigma}>\omega^{{\omega^{\delta}\cdot{(2m-1% )}}}$ and so ${\gamma(\omega^{\beta_{1}})}=\omega^{\omega^{\delta}\cdot{(2m)}+\sigma}$ . We note that $\beta_{1}\cdot 2=\omega^{\delta}m+\sigma+\omega^{\delta}m+\sigma=\omega^{% \delta}{(2m)}+\sigma$ since the remainder $\sigma$ is less than $\omega^{\delta}$ . So actually ${\gamma(\omega^{\beta_{1}})}=\omega^{\beta_{1}}\omega^{\beta_{1}}$ . Coming back to the expression of $\gamma(\alpha)$ , this term can be grouped with $\omega^{\beta_{1}}{(n-1)}$ to recover the Cantor Normal Form of $\alpha$ and we finally get $\gamma(\alpha)=\omega^{\beta_{1}}\alpha$ .

Otherwise, $\sigma=0$ , $\beta_{1}=\omega^{\delta}m$ and

{\gamma(\omega^{\beta_{1}})}={\gamma(\omega^{\omega^{\delta}m})}=\omega^{{% \omega^{\delta}\cdot{(2m-1)}}}={\omega^{\beta_{1}}{\omega^{\omega^{\delta}% \cdot{(m-1)}}}}

But then $\beta_{1}=\omega^{\delta}m>{\omega^{\delta}{(m-1)}}$ and so $\omega^{\beta_{1}}\omega^{\beta_{1}}>\omega^{\beta_{1}}\omega^{{\omega^{\delta% }\cdot{(m-1)}}}$ . Hence if $n\geq 2$ , the term ${\gamma(\omega^{\beta_{1}})}$ is eliminated in the expression of $\gamma(\alpha)$ and it remains

{\gamma(\alpha)}={\omega^{\beta_{1}}\left(\omega^{\beta_{1}}{(n-1)}+\rho\right)}

where $\rho=\omega^{\beta_{2}}n_{2}+\dots+\omega^{\beta_{k}}n_{k}$ . If instead $n=1$ , then the term $\omega^{\beta_{1}}{(n-1)}$ is zero and it remains

{\gamma(\alpha)}={\omega^{\beta_{1}}\left(\omega^{{\omega^{\delta}\cdot{(m-1)}% }}+\omega^{\beta_{2}}n_{2}+\dots+\omega^{\beta_{k}}n_{k}\right)}

∎

The canonical well-ordering of α×α (part 1)

2015-01-30T00:00:00+01:00

It is well-known that the cartesian product of two infinite sets of cardinality $\aleph_{\alpha}$ is also of cardinality $\aleph_{\alpha}$ . Equivalently, the set $\omega_{\alpha}\times\omega_{\alpha}$ can be well-ordered in order-type $\omega_{\alpha}$ . However, the standard ordering on $\omega_{\alpha}\times\omega_{\alpha}$ does not work, since it is always of order type $\omega_{\alpha}^{2}$ . Instead, we introduce for each ordinal $\alpha$ the canonical well-ordering of $\alpha\times\alpha$ as follows: ${(\xi_{1},\xi_{2})}\lhd{(\eta_{1},\eta_{2})}$ if either ${\max{(\xi_{1},\xi_{2})}}<{\max{(\eta_{1},\eta_{2})}}$ or ${\max{(\xi_{1},\xi_{2})}}={\max{(\eta_{1},\eta_{2})}}$ but $(\xi_{1},\xi_{2})$ precedes $(\eta_{1},\eta_{2})$ lexicographically. If we note $\gamma(\alpha)$ the ordinal isomorphic to that well-ordering ordering, then we can prove that $\gamma(\omega_{\alpha})=\omega_{\alpha}$ as wanted (see Corollary 0.4).

Jech’s Set Theory book contains several properties of $\gamma$ . For example, $\gamma$ is increasing : for any ordinal $\alpha$ , $\alpha\times\alpha$ is a (proper) initial segment of ${(\alpha+1)}\times{(\alpha+1)}$ and of $\lambda\times\lambda$ for any limit ordinal $\lambda>\alpha$ . As a consequence, $\forall\alpha,\alpha\leq\gamma(\alpha)$ which can also trivially be seen by the increasing function $\xi\mapsto(\xi,0)$ from $\alpha$ to $\alpha\times\alpha$ . Also, $\forall\alpha,\gamma(\alpha)<\gamma(\lambda)$ implies $\sup_{\alpha<\lambda}{\gamma(\alpha)}\leq\gamma(\lambda)$ . This can not be a strict equality, or otherwise the element ${(\xi,\eta)}\in\lambda\times\lambda$ corresponding to $\sup_{\alpha<\lambda}{\gamma(\alpha)}$ via the isomorphism between $\lambda\times\lambda$ and $\gamma(\lambda)$ would be an element larger than all $(\alpha,\alpha)$ for $\alpha<\lambda$ . Hence $\gamma$ is continuous and by exercise 2.7 of the same book, $\gamma$ has arbitrary large fixed points. Exercise 3.5 shows that $\gamma(\alpha)\leq\omega^{\alpha}$ and so $\gamma$ does not increase cardinality (see Corollary 0.2 for a much better upper bound). Hence starting at any infinite cardinal $\kappa$ the construction of exercise 2.7 provides infinitely many fixed points of $\gamma$ having cardinality $\kappa$ . Hence the infinite fixed points of $\gamma$ are not just cardinals and we can wonder what they are exactly…

I recently tried to solve Exercise I.11.7 of Kunen’s Set Theory book which suggests a nice characterization of infinite fixed points of $\gamma$ : they are the ordinals of the form $\omega^{\omega^{\alpha}}$ . However, I could not find a simple proof of this statement so instead I tried to determine the explicit expression of $\gamma(\alpha)$ , from which the result becomes obvious (see Corollary 0.3). My final calculation is summarized in Theorem 0.1, which provides a relatively nice expression of $\gamma(\alpha)$ . Recall that any $\alpha\geq 1$ can be written uniquely as $\alpha=\omega^{\beta}q+\rho$ where $\beta$ is (following Kunen’s terminology) the “logarithm in base $\omega$ of $\alpha$ ” (that we will denote $\log_{\omega}{\alpha}$ for $\alpha\geq\omega$ ) and $q,\rho$ are the quotient and remainder of the Euclidean division of $\alpha$ by $\omega^{\beta}$ . Alternatively, this can be seen from Cantor’s Normal Form: $\beta$ and $q$ are the exponent and coefficient of the largest term while $\rho$ is the sum of terms of smaller exponents.

Theorem 0.1.

For all ordinal $\alpha$ , we denote $\gamma(\alpha)$ the order-type of the canonical ordering of $\alpha\times\alpha$ . Then $\gamma$ can be calculated as follows:

1.

Finite Ordinals: For any $n<\omega$ we have

$\gamma(n)=n^{2}$

Limit Ordinals: For any limit ordinal $\alpha$ ,

(a)

If $\omega^{\log_{\omega}\left(\log_{\omega}\left(\alpha\right)\right)}$ does not divide $\log_{\omega}(\alpha)$ then

$\gamma(\alpha)=\omega^{\log_{\omega}(\alpha)}\cdot\alpha$
(b)

Otherwise, we write $\alpha={\omega^{\log_{\omega}(\alpha)}n}+\rho$ for some $n\geq 1$ . If $n\geq 2$ then

$\gamma(\alpha)=\omega^{\log_{\omega}(\alpha)}\cdot\left({\omega^{\log_{\omega}% (\alpha)}\cdot{(n-1)}}+\rho\right)$

(like the first case but we “decrement $n$ in the second factor”)

(c)

Otherwise, $\alpha={\omega^{\log_{\omega}(\alpha)}}+\rho$ and we write $\log_{\omega}(\alpha)=\omega^{\log_{\omega}\left(\log_{\omega}\left(\alpha% \right)\right)}m$ for some $m\geq 1$ . We have

\gamma(\alpha)=\omega^{\log_{\omega}(\alpha)}\cdot\left(\omega^{\omega^{\log_{% \omega}\left(\log_{\omega}\left(\alpha\right)\right)}\cdot\left(m-1\right)}+% \rho\right)

(like the first case but we “decrement $m$ in the second factor”)

3.

Infinite Successor Ordinals: For any limit ordinal $\alpha$ and $1\leq n<\omega$ we have

$\gamma(\alpha+n)=\gamma(\alpha)+{\alpha\cdot{2n}}+n$

where $\gamma(\alpha)$ is determined as in the previous point.

Proof.

In a future blog post ;-) ∎

From this theorem, we deduce several corollaries:

Corollary 0.2.

For any ordinal $\alpha$ , we have

\alpha\leq{\gamma(\alpha)}\leq{{\omega^{\log_{\omega}(\alpha)}\cdot\left({% \alpha-r}\right)}+{\alpha\cdot{2r}}}\leq\alpha\left(\alpha+r\right)

where $r=\alpha\mod\omega$ is the remainder in the Euclidean division of $\alpha$ by $\omega$ (i.e. the constant term in the Cantor Normal Form).

Proof.

The “Limit Ordinals” case is $r=0$ i.e. $\alpha\leq{\gamma(\alpha)}\leq{{\omega^{\log_{\omega}(\alpha)}\cdot\alpha}}$ , which is readily seen by the previous theorem. Then we deduce for the “Infinite Successor Ordinals” case: ${\gamma(\alpha+n)}=\gamma(\alpha)+{\alpha\cdot{2n}}+n\leq{{\omega^{\log_{% \omega}(\alpha)}\cdot\alpha}}+{\left(\alpha+n\right)\cdot{2n}}$ where ${\log_{\omega}(\alpha+n)}={\log_{\omega}(\alpha)}$ and $n=\left(\alpha+n\mod\omega\right)$ . Since $\omega^{\log_{\omega}(\alpha)}\leq\alpha$ we can write ${{\omega^{\log_{\omega}(\alpha)}\cdot\left({\alpha-r}\right)}+{\alpha\cdot{2r}% }}\leq{\alpha\cdot\left(\alpha-r+2r\right)}={\alpha\cdot{(\alpha+r)}}$ . ∎

Hence the order type of the canonical ordering $\lhd$ of $\alpha\times\alpha$ (i.e. $\gamma(\alpha)\leq\alpha{(\alpha+\omega)}$ ) is never significantly bigger than the one of the standard lexical ordering (i.e. $\alpha^{2}$ ), and even never larger for limit ordinals. Moreover, for many ordinals $\alpha$ the canonical ordering is of order type $\alpha$ :

Corollary 0.3.

The fixed points of $\gamma$ are $0,1$ and $\omega^{\omega^{\alpha}}$ for all ordinals $\alpha$ .

Proof.

For the “Finite Ordinals” case, we have $\gamma(n)=n^{2}=n$ iff $n=0$ or $n=1$ . For the “Infinite Successor Ordinals” case, we have $\alpha\cdot{2n}>\alpha$ if $n\geq 1$ so $\gamma(\alpha+n)>\alpha+n$ . Now we look at the three subcases of the “Limit Ordinals” case. For the first one, we have $\log_{\omega}(\alpha)\geq 1$ and so $\omega^{\log_{\omega}(\alpha)}\alpha\geq\omega\alpha>\alpha$ . For the second one, we note that ${\log_{\omega}(\alpha)}2>\log_{\omega}(\alpha)$ and so $\omega^{{\log_{\omega}(\alpha)}2}>\alpha$ (compare the Cantor Normal Form). Since $n-1\geq 1$ by assumption, we have $\gamma(\alpha)>\alpha$ . Similarly in the third case, if we expand the parenthesis the first term is $\omega$ raised to the power ${\omega^{\log_{\omega}\left(\log_{\omega}\left(\alpha\right)\right)}\cdot\left% (2m-1\right)}$ which is stricly greater than $\log_{\omega}(\alpha)=\omega^{\log_{\omega}\left(\log_{\omega}\left(\alpha% \right)\right)}m$ if $m\geq 2$ . Now if $m=1$ , we obtain $\gamma(\alpha)={\omega^{\log_{\omega}(\alpha)}+{\omega^{\log_{\omega}(\alpha)}% \rho}}$ and $\alpha={\omega^{\log_{\omega}(\alpha)}}+\rho$ . Hence $\gamma(\alpha)>\alpha$ if $\rho>0$ and $\gamma(\alpha)=\alpha$ if $\rho=0$ . Finally for $\alpha\geq\omega$ , $\gamma(\alpha)=\alpha$ iff $\alpha=\omega^{\omega^{\log_{\omega}\left(\log_{\omega}\left(\alpha\right)% \right)}}$ iff $\alpha=\omega^{\omega^{\beta}}$ for some ordinal $\beta$ . ∎

Finally, now that we know that infinite fixed points of $\gamma$ are of the form $\alpha=\omega^{\omega^{\beta}}$ we only need to verify that infinite cardinals $\kappa$ are of this form to prove that ${|\kappa\times\kappa|}=\kappa$ . This provides an alternative (less straightforward) proof of Theorem 3.5 from Jech’s Set Theory book.

Corollary 0.4.

Any infinite cardinal $\kappa$ is a fixed point of $\gamma$ . Hence for any infinite cardinal $\kappa_{1},\kappa_{2}$ we have

\kappa_{1}+\kappa_{2}=\kappa_{1}\kappa_{2}={\max(\kappa_{1},\kappa_{2})}

Proof.

For any cardinal infinite cardinal $\mu$ that is a fixed point of $\gamma$ , the canonical well-ordering provides a bijection between $\mu$ and $\mu\times\mu$ i.e. $\mu^{2}=\mu$ . Hence if $\mu_{1}\leq\mu_{2}$ are two fixed points, we have

\mu_{2}\leq{\mu_{1}+\mu_{2}}\leq{\mu_{1}\mu_{2}}\leq\mu_{2}^{2}=\mu_{2}

Suppose that there is a cardinal $\kappa$ which is not a fixed point of $\gamma$ and consider the smallest one. Then any infinite cardinal below $\kappa$ is a fixed point of $\gamma$ and the previous equality is still true below $\kappa$ .

Suppose $\kappa>\omega^{\log_{\omega}{\kappa}}$ and write $\kappa=\omega^{\log_{\omega}{\kappa}}+\rho$ where $\rho<\kappa$ corresponds to the remaining terms in Cantor Normal Form. Then $\aleph_{0}\leq\mu_{1}=\left|\omega^{\log_{\omega}{\kappa}}\right|<\kappa$ , $\mu_{2}={|\rho|}<\kappa$ and $\mu_{1}+\mu_{2}=\kappa$ . But the two first relations imply $\mu_{1}+\mu_{2}=\max(\mu_{1},\mu_{2})<\kappa$ which contradicts the third one. Hence $\kappa=\omega^{\log_{\omega}{\kappa}}$ .

Now suppose $\log_{\omega}{\kappa}>\omega^{\log_{\omega}{\log_{\omega}{\kappa}}}$ and write $\log_{\omega}{\kappa}=\omega^{\log_{\omega}{\kappa}}+\rho$ where $\rho<\log_{\omega}{\kappa}$ corresponds to the remaining terms in Cantor Normal Form. Then we have

\kappa=\omega^{\log_{\omega}{\kappa}}=\omega^{\omega^{\log_{\omega}{\log_{% \omega}{\kappa}}}}\omega^{\rho}

where $\omega^{\omega^{\log_{\omega}{\log_{\omega}{\kappa}}}}<\omega^{\log_{\omega}{% \kappa}}=\kappa$ and $\omega^{\rho}<\omega^{\log_{\omega}{\kappa}}=\kappa$ since $\alpha\mapsto\omega^{\alpha}$ is strictly increasing. Then $\aleph_{0}\leq\mu_{1}=\left|\omega^{\omega^{\log_{\omega}{\log_{\omega}{\kappa% }}}}\right|<\kappa$ , $\mu_{2}={|\omega^{\rho}|}<\kappa$ and $\mu_{1}\mu_{2}=\kappa$ . But again, the two first relations imply $\mu_{1}\mu_{2}=\max(\mu_{1},\mu_{2})<\kappa$ which contradicts the third one.

Finally, $\kappa=\omega^{\omega^{\log_{\omega}{\log_{\omega}{\kappa}}}}$ and so is a fixed point of $\gamma$ , a contradiction. Hence all the infinite cardinals are fixed point of $\gamma$ and so the property stated at the beginning is true for arbitrary infinite cardinals $\mu_{1},\mu_{2}$ . ∎

Decomposition of 2D-transform matrices

2013-12-01T00:00:00+01:00

I recently took a look at the description of the CSS 2D / SVG transform matrix(a, b, c, d, e, f) on MDN and I added a concrete example showing the effect of such a transform on an SVG line, in order to make this clearer for people who are not familiar with affine transformations or matrices.

This also recalled me a small algorithm to decompose an arbitrary SVG transform into a composition of basic transforms (Scale, Rotate, Translate and Skew) that I wrote 5 years ago for the Amaya SVG editor. I translated it into Javascript and I make it available here. Feel free to copy it on MDN or anywhere else. The convention used to represent transforms as 3-by-3 matrices is the one of the SVG specification.

Live demo

Enter the CSS 2D transform you want to reduce and decompose or pick one example from the list . You can also choose between LU-like or QR-like decomposition: .

CSS

Here is the reduced CSS/SVG matrix as computed by your rendering engine ? and its matrix representation:

After simplification (and modulo rounding errors), an SVG decomposition into simple transformations is ? and it renders like this:

After simplification (and modulo rounding errors), a CSS decomposition into simple transformations is ? and it renders like this:

CSS

A matrix decomposition of the original transform is:

Mathematical Description

The decomposition algorithm is based on the classical LU and QR decompositions. First remember the SVG specification: the transform matrix(a,b,c,d,e,f) is represented by the matrix

(\begin{matrix} a & c & e \\ b & d & f \\ 0 & 0 & 1 \end{matrix})

and corresponds to the affine transformation

$(\begin{matrix} x \\ y \end{matrix}) \mapsto (\begin{matrix} a & c \\ b & d \end{matrix}) (\begin{matrix} x \\ y \end{matrix}) + (\begin{matrix} e \\ f \end{matrix})$

which shows the classical factorization into a composition of a linear transformation $(\begin{matrix} a & c \\ b & d \end{matrix})$ and a translation $(\begin{matrix} e \\ f \end{matrix})$ . Now let's focus on the matrix $(\begin{matrix} a & c \\ b & d \end{matrix})$ and denote $Δ = a d - b c$ its determinant. We first consider the LDU decomposition. If $a \neq 0$ , we can use it as a pivot and apply one step of Gaussian's elimination:

$(\begin{matrix} 1 & 0 \\ - b / a & 1 \end{matrix}) (\begin{matrix} a & c \\ b & d \end{matrix}) = (\begin{matrix} a & c \\ 0 & Δ / a \end{matrix})$

and thus the LDU decomposition is

$(\begin{matrix} a & c \\ b & d \end{matrix}) = (\begin{matrix} 1 & 0 \\ b / a & 1 \end{matrix}) (\begin{matrix} a & 0 \\ 0 & Δ / a \end{matrix}) (\begin{matrix} 1 & c / a \\ 0 & 1 \end{matrix})$

Hence if $a \neq 0$ , the transform matrix(a,b,c,d,e,f) can be written translate(e,f) skewY(atan(b/a)) scale(a, Δ/a) skewX(c/a). If $a = 0$ and $b \neq 0$ then we have $Δ = - c b$ and we can write (this is approximately "LU with full pivoting"):

$(\begin{matrix} 0 & c \\ b & d \end{matrix}) = (\begin{matrix} 0 & - 1 \\ 1 & 0 \end{matrix}) (\begin{matrix} b & d \\ 0 & - c \end{matrix}) = (\begin{matrix} \cos (π / 2) & - \sin (π / 2) \\ \sin (π / 2) & \cos (π / 2) \end{matrix}) (\begin{matrix} b & 0 \\ 0 & Δ / b \end{matrix}) (\begin{matrix} 1 & d / b \\ 0 & 1 \end{matrix})$

and so the transform becomes translate(e,f) rotate(90°) scale(b, Δ/b) skewX(d/b). Finally, if $a = b = 0$ , then we already have an LU decomposition and we can just write

$(\begin{matrix} 0 & c \\ 0 & d \end{matrix}) = (\begin{matrix} c & 0 \\ 0 & d \end{matrix}) (\begin{matrix} 1 & 1 \\ 0 & 1 \end{matrix}) (\begin{matrix} 0 & 0 \\ 0 & 1 \end{matrix})$

and so the transform is translate(e,f) scale(c, d) skewX(45°) scale(0, 1).

As a consequence, we have proved that any transform matrix(a,b,c,d,e,f) can be decomposed into a product of simple transforms. However, the decomposition is not always what we want, for example scale(2) rotate(30°) will be decomposed into a product that involves skewX and skewY instead of preserving the nice factors.

We thus consider instead the QR decomposition. If $Δ \neq 0$ , then by applying the Gram–Schmidt process to the columns $(\begin{matrix} a \\ b \end{matrix}), (\begin{matrix} c \\ d \end{matrix})$ we obtain

$(\begin{matrix} a & c \\ b & d \end{matrix}) = (\begin{matrix} a / r & - b / r \\ b / r & a / r \end{matrix}) (\begin{matrix} r & (a c + b d) / r \\ 0 & Δ / r \end{matrix}) = (\begin{matrix} a / r & - b / r \\ b / r & a / r \end{matrix}) (\begin{matrix} r & 0 \\ 0 & Δ / r \end{matrix}) (\begin{matrix} 1 & (a c + b d) / r^{2} \\ 0 & 1 \end{matrix})$

where $r = \sqrt{a^{2} + b^{2}} \neq 0$ . In that case, the transform becomes translate(e,f) rotate(sign(b) * acos(a/r)) scale(r, Δ/r) skewX(atan((a c + b d)/r^2)). In particular, a similarity transform preserves orthogonality and length ratio and so $a c + b d = (\begin{matrix} a \\ b \end{matrix}) \cdot (\begin{matrix} c \\ d \end{matrix}) = 0$ and $Δ = ∥ (\begin{matrix} a \\ b \end{matrix}) ∥ ∣ (\begin{matrix} c \\ d \end{matrix}) ∥ \cos (π / 2) = r^{2}$ . Hence for a similarity transform we get translate(e,f) rotate(sign(b) * acos(a/r)) scale(r) as wanted. We also note that it is enough to assume the weaker hypothesis $r \neq 0$ (that is $a \neq 0$ or $b \neq 0$ ) in the expression above and so the decomposition applies in that case too. Similarly, if we let $s = \sqrt{c^{2} + d^{2}}$ and instead assume $c \neq 0$ or $d \neq 0$ we get

$(\begin{matrix} a & c \\ b & d \end{matrix}) = (\begin{matrix} \cos (π / 2) & - \sin (π / 2) \\ \sin (π / 2) & \cos (π / 2) \end{matrix}) (\begin{matrix} - c / s & d / s \\ - d / s & - c / s \end{matrix}) (\begin{matrix} Δ / s & 0 \\ 0 & s \end{matrix}) (\begin{matrix} 1 & 0 \\ (a c + b d) / s^{2} & 1 \end{matrix})$

Hence in that case the transform is translate(e,f) rotate(90° - sign(d) * acos(-c/s)) scale(Delta/s, s) skewY(atan((a c + b d)/s^2)). Finally if $a = b = c = d = 0$ , then the transform is just scale(0,0).

The decomposition algorithms are now easy to write. We note that none of them gives the best result in all the cases (compare for example how they factor Rotate2 and Skew1). Also, for completeness we have included the noninvertible transforms in our study (that is $Δ = 0$ ) but in practice they are not really useful (try NonInvertible).

Exercises in Set Theory: Iterated Forcing and Martin’s Axiom

2013-06-01T00:00:00+02:00

New solutions to exercises from Thomas Jech’s book “Set Theory”:

•

Chapter 16: Forcing

Last November, I tried to provide some details of the proof given in chapter 7, regarding the fact that the continuum hypothesis implies the existence of a Ramsey ultrafilter. Booth actually proved in 1970 that this works assuming only Martin’s Axiom. The missing argument is actually given in exercise 16.16. For completeness, I copy the details on this blog post.

Remember that the proof involves contructing a sequence ${(X_{\alpha})}_{\alpha<2^{\aleph_{0}}}$ of infinite subsets of $\omega$ . The induction hypothesis is that at step $\alpha<2^{\aleph_{0}}$ , for all $\beta_{1},\beta_{2}<\alpha$ we have $\beta_{1}<\beta_{2}\implies X_{\beta_{2}}\setminus X_{\beta_{1}}$ is finite. It is then easy to show the result for the successor step, since the construction satisfies $X_{\alpha+1}\subseteq X_{\alpha}$ . However at limit step, to ensure that $X_{\alpha}\setminus X_{\beta}$ is finite for all $\beta<\alpha$ , the proof relies on the continunum hypothesis. This is the only place where it is used.

Assume instead Martin’s Axiom and consider a limit step $\alpha<2^{\aleph_{0}}$ . Define the forcing notion $P_{\alpha}=\{(s,F):s\in{[\omega]}^{<\omega},F\in{[\alpha]}^{<\omega}\}$ and $(s^{\prime},F^{\prime})\leq(s,F)$ iff $s\subseteq s^{\prime}$ , $F\subseteq F^{\prime}$ and $s^{\prime}\setminus s\subseteq X_{\beta}$ for all $\beta\in F$ . It is clear that the relation is reflexive and antisymmetric. The transitivity is almost obvious, just note that if $(s_{3},F_{3})\leq(s_{2},F_{2})$ and $(s_{2},F_{2})\leq(s_{1},F_{1})$ then for all $\beta\in F_{1}\subseteq F_{2}$ we have $s_{3}\setminus s_{1}\subseteq s_{3}\setminus s_{2}\cup s_{2}\setminus s_{1}% \subseteq X_{\beta}$ .

The forcing notion satisfies ccc or even property (K): since ${[\omega]}^{<\omega}$ is countable, for any uncountable subset $W$ there is $t\in{[\omega]}^{<\omega}$ such that $Z=\{(s,F)\in W:s=t\}$ is uncountable. Then any $(t,F_{1}),(t,F_{2})\in Z$ have a common refinement $(t,F_{1}\cup F_{2})$ .

For all $n<\omega$ , define $D_{n}=\{(s,F):|s|\geq n\}$ . Let $\beta_{1}>\beta_{2}>...>\beta_{k}$ the elements of $F$ . We show by induction on $1\leq m\leq k$ that $\bigcap_{i=1}^{m}X_{\beta_{i}}$ is infinite. This is true for $m=1$ by assumption. If it is true for $m-1$ then

\bigcap_{i=1}^{m-1}X_{\beta_{i}}=\bigcap_{i=1}^{m}X_{\beta_{i}}\cup\bigcap_{i=% 1}^{m}X_{\beta_{i}}\setminus X_{\beta_{m}}

The left hand side is infinite by induction hypothesis. The second term of the right hand side is included in $X_{\beta_{1}}\setminus X_{\beta_{m}}$ and thus is finite. Hence the first term is infinite and the result is true for $m$ . Finally, for $m=k$ , we get that $\bigcap_{\beta\in F}X_{\beta}$ is infinite. Pick $x_{1},x_{2},...,x_{n}$ distinct elements from that set and define $(s^{\prime},F^{\prime})=(s\cup\{x_{1},...x_{n}\},F)$ . We have $(s^{\prime},F^{\prime})\in D_{n}$ , $s\subseteq s^{\prime}$ , $F\subseteq F^{\prime}$ and for all $\beta\in F$ , $s^{\prime}\setminus s=\{x_{1},...x_{n}\}\subseteq X_{\beta}$ . This shows that $D_{n}$ is dense. For each $\beta<\alpha$ , the set $E_{\beta}=\{(s,F):\beta\in F\}$ is also dense: for any $(s,F)$ consider $(s^{\prime},F^{\prime})=(s,F\cup\{\beta\})$ .

By Martin’s Axiom there is a generic filter $G$ for the family $\{D_{n}:n<\omega\}\cup\{E_{\beta}:\beta<\alpha\}$ of size $|\alpha|<2^{\aleph_{0}}$ . Let $X_{\alpha}=\{n<\omega:\exists(s,F)\in G,n\in s\}$ . For all $n<\omega$ , there is $(s,F)\in G\cap D_{n}$ and so $|X_{\alpha}|\geq|s|\geq n$ . Hence $X_{\alpha}$ is infinite. Let $\beta<\alpha$ and $(s_{1},F_{1})\in G\cap E_{\beta}$ . For any $x\in X_{\alpha}$ , there is $(s_{2},F_{2})\in G$ such that $x\in s_{2}$ . Hence there is $(s_{3},F_{3})\in G$ a refinement of $(s_{1},F_{2}),(s_{2},F_{2})$ . We have $x\in s_{2}\subseteq s_{3}$ and $s_{3}\subseteq s_{1}\subseteq X_{\beta}$ . Hence $X_{\alpha}\setminus X_{\beta}\subseteq s_{1}$ is finite and the induction hypothesis is true at step $\alpha$ .

Exercises in Set Theory: Applications of Forcing

2013-05-03T00:00:00+02:00

New solutions to exercises from Thomas Jech’s book “Set Theory”:

•

Chapter 15: Forcing

The exercises from this chapter was a good opportunity to play a bit more with the forcing method. Exercise 15.15 seemed a straightforward generalization of Easton’s forcing but turned out to be a bit technical. I realized that the forcing notion used in that exercise provides a result in ZFC (a bit like Exercises 15.31 and 15.32 allow to prove some theorems on Boolean Algebras by Forcing).

Remember that $\beth_{0}=\aleph_{0},\beth_{1}=2^{\beth_{0}},\beth_{2}=2^{\beth_{1}}...,\beth_% {\omega}=\sup\beth_{n},...,\beth_{\alpha+1}=2^{\beth_{\alpha}},...$ is the normal sequence built by application of the continuum function at successor step. One may wonder: is $\beth_{\alpha}$ regular?

First consider the case where $\alpha$ is limit. The case $\alpha=0$ is clear ( $\beth_{0}=\aleph_{0}$ is regular) so assume $\alpha>0$ . If $\alpha$ is an inacessible cardinal, it is easy to prove by induction that for all $\beta<\alpha$ we have $\beth_{\beta}<\alpha$ : at step $\beta=0$ we use that $\alpha$ is uncountable, at successor step that it is strong limit and at limit step that it is regular. Hence $\beth_{\alpha}=\alpha$ and so is regular. If $\alpha$ is not a cardinal then $\operatorname{cf}(\beth_{\alpha})=\operatorname{cf}(\alpha)\leq|\alpha|<\alpha% \leq\beth_{\alpha}$ so $\beth_{\alpha}$ is singular. If $\alpha$ is a cardinal but not strong limit then there is $\beta<\alpha$ such that $2^{\beta}\geq\alpha$ . Since $\beta<\alpha\leq\beth_{\alpha}$ there is $\gamma<\alpha$ such that $\beth_{\gamma}>\beta$ . Then $\beth_{\alpha}\geq\beth_{\gamma+1}=2^{\beth_{\gamma}}>2^{\beta}\geq\alpha$ . So $\operatorname{cf}(\beth_{\alpha})=\operatorname{cf}(\alpha)\leq\alpha<\beth_{\alpha}$ and $\beth_{\alpha}$ is singular. Finally, if $\alpha$ is a singular cardinal, then again $\operatorname{cf}(\beth_{\alpha})=\operatorname{cf}(\alpha)<\alpha\leq\beth_{\alpha}$ and $\beth_{\alpha}$ is singular.

What about the successor case i.e. $\beth_{\alpha+1}$ ? By Corollary 5.3 from Thomas Jech’s book any $\alpha$ , we can show that $\aleph_{\alpha+1}$ is a regular cardinal. The Generalized Continuum Hypothesis says that $\forall\alpha,\aleph_{\alpha}=\beth_{\alpha}$ . Since it holds in $L$ we can not prove in ZFC that for some $\alpha$ , $\beth_{\alpha+1}$ is singular.

The generic extension $V[G]\supseteq V$ constructed in exercise 15.15 satisfies GCH and so it’s another way to show that $\beth_{\alpha+1}$ can not be proved to be singular for some $\alpha$ . However, it provides a better result: by construction, $V[G]\models\beth_{\alpha+1}^{V}={(\beth_{\alpha}^{V})}^{+}$ and so $V[G]\models[\beth_{\alpha+1}^{V}\text{ is a regular cardinal}]$ . Since “regular cardinal” is a $\Pi_{1}$ notion we deduce that $\beth_{\alpha+1}$ is a regular cardinal in $V$ .

Now the question is: is there any “elementary” proof of the fact that $\beth_{\alpha+1}$ is regular i.e. without using the forcing method?

–update: of course, I forgot to mention that by KÃ¶nig’s theorem, $2^{\beth_{\alpha}}=\beth_{\alpha+1}\geq\operatorname{cf}(\beth_{\alpha+1})=% \operatorname{cf}(2^{\beth_{\alpha}})\geq{(\beth_{\alpha})}^{+}$ so the singularity of $\beth_{\alpha+1}$ would imply the failure of the continuum hypothesis for the cardinal $\beth_{\alpha}$ and this is not provable in ZFC.

Suslin’s Problem

2013-04-04T00:00:00+02:00

In a previous blog post, I mentioned the classical independence results regarding the Axiom of Choice and the Generalized Continuum Hypothesis. Here, I’m going to talk about a slightly less known problem that is undecidable in ZFC. It is about a characterization of the set of reals $\mathbb{R}$ and its formulation does not involve at all cardinal arithmetics or the axiom of choice, but only properties of ordered sets.

First, the set of rationals $(\mathbb{Q},<)$ is well-known to be countable. It is linearly ordered (for any $x,y\in\mathbb{Q}$ either $x<y$ or $y<x$ ), unbounded (for any $x\in\mathbb{Q}$ there is $y_{1},y_{2}\in\mathbb{Q}$ such that $x<y_{1}$ and $y_{2}<x$ ) and dense (for any $x,y\in\mathbb{Q}$ if $x<y$ we can find $z\in\mathbb{Q}$ such that $x<z<y$ ). It turns out that $\mathbb{Q}$ can be characterized by these order properties:

Lemma 0.1.

Let $(P,<)$ be a countable, dense, unbounded linearly ordered set. Then $(P,<)$ is isomorphic to $(\mathbb{Q},<)$ .

Proof.

Let $P=\{p_{n}:n\in\mathbb{N}\}$ and $\mathbb{Q}=\{q_{n}:n\in\mathbb{N}\}$ be enumerations of $P$ and $\mathbb{Q}$ . We shall construct by induction a sequence $f_{0}\subseteq f_{1}\subseteq f_{2}\subseteq...$ of functions such that for all $n\in\mathbb{N}$ , $\operatorname{dom}(f_{n})\supseteq\{p_{i}:i<n\}$ , $\operatorname{ran}(f_{n})\supseteq\{q_{i}:i<n\}$ and $\forall x,y\in\operatorname{dom}(f_{n}),x<y\Leftrightarrow f(x)<f(y)$ . Then $f=\bigcup_{n\in\mathbb{N}}f_{n}$ is a function: if $(x,y),(x,z)\in f$ then there is $n$ large enough such that $(x,y),(x,z)\in f_{n}$ and since $f_{n}$ is a function $y=z$ . Moreover, $\operatorname{dom}(f)=\bigcup_{n\in\mathbb{N}}\operatorname{dom}(f_{n})=P$ and $\operatorname{ran}(f)=\bigcup_{n\in\mathbb{N}}\operatorname{ran}(f_{n})= \mathbb{Q}$ . Finally, for any $x,y\in P$ there is $n$ large enough such that $x,y\in\operatorname{dom}(f_{n})$ and so $x<y\Leftrightarrow f_{n}(x)<f_{n}(y)\Leftrightarrow f(x)<f(y)$ . Hence $f$ is an isomorphism between $(P,<)$ and $(\mathbb{Q},<)$ .

Thus let $f_{0}=\emptyset$ . If $f_{n}$ is defined, we construct $f_{n+1}\supseteq f_{n}$ as follows. Suppose $p_{n}\notin\operatorname{dom}(f_{n})$ . If $\forall i<n,p_{n}>p_{i}$ then because $\mathbb{Q}$ is unbounded we can consider the least $n_{0}$ such that $\forall i<n,f(p_{i})<q_{n_{0}}$ and set $f_{n+1}(p_{n})=q_{n_{0}}$ . Similarly if $\forall i<n,p_{n}<p_{i}$ . Otherwise, let $i_{1},i_{2}<n$ such that $p_{i_{1}}<p_{n}<p_{i_{2}}$ with $p_{i_{1}},p_{i_{2}}$ respectively the largest and smallest possible. Because $\mathbb{Q}$ is dense we can consider the least $n_{0}$ such that $f_{n}(p_{i_{1}})<q_{n_{0}}<f_{n}(p_{i_{2}})$ and again set $f_{n+1}(p_{n})=q_{n_{0}}$ . Similarly, if $n\neq n_{0}$ we use the fact that $P$ is unbounded and dense to find $m_{0}\geq n+1$ that allows to define $f_{n+1}(p_{m_{0}})=q_{n}$ and ensures $f_{n+1}$ is order-preserving.

∎

We now notice that $\mathbb{R}$ is linearly ordered, unbounded, dense and has the least upper-bound property (that is any nonempty bounded subset of $\mathbb{R}$ has a least upper-bound). Moreover, the subset $\mathbb{Q}$ is countable and dense in $\mathbb{R}$ (that is for any $x,y\in\mathbb{R}$ such that $x<y$ we can find $z\in\mathbb{Q}$ such that $x<z<y$ ). Using the previous lemma, we deduce again that these order properties give a characterization of the set of reals:

Theorem 0.1.

Let $(R,<)$ be an unbounded, dense and linearly ordered set with the least upper-bound property. Suppose that $R$ has a dense countable subset $P$ . Then $(R,<)$ is isomorphic to $(\mathbb{R},<)$ .

Proof.

$P$ is countable by assumption and since $P\subseteq R$ it is also linearly ordered. If $x,y\in P\subseteq R$ and $x<y$ then by density of $P$ in $R$ there is $z\in P$ such that $x<z<y$ . So $P$ is actually dense. Similarly, if $x\in P\subseteq R$ then since $R$ is unbounded there is $y_{1},y_{2}\in R$ such that $y_{2}<x<y_{2}$ and again by density of $P$ in $R$ we can find $z_{1},z_{2}\in P$ such that $y_{2}<z_{2}<x<z_{1}<y_{1}$ . So $P$ is unbounded. By the previous lemma, there is an isomorphism of ordered sets $f:P\rightarrow\mathbb{Q}$ .

We define for all $x\in R$ , $f_{\star}(x)=\sup_{y\in P;y<x}f(y)$ . Because $P$ is dense in $R$ and $\mathbb{R}$ has the least upper bound property this is well-defined. If $x\in P$ then for all $y<x$ such that $y\in P$ we have $f(y)<f(x)$ and so $f_{\star}(x)\leq f(x)$ . If $f_{\star}(x)<f(x)$ we could find (by density of $\mathbb{Q}$ in $\mathbb{R}$ ) an element $q\in\mathbb{Q}$ such that $f_{\star}(x)<q<f(x)$ . For $p=f^{-1}(q)$ we get $p<x$ and so $q=f(p)\leq f_{\star}(x)$ . A contradiction. So ${f_{\star}}_{|P}=f$ .

Let $x,y\in R$ such that $x<y$ . Because $f$ is increasing we get $f_{\star}(x)\leq f_{\star}(y)$ . By density of $P$ in $R$ we can find $p_{1},p_{2}\in P$ such that $x<p_{1}<p_{2}<y$ . Again, we get $f_{\star}(x)\leq f_{\star}(p_{1})=p_{1}$ and $p_{2}=f_{\star}(p_{2})\leq f_{\star}(y)$ . Hence we actually have $f_{\star}(x)<f_{\star}(y)$ . In particular, $f_{\star}$ is one-to-one.

We shall prove that $f_{\star}$ is surjective. Of course $f_{\star}(P)=f(P)=\mathbb{Q}$ so let’s consider $r\in\mathbb{R}\setminus\mathbb{Q}$ . Define $x=\sup_{q<r:q\in\mathbb{Q}}f^{-1}(q)\in R$ . This is well-defined because $\mathbb{Q}$ is dense in $\mathbb{R}$ and $R$ has the the least upper bound property. Then for all $q<r$ such that $q\in\mathbb{Q}$ we have $f^{-1}(q)\leq x$ by definition. By density of $\mathbb{Q}$ in $\mathbb{R}$ we can actually find $q^{\prime}\in\mathbb{Q}$ such that $q<q^{\prime}<r$ and so $f^{-1}(q)<f^{-1}(q^{\prime})\leq x$ . We have $x>f^{-1}(q)\in P$ and so $f_{\star}(x)\geq f(f^{-1}(q))=q$ . Hence $f_{\star}(x)\geq r$ . Suppose that $r<f_{\star}(x)$ and consider (by density of $\mathbb{Q}$ in $\mathbb{R}$ ) some $q\in\mathbb{Q}$ such that $r<q<f_{\star}(x)$ and let $p=f^{-1}(q)$ . If $p<x$ then there is $q^{\prime}\in\mathbb{Q}$ , $q^{\prime}<r$ , such that $p\leq f^{-1}(q^{\prime})$ . Then $r<q=f(p)\leq q^{\prime}<r$ a contradiction. If instead $p\geq x$ then $f_{\star}(x)\leq f_{\star}(p)=q<f_{\star}(x)$ which is again a contradiction.

Finally, let $x,y\in R$ such that $f_{\star}(x)<f_{\star}(y)$ . Because $R$ is totally ordered either $x<y$ or $y<x$ . But the latter is impossible since we saw above that it implied $f_{\star}(y)<f_{\star}(x)$ . Hence $f_{\star}$ is an isomorphism between $(R,<)$ and $(\mathbb{R},<)$ .

∎

Now consider a totally ordered set $(R,<)$ . For any $a,b$ we define the open interval $(a,b)=\{x\in R:a<x<b\}$ . Suppose $P=\{p_{n}:n\in\mathbb{N}\}$ is a dense subset of $R$ . If ${\left((a_{i},b_{i})\right)}_{i\in I}$ is a family of pairwise disjoint open intervals then we can associate to any $(a_{i},b_{i})$ the least $n_{i}\in\mathbb{N}$ such that $p_{n_{i}}\in(a_{i},b_{i})$ (by density of $P$ in $R$ ). Since the family is disjoint the function $i\mapsto n_{i}$ obtained is one-to-one and so the family is at most countable. One naturally wonders what happens if we replace in theorem 0.1 the existence of a countable dense subset by this weaker property on open intervals:

Problem 0.1 (Suslin’s Problem).

Let $(R,<)$ be an unbounded, dense and linearly ordered set with the least upper-bound property. Suppose that any family of disjoint open intervals in $R$ is at most countable. Is $(R,<)$ isomorphic to $(\mathbb{R},<)$ ?

We have seen how this problem arises from a natural generalization of a characterization of $\mathbb{R}$ . We note that we did not use the Axiom of Choice in the above analysis and that the problem can be expressed using only definitions on ordered sets. However, in order to answer Suslin’s Problem we will need to introduce more concepts. We will assume familiarity with basic notions of Set Theory like ordinals, cardinals or the Axiom of Choice. The first five chapters of Thomas Jech’s book ‘‘Set Theory’’ should be enough. In addition, we will rely on the axiom $\mathrm{MA}_{\aleph_{1}}$ and on the Diamond Principle $\Diamond$ , that we define here:

Definition 0.1 (Martin’s Axiom $\aleph_{1}$ , Diamond Principle).

$\mathrm{MA}_{\aleph_{1}}$ and $\Diamond$ are defined as follows:

•

Let $(P,<)$ be a partially ordered set. Two elements $x,y$ are compatible if there is $z$ such that $z\leq x$ and $z\leq y$ . Note that comparable implies compatible.
•

$D$ is dense in $P$ if for any $p\in P$ there is $d\in D$ such that $d\leq p$ .
•
$G\subseteq P$ is a filter if:
- –
  
  $G\neq\emptyset$
- –
  
  For any $p,q\in P$ such that $p\leq q$ and $p\in G$ we have $q\in G$
- –
  
  Any $p,q\in G$ there is $r\in G$ such that $r\leq p,q$ .
•

$\mathrm{MA}_{\aleph_{1}}$ : Let $(P,<)$ be a partially ordered set such that any subset of paiwise incompatible elements of $P$ is at most countable. Then for any family of at most $\aleph_{1}$ dense subsets there is a filter $G\subseteq P$ that has a nonempty intersection with each element of this family.
•
A set $C\subseteq\omega_{1}$ is closed unbounded if
- –
  
  It is unbounded in the sense that for any $\alpha<\omega_{1}$ there is $\beta\in C$ such that $\alpha<\beta$
- –
  
  It is closed for the order topology, or equivalently if for any $\gamma<\omega_{1}$ and any $\gamma$ -sequence $\alpha_{0}<\alpha_{1}<...<\alpha_{\xi}<...$ of elements of $C$ , $\lim_{\xi\rightarrow\gamma}\alpha_{\xi}$ is in $C$ .
•

A set $S\subseteq\omega_{1}$ is stationary if for any $C$ closed unbounded, $S\cap C\neq\emptyset$ .
•

$\Diamond$ : There is an $\omega_{1}$ -sequence of sets $S_{\alpha}\subseteq\alpha$ such that for every $X\subseteq\omega_{1}$ the set $\{\alpha<\omega_{1}:X\cap\alpha=S_{\alpha}\}$ is stationary.

First we show that if $\mathrm{MA}_{\aleph_{1}}$ holds, then we get a positive answer to Suslin’s problem:

Theorem 0.2.

Assume Martin’s axiom $\mathrm{MA}_{\aleph_{1}}$ holds. Let $(R,<)$ be an unbounded, dense and linearly ordered set with the least upper-bound property. Suppose that any disjoint family of open intervals in $R$ is at most countable. Then $(R,<)$ is isomorphic to $(\mathbb{R},<)$ .

Proof.

Suppose $(R,<)$ is not isomorphic to $(\mathbb{R},<)$ and in particular does not have any countable dense subset (otherwise we could apply theorem 0.1). We define closed intervals $I_{\alpha}\subseteq R$ by induction on $\alpha<\omega_{1}$ . If $I_{\beta}=[a_{\beta},b_{\beta}]$ is defined for all $\beta<\alpha$ then the set $C=\{a_{\beta}:\beta<\alpha\}\cup\{b_{\beta}:\beta<\alpha\}$ is countable and thus is not dense in $R$ . Then there is $a_{\alpha}<b_{\alpha}$ such that $I_{\alpha}=[a_{\alpha},b_{\alpha}]$ is disjoint from $C$ . We define the set $S=\{I_{\alpha},\alpha<\omega_{1}\}$ . Clearly, $|S|=\aleph_{1}$ and $(S,\subsetneq)$ is partially ordered. We note that if $\beta<\alpha<\omega_{1}$ then by construction $a_{\beta},b_{\beta}\notin I_{\alpha}$ and so either $I_{\alpha}\cap I_{\beta}=\emptyset$ or $I_{\alpha}\subsetneq I_{\beta}$ . In particular, comparable is the same as compatible in $S$ and any family of pairwise incomparable/incompatible elements of $S$ is a family of pairwise disjoint intervals of $R$ so at most countable.

Let $\alpha<\omega_{1}$ and define $P_{I_{\alpha}}=\{I\in S:I\supsetneq I_{\alpha}\}$ . Let $X\subseteq P_{I_{\alpha}}$ nonempty. Let $\beta$ the least ordinal such that $I_{\beta}\in X$ . If $I_{\gamma}\in X$ then $\beta\leq\gamma$ and $I_{\gamma}\cap I_{\beta}\supseteq I_{\alpha}\neq\emptyset$ . Hence by the previous remark $I_{\beta}\supseteq I_{\gamma}$ . So $P_{I_{\alpha}}$ is well-ordered by $\supseteq$ and we define $o(I_{\alpha})$ the order-type of $P_{I_{\alpha}}$ . We note that the set $P_{I_{\alpha}}$ can be enumerated by $I_{\alpha_{1}}\supsetneq I_{\alpha_{2}}\supsetneq...\supsetneq I_{\alpha}$ for some $\alpha_{\xi}\leq\alpha$ and so $o(I_{\alpha})<\omega_{1}$ . Moreover for any $\alpha,\beta<\omega$ , if $I_{\alpha}\supsetneq I_{\beta}$ then $P_{I_{\alpha}}$ is an initial segment of $P_{I_{\beta}}$ and so $o(I_{\alpha})<o(I_{\beta})$ . Hence for each $\alpha<\omega_{1}$ the set $L_{\alpha}=\{I\in S:o(I)=\alpha\}$ has pairwise incomparable elements and so is at most countable.

For any $I\in S$ , define $S_{I}=\{J\in S:J\subseteq I\}$ and let $T=\{I\in S:|S_{I}|=\aleph_{1}\}$ . Let $\alpha<\omega_{1}$ and suppose that $L_{\alpha}\cap T=\emptyset$ . Then $S=\bigcup_{\beta<\alpha}L_{\beta}\cup\bigcup_{I\in L_{\alpha}}S_{I}$ and since the $L_{\beta}$ are at most countable for $\beta<\omega_{1}$ and the $S_{I}$ are at most countable for $I\in L_{\alpha}$ we would have $S$ at most countable, a contradiction. So for each $\alpha<\omega_{1}$ , there is $I\in T\cap L_{\alpha}$ and in particular $|T|=\aleph_{1}$ . We note that if $I\in T$ and $J\supseteq I$ then $S_{I}\subseteq S_{J}$ and so $J\in T$ . In particular, $P_{I}=\{J\in T:J\supsetneq I\}$ and thus without loss of generality we may assume that $S=T$ .

For any $\alpha<\omega_{1}$ let $D_{\alpha}=\{I\in D_{\alpha}:o(I)>\alpha\}$ . For any $I\in S$ , $|S_{I}|=\aleph_{1}$ and $S_{I}=\left(\bigcup_{\beta\leq\alpha}S_{I}\cap L_{\beta}\right)\cup S_{I}\cap D _{\alpha}$ . The first term is at most countable and so the second is uncountable and a fortiori nonempty. So $D_{\alpha}$ is a dense subset of $S$ .

Using $\mathrm{MA}_{\aleph_{1}}$ , we find $G\subseteq S$ a filter that intersects each $D_{\alpha}$ . By definition, elements of a filter are pairwise compatible and so pairwise comparable. Let us construct by induction on $\alpha<\omega_{1}$ , some sets $J_{\alpha}\in G$ . If $J_{\beta}$ is constructed for any $\beta<\alpha$ then $\gamma=\sup_{\beta<\alpha}o(J_{\beta})<\omega_{1}$ and we can pick $J_{\alpha}\in G\cap D_{\gamma}$ . We obtain a decreasing $\omega_{1}$ -sequence of intervals $J_{0}\supsetneq J_{1}\supsetneq...\supsetneq J_{\alpha}\supsetneq...$ . If $J_{\alpha}=[x_{\alpha},y_{\alpha}]$ then this gives an increasing sequence $x_{0}<x_{1}<x_{2}<...<x_{\alpha}<...$ . The sets $(x_{\alpha},x_{\alpha+1})$ form an uncountable family of disjoint open intervals. A contradiction.

∎

Finally, we show that the $\Diamond$ principle provides a negative answer to Suslin’s problem:

Theorem 0.3.

Assume the $\Diamond$ principle holds. Then there is a linearly ordered set $(R,<)$ not isomorphic to $(\mathbb{R},<)$ , unbounded, dense, that has the least upper-bound property and such that any family of disjoint open intervals is at most countable.

Proof.

Let ${(S_{\alpha})}_{\alpha<\omega_{1}}$ be a $\Diamond$ -sequence. We first construct a partial ordering $\prec$ of $T=\omega_{1}$ . We define for all $1\leq\alpha<\omega_{1}$ an ordering $(T_{\alpha},\prec)$ on initial segments $T_{\alpha}\subseteq\omega_{1}$ and obtain the ordering $\prec$ on $\omega_{1}=T=\bigcup_{\alpha<\omega_{1}}T_{\alpha}$ .

Each $(T_{\alpha},\prec)$ will be a tree i.e. for any $x$ in the tree the set $P_{x}=\{y:y\prec x\}$ is well-ordered. As in the proof of 0.2 we can define $o(x)$ the order-type of $P_{x}$ . The level $\alpha$ is the set of elements such that $o(x)=\alpha$ . The height of a tree is defined as the supremium of the $o(x)+1$ . In a tree, a branch is a maximal linearly ordered subset and an antichain a subset of pairwise incomparable elements. A branch is also well-ordered and so we can define its length as its order-type. $T_{\alpha}$ is constructed such that its height is $\alpha$ and for each $x\in T_{\alpha}$ there is some $y\succ x$ at each higher level less than $\alpha$ .

We let $T_{1}=\{0\}$ . If $\alpha$ is a limit ordinal then $(T_{\alpha},\prec)$ is the union of $(T_{\beta},\prec)$ for $\beta<\alpha$ . If $\alpha=\beta+1$ is a successor ordinal, then the highest level of $T_{\alpha}$ is $L_{\beta}=\{x\in T_{\alpha}:o(x)=\beta\}$ . $T_{\alpha+1}$ is obtained by adding $\aleph_{0}$ immediate successors to each element of $L_{\beta}$ . These successors are taken from $\omega_{1}$ in a way that $T_{\alpha+1}$ is an initial segment of $\omega_{1}$ .

Let $\alpha$ is a limit ordinal. Let $A=S_{\alpha}$ if it is a maximal antichain in $(T_{\alpha},\prec)$ and take $A$ an arbitrary maximal antichain of $T_{\alpha}$ otherwise. Then for each $t\in T_{\alpha}$ there is $a\in A$ such that either $a\prec t$ or $t\prec a$ . Let $b_{t}$ be a branch that contains $a,t$ . We construct $(T_{\alpha+1},\prec)$ by adding for each branch $b_{t}$ some $x_{b_{t}}$ that is greater than all the elements of $b_{t}$ . We can choose $b_{t}$ in way that it contains an element of each level less than $\alpha$ and so $o(x_{b_{t}})=\alpha$ and the height of $T_{\alpha+1}$ is $\alpha+1$ . We note that each $t\in T_{\alpha+1}$ is either in $T_{\alpha}$ (so comparable with some element of $A$ ) or greater than (a fortiori comparable with) one element of $A$ . So $A$ is an antichain in $(T_{\alpha+1},\prec)$ .

Now consider a maximal antichain $A\subseteq T=\omega_{1}$ and $C$ the set of ordinals $\alpha<\omega_{1}$ such that ‘‘ $A\cap T_{\alpha}$ is a maximal antichain in $T_{\alpha}$ and $T_{\alpha}=\alpha$ ’’. Let $\alpha_{0}<\alpha_{1}<...<\alpha_{\xi}<...$ ( $\xi<\gamma<\omega_{1}$ ) be a sequence of elements in $C$ and consider the limit ordinal $\lambda=\lim_{\xi<\gamma}\alpha_{\xi}$ . By construction, $T_{\lambda}=\bigcup_{\alpha<\lambda}T_{\alpha}=\bigcup_{\xi<\gamma}T_{\alpha_{ \xi}}=\bigcup_{\xi<\gamma}\alpha_{\xi}=\lambda$ . If $x\in T_{\lambda}$ , there is $\xi<\gamma$ such that $x\in T_{\alpha_{\xi}}$ and so $x$ is comparable with some $y\in A\cap T_{\alpha_{\xi}}\subseteq A\cap T_{\lambda}$ . So $A\cap T_{\lambda}$ is a maximal antichain in $T_{\lambda}$ . Finally $\lambda\in C$ and $C$ is closed.

We note that $T_{1}=\{0\}\geq 1$ . If $T_{\alpha}\geq\alpha$ then $T_{\alpha+1}$ is obtained by adding at least one element at the end of the initial segment $T_{\alpha}$ and so $T_{\alpha+1}\geq\alpha+1$ . Finally if $\lambda>0$ is limit and $T_{\alpha}\geq\alpha$ for each $\alpha<\lambda$ then $T_{\lambda}=\bigcup_{\alpha<\lambda}T_{\alpha}\geq\sup_{\alpha<\lambda}\alpha=\lambda$ . Moreover by definition, each $T_{\alpha}$ is at most countable. Let’s come back to the closed set $C$ above. Let $\alpha_{0}<\omega_{1}$ be arbitrary. For each $n<\omega$ , we let $\alpha_{2n+1}$ be the limit of the sequence $\alpha_{0}\leq T_{\alpha_{0}}\leq T_{T_{\alpha_{0}}}\leq T_{T_{T_{\alpha_{0}}} }...$ . By definition, $T_{\alpha_{2n+1}}=\bigcup_{\xi<\alpha_{2n+1}}T_{\xi}=\alpha_{0}\cup T_{\alpha_ {0}}\cup T_{T_{\alpha_{0}}}...=\alpha_{2n+1}$ . Because $A$ is a maximal antichain in $T$ , for each $x\in T_{\alpha_{2n+1}}$ we can find some $\alpha_{x}\geq\alpha_{2n+1}$ and $a_{x}\in A_{\alpha_{x}}$ that is comparable with $x$ . Because $T_{\alpha_{2n+1}}$ is countable we can define $\alpha_{2n+2}=\sup_{x\in T_{\alpha_{2n+1}}}\alpha_{x}<\omega_{1}$ . Then any $x\in T_{\alpha_{2n+1}}$ is comparable with some element of $a_{x}\in A\cap T_{\alpha_{2n+2}}$ . Let $\lambda=\lim_{n<\omega}\alpha_{n}$ . With the same method as to prove the fact that $C$ is closed, the equality $\lambda=\lim_{n<\omega}\alpha_{2n+1}$ shows that $T_{\lambda}=\lambda$ while the equality $\lambda=\lim_{n<\omega}\alpha_{2n+2}$ shows that $A\cap T_{\lambda}$ is a maximal antichain in $T_{\lambda}$ . So $\lambda\in C$ and $C$ is closed unbounded.

Using the Diamond principle, $\{\alpha<\omega_{1}:A\cap\alpha=S_{\alpha}\}\cap C\neq\emptyset$ . If $\alpha$ is in the intersection, then $S_{\alpha}=A\cap T_{\alpha}$ is a maximal antichain in $T_{\alpha}$ . By construction, it is also a maximal antichain in $T_{\alpha+1}$ . Each element of $a\in A\cap T_{\alpha+1}$ is at level $o(a)\leq\alpha$ . Any $t\in T_{\alpha+1}$ is comparable with some element of $A\cap T_{\alpha+1}$ . Moreover, by contruction any $t^{\prime}\in T\setminus T_{\alpha+1}$ has some predecessor $t\in T_{\alpha+1}$ at level $\alpha$ and there is $a\in A\cap T_{\alpha+1}$ that is comparable with $t$ . Necessarily, $o(a)<o(t)=\alpha$ and so $a\prec t\preceq t^{\prime}$ . Thus $A\cap T_{\alpha+1}$ is maximal in $T$ and $A=A\cap T_{\alpha+1}\subseteq T_{\alpha+1}$ is at most countable.

Let $B$ be a branch in $T$ . It is clear that $B$ is nonempty. Actually it is infinite: otherwise there is some limit ordinal $\alpha>0$ such that $B\subseteq T_{\alpha}$ and by construction we can find some $y\succ\max B$ contradicting the maximality of $B$ . By construction, each $x\in T$ has infinitely many successors at the next level and so these successors are pairwise incomparable. Hence if $x\in B$ then we can pick one $z_{x}\notin B$ among these succcessors. Let $x\prec y$ be two elements of $B$ . Then $o(z_{x})=o(x)+1>o(y)+1=o(z_{y})$ so $z_{x}\preceq z_{y}$ is impossible. Suppose $z_{y}\prec z_{x}$ . We also have $x\prec z_{x}$ by definition. If $z_{y}\preceq x$ then we would have $z_{y}\in B$ because $B$ is a branch. If $x\prec z_{y}$ we would get $o(x)<o(z_{y})=o(y)+1\leq o(x)$ . Hence $z_{x},z_{y}$ are incomparable (in particular distinct) and the set $\{z_{x}:x\in B\}$ is an antichain in $T$ and so $B$ is countable.

Finally, we define $S$ the set of all branches of $T$ . By construction, each $x\in T$ has countably many immediate successors and we order them as $\mathbb{Q}$ . Let $B_{1},B_{2}$ be two branches and $\alpha$ the least level where they differ. The level 0 is $T_{1}=\{0\}$ so $\alpha>0$ . If $\alpha$ is limit then the restriction of the branches $B_{1},B_{2}$ to $T_{\alpha}$ is the same branch $b$ and $T_{\alpha+1}$ has been contructed in a way that there is only one possible element at level $\alpha$ to extend this branch and this is a contradiction. So $\alpha$ is actually a successor ordinal $\beta+1$ . Hence if $B_{1}(\alpha)\in B_{1}$ and $B_{2}(\alpha)\in B_{2}$ are the immediate successors of the point in $B_{1}\cap B_{2}$ at level $\beta$ , we order $B_{1},B_{2}$ according to whether $B_{1}(\alpha)<B_{2}(\alpha)$ or $B_{1}(\alpha)>B_{2}(\alpha)$ , using the $\mathbb{Q}$ -isomorphic order we just defined. Clearly, this gives a linear ordering $<_{S}$ . It is also unbounded: for any $B\in S$ , if $B(1)\in B$ is the element at level 1 pick $x$ greater (or smaller) than for the $\mathbb{Q}$ -isomorphic order on successors of $B(0)=0$ and consider a branch extending $0\prec x$ .

Now consider two branches $B_{1}<_{S}B_{2}$ . Let $\alpha=\beta+1$ be as above. We can find $x$ an immediate successor of $B_{1}(\beta)$ such that $B_{1}(\alpha)<x<B_{2}(\alpha)$ for the $\mathbb{Q}$ -isomorphic order on immediate successors of $B_{1}(\beta)$ . Let $I_{x}=\{B\in S:x\in B\}$ . Any $B\in I_{x}$ contains $\{y\in T:y\prec x\}=\{y\in T:y\prec B_{1}(\beta)\}=\{y\in T:y\prec B_{2}(\beta)\}$ . Moreover $x=B(\alpha)$ and so $B_{1}<B<B_{2}$ . $I_{x}$ is nonempty (we can extend $\{y\in T:y\preceq x\}$ to a maximal branch) and so $S$ is dense. If $I_{x}\cap I_{y}=\emptyset$ then $x,y$ are incomparable. So from any collection of disjoint open intervals ${(B_{1i},B_{2i})}_{i\in I}$ we get an antichain $\{x_{i}:i\in I\}$ and so $I$ is at most countable.

Let $C$ be a countable set of branches in $T$ . Since these branches are countable, we can find $\alpha<\omega_{1}$ larger than the length of any branches in $C$ . If $x\in T$ is at level greater than $\alpha$ then for all $B\in C$ , $x\notin B$ so $B\notin I_{x}$ . Finally $C\cap I_{x}=\emptyset$ and $C$ is not dense.

Now let $R$ be the Dedekind-MacNeille completion of $S$ . It is unbounded, linearly ordered, has the least upper-bound property and $S$ is dense in $R$ . Using the fact that $S$ is dense in $R$ we deduce that $R$ is dense. Similarly, if ${(a_{i},b_{i})}_{i\in I}$ is any collection of disjoint open intervals in $R$ we can find $c_{i},d_{i}\in S$ such that $a_{i}<c_{i}<d_{i}<b_{i}$ . Then ${(c_{i},d_{i})}_{i\in I}$ is a collection of disjoint open intervals in $S$ and so $I$ is at most countable.

Finally, it remains to prove that $R$ is not isomorphic to $\mathbb{R}$ and it suffices to show that $R$ does not have any countable dense subset $C$ . If $C$ is a countable subset of $R$ , then for any elements $a<b$ in $C$ we pick $c\in S$ such that $a<c<b$ . This gives a countable subset $C^{\prime}$ of $S$ . If $a<b$ are elements of $S$ , we can find $c,d\in C$ such that $a<c<d<b$ and so $e\in C^{\prime}$ such that $a<c<e<d<b$ . Thus $C^{\prime}$ would be a countable dense subset of $S$ . A contradiction.

∎

The $\Diamond$ principle holds in the model $L$ of constructible sets. Using iterated forcing, we can construct a model of ZFC in which $\mathrm{MA}_{\aleph_{1}}$ holds. Using theorems 0.2 and 0.3 we deduce that both the positive and negative answers to Suslin’s problem are consistent with ZFC and so Suslin’s problem is undecidable in ZFC. It’s remarkable that a problem that only involves linearly ordered set can be solved using sophisticated methods from Set Theory. The above proofs follow chapters 4, 9, 15 and 16 from Thomas Jech’s book ‘‘Set Theory’’. In particular:

•

Lemma 0.1 and theorem 0.1 are based on the sketch given in theorem 4.3.
•

Theorem 0.2 is based on the proofs of theorem 16.16 and lemma 9.14 (a).
•

Theorem 0.3 is based on the proofs of lemma 15.24, lemma 15.25, theorem 15.26, lemma 15.27 and lemma 9.14 (b).
•

Theorem 0.2 implicitely uses theorem 4.4 about the Dedekind-MacNeille completion of a dense unbounded linearly ordered.
•

In addition, theorem 0.2 also contains the proof of lemma 8.2 (the intersection of two closed unbounded sets is closed unbounded) the solutions to exercise 2.7 (any normal sequence has arbitrarily large fixed points) and exercise 9.7 (if all the antichains of a normal $\omega_{1}$ -tree are at most countable then so are its branches).
•

Finally, $\Diamond$ and $\mathrm{MA}_{\aleph_{1}}$ are proved to be consistent with ZFC in theorems 13.21 and 16.13.

Exercises in Set Theory: Classical Independence Results

2013-03-20T00:00:00+01:00

I worked on Chapter 13 and 14 of Thomas Jech’s book ‘‘Set Theory’’.

Doing the exercises from these chapters gave me the opportunity to come back to the ‘‘classical’’ results about the independence of the Axiom of Choice and (Generalized) Continuum Hypothesis by Kurt Gödel and Paul Cohen. It’s funny to note that it’s easier to prove that AC holds in $L$ (essentially, the definition by ordinal induction provides the well-ordering of the class of contructible sets) than to prove that GCH holds in $L$ (you rely on AC in $L$ and on the technical condensation lemma). Actually, I believe Gödel found his proof for AC one or two years after the one for GCH. On the other hand, it is easy to make GCH fails (just add $\aleph_{2}$ Cohen reals by Forcing) but more difficult to make AC fails (e.g. AC is preserved by Forcing). This can be interpreted as AC being more ‘‘natural’’ than GCH.

After reading the chapters again and now I analyzed in details the claims, I’m now convinced about the correctness of the proof. There are only two points I didn’t verify precisely about the Forcing method (namely that all axioms of predicate calculus and rules of inference are compatible with the Forcing method ; that the Forcing/Generic Model theorems can be transported from the Boolean Algebra case to the general case) but these do not seem too difficult. Here are some notes about claims that were not obvious to me at the first reading. As usual, I hope they might be useful to the readers of that blog:

1.

In the first page of chapter 13, it is claimed that for any set $M$ , $M\in\mathrm{def}(M)$ and $M\subseteq\mathrm{def}(M)\subseteq\operatorname{\mathcal{P}}(M)$ . The first statement is always true because $M=\{x\in M:(M,\in)\models x=x\}\in\mathrm{def}(M)$ and ${(x=x)}^{(M,\in)}$ is $x=x$ by definition. However, the second statement can only be true if $M$ is transitive (since that implies $M\subseteq\operatorname{\mathcal{P}}(M)$ ). Indeed, if $M$ is transitive then for all $a\in M$ we have $a\subseteq M$ and since $x\in a$ is $\Delta_{0}$ we get $a=\{x\in M:(M,\in)\models x\in a\}\in\mathrm{def}(M)$ . If moreover we consider $x\in X\in\mathrm{def}(M)$ then $x\in X\subseteq M$ so $x\in M\subseteq\mathrm{def}(M)$ and $\mathrm{def}(M)$ is also transitive. Hence the transitivity of the $L_{\alpha}$ can still be shown by ordinal induction.
2.

The proof of lemma 13.7 can not be done exactly by induction on the complexity of $G$ , as suggested. For example to prove (ii) for $G=G_{2}=\cdot\times\cdot$ , we would consider $\exists u\in F(...)\times H(...)\varphi(u)\Leftrightarrow\exists a\in F(...),% \exists b\in H(...),\varphi((a,b))$ and would like to say that $\varphi((a,b))$ is $\Delta_{0}$ . Nevertheless, we can not deduce that from the induction hypothesis. Hence the right thing to do is to prove the lemma for $G_{1}=\{\cdot,\cdot\}$ first and deduce the lemma for $G=(\cdot,\cdot)$ (and $G^{\prime}=(\cdot,\cdot,\cdot)$ ). Then we can proceed by induction.
3.

In the proof of theorem 13.18, it is mentioned that the assumption
1. (i)
  
  $x<_{\alpha}y$ implies $x<_{\beta}y$
2. (ii)
  
  $x\in L_{\alpha}$ and $y\in L_{\beta}\setminus L_{\alpha}$ implies $x<_{\beta}y$
implies that if $x\in y\in L_{\alpha}$ then $x<_{\alpha}y$ . To show that, we consider $\beta\leq\alpha$ the least ordinal such that $y\in L_{\beta}$ . In particular, $\beta$ is not limit ( $L_{0}=\emptyset$ and if $y\in L_{\beta}$ for some limit $\beta>0$ then there is $\gamma<\beta$ such that $y\in L_{\gamma}$ ) and we can write it $\beta=\gamma+1$ . We have $y\in L_{\beta}=L_{\gamma+1}$ so there is a formula $\varphi$ and elements $a_{1},...,a_{n}\in L_{\gamma}$ such that $x\in y=\{z\in L_{\gamma}:(L_{\gamma},\in)\models\varphi(z,a_{1},...,a_{n})\}$ . Hence $x\in L_{\gamma}$ . Moreover by minimality of $\beta$ , $y\in L_{\beta}\setminus L_{\gamma}$ so by (ii) we have $x<_{\beta}y$ and by (i) $x<_{\alpha}y$ .

In lemma 14.18, we have expressions that seem ill-defined for example $a_{u}(t)$ where $t\notin\operatorname{dom}(a_{u})$ . This happens in other places, like lemma 14.17 or definition 14.27. The trick is to understand that the functions are extended by 0. Indeed, for any $x,y\in V^{B}$ if $x\subseteq y$ and $\forall t\in\operatorname{dom}(y)\setminus\operatorname{dom}(x),y(t)=0$ then

\begin{gathered}\displaystyle\|y\subseteq x\|\\ \displaystyle=\prod_{t\in\operatorname{dom}(y)}\left(-y(t)+{\|t\in x\|}\right)% \\ \displaystyle=\prod_{t\in\operatorname{dom}(x)}\left(-x(t)+{\|t\in x\|}\right)% \\ \displaystyle=\|x\subseteq x\|=1\end{gathered}

and similarly we get $\|x=y\|=1$ . Then we can use the inequality page 207 ( $\|\varphi(x)\|=\|x=y\|\cdot\|\varphi(x)\|\leq\|\varphi(y)\|=\|x=y\|\cdot\|% \varphi(y)\|\leq\|\varphi(x)\|$ ) to replace $x$ by its extension $y$ .

5.

In lemma 14.23, the inequality

$\|x\text{ is an ordinal}\|\leq\|x\in\check{\alpha}\|+\|x=\check{\alpha}\|+\|% \check{\alpha}\in x\|$

seems obvious but I don’t believe that it can be proved so easily at that point. For example the proof from chapter 2 requires at least the Separation axiom and the $\Delta_{0}$ formulation from chapter 10 is based on the Axiom of Regularity. To solve that issue, it seems to me that the lemma should be moved after the proof that axioms of ZFC are valid in $V^{B}$ . This is not an issue since lemma 14.23 is only used much later in lemma 14.31.

Many details could be added to the proof of theorem 14.24, but let’s just mention Powerset. For any $u\in V^{B}$ , some $u^{\prime}\in\operatorname{dom}(Y)$ is defined and satisfies $\|u\subseteq X\|\leq\|u=u^{\prime}\|$ (this follows from the definitions, using the Boolean inequality $-a+b\leq-a+b\cdot a$ to conclude). Since moreover $\forall t\in\operatorname{dom}(Y),Y(t)=1$ we get

	$\displaystyle\\|u\subseteq X\\|\implies\\|u\in Y\\|$	$\displaystyle\geq-\\|u=u^{\prime}\\|+\sum_{t\in\operatorname{dom}(Y)}\left(\\|u=t% \\|\cdot Y(t)\right)$
		$\displaystyle=\sum_{t\in\operatorname{dom}(Y)}-\left(\\|u\neq t\\|\cdot\\|u=u^{% \prime}\\|\right)$
		$\displaystyle\geq\sum_{t\in\operatorname{dom}(Y)}\\|u^{\prime}=t\\|=1$

In theorem 14.34, we prove that any $\kappa$ regular in $V$ remains regular in $V[G]$ (the hard case is really $\kappa$ uncountable and this assumption is implicitely used later to say that $\bigcup_{\alpha<\lambda}A_{\alpha}$ is bounded). It may not be obvious why this is enough. First recall that for any ordinal $\alpha$ , $\operatorname{cf}^{V[G]}(\alpha)\leq\operatorname{cf}^{V}(\alpha)$ , ${|\alpha|}^{V[G]}\leq{|\alpha|}^{V}$ , and any (regular) cardinal in $V[G]$ is a (regular) cardinal in $V$ . Next we have,

	$\displaystyle\exists\alpha\in\mathrm{Ord},\operatorname{cf}^{V[G]}(\alpha)\leq% \operatorname{cf}^{V}(\alpha)$	$\displaystyle\implies\exists\alpha\in\mathrm{Ord},\operatorname{cf}^{V[G]}(% \operatorname{cf}^{V}(\alpha))\leq\operatorname{cf}^{V[G]}(\alpha)<% \operatorname{cf}^{V}(\alpha)$
		$\displaystyle\implies\exists\beta\textrm{ regular cardinal in }V,\textrm{ not % regular cardinal in }V[G]$
		$\displaystyle\implies\exists\beta\in\mathrm{Ord},\operatorname{cf}^{V[G]}(% \beta)<\beta=\operatorname{cf}^{V}(\beta)$

that is $\forall\alpha\in\mathrm{Ord},\operatorname{cf}^{V[G]}(\alpha)=\operatorname{cf% }^{V}(\alpha)$ is equivalent to ‘‘ $V$ and $V[G]$ have the same regular cardinals’’. Similarly, we can prove that $\forall\alpha\in\mathrm{Ord},{|\alpha|}^{V[G]}={|\alpha|}^{V}$ is equivalent to ‘‘ $V$ and $V[G]$ have the same cardinals’’.

The proof of theorem 14.34 shows that ‘‘ $V$ and $V[G]$ have the same regular cardinals’’ and so to complete the proof, it is enough to show that $\forall\alpha,\operatorname{cf}^{V[G]}(\alpha)=\operatorname{cf}^{V}(\alpha)$ implies $\forall\alpha,{|\alpha|}^{V[G]}={|\alpha|}^{V}$ . So suppose $\forall\alpha,\operatorname{cf}^{V[G]}(\alpha)=\operatorname{cf}^{V}(\alpha)$ and assume that there is $\alpha$ such that ${|\alpha|}^{V[G]}<{|\alpha|}^{V}$ . Consider the least such $\alpha$ . If $\beta={|\alpha|}^{V}$ then $\beta\leq\alpha$ so ${|\beta|}^{V[G]}\leq{|\alpha|}^{V[G]}<{|\alpha|}^{V}=\beta$ . By minimality of $\alpha$ , $\beta=\alpha$ and so $\alpha$ is a cardinal in $V$ . $\alpha$ is actually regular in $V$ . Otherwise, suppose $\operatorname{cf}^{V}(\alpha)<\alpha$ and let $\alpha=\bigcup_{\beta<\operatorname{cf}^{V}(\alpha)}X_{\beta}$ such that ${|X_{\beta}|}^{V}<{|\alpha|}^{V}$ . By minimality of $\alpha$ , we have ${|\operatorname{cf}^{V}(\alpha)|}^{V[G]}={|\operatorname{cf}^{V}(\alpha)|}^{V}$ and ${|X_{\beta}|}^{V[G]}={|X_{\beta}|}^{V}$ . Then ${|\alpha|}^{V[G]}={|\operatorname{cf}^{V}(\alpha)|}^{V[G]}\sup_{\beta<% \operatorname{cf}^{V}(\alpha)}{|X_{\beta}|}^{V[G]}={|\operatorname{cf}^{V}(% \alpha)|}^{V}\sup_{\beta<\operatorname{cf}^{V}(\alpha)}{|X_{\beta}|}^{V}={|% \alpha|}^{V}$ , a contradiction. Finally, we get $\operatorname{cf}^{V}(\alpha)=\alpha={|\alpha|}^{V}>{|\alpha|}^{V[G]}\geq% \operatorname{cf}^{V[G]}(\alpha)$ . This is again a contradiction and so $\forall\alpha,{|\alpha|}^{V[G]}={|\alpha|}^{V}$ .

Analysis of Lithium's algorithm

2013-01-22T00:00:00+01:00

I’ve recently been working on automated testcase reduction tools and thus I had the chance to study Jesse Ruderman’s Lithium tool, itself inspired from the ddmin algorithm. This paper contains good ideas, like for example the fact that the reduction could be improved if we rely on the testcase structure like XML nodes or grammar tokens instead of just characters/lines. However, the authors of the ddmin paper really don’t analyse precisely the complexity of the algorithm, except the best and worst case and there is a large gap between the two. Jesse’s analysis is much better and in particular introduces the concepts of monotonic testcase and clustered reduction where the algorithm performs the best and which intuitively seems the usual testcases that we meet in practice. However, the monotonic+clustered case complexity is only “guessed” and the bound $O(M\log_{2}(N))$ for a monotonic testcase (of size $N$ with final reduction of size $M$ ) is not optimal. For example if the final reduction is relatively small compared to $N$ , say $M=\frac{N}{\log_{2}(N)}=o(N)$ then $M\log_{2}(N)=N=\Omega(N)$ and we can’t say that the number of verifications is small compared to $N$ . In particular, Jesse can not deduce from his bound that Lithium’s algorithm is better than an approach based on $M$ binary search executions! In this blog post, I shall give the optimal bound for the monotonic case and formalize that in some sense the clustered reduction is near the best case. I’ll also compare Lithium’s algorithm with the binary search approach and with the ddmin algorithm. I shall explain that Lithium is the best in the monotonic case (or actually matches the ddmin in that case).

Thus suppose that we are given a large testcase exhibiting an unwanted behavior. We want to find a smaller test case exhibiting the same behavior and one way is to isolate subtestcases that can not be reduced any further. A testcase can be quite general so here are basic definitions to formalize a bit the problem:

•

A testcase $S$ is a nonempty finite sets of elements (lines, characters, tree nodes, user actions) exhibiting an “interesting” behavior (crash, hang and other bugs…)
•

A reduction $T$ of $S$ is a testcase $T\subseteq S$ with the same “interesting” behavior as $S$ .
•

A testcase $S$ is minimal if $\forall T\subsetneq S,T\text{ is not a reduction of }S$ .

Note that by definition, $S$ is a reduction of itself and $\emptyset$ is not a reduction of $S$ . Also the relation “is a reduction of” is transitive that is a reduction of a reduction of $S$ is a reduction of $S$ .

We assume that verifying one subset to check if it has the “interesting” behavior is what takes the most time (think e.g. testing a hang or user actions) so we want to optimize the number of testcases verified. Moreover, the original testcase $S$ is large and so a fast reduction algorithm would be to have a complexity in $o(|S|)$ . Of course, we also expect to find a small reduction $T\subseteq S$ that is $|T|=o(|S|)$ .

Without information on the structure on a given testcase $S$ or on the properties of the reduction $T$ , we must consider the $2^{|S|}-2$ subsets $\emptyset\neq T\neq S$ , to find a minimal reduction. And we only know how to do that in $O\left(2^{|S|}\right)$ operations (or $O\left(2^{|S|/2}\right)$ with Grover’s algorithm ;-)). Similarly, even to determine whether $T\subseteq S$ is minimal would require testing $2^{|T|}-2$ subsets which is not necessarily $o(|S|)$ (e.g. $|T|=\log_{2}{(|S|)}=o(|S|)$ ). Hence we consider the following definitions:

•

For any integer $n\geq 1$ , $S$ is $n$ -minimal if $\forall T\subsetneq S,|S-T|\leq n\implies T\text{ is not a reduction of }S$ .
•

In particular, $S$ is $1$ -minimal if $\forall x\in S,S\setminus\{x\}\text{ is not a reduction of }S$ .
•

$S$ is monotonic if $\forall T_{1}\subseteq T_{2}\subseteq S,\,T_{1}\text{ is a reduction of }S% \implies T_{2}\text{ is a reduction of }S$ .

Finding a $n$ -minimal reduction will give a minimal testcase that is no longer interesting if we remove any portion of size at most $n$ . Clearly, $S$ is minimal if it is $n$ -minimal for all $n$ . Moreover, $S$ is always $n$ -minimal for any $n\geq|S|$ . We still need to test exponentially many subsets to find a $n$ -minimal reduction. To decide whether $T\subseteq S$ is $n$ -minimal, we need to consider subsets obtained by removing portions of size $1,2,...,\min(n,|T|-1)$ that is $\sum_{k=1}^{\min(n,|T|-1)}\binom{|T|}{k}$ subsets. In particular whether $T$ is $1$ -minimal is $O(|T|)$ and so $o(|S|)$ if $T=o(|S|)$ . If $S$ is monotonic then so is any reduction $T$ of $S$ . Moreover, if $T\subsetneq S$ is a reduction of $S$ and $x\in S\setminus T$ , then $T\subseteq S\setminus\{x\}\subsetneq S$ and so $S\setminus\{x\}$ is a reduction of $S$ . Hence when $S$ is monotonic, $S$ is $1$ -minimal if and only if it is minimal. We will target $1$ -minimal reduction in what follows.

Let’s consider Lithium’s algorithm. We assume that $S$ is ordered and so can be identified with the interval $[1,|S|]$ (think for example line numbers). For simplicity, let’s first assume that the size of the original testcase is a power of two, that is $|S|=N=2^{n}$ . Lithium starts by $n-1$ steps $k=1,2,...,n-1$ . At step $k$ , we consider the chunks among the intervals $[1+j2^{n-k},(j+1)2^{n-k}]\ (0\leq j<2^{k})$ of size $2^{n-k}$ . Lithium verifies if removing each chunk provides a reduction. If so, it permanently removes that chunk and tries another chunk. Because $\emptyset$ is not a reduction of $S$ , we immediately increment $k$ if it remains only one chunk. The $n$ -th step is the same, with chunk of size 1 but we stop only when we are sure that the current testcase $T$ is $1$ -minimal that is when after $|T|$ attempts, we have not reduced $T$ any further. If $N$ is not a power of 2 then $2^{n-1}<N<2^{n}$ where $n=\lceil\log_{2}(N)\rceil$ . In that case, we apply the same algorithm as $2^{n}$ (i.e. as if there were $2^{n}-N$ dummy elements at the end) except that we don’t need to remove the chunks that are entirely in that additional part. This saves testing at most $n$ subtests (those that would be obtained by removing the dummy chunks at the end of sizes $2^{n-1},2^{n-2},...,1$ ). Hence in general if $C_{N}$ is the number of subsets of $S$ tried by Lithium, we have $C_{2^{n}}-n\leq C_{N}\leq C_{2^{n}}$ . Let $M$ be the size of the $1$ -minimal testcase found by Lithium and $m=\lceil\log_{2}(M)\rceil$ .

Lithium will always perform the $n-1$ initial steps above and check at least one subset at each step. At the end, it needs to do $M$ operations to be sure that the testcase is $1$ -minimal. So $C_{N}\geq\lceil\log_{2}(N)\rceil+M-1=\Omega(\log_{2}(N)+M)$ . Now, consider the case where $S$ monotonic and has one minimal reduction $T=[1,M]$ . Then $T$ is included in the chunk $[1,2^{m}]$ from step $k=n-m$ . Because $S$ is monotonic, this means that at step $k=1$ , we do two verifications and the second chunk is removed because it does not contain the $T$ (and the third one too if $N$ is not a power of two), at step $k=2$ it remains two chunks, we do two verifications and the second chunk is removed etc until $k=n-m$ . For $k>n-m$ , the number of chunk can grow again: 2, 4, 8… that is we handle at most $2^{1+k-(n-m)}$ chunks from step $n-m+1$ to $n-1$ . At step $k=n$ , a first round of at most $2^{m}$ verifications ensure that the testcase is of size $M$ and a second round of $M$ verifications ensure that it is $1$ -minimal. So $C_{N}\leq 1+\left(\sum_{k=1}^{n-m}2\right)+\left(\sum_{k=n-m+1}^{n}2^{1+k-(n-m% )}\right)+2^{m}+M=1+2(n-m)+2^{m}-1+2^{m}+M$ and after simplification $C_{N}=O(\log_{2}(N)+M)$ . Hence the lower bound $\Omega(\log_{2}(N)+M)$ is optimal. The previous example suggests the following generalization: a testcase $T$ is $C$ -clustered if it can be written as the union of $C$ nonempty closed intervals $T=I_{1}\cup I_{2}\cup...\cup I_{C}$ . If the minimal testcase found by Lithium is $C$ -clustered, each $I_{j}$ is of length at most $M\leq 2^{m}$ and so $I_{j}$ intersects at most 2 chunks of length $2^{m}$ from the step $k=n-m$ . So $T$ intersects at most $2C$ chunks from the step $k=n-m$ and a fortiori from all the steps $k\leq n-m$ . Suppose that $S$ is monotonic. Then if $c$ is a chunk that does not contain any element of $T$ then $T\setminus c$ is a reduction of $T$ and so Lithium will remove the chunk $c$ . Hence at each step $k\leq n-m$ , at most $2C$ chunks survive and so there are at most $4C$ chunks at the next step. A computation similar to what we have done for $T=[1,M]$ shows that $C_{N}=O(C(\log_{2}(N)+M))$ if the final testcase found by Lithium is $C$ -clustered. Note that we always have $M=o(N)$ and $\log_{2}(N)=o(N)$ . So if $C=O(1)$ then $C_{N}=O(\log_{2}(N)+M)=o(N)$ is small as wanted. Also, the final testcase is always $M$ -clustered (union of intervals that are singletons!) so we found that the monotonic case is $O(M(\log_{2}(N)+M))$ . We shall give a better bound below.

Now, for each step $k=1,2,...,n-1$ , Lithium splits the testcase in at most $2^{k}$ chunk and try to remove each chunk. Then it does at most $N$ steps before stopping or removing one chunk (so the testcase becomes of size at most $N-1$ ), then it does at most $N-1$ steps before stopping or removing one more chunk (so the testcase becomes of size at most $N-1$ ), …, then it does at most $M+1$ steps before stopping or removing one more chunk (so the testcase becomes of size at most $M$ ). Then the testcase is exactly of size $M$ and Lithium does at most $M$ additional verifications. This gives $C_{N}\leq\sum_{k=1}^{n-1}2^{k}+\sum_{k=M}^{N}k=2^{n}-2+\frac{N(N+1)-M(M-1)}{2}% =O(N^{2})$ verifications. This bound is optimal if $1\leq M\leq 2^{n-1}$ (this is asymptotically true since we assume $M=o(N)$ ): consider the cases where the proper reductions of $S$ are exactly the segments $[1,k]\ (2^{n-1}+1\leq k\leq 2^{n}-1)$ and $[k,2^{n-1}+1]\ (2\leq k\leq 2^{n-1}-M+2)$ . The testcase will be preserved during the first phase. Then we will keep browsing at least the first half to remove elements at position $2^{n-1}+2\leq k\leq 2^{n}-1$ . So $C_{N}\geq\sum_{k=2^{n-1}+2}^{2^{n}-1}2^{n-1}=2^{n-1}\left(2^{n-1}-2\right)=% \Omega(N^{2})$ .

We now come back to the case where $S$ is monotonic. We will prove that the worst case is $C_{N}=\Theta\left(M\log_{2}(\frac{N}{M})\right)$ and so our assumption $M=o(N)$ gives $M\log_{2}(\frac{N}{M})=\left(-\frac{M}{N}\log_{2}(\frac{M}{N})\right)N=o(N)$ as we expected. During the steps $1\leq k\leq m$ , we test at most $2^{k}$ chunks. When $k=m$ , $2^{m}\geq M$ chunks but at most $M$ distinct chunks contain an element from the final reduction. By monocity, at most $M$ chunks will survive and there are at most $2M$ chunks at step $m+1$ . Again, only $M$ chunks will survive at step $m+2$ and so on until $k=n-1$ . A the final step, it remains at most $2M$ elements. Again by monocity a first round of $2M$ tests will make $M$ elements survive and we finally need $M$ additional tests to ensure that the test case is minimal. Hence $C_{N}\leq{\sum_{k=1}^{m}2^{k}}+{\sum_{k=m+1}^{n}2M}+M=2^{m+1}-3+M(2(n-m)+1)=O(% M\log_{2}(\frac{N}{M}))$ . This bound is optimal: if $M=2^{m}$ , consider the case where $T=\{j2^{n-m}+1:0\leq j<2^{m}\}$ is the only minimal testcase (and $S$ monotonic) ; if $M$ is not a power of two, consider the same $T$ with $2^{m}-M$ points removed at odd positions. Then for each step $1\leq k\leq m-1$ , no chunks inside $[1,2^{n}]$ are removed. Then some chunks in $[1,2^{n}]$ are removed (none if $M$ is a power of two) at step $m$ and it remains $M$ chunks. Then for steps $m+1\leq k\leq n-1$ there are always exactly $2M$ chunks to handle. So $C_{N}\geq\sum_{m+1\leq k\leq n-1}2M=2M(n-m-2)=2M(\log_{2}(\frac{N}{M})-2)=% \Omega(M\log_{2}(\frac{N}{M}))$ .

We note that we have used two different methods to bound the number of verifications in the general monotonic case, or when the testcase is $C$ -clustered. One naturally wonders what happens when we combine the two techniques. So let $c=\lceil\log_{2}C\rceil\leq m$ . From step $1$ to $c$ , the best bound we found was $O(2^{k})$ ; from step $c$ to $m$ , it was $O(C)$ ; from step $m$ to $n-m$ it was $O(C)$ again ; from step $n-m$ to $n-c$ , it was $O(2^{1-k-(n-m)}C)$ and finally from step $n-c$ to $n$ , including final verifications, it was $O(M)$ . Taking the sum, we get $C_{N}=O(2^{c}+((n-m)-c)C+2^{(n-c-(n-m))}C+(n-(n-c))M)=O(C\left(1+\log_{2}{% \left(\frac{N}{MC}\right)}\right)+M(1+\log_{2}{C}))$ Because $C=O(M)$ , this becomes $C_{N}=O(C\log_{2}{\left(\frac{N}{MC}\right)}+M(1+\log_{2}{C}))$ . If $C=O(1)$ , then we get $C_{N}=O(\log_{2}(N)+M)$ . At the opposite, if $C=\Omega(M)$ , we get $C_{N}=\Omega(M\log_{2}{\left(\frac{N}{M}\right)})$ . If $C$ is not $O(1)$ but $C=o(M)$ then $1=o(\log_{2}{C})$ and $C\log_{2}(C)=o(M\log_{2}{C})$ and so the expression can be simplified to $C_{N}=O(C\log_{2}{\left(\frac{N}{M}\right)}+M\log_{2}{C})$ . Hence we have obtained an intermediate result between the worst and best monotonic cases and shown how the role played by the number of clusters: the less the final testcase is clustered, the faster Lithium finds it. The results are summarized in the following table:

	Number of tests
Best case	$\Theta(\log_{2}(N)+M)$
$S$ is monotonic ; $T$ is $O(1)$ -clustered	$O(\log_{2}(N)+M)$
$S$ is monotonic ; $T$ is $C$ -clustered ( $C=o(M)$ and unbounded)	$O(C\log_{2}{\left(\frac{N}{M}\right)}+M\log_{2}{C})$
$S$ is monotonic	$O\left(M\log_{2}\left(\frac{N}{M}\right)\right);o(N)$
Worst case	$\Theta(N^{2})$

Figure 0.1: Performance of Lithium’s algorithm for some initial testcase

S

of size

N

and final reduction

T

of size

M=o(N)

T

C

-clustered if it is the union of

C

intervals.

In the ddmin algorithm, at each step we add a preliminary round where we try to immediately reduce to a single chunk (or equivalently to remove the complement of a chunk). Actually, the ddmin algorithm only does this preliminary round at steps where there are more than 2 chunks for otherwise it would do twice the same work. For each step $k>1$ , if one chunk $c_{1}$ is a reduction of $S$ then $c_{1}\subseteq c_{2}$ for some chunk $c_{2}$ at the previous step $k-1$ . Now if $S$ is monotonic then, at level $k-1$ , removing all but the chunk $c_{2}$ gives a subset that contains $c_{1}$ and so a reduction of $S$ by monocity. Hence on chunk survive at level $k$ and there are exactly 2 chunks at level $k$ and so the ddmin algorithm is exactly Lithium’s algorithm when $S$ is monotonic. The ddmin algorithm keeps in memory the subsets that we didn’t find interesting in order to avoid repeating them. However, if we only reduce to the complement of a chunk, then we can never repeat the same subset and so this additional work is useless. That’s the case if $S$ is monotonic.

Finally, if $S$ is monotonic Jesse proposes a simpler approach based on a binary search. Suppose first that there is only one minimal testcase $T$ . If $k\geq\min T$ then $[1,k]$ intersects $T$ and so $U_{k}=S\setminus[1,k]\neq T$ . Then $U_{k}$ is not a reduction of $S$ for otherwise a minimal reduction of $U_{k}$ would be a minimal reduction of $S$ distinct from $T$ which we exclude by hypothesis. If instead $k<\min T$ then $[1,k]$ does not intersect $T$ and $U_{k}=S\setminus[1,k]\supseteq T\setminus[1,k]\supseteq T$ is a reduction of $S$ because $S$ is monotonic. So we can use a binary search to find $\min T$ by testing at most $\log_{2}(N)$ testcases (modulo some constant). Then we try with intervals $[1+\min T,k]$ to find the second least element of $T$ in at most $\log_{2}(N)$ . We continue until we find the $M$ -th element of $T$ . Clearly, this gives $M\log_{2}(N)$ verifications which sounds equivalent to Jesse’s bound with even a better constant factor. Note that the algorithm still works if we remove the assumption that there is only one minimal testcase $T$ . We start by $S=S_{1}$ and find $x_{1}=\max\{\min T:T\text{ is a minimal reduction of }S_{1}\}$ : if $k<x_{1}$ then $S\setminus[1;k]$ contains at least one mininal reduction with least element $x_{1}$ and so is a reduction because $S$ is monotonic. If $k\geq x_{1}$ then $S\setminus[1;k]$ is not a reduction of $S$ or a minimal reduction of $S\setminus[1;k]$ would be a minimal reduction of $S$ whose least element is greater than $x_{1}$ . So $S_{2}=S_{1}\setminus[1;x_{1}-1]$ is a reduction of $S_{1}$ . The algorithm continues to find $x_{2}=\max\{\min(T\setminus\{x_{1}\}):T\text{ is a minimal reduction of }S_{2}% ,\min(T)=x_{1}\}$ etc and finally returns a minimal reduction of $S$ . However, it is not clear that this approach can work if $S$ is not monotonic while we can hope that Lithium is still efficient if $S$ is “almost” monotonic. We remark that when there is only one minimal testcase $T=[1,M]$ , the binary search approach would require something like $\sum_{k=0}^{M-1}\log_{2}(N-k)=M\log_{2}(N)+\sum_{k=0}^{M-1}\log_{2}\left(1-% \frac{k}{N}\right)\geq M\log_{2}(N)+M\log_{2}\left(1-\frac{M}{N}\right)=M\log_% {2}(N)+o(M)$ . So that would be the worst case of the binary search approach whereas Lithium handles this case very nicely in $\Theta(\log_{2}(N)+M)$ ! In general, if there is only one minimal testcase $T$ of size $M\leq\frac{N}{2}$ then $\max(T)$ can be anywhere between $[M,N]$ and if $T$ is placed at random, $\max(T)\leq\frac{3}{4}N$ with probability at least $\frac{1}{2}$ . So the average complexity of the binary search approach in that case will be at least $\frac{1}{2}\sum_{k=0}^{M-1}\log_{2}(\frac{1}{4}N)=\frac{1}{2}M\log_{2}(N)+o(M)$ which is still not as good as Lithium’s optimal worst case of $O(M\log_{2}(\frac{N}{M}))$ …

Exercises in Set Theory: Ultrafilters and Boolean algebras

2012-11-27T00:00:00+01:00

Since my previous blog post, I have been working on exercises from chapter 7 of Thomas Jech’s book "Set Theory". I’ve published my solutions but I’m not yet done with exercises 7.22 and 7.28 which seem to be generalizations of theorems 4.3 and 3.2. I guess that before coming back to these two exercises, I’ll consider the content and exercises of subsequent chapters…

I had the opportunity to read chapter 7 again and to study more precisely the correctness of some claims (you know, those that are "clearly seen to be obvious via an easy proof based on a straightforward argument that makes them trivial" but actually take time before really being understood) I describe below the two statements for which the ratio between the time I spent on them and the length of the proof provided was the largest. I indicate the arguments I used to convince myself. I also had trouble to understand the proof of theorem 7.15 but most of the required arguments can actually be found in the exercices.

The first part of the chapter essentially deals with ultrafilters on an infinite set. In particular, a nonprincipal ultrafilter $U$ on $\omega$ is a Ramsey ultrafilter if for every partition $\{A_{n}|n<\omega\}$ of $\omega$ into $\aleph_{0}$ pieces such that $\forall n,A_{n}\notin U$ , there exists $X\in U$ such that $\forall n,\left|X\cap A_{n}\right|=1$ . We have:

Theorem 1.

The continuum hypothesis implies the existence of a Ramsey ultrafilter.

Proof.

The proof given in the book is to consider an enumeration $\mathcal{A}_{\alpha},\alpha<\omega_{1}$ of all partitions of $\omega$ and to construct by induction a sequence ${(X_{\alpha})}_{\alpha<\omega_{1}}$ of infinite subsets. $X_{\alpha+1}\subseteq X_{\alpha}$ is chosen to be included in some element of $\mathcal{A}_{\alpha}$ or to intersect each of them at, at most, one element. For $\alpha$ limit, $X_{\alpha}$ is chosen such that $X_{\alpha}\setminus X_{\beta}$ is finite for all $\beta<\alpha$ . Such a $X_{\alpha}$ is claimed to exist because $\alpha<\omega_{1}$ is countable. Finally, $D=\{X:X\supseteq X_{\alpha}\text{ for some }\alpha<\omega_{1}\}$ is claimed to be a Ramsey ultrafilter.

∎

There were several points that were not obvious to me. First, it’s good to count the number of partitions of $\omega$ . On the one hand, given any $A_{0}\subseteq\omega$ which is neither empty nor cofinite we can define a partition of $\omega$ by $\forall n<\omega,A_{n+1}=\left\{\min{\left({\omega\setminus{\cup_{m\leq n}A_{m% }}}\right)}\right\}$ and so there are at least $2^{\aleph_{0}}$ such partitions (cofinite subsets are in bijection with finite subsets and $|\omega^{<\omega}|=\aleph_{0}$ ). On the other hand, a partition is determined by the family of subsets $\{A_{n}|n<\omega\}$ and so there are at most ${\left(2^{\aleph_{0}}\right)}^{\aleph_{0}}=2^{\aleph_{0}}$ such partitions. Since we assume the continuum hypothesis $2^{\aleph_{0}}=\aleph_{1}$ we get the enumeration $\mathcal{A}_{\alpha},\alpha<\omega_{1}$ .

Next, I’m not even sure that as it is defined $D$ is a filter. Because $X_{\alpha}$ is infinite $\emptyset\notin D$ and $Y\supseteq X\supseteq X_{\alpha}\implies Y\in D$ but if $X\supseteq X_{\alpha}$ and $Y\supseteq X_{\beta}$ , it does not seem really clear which $X_{\gamma}\subseteq X\cap Y$ . However, if $U$ is a nonprincipal ultrafilter a remark from chapter 10 suggests that $U$ does not contain any finite set. By exercise 7.3, this is equivalent to the fact that $U$ does not contain any singleton $\{\alpha\}$ . This latter point is easy to verify: if we assume the contrary, because $U$ is nonprincipal, there exists $X\in U$ such that $X\nsupseteq\{\alpha\}$ and so $\kappa\setminus X\supseteq\{\alpha\}$ would be in $U$ . Hence any ultrafilter containing $D$ should also contain $E=\{X:\exists\alpha<\omega_{1},\exists x\in\kappa^{<\omega},X\supseteq X_{% \alpha}\setminus x\}$ . $E$ is a filter: the arguments above still hold and $X\supseteq X_{\alpha}\setminus x\wedge Y\supseteq X_{\beta}\setminus y\implies X% \cap Y\supseteq X_{\alpha}\cap X_{\beta}-(x\cup y)\supseteq X_{\alpha}% \setminus(X_{\alpha}\setminus X_{\beta}\cup x\cup y)$ and so we only need to verify that $X_{\alpha}\setminus X_{\beta}$ is finite if $\beta<\alpha$ . This is already what is claimed for $\alpha$ a limit ordinal. In general if $\gamma$ is the greatest limit ordinal such that $\gamma\leq\alpha$ then by construction, $X_{\gamma}\supseteq X_{\gamma+1}\supseteq...\supseteq X_{\alpha}$ and so $X_{\alpha}\setminus X_{\beta}$ is empty if $\gamma\leq\beta\leq\alpha$ and $X_{\alpha}\setminus X_{\beta}\subseteq X_{\gamma}\setminus X_{\beta}$ is finite if $\beta<\gamma$ . So we can replace $D$ by a "ultrafilter extending $E$ ". It’s not too difficult to check that a ultrafilter containing $D$ must be Ramsey. For any partition $\mathcal{A}_{\alpha}$ then by construction either $X_{\alpha+1}\subseteq A_{n}$ for some $n<\omega$ and so $A_{n}\in D$ or $\forall n<\omega,|X_{\alpha+1}\cap A_{n}|\leq 1$ . In the latter case we can find $X\supseteq X_{\alpha+1}$ (so $X\in D$ ) such that $\forall n,|X\cap A_{n}|=1$ (we pick one element from each $A_{n}$ such that $X_{\alpha+1}\cap A_{n}=\emptyset$ and put these elements into $X$ ).

Finally, the most difficult part was to understand how the sequence ${(X_{\alpha})}_{\alpha<\omega_{1}}$ can be built. We can choose $X_{0}$ arbitrarily as this set is not involved in the arguments above. For any $\alpha<\omega_{1}$ , if there exists some $A\in\mathcal{A}_{\alpha}$ such that $X_{\alpha}\cap A$ is infinite, we just set $X_{\alpha+1}=X_{\alpha}\cap A$ . Otherwise, since $X_{\alpha}=\cup_{A\in\mathcal{A}_{\alpha}}(X_{\alpha}\cap A)$ is infinite, $X_{\alpha}\cap A$ is nonempty for infinitely many $A\in\mathcal{A}_{\alpha}$ . Pick one element from each nonempty set to form the set $X_{\alpha+1}$ . By definition, $\forall A\in\mathcal{A}_{\alpha},\left|X_{\alpha+1}\cap A\right|\leq 1$ . Now we consider the case $\alpha<\omega_{1}$ limit. Let $\beta_{1}<\beta_{2}<...\beta_{n}<...$ be a cofinal $\omega$ -sequence in $\alpha$ . Note that it is the only place where we use the continuum hypothesis! For each $n<\omega$ , we define $Y_{n}=\bigcap_{i<n}X_{\beta_{i}}$ . We have $X_{\beta_{n}}$ infinite and $X_{\beta_{n}}\setminus X_{\beta_{n-1}}$ finite so $X_{\beta_{n}}\cap X_{\beta_{n-1}}$ is infinite. Similarly, $X_{\beta_{n}}\setminus X_{\beta_{n-1}}$ and $X_{\beta_{n-1}}\setminus X_{\beta_{n-2}}$ are finite so $X_{\beta_{n}}\cap X_{\beta_{n-1}}\cap X_{\beta_{n-2}}$ is infinite etc and finally $Y_{n}$ is infinite. Thus we can use a classical technique from Cantor (I can’t find a reference) and construct the set $X_{\alpha}=\{x_{n}|n<\omega\}$ by picking each $x_{n}\in Y_{n}\setminus\{x_{0},x_{1},...,x_{n-1}\}$ so that $X_{\alpha}\setminus X_{\beta_{n}}\subseteq\{x_{0},x_{1},...,x_{n-1}\}$ is finite. Moreover for each $\beta<\alpha$ consider some $\beta_{n}>\beta$ : $|X_{\alpha}\setminus X_{\beta}|\leq{|X_{\alpha}\setminus X_{\beta}|}+{|X_{% \beta}\setminus X_{\beta_{n}}|}$ is finite.

The second part of the chapter is about Boolean algebras. A Boolean algebra $C$ is a completion of a Boolean algebra $B$ if it is complete (every subset $X\subseteq C$ has a least upper bound $\sum X$ ) and if $B$ is a dense subalgebra of $C$ (for any $0<c\in C$ there exists $b\in B$ such that $0<b\leq c$ ). I got stuck on the apparently easy proof of the following lemma:

Lemma 1.

The completion of a Boolean algebra $B$ is unique up to isomorphism

Proof.

Let $C,D$ are two completions of $B$ . It is indicated in the book that the density of $B$ in $C$ and $D$ is used to prove that $\pi(c)=\sum^{D}\{u\in B|u\leq c\}$ defines an isomorphism between $C$ and $D$ . The example of $c\neq 0\implies\pi(c)\neq 0$ is given: indeed if $0<c$ we can find an element $b\in B$ such that $0<b\leq c$ and so $0<b\leq\pi(c)$ . ∎

Two or three pages before, it is mentioned that "if $\pi:C\rightarrow D$ is a one-to-one mapping such that $\forall u,v\in C,u\leq v\Leftrightarrow\pi(u)\leq\pi(v)$ then $\pi$ is an isomorphism". That seems to be the most natural method to prove the lemma above. However, this statement is false: consider the morphism of Boolean algebra $\pi:\{0,1\}\rightarrow\operatorname{\mathcal{P}}(\{0,1\})$ which is one-to-one but not surjective. Actually, the property $\forall u,v\in C,u\leq v\Leftrightarrow\pi(u)\leq\pi(v)$ implies the fact that $\pi$ is one-to-one, which becomes vacuous. So I suspect the author rather means "surjective" instead of "one-to-one". Indeed, in that case $\pi$ is a bijection that preserves the order $\leq$ in both directions and since all the operations of the Boolean algebra can be described in terms of this order, $\pi$ is an isomorphism.

So first, given $d\in D$ , we want to find $c\in C$ such that $\pi(c)=d$ . By symmetry, the natural candidate is $c=\sum^{C}\{u\in B|u\leq d\}$ . For all $u\leq d$ , we have $u\leq c$ and so $d\leq\sum^{D}\{u\in B|u\leq d\}\leq\sum^{D}\{u\in B|u\leq c\}=\pi(c)$ . If $d<\pi(c)$ then by density we can find $b\in B$ such that $0<b\leq\pi(c)-d$ . Hence $b\leq\pi(c)$ and $b\leq-d$ . The latter implies that for all $u\leq d$ we have $u\leq-b$ an so $\pi(c)\leq-b$ . Combining this with the former, we would have $b\leq\pi(c)\cdot-\pi(c)=0$ . Thus $\pi$ is surjective.

It remains to prove $\forall u,v\in C,u\leq v\Leftrightarrow\pi(u)\leq\pi(v)$ . If $u\leq v$ then $\pi(u)=\sum^{D}\{b\in B|b\leq u\}\leq\sum^{D}\{b\in B|b\leq v\}\leq\pi(v)$ . Let’s consider the contrapositive of the converse implication: suppose $u\nleq v$ . Then $u\cdot-v\neq 0$ and $0<\pi(u\cdot-v)$ (we use the hint given in the book). If we are able to prove $\pi(u\cdot-v)\leq\pi(u)\cdot-\pi(v)$ then we would have $0<\pi(u)\cdot-\pi(v)$ and so $\pi(u)\nleq\pi(v)$ as desired. It suffices to show two inequalities on the $\cdot$ and $-$ operations. Note that $\forall b\in B,\pi(b)=b$ . Let $w\in C$ . Let $w_{1},w_{2}\in C$ . Then for all $b\in B$ , $b\leq w_{1}\cdot w_{2}\implies b\leq w_{1},w_{2}\implies b=\pi(b)\leq\pi(w_{1}% ),\pi(w_{1})\implies b\leq\pi(w_{1})\cdot\pi(w_{2})$ so $\pi(w_{1}\cdot w_{2})\leq\pi(w_{1})\cdot\pi(w_{2})$ . Similarly, we have for all $b\in B$ , $b\leq-w\implies-b\geq w\implies-b=\pi(-b)\geq\pi(w)\implies b\leq-\pi(w)$ . So $\pi(-w)\leq-\pi(w)$ .

As a conclusion, I notice a similarity between these too cases: both have a sketchy proof based on a statement that seems incorrect (" $D$ is a filter", "any one-to-one mapping between Boolean algebra that preserves the order $\leq$ in both directions is an isomorphism"). Sometimes, I’m wondering if these kinds of inaccuracy are not intentional in order to force readers to try and find the proof themselves :-)

Master Thesis, LaTeXML and Quantum Groups (part 2)

2012-09-13T00:00:00+02:00

In this second part, I will try to explain briefly my contribution to Quantum Groups. I studied Lusztig’s results on the so-called restricted specialization of Quantum Group at a root of unity. As I indicated in a previous blog post, the case treated in the literature contains some restrictions on $l$ , for example $l$ is supposed to be odd. Basically, I generalized most results to an arbitrary value of $l$ (assuming only $l\geq 7$ , in order to factorize irreducible modules).

Let’s start with some notations. Let $\mathfrak{g}$ be a simple Lie algebra and $A={\left(a_{{ij}}\right)}_{{1\leq i,j\leq n}}$ its Cartan matrix. This matrix is symmetrizable and we denote by $D={(d_{i})}_{{1\leq i\leq n}}$ the associated diagonal matrix. Finally $\alpha _{1},\alpha _{2},...,\alpha _{n}$ are the simple roots of $\mathfrak{g}$ and $P$ the weight lattice. As it is well-known, we can consider the universal envelopping algebra $U(\mathfrak{g})$ which has the same representation theory as $\mathfrak{g}$ .

The first step is to define the quantum enveloping algebras $U_{q}(\mathfrak{g})$ as a deformation of $U(\mathfrak{g})$ . This is an algebra over ${\mathbb{Q}}(q)$ , the field of rational functions in the parameter $q$ . We let $q_{i}=q^{{d_{i}}}$ and for any integer $n\in{\mathbb{Z}}$ , we define a quantum version ${\left[{n}\right]}_{{q_{i}}}=\frac{q_{i}^{n}-q_{i}^{{-n}}}{q_{i}-q_{i}^{{-1}}}$ which obviously converges to the classical $n$ when $q\rightarrow 1$ . Similarly, one defines quantum versions ${\left[{n}\right]}_{{q_{i}}}!$ , ${\genfrac{[}{]}{0.0pt}{}{n}{k}}_{{q_{i}}}$ of the classical factorial and binomial coefficient. Then we define $U_{q}(\mathfrak{g})$ by generators $X_{i}^{\pm},K_{i}^{{\pm 1}}$ and relations $K_{i}K_{j}=K_{j}K_{i}$ , $K_{i}K_{i}^{{-1}}=K_{i}^{{-1}}K_{i}=1$ , $K_{i}X_{j}^{\pm}K_{i}^{{-1}}=q_{i}^{{\pm a_{{ij}}}}X_{j}^{\pm}$ ,

[X_{i}^{+},X_{j}^{-}]=X_{i}^{+}X_{j}^{-}-X_{j}^{-}X_{i}^{+}=\delta _{{i,j}}\frac{K_{i}-K_{i}^{{-1}}}{q_{i}-q_{i}^{{-1}}}

and for $i\neq j$ ,

\sum _{{r=0}}^{{1-a_{{ij}}}}(-1)^{r}{\genfrac{[}{]}{0.0pt}{}{1-a_{{ij}}}{r}}_{{q_{i}}}(X_{i}^{\pm})^{{1-a_{{ij}}-r}}X_{j}^{\pm}(X_{i}^{\pm})^{r}=0

If we think ” $K_{i}=e^{{(q-1)d_{i}H_{i}}}$ ”, a first order expansion of the above relations at $q\rightarrow 1$ gives the well-known Chevalley-Serre relations of $U(\mathfrak{g})$ . Actually, $U_{q}(\mathfrak{g})$ has the classical properties of $U(\mathfrak{g})$ , especially for the representation theory.

Now, we would like to specialize $q$ at an arbitrary complex number $\epsilon$ . This makes sense if $\epsilon\neq 0$ and is not a root of unity and in that case we again find a representation theory similar to the classical case. With some technical work, one can still define a specialization at a root of unity $\epsilon$ of order $l$ . To do that, one defines for all $k\in{\mathbb{N}}$ , the reduced power ${(X_{i}^{\pm})}^{{(k)}}={\left(X_{i}^{\pm}\right)}^{k}/{\left[{k}\right]}_{{q_{i}}}!$ and $\mathcal{A}={\mathbb{Z}}[q,q^{{-1}}]$ the ring of Laurent polynomials. Then $U_{\mathcal{A}}$ is the sub- $\mathcal{A}$ -algebra of $U_{q}(\mathfrak{g})$ generated by the $K_{i}^{{\pm 1}}$ and the ${(X_{i}^{\pm})}^{{(k)}}$ . One can define a $\mathcal{A}$ -basis of $U_{\mathcal{A}}$ (this is not trivial) from which we deduce the restricted specialization ${U_{\epsilon}^{{\mathrm{res}}}}(\mathfrak{g})$ .

At that point, one naturally wonders where we have a singularity, i.e. given $k\in{\mathbb{Z}}$ , when does $\epsilon _{i}^{k}-\epsilon _{i}^{{-k}}=0$ happen? Using basic arithmetic, that’s the case if and only if $k\equiv 0\mod m_{i}$ where $m=l/\gcd(l,2)$ and $m_{i}=m/\gcd(m,d_{i})$ . These $m_{i}$ ’s act like periods: we get ${\left(X_{i}^{\pm}\right)}^{{m_{i}}}=0$ (when $m$ is greater than one) and ${\left(K_{i}\right)}^{{2m_{i}}}=1$ . As a consequence, the important elements to consider become $X_{i}^{\pm}$ , ${(X_{i}^{\pm})}^{{(m_{i})}}$ , $K_{i}$ (which generate the whole algebra) as well as ${\genfrac{[}{]}{0.0pt}{}{{K_{i}};{0}}{m_{i}}}_{{\epsilon _{i}}}=\left[{(X_{i}^{+})}^{{(m_{i})}},{(X_{i}^{-})}^{{(m_{i})}}\right]$ (which, together with the $K_{i}$ , form a ”Cartan subalgebra”). Note that if $d_{i}$ is prime to $m$ then $m_{i}=m$ and if moreover $l$ is odd, $m_{i}=l$ . Not surprisingly, these are exactly assumptions used in the literature. The only difference in the most general case is that new signs ${(-1)}^{{m_{i}+1}}$ , $\epsilon _{i}^{{m_{i}}}={(-1)}^{{(l+1)d_{i}/\delta _{i}}}$ appear in the expressions.

If $V$ is a finite-dimensional ${U_{\epsilon}^{{\mathrm{res}}}}$ -module, we can define for all $\sigma\in{\{-1,1\}}^{n}$ and $\lambda\in P$ the weight space

\displaystyle V_{{\sigma,\lambda}}=\bigcap _{{1\leq i\leq n}}{\operatorname{Ker}{\mathop{\left(K_{i}-\sigma _{i}\epsilon _{i}^{{\lambda(\alpha _{i}^{\vee})}}1\right)}}\cap\bigcup _{{T\geq 1}}\operatorname{Ker}{\mathop{{\left({\genfrac{[}{]}{0.0pt}{}{{K_{i}};{0}}{m_{i}}}_{{\epsilon _{i}}}-\Delta _{i}(\lambda)\left\lfloor\frac{\lambda(\alpha _{i}^{\vee})}{m_{i}}\right\rfloor 1\right)}^{{T}}}}}

where $\Delta _{i}(\lambda)={(-1)}^{{m_{i}+1}}\sigma _{i}^{{m_{i}}}{(\epsilon _{i}^{{m_{i}}})}^{{\lambda(\alpha _{i}^{\vee})+1}}$ . Then $V=\bigoplus _{{\sigma _{,}\lambda}}V_{{\sigma,\lambda}}$ and $X_{j}^{\pm}V_{{\sigma\lambda}}\subseteq V_{{\sigma,\lambda\pm\alpha _{j}}}$ ; ${(X_{j}^{\pm})}^{{(m_{j})}}V_{{\sigma\lambda}}\subseteq V_{{\sigma,\lambda\pm m_{j}\alpha _{j}}}$ . In the same way as for the $U_{q}(\mathfrak{g})$ -modules, we deduce a classification of irreducible ${U_{\epsilon}^{{\mathrm{res}}}}$ -modules. They are all isomorphic to some ${V_{\epsilon}^{{\mathrm{res}}}}(\sigma,\lambda)$ for a type $\sigma$ and a highest weight $\lambda\in P^{+}$ . Note that the expression above is not too surprising: if we specialize a $U_{q}(\mathfrak{g})$ -module and consider the euclidean division of $\lambda(\alpha _{i}^{\vee})$ by $m_{i}$ then intuitively the remainder goes in the $K_{i}$ part while the quotient goes in the ${\genfrac{[}{]}{0.0pt}{}{{K_{i}};{0}}{m_{i}}}_{{\epsilon _{i}}}$ part. But specializing an irreducible $U_{q}(\mathfrak{g})$ -module does not necessarily provide an irreducible ${U_{\epsilon}^{{\mathrm{res}}}}$ -module and actually one can get by this method a reducible ${U_{\epsilon}^{{\mathrm{res}}}}$ -module which is not completely reducible. This latter property is a major difference with the classical case where all representations are semisimple.

Finally, thanks to the Hopf algebra structure of ${U_{\epsilon}^{{\mathrm{res}}}}$ , one can define tensor products of ${U_{\epsilon}^{{\mathrm{res}}}}$ -modules. We suppose $1\leq d_{i}<m$ and consider $\sigma\in\{-1,+1\}^{n},\lambda\in P^{+}$ . We note $\lambda=\lambda _{0}+\lambda _{1}$ where for all $1\leq i\leq n$ , $0\leq\lambda _{0}(\alpha _{i}^{\vee})<m_{i}$ and $\lambda _{1}(\alpha _{i}^{\vee})\equiv 0\mod m_{i}$ . Then

{V_{\epsilon}^{{\mathrm{res}}}}(\sigma,\lambda)\cong{V_{\epsilon}^{{\mathrm{res}}}}(\sigma,0)\otimes{V_{\epsilon}^{{\mathrm{res}}}}(1,\lambda _{0})\otimes{V_{\epsilon}^{{\mathrm{res}}}}(1,\lambda _{1})

${V_{\epsilon}^{{\mathrm{res}}}}(\sigma,0)$ is one-dimensional and ${V_{\epsilon}^{{\mathrm{res}}}}(1,\lambda _{0})$ becomes a ${U_{\epsilon}^{{\mathrm{fin}}}}$ -module where ${U_{\epsilon}^{{\mathrm{fin}}}}$ is the finite-dimensional subalgebras of ${U_{\epsilon}^{{\mathrm{res}}}}$ generated by the $X_{i}^{\pm},K_{i}^{{\pm 1}}$ . What about ${V_{\epsilon}^{{\mathrm{res}}}}(1,\lambda _{1})$ ? Lusztig’s answer in the case $l$ odd and prime with the $d_{i}$ ’s is that it is the pullback of the irreducible $U(\mathfrak{g})$ -module $V(\lambda _{1}/l)$ by an algebra morphism $\mathrm{Fr}:{U_{\epsilon}^{{\mathrm{res}}}}(\mathfrak{g})\rightarrow U(\mathfrak{g})$ . This does not seem to be possible in general and I suggested instead to consider the pullback of an irreducible $U_{{\epsilon^{{\prime}}}}(\mathfrak{g})$ -module by a morphism $\mathrm{Fr}:{U_{\epsilon}^{{\mathrm{res}}}}(\mathfrak{g})\rightarrow U_{{\epsilon^{{\prime}}}}(\mathfrak{g})$ where the order $l^{{\prime}}$ of $\epsilon^{{\prime}}$ is not too big. For example $\epsilon^{{\prime}}=1$ works in Lusztig’s framework, which is consistent with the fact that $U_{1}(\mathfrak{g})$ is close to $U(\mathfrak{g})$ . In general, you need to consider the parity of $l$ , whether it is a multiple of a $d_{i}$ and also whether the 2-adic order of $m$ is 0, 1 or 2. So I conjectured that $l^{{\prime}}$ can always be taken among the divisors of 24. I explained how to get this result under the assumption that $\mathrm{Fr}$ exists. There is a natural way one would like to define such a morphism but because I did not study in details the $\mathcal{A}$ -basis of $U_{\mathcal{A}}$ mentioned above, I can not state this for sure.

Apart from how to build $\mathrm{Fr}$ precisely, my work opens new perspectives. According to a paper of Sawin, the new cases $l$ even and multiple of $d_{i}$ are exactly those used for applications in physics and constructions of topological invariants. Hence my work could give new results in these areas. Also, one can rely on my work to study other generalizations: use of a symetrizable Kac-Moody algebra $\mathfrak{g}$ , tilting modules for an arbitrary order $l$ , non-restricted specialization for an arbitrary order $l$ etc

Quantum Groups at root of unity

2012-06-23T00:00:00+02:00

{% raw %}

All the papers or books I read so far on quantum groups about Lusztig’s restricted specialization consider only primitive roots of unity of odd order and additional conditions. In most cases, it is claimed that these restrictions could be removed without too much harm but details are not given. So I have tried to do the calculation myself. Below is what I find for the finite dimensional representations of ${U_{\epsilon}^{{\mathrm{res}}}}(\mathfrak{g})$ . Note that most authors consider the case $l=m=m_{i}$ odd. Then ${(-1)}^{{m_{i}}}=\epsilon _{i}^{{m_{i}}}=1$ and we can even restrict the study to the representations for which $\sigma$ is always 1. Indeed, all these assumptions make the expression below much simpler.

We consider $\mathfrak{g}$ a simple Lie algebra of rank $n$ and use standard notations $\alpha _{i}$ for roots, $\alpha _{i}^{\vee}$ for coroots and $P(\mathfrak{g})$ for the weight space. Let $l>2$ be an integer and $\epsilon$ a primitive $l$ -root of the unity. We let $m=\frac{l}{2}$ if $l$ is even and $m=l$ otherwise. For all $1\leq i\leq n$ , define $\epsilon _{i}=\epsilon^{{d_{i}}}$ , $\delta _{i}=\gcd(d_{i},m)$ , and $m_{i}=\frac{m}{\delta _{i}}$ . We assume that no $d_{i}$ is a multiple of $m$ , which is obviously true for $l$ large enough.

We denote ${U_{\epsilon}^{{\mathrm{res}}}}(\mathfrak{g})$ the restricted specialization as defined in Chari and Pressley’s guide to quantum groups and keep their notations for the elements $K_{i}$ , ${\genfrac{[}{]}{0.0pt}{0}{{K_{i}};{c}}{r}}_{{\epsilon _{i}}}$ , $X_{i}^{{\pm}}$ and ${(X_{i}^{\pm})}^{{(r)}}$ of ${U_{\epsilon}^{{\mathrm{res}}}}(\mathfrak{g})$ . Finally, we let $V$ be a finite ${U_{\epsilon}^{{\mathrm{res}}}}(\mathfrak{g})$ -module.

For any $\lambda\in P(\mathfrak{g})$ and $\sigma\in{\{-1,1\}}^{n}$ define

V_{{\sigma,\lambda}}=\bigcap _{{1\leq i\leq n}}{\operatorname{Ker}{\left(K_{i}-\sigma _{i}\epsilon _{i}^{{\lambda(\alpha _{i}^{\vee})}}1\right)}\cap\operatorname{Ker}{{\left({\genfrac{[}{]}{0.0pt}{0}{{K_{i}};{0}}{m_{i}}}_{{\epsilon _{i}}}-\Delta _{i}(\lambda)\left\lfloor\frac{\lambda(\alpha _{i}^{\vee})}{m_{i}}\right\rfloor 1\right)}}}

where $\forall i,\Delta _{i}(\lambda)={(-1)}^{{m_{i}+1}}\sigma _{i}^{{m_{i}}}{(\epsilon _{i}^{{m_{i}}})}^{{\lambda(\alpha _{i}^{\vee})+1}}$

For any $1\leq j\leq n$ , we have

X_{j}^{\pm}V_{{\sigma\lambda}}\subseteq V_{{\sigma,\lambda\pm\alpha _{j}}}

{(X_{j}^{\pm})}^{{(m_{j})}}V_{{\sigma\lambda}}\subseteq V_{{\sigma,\lambda\pm m_{j}\alpha _{j}}}

Moreover if $V$ is simple, $V=\bigoplus _{{\sigma,\lambda}}V_{{\sigma,\lambda}}$ .

$\square$

Cyclic HSP based on Subgroup Reduction

2012-04-22T00:00:00+02:00

In my master thesis on quantum computing, I designed a version of the Abelian HSP algorithm based on subgroup reduction. In this blog post, I wondered whether this algorithm could be extended to the Dedekindian HSP and if it would be more efficient than the standard one. I’m going to prove that the latter is false in the cyclic case, where the two algorithms have exactly the same associated Markov chain.

Recall that the HSP is - given a finite group $G$ , an unknown subgroup $H$ and an "oracle" $f$ hiding $H$ (i.e. a function on $G$ which factorizes on the coset space $G/H$ in a function with pairwise distinct values) - to determine the subgroup $H$ using a number of oracle calls polynomial in the size of $G$ .

In the standard Dedekindian HSP algorithm, $G$ is a dedekindian group and we use Fourier Sampling to measure a polynomial number of representations $\rho_{1},...\rho_{m}$ of $G$ such that $H\subseteq\operatorname{Ker}{\rho_{k}}$ . One proves that with high probability $H=\cap_{k}\operatorname{Ker}{\rho_{k}}$ . In my alternative algorithm, we measure $\rho_{1}$ and set $G_{1}=\operatorname{Ker}{\rho_{1}}$ . Then instead of continuing to work on the whole group $G$ , the idea is to move to a HSP problem on $G_{1}$ with hidden subgroup $H$ . Then do a Fourier Sampling on $G_{1}$ to move to a new subgroup $G_{2}$ and so forth. If $H\subsetneq G_{k}$ then there is a probability at least one half, that $G_{k+1}$ is at least twice smaller and so we see that we reach $H$ in an exponentially fast way.

Let’s now consider the cyclic HSP. We are given a cyclic group $G=\mathbb{Z}_{N}=\mathbb{Z}/{N\mathbb{Z}}$ and a hidden subgroup $H=d\mathbb{Z}_{N}$ for some $1\leq d\leq N$ (we exclude the case $H=0$ , which is easy) and an oracle $f$ . With this setting, one call to $f$ allows to get a Fourier sample $\lambda\frac{N}{d}$ for some random $\lambda\in\{0,1,...,d-1\}$ , with uniform probability. In the standard cyclic HSP, we repeat Fourier sampling to get $\lambda_{1}\frac{N}{d},\lambda_{2}\frac{N}{d},...\lambda_{m}\frac{N}{d}$ and deduce $d$ . For example, one can show that

P(\frac{N}{\gcd{\left(\lambda_{k}\frac{N}{d}\right)}_{1\leq k\leq m}}=d)\geq 1% -2^{-m/2}

In the algorithm I proposed, we build a sequence of polynomial length $H\subseteq G_{m}\subseteq G_{m-1}\subseteq...\subseteq G_{1}=G=\mathbb{Z}_{N}$ such that with high probability, $H=G_{m}$ . Clearly, all the $G_{k}$ ’s are cyclic groups and can be written $G_{k}=a_{k}\mathbb{Z}_{N}$ . Since they contain $H=d\mathbb{Z}_{N}$ , we have $a_{k}|d$ . From $a_{k}$ , we set $d_{k}=\frac{d}{a}_{k}$ and $N_{k}=\frac{N}{a}_{k}$ . We have the isomorphism $\phi_{k}:G_{k}\overset{\sim}{\to}\mathbb{Z}_{N_{k}}$ defined by $\phi_{k}(x)=\frac{x}{a}_{k}$ . Hence we can now work on $\mathbb{Z}_{N_{k}}$ , with a hidden subgroup $H_{k}=\phi_{k}(H)=d_{k}\mathbb{Z}_{N_{k}}$ and oracle $f_{k}=f\circ\phi_{k}^{-1}$ .

We start with $G_{1}=G=\mathbb{Z}_{N},H_{1}=H=d\mathbb{Z}_{N},a_{1}=1,d_{1}=d,N_{1}=N$ . At step $k$ , we perform a Fourier sampling that provides a random $\lambda_{k}\frac{N_{k}}{d_{k}}$ for some $\lambda_{k}\in\{0,1,...,d_{k}-1\}$ . Then, applying the Continued Fraction Algorithm on $\lambda_{k}\frac{N_{k}}{d_{k}}\frac{a_{k}}{N}=\frac{\lambda_{k}}{d_{k}}$ we get an irreducible fraction $\frac{p_{k}}{q_{k}}$ and a divisor $q_{k}$ of $d_{k}$ . Set $a_{k+1}=q_{k}a_{k}$ . Then $G_{k+1}=a_{k+1}\mathbb{Z}_{N}$ is a subgroup of $G_{k}$ . Because $q_{k}|d_{k}$ , we have ${a_{k}q_{k}}|{a_{k}d_{k}}=d$ and so $H$ is a subgroup of $G_{k+1}$ .

The general remark about the algorithm being exponentially fast is easy to see in the cyclic case. Indeed if $q_{k}=1$ above, then $\lambda_{k}=p_{k}d_{k}$ which happens only if $\lambda_{k}=0$ . If at step $k$ , we have $H\subsetneq G_{k}$ then $\lambda_{k}$ can take a non-zero value with probability at least one half. In that case $q_{k}\geq 2$ and $a_{k+1}\geq 2a_{k}$ . Hence $a_{k}$ grows exponentially fast until it becomes stationary and thus $a_{m}=d$ with very high probability.

We can prove that we get the same procedure using only Fourier sampling on the initial group $G$ . Suppose we have measured Fourier samples $\lambda_{1}\frac{N}{d},\lambda_{2}\frac{N}{d},...\lambda_{m}\frac{N}{d}$ . We want to construct the same sequence $a_{k}$ . Set $a_{1}=1$ . For each $1\leq k<m$ , we let $d_{k}=\frac{d}{a}_{k}$ and compute the irreducible fraction $\lambda_{k}\frac{N}{d}\frac{a_{k}}{N}=\frac{\lambda_{k}}{d_{k}}=\frac{p_{k}}{q% _{k}}$ . As above, we have ${a_{k}q_{k}}|{a_{k}d_{k}}=d$ and set $a_{k+1}={q_{k}a_{k}}$ . Now, write the euclidean division $\lambda_{k}=d_{k}\mu_{k}+\nu_{k}$ where $\mu_{k}\in\{0,...a_{k}-1\}$ and $\nu_{k}\in\{0,...d_{k}-1\}$ (with uniform probability for $\mu_{k}$ and $\nu_{k}$ too). We have $\frac{p_{k}}{q_{k}}=\frac{\lambda_{k}}{d_{k}}=\mu_{k}+\frac{\nu_{k}}{d_{k}}$ so $q_{k}$ is actually the denominator obtained when we compute the irreducible fraction $\frac{\nu_{k}}{d_{k}}$ . With this description, it becomes obvious that the two algorithms are equivalent (where $\nu_{k}$ corresponds to $\lambda_{k}$ in the previous algorithm).

I conjecture that the same phenomenon happens for a general Dedekindian group, but this may be more difficult to write it down…

Shelah’s PCF theory

2012-03-15T00:00:00+01:00

I have recently started to read a bit about Shelah’s theory of possible cofinalities. It is a quite interesting topic in Set Theory that I have wanted to study for a long time, but I did not really find time until now. I am going to give an overview in this blog post for people who are interested but also mostly to help me organizing my ideas and understanding of this subject.

First, this theory originated from (infinite) cardinal arithmetics, which is itself the Cantor’s invention that marked the birth of Set Theory. The addition and multiplication of infinite cardinals are trivial, since we have

\forall{κ,λ>=ℵ_{0}},\,κ+λ=κ.λ=\max{\{κ,λ\}}

The exponentiation is much more difficult (and interesting!). Here is Theorem 5.20 from Thomas Jech’s book "Set Theory":

Theorem 0.1

Let $λ$ be an infinite cardinal. Then for all infinite cardinal $κ$ , the value of $κ^{λ}$ is computed as follows, by induction on $κ$ :

1.

If $κ\leq λ$ then $κ^{λ}=2^{λ}$ .
2.

If there exists some $\mu<κ$ such that $\mu^{λ}\geq κ$ , then $κ^{λ}=\mu^{λ}$
3.
If $κ>λ$ and $\mu^{λ}<κ$ for all $\mu<κ$ , then
1. (a)
  
  if $\operatorname{cf}(κ)>λ$ , then $κ^{λ}=κ$ ,
2. (b)
  
  if $\operatorname{cf}(κ)\leq λ$ , then $κ^{λ}=κ^{\operatorname{cf}κ}$

This theorem reduces the problem of exponentiation to the determination of the continuum function $κ\mapsto 2^{κ}$ and, on singular cardinals $κ$ , of the gimel function $κ\mapsto\operatorname{\gimel}(κ)=κ^{\operatorname{cf}κ}$ . There are simular reductions of infinite sums and products to $\sup$ and exponentation operations, so computing these two functions is the crucial point of cardinal arithmetics.

Set theorists proved by forcing that we can not really say more about the value of the continuum for regular cardinals. Indeed, the only constraints are that the function must be increasing and must satisfy some consequences of König’s theorem (namely $2^{κ}>κ$ and $\operatorname{cf}(2^{κ})>κ$ ).

At the opposite it turns out that more properties can be proved for the singular case. For instance, we have the striking bound of $\operatorname{\gimel}(ℵ_{ω})$ mentioned on Shelah’s Web Archive:

Theorem 0.2

\operatorname{\gimel}(ℵ_{ω})=ℵ_{ω}^{ℵ_{0}}\leq 2^{ℵ_{0}}+ℵ_{ω_{4}}

Shelah developed his PCF theory in order to obtain these kinds of results. The basic idea is to consider an "interval" of regular cardinals $A$ i.e. with the property that any regular cardinal between two elements of $A$ is itself an element of $A$ . Then we define the set of possible cofinalities of ultraproducts of $A$ :

\operatorname{pcf}(A)=\left\{\operatorname{cf}{\left(\prod A/U\right)}|U\text{ is an ultrafilter on }A\right\}

where $\prod A/U$ is a totally ordered set: an ordinal function is smaller than another if it is smaller "almost everywhere" (i.e. on an element of the ultrafilter $U$ ). The cofinality of such a totally ordered set is defined in a similar way as for cardinals: it is shortest length of an unbounded sequence.

Now, $\prod A/U$ is a topological space (a basis is naturally given by the open intervals with respect to the total order) and so we can use topological methods to get information on it. In particular, this provides information on $\prod A$ , which in turn provides infomation on $\left|\prod A\right|$ , the infinite product of the (regular) cardinals in $A$ .

For example, if $A={\left\{\aleph_{n}\right\}}_{n=0}^{\infty}$ and if we assume that $\aleph_{\omega}$ is strong limit then one can show that $\operatorname{pcf}(A)$ is itself an interval of regular cardinals and has a maximum element which is $2^{\aleph_{\omega}}$ . So its elements are in the sequence of cardinals

\min{\operatorname{pcf}(A)}=\aleph_{\alpha},\aleph_{\alpha+1},\aleph_{\alpha+2 },\ldots,\aleph_{\alpha+\lambda}=\max{\operatorname{pcf}(A)}=2^{\aleph_{\omega}}

and obviouly, $\alpha+\lambda\leq{\left|\operatorname{pcf}A\right|}$ . But the size of $\operatorname{pcf}A$ is not greater than the number of ultrafilters of $A$ , which are sets of subsets of the countable set $A$ i.e. there are at most $2^{2^{\aleph_{0}}}$ many of them. Since we assume that $\aleph_{\omega}$ is a strong limit cardinal, we have $\alpha+\lambda\leq 2^{2^{\aleph_{0}}}<\aleph_{\omega}$ and finally

Theorem 0.3

If $\aleph_{\omega}$ is a strong limit cardinal then

2^{\aleph_{\omega}}<\aleph_{\aleph_{\omega}}

These are of course just two examples of what we can say about the values of the continuum and gimel functions at a singular cardinal (here for $\aleph_{\omega}$ , the smallest infinite singular cardinal). Several other results are known and the PCF theory also applies to other areas than cardinal arithmetics. Many beautiful topics that I would like to learn about…

Maximal Subgroup Reduction and HSP over Projective Linear Groups

2010-12-10T00:00:00+01:00

-- updated the 12th of December.

Among the recent algorithms for the Hidden Subgroup Problem, two are particularly interesting: one to find hidden subgroups of nil-2 groups by Ivanyos, Sanselme and Santha ; and another one to find conjugate stabilizer subgroups of $PSL (2, q)$ (and also those of the related groups $SL (2, q)$ and $PGL (2, q)$ ) by Denney, Moore and Russell. The idea of both algorithms is to reduce the initial problem to a HSP over a group for which we already know an efficient algorithm. However, the former only relies on Fourier Sampling over abelian groups while the latter requires Strong Fourier Sampling over some semidirect products of abelian groups. Also, considering the general approach to HSP that I have described in my Master Project, the technique of the former is only likely to work for solvable groups while the composition series of groups considered in the latter contain non-abelian simple groups... but can their ideas be combined together to get new HSP algorithms?

First, let's recall the concept of quantum hiding functions, which is a key ingredient of the first paper. In general, HSP algorithms use a classical oracle $f$ over $G$ which hides a subgroup $H$ i.e. is constant over each coset of $H$ and takes different values on different cosets. The quantum counterpart of $f$ is implemented by a quantum operator $U_{f}$ which takes $∣ g ⟩ ∣ 0 ⟩$ to $∣ g ⟩ ∣ f (g) ⟩$ . Some HSP algorithms use the so-called Strong Fourier Sampling: we create a uniform superposition of the states $∣ g ⟩ ∣ f (g) ⟩$ over all $g \in G$ , apply a Fourier Tranform on the first register and measure it. The idea of quantum hiding functions is to replace $U_{f}$ by any other quantum operator sending $∣ g ⟩ ∣ 0 ⟩$ to $∣ g ⟩ ∣ Ψ_{g} ⟩$ . In order to make the procedure works, we only need the $∣ Ψ_{g} ⟩$ 's to be unit states taking the same value over one coset of $H$ and orthogonal values on different cosets. These states may even change at each sampling.

To solve the general HSP and in particular the difficult case of simple groups, I have proposed maximal subgroup reduction. Basically, the idea is to find a maximal subgroup $K$ containing $H$ (there is always one except in the trivial case $H = G$ , which can be checked directly) using an HSP algorithm over $G$ and thus we reduce the problem to HSP over $K$ . Nevertheless, It is not clear at all how to build an efficient oracle hiding $K$ from an oracle $f$ hiding $H$ : the natural choice for $f (g)$ would be something gathering the values of $f$ over all the cosets of $H$ inside the coset $g K$ ... but unfortunately there can be exponentially many of them! Hence, it would maybe help to consider quantum hiding functions instead. For example, the states

$∣ Ψ_{g} ⟩ = \frac{1}{\sqrt{∣ K ∣}} \sum_{k \in K} ∣ f (g k) ⟩$ satisfy the expected properties. However it is not clear how to produce $∣ Ψ_{g} ⟩$ efficiently and even less without knowing the subgroup $K$ . But it is open whether we can find another method, maybe involving other states $∣ Ψ_{g} ⟩$ .

Let's turn our attention to HSP over $G = PSL (2, q)$ (or the other groups described in the paper of Denney, Moore and Russell) and see whether we can apply a maximal subgroup reduction. Note that the group $G$ acts on the projective line $F_{q} \cup {\infty}$ and thus we can define the stabilizer $G_{a}$ of a point $a$ , which is a maximal subgroup. Now, we assume that the hidden subgroup $H$ is included in some $G_{a}$ and that we have a way to build a quantum function hiding $G_{a}$ from an oracle hiding $H$ . Note that we have the inclusions

$G_{a} \cap G_{\infty} \subseteq G_{\infty} \subseteq G$

The idea of the paper is to restrict an oracle hiding $G_{a}$ to the subgroup $G_{\infty}$ and thus get a HSP over $G_{\infty}$ with hidden subgroup $G_{a} \cap G_{\infty}$ . Now, $G_{\infty}$ is isomorphic to some "friendly" semidirect product of abelian groups and thus the authors are able to determine $G_{a} \cap G_{\infty}$ by Strong Fourier Sampling and thus find the stabilizer $G_{a}$ . Note that their algorithm still works if we use quantum hiding function instead! Hence, with the hypothesis above, we are able to determine a maximal subgroup $G_{a}$ containing $H$ . To complete the algorithm, we need to find the hidden subgroup $H$ of $G_{a} ≅ G_{\infty}$ . However, the paper describes only the case of $G_{a} \cap G_{\infty} \subseteq G_{\infty}$ and I am not sure that the authors mean they have an algorithm for finding arbitrary subgroups of $G_{\infty}$ ...

Another open question: can we generalize this maximal subgroup technique to solve HSP over $PSL (2, q)$ ? We would like a way to distinguish among exponentially many (maximal) subgroups. Note that the list of (maximal) subgroups of $PSL (2, q)$ and $PGL (2, q)$ are known (see Dickson's book). Among the exponentially large subgroups (i.e. for which we would need an efficient HSP algorithm), there are subgroups isomorphic to $PSL (2, q_{0})$ , $PGL (2, q_{0})$ or to a semidirect product of a p-group and a cyclic group (including the famous dihedral group!). Note also that $PSL (2, q)$ is a normal subgroup of $PGL (2, q)$ , and so we could also imagine the use of a quotient reduction...

Can transfinite numbers be extended to more general algebraic structures?

2010-12-01T00:00:00+01:00

Infinite ordinals and cardinals are beautiful mathematical objects, extending in a natural way $ℕ$ and sharing most of its properties. I have always wondered whether it would be possible to generalize the constructions of $ℤ$ or $ℚ$ to get transfinite integers or rationals. In general, putting a group structure on classes extending $Ord$ or $Card$ is not possible as I indicated some years ago. Basically, we would have $1 + ℵ_{0} = ℵ_{0}$ (and also $1 + ω = ω$ ) which would imply $1 = 0$ . However, we can consider a slightly modified problem with weaker constraints, as follows. First we assume we have a class of cardinals $C \subseteq Card$ containing zero and stable by addition.

$(1) {\begin{array}{ll} 0 \in C \\ \forall κ, λ \in C, κ + λ \in C \end{array}$

We define the class $Z_{C} = C \cup C^{-}$ , where $C^{-} = {- κ ∣ κ \in C, κ \neq 0}$ is a representation of "negative" cardinals. As we have seen, the class $Z_{C}$ need not be a group. Nevertheless, we can try to find an equivalence class $R$ over $Z_{C}$ such that $G_{C} = \frac{Z_{C}}{R}$ is a group. Moreover, we want this equivalence class to be compatible with addition and opposite i.e.

(2) \begin{matrix} \forall κ, λ \in C, \bar{κ + λ} = \bar{κ} + \bar{λ} \\ \forall κ \in C, \bar{- κ} = - \bar{κ} \end{matrix}

Of course this implies that $\bar{0}$ is the identity element $0$ of the expected group. Similarly, we can settle the same problem for a class $C \subseteq Ord$ and + is now understood as the ordinal addition. Can we follow this schema to contruct interesting infinite algebraic structures?

For $C \subseteq Card$ , we have for any infinite $κ \in C$ the relation $\bar{κ} = \bar{κ + κ} = \bar{κ} + \bar{κ}$ so $\bar{κ} = 0$ . Hence $G_{C \cap ω} = G_{C}$ and the initial problem is reduced to the case where $C \subseteq ω$ contains only finite cardinals (= finite ordinals) and so the problem is still not really exciting. What about the case $C \subseteq Ord$ ? In general it is possible to have $β_{1} + γ = β_{2} + γ$ without $β_{1}, β_{2}$ being equal and so for any $α$ such that $β_{1} α, β_{2} α, γα \in C$ , we get that $\bar{β_{1} α} = \bar{β_{2} α}$ . In particular for any infinite ordinal $γ$ and $α \in C$ such that $γα \in C$ we obtain $\bar{α} = 0$ because $1 + γ = γ$ (if this is not obvious to you, a more general statement is proved below). As a consequence, more general assumptions on the class $C$ strongly limit the structure of the group. For example if $C$ is closed under an infinite sum ( $C = Ord$ for example) then $G_{C}$ is trivial. Similarly, if we require $C$ to be stable by multiplication (in order to define a ring structure for example) then $G_{C}$ is trivial whenever $C$ contains an infinite ordinal $γ$ (hence the remaining case is again $C \subseteq ω$ ). If $G_{C}$ is not trivial, we denote $α_{0} \in C$ the least element such that $\bar{α_{0}} \neq 0$ (in particular $α_{0} > 0$ ).

One additional natural hypothesis should be added. We know that for any ordinal $α < β$ there exists a unique ordinal $γ$ such that $α = β + γ$ . We would very like $\bar{γ}$ to match the difference " $- \bar{β} + \bar{α}$ ". For that purpose, we only require $γ$ to belong to $C$ . Hence we assume that

$(3) \forall α, β \in C, \forall γ \in Ord, α = β + γ \Rightarrow γ \in C$

Let's come back to the case $C \subseteq ω$ with the new assumption (3) and suppose that $G_{C}$ is not trivial. Then $α_{0} < ω$ and any finite $m \in C$ and be written $m = α_{0} q + r$ with $r < α_{0}$ . By (3), $r \in C$ and because $C$ is stable by finite sums, $\bar{m} = \bar{α_{0}} q + \bar{r} = \bar{α_{0}} q$ . Thus $G_{C}$ is the monogenic group generated by $\bar{α_{0}}$ .

What about the general case? Unfortunately, it turns out that if $G_{C}$ is not trivial it is still a monogenic group generated by $\bar{α_{0}}$ . To prove this, we need to recall some equalities on ordinals. First, for any $α > 0$ , we have $1 + ω^{α} = ω^{α}$ . This is clearly true for $α = 1$ and we prove the general case by induction on $α$ : $1 + ω^{α + 1} = 1 + ω ω^{α} = 1 + (1 + ω) ω^{α} = (1 + ω^{α}) {+ ω}^{α + 1} = ω^{α} + ω^{α + 1} = (1 + ω) ω^{α} = ω ω^{α} = ω^{α + 1}$ and $1 + ω^{λ} = sup_{0 < α < λ} ({1 + ω}^{α}) = sup_{0 < α < λ} ω^{α} = ω^{λ}$ . Now, if we have two ordinals $α < β$ and if $γ > 0$ is such that $β = α + γ$ we have $ω^{α} + ω^{β} = ω^{α} (1 + ω^{γ}) = ω^{α} ω^{γ} = ω^{β}$ . Finally, if we have two ordinals $γ_{1} < γ_{2}$ we can write their Cantor Normal Forms:

$(4) \begin{matrix} γ_{1} = ω^{β_{1}^{1}} k_{1}^{1} + ω^{β_{2}^{1}} k_{2}^{1} + \dots + ω^{β_{n}^{1}} k_{n}^{1} \\ γ_{2} = ω^{β_{1}^{2}} k_{1}^{2} + ω^{β_{2}^{2}} k_{2}^{2} + \dots + ω^{β_{n}^{2}} k_{m}^{2} \end{matrix}$

where $β_{1}^{2} \geq β_{1}^{i} > β_{2}^{i} > \dots > β_{2}^{i}$ and $k_{j}^{i}$ are positive integers. Using the equality $ω^{α} + ω^{β} = ω^{β}$ for $α < β$ it is clear that $γ_{1} + γ_{2} = γ_{2}$ if $β_{1}^{2} > β_{1}^{1}$ .

Now our element $α_{0} \in C$ can be written $α_{0} = ω^{β} k + γ$ where $ω^{β} k$ is the first term of its Cantor Normal Form. For any $δ \in C$ , we can write its euclidean division by $α_{0}$ : $δ = α_{0} l + ε$ where $ε < α_{0}$ . By the previous discussion, if $l \geq ω$ then we can write the Cantor Normal Forms of the two elements $α_{0} < δ$ as in (4) and see that $β_{1}^{2} > β_{1}^{1}$ . Hence $α_{0} + δ = δ$ and so $\bar{α_{0}} = 0$ , which is a contradiction. So $l < ω$ and because $C$ is stable by finite sums, $α_{0} l \in C$ and by the property (3) we get $ε \in C$ . This means that $\bar{δ} = \bar{α_{0}} l + \bar{ε} = \bar{α_{0}} l$ and hence $G_{C}$ is generated by $\bar{α_{0}}$ as claimed above.

As a conclusion, if $C \subseteq Card$ satisfies the properties (1), (2) above and $G_{C} = \frac{Z_{C}}{R}$ is a group then $C \subseteq ω \subseteq Ord$ . If $C \subseteq Ord$ satisfies properties (1), (2), (3) above and $G_{C} = \frac{Z_{C}}{R}$ is a group then $G_{C}$ is isomorphic to $ℤ$ or to $\frac{ℤ}{n ℤ}$ . Conversely, we can build these groups from $C = ω$ by defining the relation $R$ as the equality (respectively the equality modulo $n$ ). But we do not get any new groups...

Two Open Problems for the Subgroup-Reduction based Dedekindian HSP Algorithm

2010-08-05T00:00:00+02:00

One of my main idea after having studied the Hidden Subgroup Problem is that subgroup simplification is likely to play an important role. Basically, rather than directly finding the hidden subgroup $H$ by working on the whole initial group $G$ , we only try to get partial information on $H$ in a first time. This information allows us to move to a simpler HSP problem and we can iterate this process until the complete determination of $H$ . Several reductions of this kind exist and I think the sophisticated solution to HSP over 2-nil groups illustrates well how this technique can be efficient.

Using only subgroup reduction, I've been able to design an alternative algorithm for the Dedekindian HSP i.e. over groups that have only normal subgroups. Recall that the standard Dedekindian HSP algorithm is to use Weak Fourier Sampling, measure a polynomial number of representations $ρ_{1}, \dots, ρ_{m}$ and then get with high probability the hidden subgroup as the intersection of kernels $H = ⋂_{i} Ker ρ_{i}$ . When the group is dedekindian, we always have $H \subseteq Ker ρ_{i}$ . Hence my alternative algorithm is rather to start by measuring one representation $ρ_{1}$ , move the problem to HSP over $Ker ρ_{1}$ and iterate this procedure. I've been able to show that we reach the group $H$ after a polynomial number of steps, the idea being that when we measure a non-trivial representation the size of the underlying group becomes at least twice smaller. One difficulty of this approach is to determine the structure of the new group $Ker ρ_{i}$ so that we can work on it. However, for the cyclic case this is determination is trivial and for the abelian case I've used the group decomposition algorithm, based on the cyclic HSP. Finally I've two open questions:

Can my algorithm work for the Hamiltonian HSP i.e. over non-abelian dedekindian groups?
Is my algorithm more efficient than the standard Dedekindian HSP?

For the first question, I'm pretty sure that the answer is positive, but I admit that I've not really thought about it. For the second one, it depends on what we mean by more efficient. The decomposition of the kernel seems to increase the time complexity but since we are working on smaller and smaller groups, we decrease the space complexity. However, if we are only considering the numbers of sample, my conjecture is that both algorithms have the same complexity and more precisely yield the same markov process. In order to illustrate this, let's consider the cyclic case $G = ℤ_{18}$ and $H = 6 ℤ_{18}$ . The markov chain of the my alternative algorithm is given by the graph below, where the edge labels are of course probabilities and the node labels are the underlying group. We start at $G = ℤ_{18}$ and want to reach $H ≅ ℤ_{3}$ .

One can think that moving to smaller and smaller subgroups will be faster than the standard algorithm which always works on $ℤ_{18}$ . However, it turns out that the markov chain of the standard algorithm is exactly the same. The point being that while it is true that working over $ℤ_{9}$ or $ℤ_{6}$ provides less possibilities of samples (3 and 2 respectively, instead of 6 for $ℤ_{18}$ ) the repartition of "good" and "bad" samples is the same and thus we get the same transition probabilities. I guess it would be possible to formalize this for a general cyclic group. The general abelian case seems more difficult, but I'm sure that the same phenomenon can be observed.

Thoughts on the Dihedral Hidden Subgroup Problem (part 3)

2010-07-05T00:00:00+02:00

Back to the DHSP again (see part 1 and part 2 in previous blog posts)... This time, I've tried to approximate the value of $d$ rather than finding its parity. For convenience, we assume $N$ to be even and define $N' = 2 M = \frac{N}{2}$ . We consider for $a \in ℤ_{N}$ and $b \in ℤ_{2}$ the sets of $N'$ successive values $F_{a}^{b} = {f (a + i, b), i \in ℤ_{N'}}$

We suppose that we can efficiently create uniform superposition over these states. As in part 2, we have $F_{a}^{1} = F_{a - d}^{0} = F_{s}^{0}$ with $- N < s < N$ and we measure the tensor product of the uniform superpositions over $F_{a}^{1} = F_{s}^{0}$ and $F_{0}^{0}$ in the basis ${∣ x ⟩ ∣ x ⟩}_{x \in ℤ_{N}}$ , ${\frac{1}{\sqrt{2}} (∣ x ⟩ ∣ y ⟩ + ∣ x ⟩ ∣ y ⟩)}_{x < y \in ℤ_{N}}$ , ${\frac{1}{\sqrt{2}} (∣ x ⟩ ∣ y ⟩ - ∣ x ⟩ ∣ y ⟩)}_{x < y \in ℤ_{N}}$ . We define $l = ∣ F_{0}^{0} \cap F_{a}^{1} ∣$ the number of common elements.

The figure above shows that in the case $0 \leq s \leq N'$ , we have $l = N' - s$ . In general, the expression is

$l = ∣ N' - ∣ s ∣ ∣ = ∣ N' - ∣ d - a ∣ ∣$

Using an analysis similar to the one part 2, we get the probability to measure an element in ${\frac{1}{\sqrt{2}} (∣ x ⟩ ∣ y ⟩ - ∣ x ⟩ ∣ y ⟩)}_{x < y \in ℤ_{N}}$ :

$P = \frac{1}{2} [1 - {(\frac{l}{N'})}^{2}]$

Repeating the measurements, we can approximate $P$ , and consequently $l, s$ and finally $d$ . We can also play with the value $a$ , to improve the approximation. Unfortunately, using a number of repetitions polynomial in $\log N$ , it does not seem possible to bound the error better than $O (N)$ . It seems however that it is still a better result than the one that I obtained with the standard Coset Sampling method. Hence it is another hint of the fact that using a superposition over several oracles values would yield improvement.

I'm still wondering how to create efficiently an uniform superposition over several oracle values. One thing I've read in many papers, is given a superposition over elements $∣ x ⟩ ∣ f (x) ⟩$ , erase or uncompute the first register to get a superposition over elements $∣ f (x) ⟩$ . However, I'm not sure what are the requirements to apply such an operation. If it could work for the algorithm of part 2, then this would solve the case $N = 2^{n}$ .

Thoughts on the Dihedral Hidden Subgroup Problem (part 2)

2010-06-20T00:00:00+02:00

In a previous post, I've proposed a way to solve the Dihedral Hidden Subgroup Problem using a superposition of oracle evaluations. However, it seems to require a way to inverse the functions $x \mapsto f (x, .)$ which is a bit cheating since we do not have this feature for the classical problem. In this post, we will consider the case $M = \frac{N}{N'} = 2$ , $d = 2 d' + b$ and try to determine the parity $b$ of $d$ . If $N$ is a power of two, this is enough to find $d$ recursively. As in the previous post, the main idea is to consider the superpositions

$∣ ψ_{0} ⟩ = \frac{1}{\sqrt{N'}} \sum_{i = 0}^{N' - 1} ∣ f (2 i, 0) ⟩$

and

$∣ ϕ_{0} ⟩ = \frac{1}{\sqrt{N'}} \sum_{i = 0}^{N' - 1} ∣ f (2 i, 1) ⟩$

Using the equality $f (g (d, 1)) = f (g)$ we can rewrite the latter state

$∣ ϕ_{0} ⟩ = \frac{1}{\sqrt{N'}} \sum_{i = 0}^{N' - 1} ∣ f (2 i - b, 0) ⟩$

If $b = 0$ , the latter state is the same superposition as the former state, over $N'$ values of $ℤ_{N}$ . Otherwise, $b = 1$ and the latter state is the superposition over the $N'$ other values of $ℤ_{N}$ . We let $f_{i}^{b} = f (2 i - b, 0)$ and measure the product state $∣ ψ_{0} ⟩ ∣ ϕ_{0} ⟩$ in the basis ${∣ x ⟩ ∣ x ⟩}_{x \in ℤ_{N}}$ , ${∣ x ⟩ ∣ y ⟩ + ∣ x ⟩ ∣ y ⟩}_{x < y \in ℤ_{N}}$ , ${∣ x ⟩ ∣ y ⟩ - ∣ x ⟩ ∣ y ⟩}_{x < y \in ℤ_{N}}$ . It turns out that this allows to distinguish the two cases:

If $b = 0$ , the product contains vectors with the same coordinates $∣ f_{i}^{0} ⟩ ∣ f_{i}^{0} ⟩$ for $i \in ℤ_{N}$ . For the other vectors, we have a symmetry between the two coordinates. Hence we can group them by pairs $∣ f_{i_{1}}^{0} ⟩ ∣ f_{i_{2}}^{0} ⟩ + ∣ f_{i_{2}}^{0} ⟩ ∣ f_{i_{1}}^{0} ⟩$ for $i_{1} < i_{2} \in ℤ_{N}$ . Finally, the product state belongs to the space spanned by ${∣ x ⟩ ∣ x ⟩}_{x \in ℤ_{N}}$ and ${∣ x ⟩ ∣ y ⟩ + ∣ x ⟩ ∣ y ⟩}_{x < y \in ℤ_{N}}$ .
If $b = 1$ , the product contains vectors $∣ f_{i_{1}}^{0} ⟩ ∣ f_{i_{2}}^{1} ⟩$ where the two coordinates are not equal. We never have two symmetric vectors i.e. which are the same after permutating the two coordinates. Moreover, each vector $∣ f_{i_{1}}^{0} ⟩ ∣ f_{i_{2}}^{1} ⟩$ is the expression of a sum/difference (according to the respective order of $f_{i_{1}}^{0}$ and $f_{i_{2}}^{1}$ ) of two vectors $∣ x ⟩ ∣ y ⟩ + ∣ x ⟩ ∣ y ⟩$ and $∣ x ⟩ ∣ y ⟩ - ∣ x ⟩ ∣ y ⟩$ . So the product state belongs half to ${∣ x ⟩ ∣ y ⟩ + ∣ x ⟩ ∣ y ⟩}_{x < y \in ℤ_{N}}$ and half to ${∣ x ⟩ ∣ y ⟩ - ∣ x ⟩ ∣ y ⟩}_{x < y \in ℤ_{N}}$ .

Suppose that we have a procedure to create many states $∣ ψ_{0} ⟩$ and $∣ ϕ_{0} ⟩$ . We measure a state in the space ${∣ x ⟩ ∣ y ⟩ - ∣ x ⟩ ∣ y ⟩}_{x < y \in ℤ_{N}}$ with probability $\frac{b}{2}$ . So repeating the procedure a constant number of times allow to determine $b$ with high probability.

The only gap in the previous algorithm is whether we can create the states $∣ ψ_{0} ⟩$ and $∣ ϕ_{0} ⟩$ . As in the previous post, this is possible if the gates $V_{f, .}$ are available. Otherwise, we can create a superposition of states over $ℤ_{N'}$ and use the gate $U_{f}$ as well as a Quantum Fourier Transform to randomize the first coordinate. We get two random numbers $j_{1}, j_{2} \in ℤ_{N'}$ and the states

$∣ ψ_{j_{1}} ⟩ = \frac{1}{\sqrt{N'}} \sum_{i = 0}^{N' - 1} ⅇ^{\frac{2 ⅈ π j_{1} i}{N'}} ∣ f (2 i, 0) ⟩$

and $∣ ϕ_{j_{2}} ⟩ = \frac{ⅇ^{\frac{2 ⅈ π j_{2} d'}{N'}}}{\sqrt{N'}} \sum_{i = 0}^{N' - 1} ⅇ^{\frac{2 ⅈ π j_{2} i}{N'}} ∣ f (2 i - b, 0) ⟩$

Next, we can apply the same measurement as above. After a bit of calculation,we get the probabilities:

Space	case $b = 0$	case $b = 1$
${∣ x ⟩ ∣ x ⟩}_{x \in ℤ_{N}}$	$\frac{1}{N'}$	0
${∣ x ⟩ ∣ y ⟩ + ∣ x ⟩ ∣ y ⟩}_{x < y \in ℤ_{N}}$	$\frac{2}{{N'}^{2}} \sum_{i_{2} = 0}^{N' - 1} \sum_{i_{1} = 0}^{i_{2} - 1} \cos^{2} \frac{π}{N'} (j_{2} - j_{1}) (i_{2} - i_{1})$	$\frac{1}{2}$
${∣ x ⟩ ∣ y ⟩ - ∣ x ⟩ ∣ y ⟩}_{x < y \in ℤ_{N}}$	$\frac{2}{{N'}^{2}} \sum_{i_{2} = 0}^{N' - 1} \sum_{i_{1} = 0}^{i_{2} - 1} \sin^{2} \frac{π}{N'} (j_{2} - j_{1}) (i_{2} - i_{1})$	$\frac{1}{2}$

If we repeat the procedure several times, we need to take the means over the choice of $j_{1}, j_{2} \in ℤ_{N'}$ . Using some trigonometric identities, it is possible to write the sums of the first column $\frac{1}{2} + S + o (\frac{1}{N'})$ where $S$ is a sum of cosinus. Unfortunately, evaluating $S$ with a computer suggests that $S = Θ (\frac{1}{N'})$ . Hence, we can not distinguish the two cases with only a polynomial (in $\log N$ ) number of samplings.

It is still open whether we can find a variation to make such an algorithm efficient...

Thoughts on the Dihedral Hidden Subgroup Problem

2010-06-16T00:00:00+02:00

This morning, I've been thinking again about a way to solve the Hidden Subgroup Problem for the dihedral group $D_{N} = ℤ_{N} ⋊_{φ} ℤ_{2}$ (where $φ (1)$ is the inversion of $ℤ_{N}$ ). It is well-known that there exists a reduction to the case where the hidden subgroup is $H = ⟨ (d, 1) ⟩$ for some $d \in ℤ_{N}$ . In that case, if we fix $b \in {0, 1}$ , the elements $(i, b)$ for $i \in ℤ_{N}$ form a complete set of coset representatives, so $f$ is one-to-one over $ℤ_{N} \times {b}$ . Thus, $x \mapsto f (x, b)$ is a permutation of $ℤ_{N}$ , a property that will be used below to construct a unitary transform.

First let's recall the classical inductive method to determine $d$ . If $M$ is a divisor of $N$ , let $N' = \frac{N}{M}$ and write the euclidean division $d = M d' + r$ . If we are able to determine $r$ in polynomial time, then we can define the oracle $f' (x, y) = f (M x + r, y)$ for $x \in ℤ_{N'}$ and $y \in ℤ_{2}$ and obtain a dihedral HSP with hidden subgroup $⟨ (d', 1) ⟩$ . Since the prime factorization has only $O (\log N)$ factors, we can repeat this inductive step to get an efficient algorithm. Of course, this is likely to be useful only when the factors are small for otherwise it may be as much difficult to determine $r$ as to determine the whole $d$ . One interesting case is $N = 2^{n}$ and $M = 2$ : at each step, $N$ remains a power of 2 and we determine the parity of $d$ .

For $0 \leq l < M$ , we consider the $N'$ -dimensional subspace spanned by the vectors $∣ l N' + m ⟩$ for $0 \leq m < N'$ . Since these $M$ subspaces are orthogonal, we can construct a unitary transform by applying a Quantum Fourier Transform $F_{N'}$ on each subspace separately. Finally, we apply the permutation mentionned at the beginning for $b = 0$ . We get a unitary transform $W$ below where $x = M i + k$ is the euclidean division of $x$ by $M$ :

$\forall x \in Z_{N}, W ∣ x ⟩ = \frac{1}{\sqrt{N'}} \sum_{j = 0}^{N' - 1} ⅇ^{2 ⅈ π \frac{i j}{N'}} ∣ f (M j + k, 0) ⟩$

Now, let's consider the superposition (again, we see that it is well-defined by considering the permutation mentionned at the beginning for $b = 1$ ):

$\frac{1}{\sqrt{N'}} \sum_{i = 0}^{N' - 1} ∣ f (M i, 1) ⟩$

The state can be rewritten, using the fact that $f$ is constant on $H = ⟨ (d, 1) ⟩$ :

$\begin{aligned} \frac{1}{\sqrt{N'}} \sum_{i = 0}^{N' - 1} ∣ f (M i, 1) ⟩ & = \frac{1}{\sqrt{N'}} \sum_{i = 0}^{N' - 1} ∣ f (M i - d, 0) (d, 1) ⟩ \\ = \frac{1}{\sqrt{N'}} \sum_{i = 0}^{N' - 1} ∣ f (M i - d, 0) ⟩ \\ = \frac{1}{\sqrt{N'}} \sum_{i = 0}^{N' - 1} ∣ f (M i - (M d' + r), 0) ⟩ \\ = \frac{1}{\sqrt{N'}} \sum_{i = 0}^{N' - 1} ∣ f (M (i - d' + (N' - 1)) + (M - r), 0) ⟩ \\ = \frac{1}{\sqrt{N'}} \sum_{i = 0}^{N' - 1} ∣ f (M i + (M - r), 0) ⟩ \end{aligned}$

Applying $W^{- 1}$ to this state gives $∣ M - r \mod (M) ⟩$ and so the value of $r$ by a measurement in the standard basis.

Two questions remain: can we implement $W$ and create the state $∣ Ψ ⟩$ efficiently? If $N = 2^{n}$ and $M = 2$ , there are many simplifications because we are working with qubits. In that case, it is easy to see that the answer to the previous questions is affirmative if we have gates $V_{f, b} ∣ x ⟩ = ∣ f (x, b) ⟩$ implementing the permutation mentioned at the beginning. However, it is not clear how to build $V_{f, b}$ from the gate $U_{f}$ generally given in the classical presentation of the problem...

update: OK, there is also the trivial solution $N = M, N' = 1$ where the algorithm above is just computing $V_{f, 0}^{- 1} V_{f, 1} ∣ 0 ⟩ = ∣ N - d \mod (N) ⟩$ . However, I'm still wondering if there is a way to modify the previous method to work with $U_{f}$ .

Two contributions to the Hidden Subgroup Problem

2010-05-06T00:00:00+02:00

Today, I have finished the preliminary version of my report on the Hidden Subgroup Problem (HSP). For those who have never heard about it before, we are given a finite group $G$ , a subgroup $H$ and a function $f : G \to S$ . This function is constant on each (left) coset of $H$ but takes different values on distinct cosets. Said otherwise we have:

$\forall g_{1}, g_{2} \in G, f (g_{1}) = f (g_{2}) \Leftrightarrow g_{1} H = g_{2} H$ The purpose is to identify the "hidden" subgroup $H$ (for instance we can return a generating set) with a complexity polynomial in $\log ∣ G ∣$ . Although it may seem really abstract, this problem allows to describe various quantum algorithms exponentially faster than their known classical versions. This includes the famous Shor's algorithm for integer factorization, which can break the ubiquitous RSA cryptosystem. Shor's algorithm uses a solution to HSP over a cyclic group but we have quantum algorithms for the larger class of dedekindian groups as well as miscellaneous other groups. Unfortunately, these algorithms are not enough to solve HSP over the dihedral group or the symmetric group. These two cases are particularly interesting since the former could solve a lattice problem used in cryptography to establish security proofs while a solution to the latter provides an efficient algorithm for the graph isomorphism problem.

For the moment, I have essentially reviewed the major results on dedekindian, dihedral and symmetric groups. Even if there is not many new things for researchers working on that topic, I hope it could serve as an introduction to the subject and be a useful review of the state-of-the-art. In particular, I plan to complete the Table of Hidden Subgroup Problems which gives a good overview. There are at least two contributions I brought to the topic that I would like to discuss briefly in this blog post.

First, I have studied how Regev's algorithm works. As stated in his paper, we have a solution to the $poly (\log N)$ -uniqueSVP if we can find a hidden subgroup $H = ⟨ (d, 1) ⟩$ in the dihedral group $D_{N}$ using coset sampling on the whole group. This point is often overlooked in several papers, where the reader could understand that an arbitrary solution to the dihedral HSP works. Fortunately, Regev's algorithm is sufficiently close to the coset sampling procedure so that we can modify it to include some kinds of recursive algorithm. With such a modification, a solution like Kuperberg's algorithm (where we apply coset sampling on subgroups of $D_{N}$ ) becomes acceptable. Actually, we can also modify the algorithm to work with the generalized dihedral group. Another remark is about the degree of the approximation given by Regev's algorithm: I have shown that we have a solution to the $Θ (n^{\frac{1}{2} + 2 D})$ -uniqueSVP if the query complexity of the HSP algorithm used is $O ({(\log N)}^{D})$ , thus providing a more accurate statement.

Recently, I have considered a general approach to the HSP using the classical description of finite group via composition series and simple groups. The idea is that if $N$ is a non-trivial normal subgroup of $G$ we can split the problems on two HSPs over $\frac{G}{N}$ and $N$ . We can repeat the procedure until we reach simple groups. Because the length of a composition series is polynomial in $\log ∣ G ∣$ , we get a solution to the general HSP under two conditions: we have algorithms for HSP over simple groups and we can build efficient oracles on the "reduced" groups. I've succeeded to find an alternative algorithm for the abelian case based on that framework but for the dihedral/symmetric, it seems that the two previous assumptions are exactly what we are stuck on. However, it could still be useful to solve HSP over other groups.

Now, my priority is to review the results we have for some semidirect products but I hope to have time to study various open issues such that HSP over simple groups, HSP-based algorithms etc

	$\displaystyle\\|u\subseteq X\\|\implies\\|u\in Y\\|$	$\displaystyle\geq-\\|u=u^{\prime}\\|+\sum_{t\in\operatorname{dom}(Y)}\left(\\|u=t% \\|\cdot Y(t)\right)$
		$\displaystyle=\sum_{t\in\operatorname{dom}(Y)}-\left(\\|u\neq t\\|\cdot\\|u=u^{% \prime}\\|\right)$
		$\displaystyle\geq\sum_{t\in\operatorname{dom}(Y)}\\|u^{\prime}=t\\|=1$

Frédéric Wang Nélar

Stage d'implémentation des normes Web (session 2026)

Stage d'implémentation des normes Web

Five testharness.js mistakes

Introduction

1. Multiple tests in one loop

Proposed fix

2. Cleanup between tests

Proposed fix

3. Checking whether an exception is thrown

Proposed fix

4. Waiting for an event listener to be called

Proposed fix

5. Dealing with asynchronous resources

Proposed fix

My recent contributions to Gecko (3/3)

Introduction

Fetch priority

Implementation-defined prioritization

Finishing the implementation work

Internal WPT tests

Upstreamable WPT tests

Conclusion

Acknowledgments

My recent contributions to Gecko (2/3)

Introduction

The ‘content-visibility’ property

Viewport distance for content-visibility: auto

Dynamic changes to CSS ‘contain’ and ‘content-visibility’

Conclusion

My recent contributions to Gecko (1/3)

Introduction

Registered custom properties

Interpolation of registered custom properties

Custom properties in the cascade

Custom properties in animations

Conclusion

Flygskam and o caminho da Corunha

Prolegomenon

Paris as a departure or connection

Estimate of CO2 emissions

More cities and trains

Notes and references

Time travel debugging of WebKit with rr

Introduction

CMake configuration

Building and running WebKit

Using rr for WebKit debugging

Restarting playback from a known point in time

Older traces and parallel debugging

Infinite version of the Set card game

The Set Game

Trivial cases (λ≤2\lambda \leq 2 or μ≤1\mu \leq 1)

Not enough cards on the table (κ≤μ\kappa \leq \mu)

Finite number of values (λ<ℵ0\lambda < \aleph_0)

Singular number of values (cf(λ)<λ\mi{cf}(\lambda) < \lambda)

Finite number of features (μ<ℵ0\mu < \aleph_0)

Summary and open questions

MathML in Chrome 109

TL;DR

Timeline

A simple fraction

TEX-based layout

Exposing MathML magic

Integration with the web platform

Web Platform tests

Accessibility

Conclusion

Update on OpenType MATH fonts

Short blog post from Madrid's hotel room

Igalia's contribution to the Mozilla project and Open Prioritization

Igalia’s contribution to browser repositories

Open Prioritization

Two Crowdfunding Tasks for Firefox

Conclusion

Contributions to Web Platform Interoperability (First Half of 2020)

Default Aspect Ratio of Images

Scrolling

Intersection and Resize Observers

Resource Loading

Viewport distance for `content-visibility: auto`

Estimate of CO₂ emissions

Trivial cases ( $\lambda \leq 2$ or $\mu \leq 1$ )

Not enough cards on the table ( $\kappa \leq \mu$ )

Finite number of values ( $\lambda < \aleph_0$ )

Singular number of values ( $\mi{cf}(\lambda) < \lambda$ )

Finite number of features ( $\mu < \aleph_0$ )

T_EX-based layout

Script for $\alpha < \omega$