Web development: Ink API reduces latency when drawing with a digital pen

The Microsoft Edge team has introduced a new web API for drawing with a digital pen in the browser. The Ink API is initially available through Edge’s dev channel releases, but the team appears to have implemented it upstream in the Chromium open-source project. On the operating system side, it works with another new API for the upcoming Windows 11. For other operating systems such as Windows 10 and Linux, a polyfill in the Edge browser does the job.

The Ink API is intended to significantly reduce latencies when drawing with a pen as an input device in the browser. The announcement on the Microsoft Edge blog promises an improvement of up to 240%. One of the motivations of the project is probably to optimize the use of the stylus on Microsoft’s Surface devices.

Chromium, which powers Microsoft Edge and Google Chrome, among others, traditionally processes pen events in the browser process and then passes them to the JavaScript event loop. Depending on the load on the application’s main thread, there are noticeable delays between the physical input and the arrival of the event in the application.

The new Ink API uses a new Windows 11 API for drawing lines, which forwards events directly to the OS composer. This deals with other pen input points outside of the browser event loop and draws the appropriate lines directly.

The Ink API processes pen events and initially draws lines independent of the web application’s canvas and event loop.

(Picture: Microsoft)

For Windows 10 and Linux, a polyfill takes over the work of the Windows 11 API and attempts to collect inputs based on past and last known events. PointerEvent plan to draw the line. The team intends to further optimize the algorithm for forecasting in the coming months.

The Ink API is implemented upstream in Chromium and enabled by default in all Chromium-based browsers. If you want to use it in your own web applications, all you need is a InkPresenterRequest the object from the browser and then feed it with pointer events, as the following code sample from the blog shows:

try {
   let canvas = document.querySelector("#canvas");
   let presenter = 
     await navigator.ink.requestPresenter({presentationArea: canvas});

   window.addEventListener('pointermove', function(event) {
      // Gather all of the points that have come 
      //  from the queue of events.
      let points = event.getCoalescedEvents();

      points.forEach( p => {
         // ... Call renderInkStroke() for your application ...
      });

      //... Render the ink strong belonging to the dispatched event

      // Update the presenter with the last rendered point 
      // and give it a style
      presenter.updateInkTrailStartPoint(event, {
         color: "#7851A9",
         diameter: event.pressure * 4
      });
   });

the API specification is an unofficial project of the Web Platform Incubator Community Group (WICG) of the World Wide Web Consortium (W3C). It is explicitly not a W3C standard and is currently not intended as such. If you want to test the Ink API preview, you first need the latest Microsoft Edge preview from the dev channel as well as an insider preview build of Windows 11.

more details can be found on the Microsoft Edge remove blog.


(rm)

James S. Joseph