Defining requirements for an all-new digital pen architecture, from scratch

Overview

The Surface Pro 3 pen was a complete architectural shift from the previous Wacom-based hardware stack. There was no existing playbook for evaluating a physical pen artifact — most prior research (including Microsoft's own) focused narrowly on latency.

Started with a literature review to see what had actually been studied. Latency had decent coverage; almost nothing existed on the pen as a physical object — size, grip, weight — either digitally or for analog pens and pencils.

Approach

Built a physical library: purchased a wide range of highly-rated pens and measured their characteristics directly (size, weight, balance).

Went to domain experts outside HCI research — digital artists using Wacom tablets, game/model creators, and pen shopkeepers/enthusiasts — to surface the dimensions people actually use to judge a pen as good or bad. Cross-checked against online pen reviews for the same attributes.

Worked closely with engineering to understand the new hardware architecture and its tradeoffs, so research could flag "proceed as-is" vs. "pause and fix" early enough to matter.

Devised a novel comparison method: since the existing digital pen used a conductive tip detected via the screen's touch digitizer, realized a graphene tip would conduct (working with the digitizer) while also writing on real paper — so overlaid paper on the screen, had people write, and directly compared the digitized trace against the physical ink trace.

Owned the pen experience end-to-end, not just research: hardware design, reliability, firmware, roadmap, software, and ecosystem (apps, partners). Ran 30+ studies total, developing 3 canonical tasks grounded in a note-taking scenario — handwriting, drawing, and tracing/etching — and running weekly studies against custom-made prototypes, inventing new ergonomics and UX metrics as none of the existing ones fit.

Backed the case for the barrel button redesign with hard data, not just opinion: logged pen usage from 23 Surface Pro users over 6+ days (median 3 button presses per 24h vs. 2 eraser-tip uses), and audited 70 Windows Store apps — 84% didn't respond to the pen button at all. User interviews echoed the same finding across platforms: "I don't understand what it does," "I don't use the select button ever."

Also applied the same rapid-cycle research model elsewhere in the pen ecosystem: Surface Music Kit got a usability study every two weeks against new builds, alternating focus between expert and novice musicians to resolve tradeoffs between the two; ClickNote (the locked-device note-capture feature) was validated through biweekly studies checking both standalone comprehension and how it compared to alternative capture methods.

Impact

Informed product specs across every critical pen dimension: end-to-end ink latency, pen-to-tablet friction, button placement and force, weight and balance, girth and length, and tip material.

Co-invented two patented technologies: a locked-device note-capture experience (ClickNote, US 9,372,571 and continuations) and a pen-to-digitizer binding protocol transmitting richer orientation/grip data than Bluetooth could support (US 9,232,409 and continuation).

Redesigned button force reduced inadvertent actuations by 80%; a new pen variant with no button at all was added to the lineup as a direct result of the button research.

Findings and methods carried forward into Surface Pro 4 and Surface Book pen requirements, and into DirectInk, the OS-level inking engine.

Published the work at CHI 2017 ("As We May Ink?"), grounded in a companion study of 45 people's analog-vs-digital pen habits (670+ coded usage instances) — now cited 45+ times.

Authored "The State of Pen," a summary of everything learned about digital pen and ink, referenced at Microsoft for years afterward.