Digital remark-taking is geting well-understandnity, recommending a durable, editable, and easily indexable way of storing remarks in a vectorized create. However, a substantial gap remains between digital remark-taking and traditional pen-and-paper remark-taking, a train still prefered by a convey inantity of people.
Bridging this gap by altering a remark apshowr’s physical writing into a digital create is a process called derfinishering. The result is a sequence of strokes, or trajectories of a writing instrument appreciate a pen or finger, enrolled as points and stored digihighy. This is also understandn as an “online” recurrentation of writing, or “digital ink”.
The conversion to digital ink recommends participaters who still prefer traditional handwritten remarks access to their remarks in a digital create. Instead of sshow using chooseical character recognition (OCR), which would allow the writing to be transcribed to a text record, by capturing the handwritten records as a accumulateion of strokes, it’s possible to recreate them in a create that can be edited freely by hand in a way that is more organic. It allows the participater to create records with a down-to-earth watch that seizes their handwriting style, rather than sshow a accumulateion of text. This recurrentation allows the participater to tardyr study, alter or finish their handwritten remarks, which gives their remarks increased durability, seamless organization and integration with other digital satisfied (images, text, joins) or digital aidance.
For these reasons, this field has geted meaningful interest in both academia and industry, with gentleware solutions that digitize handwriting and challengingware solutions that leverage inincreateigent pens or one-of-a-kind paper for seize. The need for compriseitional challengingware and accompanying gentleware stack is, however, an obstacle for wider adchooseion, as it creates both onboarding friction and carries compriseitional expense for the participater.
With this in mind, in “InkSight: Offline-to-Online Handwriting Conversion by Lgeting to Read and Write”, we give an approach to derfinishering that can apshow a picture of a handwritten remark and reshift the strokes that created the writing without the need for one-of-a-kindized supplyment. We also erase the reliance on standard geometric creates, where gradients, contours, and shapes in an image are participated to reshift writing strokes. Instead, we train the model to create an comfervent of “reading”, so it can determine written words, and “writing”, so it can output strokes that watch appreciate handwriting. This results in a more strong model that carry outs well atraverse diverse scenarios and materializeances, including challenging airying conditions, occlusions, etc. We split the model inference code and results in a supplementary Colab.
