Document Analysis
Abstract
1 / Domain Adaptation for HTR
We will extend our existing adaptation pipeline to page-level HTR, investigating multilingual transfer, language-aware feature learning, page context, and joint adaptation of segmentation and recognition. We will also study how to improve performance in low-resource languages and historical collections where annotations are scarce and visual degradation is common. The final goal is a general adaptation framework that can be applied to modern and historical handwritten pages with minimal additional annotation.
2 / Handwriting Reconstruction From an IMU-based Digital Pen
Handwriting reconstruction from an IMU-based digital pen will continue , as our current shared-representation and domain-specific-head design has shown promise. We will improve generalization across writers, pen motion styles, paper surfaces, and recording conditions, while exploring more flexible adaptation strategies, stronger sequence modeling, and domain-aware training schemes that explicitly model the variability introduced by the physical writing support. The goal is a more robust reconstruction system which will create a bridge between handwriting analysis and sensor-based trajectory estimation.
3 / OSR for Text and Beyond
Our previous methods have demonstrated that our existing models can recognize some minor scripts that commercial VLMs from early 2026 cannot, but they are stuck on word level text. This stage aims to expand beyond word level and text, trying to model human’s perception of ‘same type of pattern’ on paragraph level and on other tasks like general purpose image classification and instance segmentation.
4 / Extended Pipeline on Recognition of Hieratic Characters
We extend a pipeline for the recognition of ancient Egyptian hieratic characters, investigating the capacity of models such as YOLO, SAM3 and DINOv3 to suggest character candidates from a given document image. Subsequently, we will establish connections between neighbouring characters and incorporate a learned language model to adjust the proposed characters based on their visual and linguistic context.
5 / Efficient layout analysis and understanding of complex historical documents
We will develop methods for historical document segmentation, complex layout analysis, table recognition, and text-line extraction to support HTR. A central objective is to develop compact versions of models like HI-SAM and benchmark them against YOLO, U-Net, Doc-UFCN, and lightweight vision Transformers. We will combine segmentation with geometric and spatial relationships to reconstruct document structure and reading order. Recognition models will assess how layout errors affect transcription. Experiments will include architecture optimization, cross-dataset evaluation, ablation studies, and end-to-end validation.
6 Ensemble-based self-training framework for adapting HTR systems
We will develop an ensemble-based self-training framework for adapting HTR systems to previously unseen datasets without manual annotations. The project will investigate how predictions from complementary recognition models can be combined to identify reliable transcriptions and reduce the influence of incorrect outputs. Selected pseudo-labeled samples will be used to iteratively adapt the models to new writing styles, languages, and document conditions. The final goal is a robust and scalable framework for improving HTR performance on low-resource and historical collections while minimizing labeling effort.
