SmolDocling, from Hugging Face and IBM Research, is the ultra-compact (256M) open VLM for end-to-end document conversion. Extracts text, layout, tables, code, and more from images.
Check out SmolDocling, a new open-source vision-language model from Hugging Face and IBM Research! True to its name, it's incredibly small – only 256M parameters! – yet it's designed for full, end-to-end document conversion.
You feed it an image of a document page (a scanned PDF, a photo, etc.), and it outputs a structured representation (called "DocTags") that includes everything:
📝 Text (OCR): It extracts the text, of course. 📑 Layout: It understands the page layout (paragraphs, headings, lists, etc.). 📊 Tables: It extracts table structure and content. 💻 Code: It recognizes and formats code blocks (with indentation!). ➕ Equations: It handles mathematical formulas. 🖼️ Figures: It identifies figures and links captions.
The key is that it does all of this in a single model, end-to-end, unlike traditional approaches that use separate OCR, layout analysis, and table extraction tools. And it does it with a model that's tiny compared to most VLMs.
It's built on SmolVLM (also open-source) and achieves competitive results with models many times its size.
@hamza_afzal_butt Good question! It's primarily English-focused, but the OCR should handle other languages. Best to test it with your specific documents, though, as mixed-language performance isn't specifically benchmarked.
Report
Automated document parsing is a great solution! 👀
Report
I used Docling a couple of months ago, it was already cool, now this mini version sounds even cooler!
Report
This being able to beat some of the classical OCR solutions in time as well as accuracy was not what i was thinking i would get to see, but recently in a project of doing on screen OCR for valuable info extraction of what is user doing currently on PC and all to be stored locally, i went through 30-40 models and classical OCR tools and found this to be a good balance
Replies
Flowtica Scribe
Hi everyone!
Check out SmolDocling, a new open-source vision-language model from Hugging Face and IBM Research! True to its name, it's incredibly small – only 256M parameters! – yet it's designed for full, end-to-end document conversion.
You feed it an image of a document page (a scanned PDF, a photo, etc.), and it outputs a structured representation (called "DocTags") that includes everything:
📝 Text (OCR): It extracts the text, of course.
📑 Layout: It understands the page layout (paragraphs, headings, lists, etc.).
📊 Tables: It extracts table structure and content.
💻 Code: It recognizes and formats code blocks (with indentation!).
➕ Equations: It handles mathematical formulas.
🖼️ Figures: It identifies figures and links captions.
The key is that it does all of this in a single model, end-to-end, unlike traditional approaches that use separate OCR, layout analysis, and table extraction tools. And it does it with a model that's tiny compared to most VLMs.
It's built on SmolVLM (also open-source) and achieves competitive results with models many times its size.
You can try SmolDocling yourself here.
Flowtica Scribe
@hamza_afzal_butt Good question! It's primarily English-focused, but the OCR should handle other languages. Best to test it with your specific documents, though, as mixed-language performance isn't specifically benchmarked.
Automated document parsing is a great solution! 👀
I used Docling a couple of months ago, it was already cool, now this mini version sounds even cooler!
This being able to beat some of the classical OCR solutions in time as well as accuracy was not what i was thinking i would get to see, but recently in a project of doing on screen OCR for valuable info extraction of what is user doing currently on PC and all to be stored locally, i went through 30-40 models and classical OCR tools and found this to be a good balance