STUDY OF MACHINE LEARNING TOOLS AND ALGORITHMS FOR RECOGNITION AND DIGITALISATION OF SALES RECEIPTS
DOI:
https://doi.org/10.33042/2522-1809-2023-6-180-7-11Keywords:
dataset, neural network, digital technologies, binarisation, sales receipt, classification, OCRAbstract
This article discusses the issue of processing images of sales receipts for subsequent text information extraction using OCR methods. This application is helpful for maintaining a family budget or for conducting accounting in small companies. The main problem with recognising receipts is the low quality of ink and printing paper, which is why it wrinkles and tears easily, and printed letters quickly fade. The study is based on a series of algorithms based on stepwise methods and integrated image transformation methods that can significantly improve the resulting character recognition. The step-by-step methods localise the text, carry out classification, segmentation, and text recognition, and remove the background part at each algorithm stage. Since they do not depend on the size of the dictionary, they can be used to recognise text from images regardless of its size. To solve the problem, we proposed a unique algorithm for image normalisation, which includes finding a receipt in the image, processing the resulting image area, removing shooting defects and media defects, and using a neural network to process and restore characters. We used the EAST (Efficient and Accurate Scene Text Detector) algorithm implemented using a convolutional neural network (CNN) for the text-finding process. Based on a comparison of the performance of the models in terms of their size and H-mean value, we selected the ddrnet23-slim neural network for the test images. The developed application can significantly increase the accuracy of text information recognition and, simultaneously, is small in size. The developed system recognises characters with reasonably high accuracy and shows the accuracy of the recognition result at a level of 97% and higher. The proposed system can be used: to detect and recognise characters by automatically scanning and updating invoice fields in the database; to extract text from an image and automatically convert it to digital format and update it in the database; as a tool for detecting, recognising, and understanding texts.
References
Borisyuk, F., Gordo, A., & Sivakumar, V. (2018). Rosetta: Large scale system for text detection and recognition in images. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD) (pp. 71–79). Association for Computing Machinery. DOI: 10.1145/3219819.3219861
Li, H., Wang, P., Shen, C., & Zhang, G. (2019). Show, attend and read: A simple and strong baseline for irregular text recognition. Proceedings of the 33rd AAAI Conference on Artificial Intelligence (pp. 8610–8617). AAAI Press. DOI: 10.1609/aaai.v33i01.33018610
Kim, W., & Kim, C. (2009). A new approach for overlay text detection and extraction from complex video scene. IEEE Transactions on Image Processing, 18(2), 401–411. DOI: 10.1109/tip.2008.2008225
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV) (pp. 2961–2969). Institute of Electrical and Electronics Engineers (IEEE). DOI: 10.1109/iccv.2017.322
Wang, W., Xie, E., Song, X., Zang, Y., Wang, W., Lu, T., Yu, G., & Shen, C. (2019). Efficient and accurate arbitrary-shaped text detection with pixel aggregation network. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 8439–8448). Institute of Electrical and Electronics Engineers (IEEE). DOI: 10.1109/iccv.2019.00853
Long, S., Ruan, J., Zhang, W., He, X., Wu, W., & Yao, C. (2018). TextSnake: A flexible representation for detecting text of arbitrary shapes. Proceedings of the 15th European Conference on Computer Vision (ECCV) (pp. 19–35). Springer. DOI: 10.1007/978-3-030-01216-8_2
Zhu, Y., Chen, J., Liang, L., Kuang, Z., Jin, L., & Zhang, W. (2021). Fourier contour embedding for arbitrary-shaped text detection. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 3122–3130). Institute of Electrical and Electronics Engineers (IEEE). DOI: 10.1109/CVPR46437.2021.00314
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 770–778). Institute of Electrical and Electronics Engineers (IEEE). DOI: 10.1109/cvpr.2016.90
Hong, Y., Pan, H., Sun, W., & Jia, Y. (2023). Deep dual-resolution networks for real-time and accurate semantic segmentation of road scenes. IEEE Transactions on Intelligent Transportation Systems, 24(3), 3448–3460. DOI: 10.1109/TITS.2022.3228042
Elagouni, K., Garcia, C., & Sbillot, P. (2011). A comprehensive neural-based approach for text recognition in videos using natural language processing. Proceedings of the 1st ACM International Conference on Multimedia Retrieval. Association for Computing Machinery. DOI: 10.1145/1991996.1992019
Matsuha, O. M. (2021). Navchalnyi posibnyk do vyvchennia kursu «Informatsiini tekhnolohii rozpiznavannia obraziv». RVV DNU.
Chaitanya, R. K., & Ashwini, B. B. (2019). Text Detection and Recognition: A Review. International Research Journal of Engineering and Technology (IRJET), 4(6), 179–185. Retrieved from https://www.irjet.net/archives/V4/i6/IRJET-V4I629.pdf
OpenCV team. (2023). OpenCV. Retrieved from https://opencv.org/
Downloads
Published
How to Cite
Issue
Section
License
The authors who publish in this collection agree with the following terms:
• The authors reserve the right to authorship of their work and give the magazine the right to first publish this work under the terms of license CC BY-NC-ND 4.0 (with the Designation of Authorship - Non-Commercial - Without Derivatives 4.0 International), which allows others to freely distribute the published work with a mandatory reference to the authors of the original work and the first publication of the work in this magazine.
• Authors have the right to make independent extra-exclusive work agreements in the form in which they were published by this magazine (for example, posting work in an electronic repository of an institution or publishing as part of a monograph), provided that the link to the first publication of the work in this journal is maintained. .
• Journal policy allows and encourages the publication of manuscripts on the Internet (for example, in institutions' repositories or on personal websites), both before the publication of this manuscript and during its editorial work, as it contributes to the emergence of productive scientific discussion and positively affects the efficiency and dynamics of the citation of the published work (see The Effect of Open Access).
