Deep Learning Optimization for Document Text Classification Using Gated Recurrent Unit (G R U)

Afodia Ayuba, Abdulsalam Ya'u Gital, Ali Usman A., Abubakar Umar, Abubakar Muhammad, Mustapha Abdulrahman Lawal, Abdulkadir Hamidu Alkali

Abstract


Text classification has become a challenge in big organizations in the area of managing large amount of data online. The classification usually applied in the tasks of Natural Language Processing. The task of organizing large documents in an organization has become quite challenging because the internal content of the files are not known. Also, manually organizing each and every file in a large collection is not practically possible as it may take hours to categorize a file based on its contents. In addition, the accuracy of classification cannot be guaranteed, due to the text being a set of data that are naturally sequential. The research work, attempts to address the issue of analysing the document efficiently using multi-level text classification approach based on GRU. The recurrent network will be used to capture long sequence data required for natural language understanding. The performance results of benchmark datasets Factory Report, Bookmarks, Reuters, EUR-Lex and RCV show that the proposed model performs better than existing algorithms in terms of the accuracy in cases of 3 datasets. These include EUR-lex, Bookmarks and RCV where the proposed model attained 74.3%, 100% and 88.44% as against the DSRM-DNN with 62.66%, 47.82 and 86.83% respectively. However, the DSRM-DNN attained classification accuracy of 91.99% for the case of Reuter’s datasets demonstrating its superiority against the proposed model which attained 89.83%. Additionally, for the case of algorithm run time, the previous study ignored the computational time of the algorithms which served as index for evaluating the quality of the model. However, this study by included the computational time which attained the better run time for 15 seconds for all the bookmark datasets. On an average, we can conclude that it takes less than two (2) minutes for the proposed model to run on all the datasets. The result shows that our proposed model performs better compared to the DSRM-DNN model.


Full Text:

PDF

References


Akhter, M. P., Jiangbin, Z., Naqvi, I. R., Abdelmajeedb, M., & Fayyaz, M. (2020). Exploring deep learning approaches for Urdu text classification in product manufacturing. Enterprise Information Systems: 1-26. https://doi.org/10.1080/17517575.2020.1755455

Boukil, S., Biniz, M., El Adnani ,F., Cherrat, Loubna, & El Moutaouakki, A.E. (2018). "Arabic text classification using deep lesarning technics."International Journal of Grid and Distributed Computing 11 (9), 103-114. http://dx.doi.org/10.14257/ijgdc.2018.11.9.09103-114.

Chen, X., Ouyang, C., Liu, Y., Luo, L., & Yang, X. (2018). A Hybrid Deep Learning Model for Text Classification. 2018 14th International Conference on Semantics, Knowledge and Grids (SKG). https://doi:10.1109/skg.2018.00014

Du, J., Gui, L., Xu, R., & He, Y. (2017). A convolutional attention model for text classification. National CCF conference on natural language processing and Chinese computing, Springer. 10619, 183–195. https://doi.org/10.1007/978-3-319-73618-1_16

Gao, M., Li, T., & Huang, P. (2018). Text classification research based on improved Word2vec and CNN. International Conference on Service-Oriented Computing, Springer, 11434, 126–135 https://doi.org/10.1007/978-3-030-17642-6_11

Guo, L., Zhang, D., Wang, L., Wang, H., and Cui, B. (2018). CRAN: a hybrid CNN-RNN attention-based model for text Sclassification. International Conference on Conceptual Modeling, Springer, 11157, 571–585. https://doi.org/10.1007/978-3-030-00847-5_42

Hassan, A., & Mahmood, A. (2018). Convolutional Recurrent Deep Learning Model for Sentence Classification. IEEE Access, 6, 13949–13957. https://doi:10.1109/access.2018.28148

Hassanzadeh, H., Nguyen, A., Karimi, S., Chu, K. (2018). "Transferability of artificial neural networks for clinical document classification across hospitals: A case study on abnormality detection radiology reports." Journal of biomedical informatics 85,68-79. https://doi.org/10.1016/j.jbi.2018.07.017

Joo, W., Choi, K., & Kim Y.(2019). "Deep learning model for unstructured knowledge classification using structural features." Personal and Ubiquitous Computing: 1-12. https://doi.org/10.1007/s00779-019-01244-x

Joshi, A. V. (2020). Machine learning and artificial intelligence, Springer.

Kilimci, Z. H., & Akyokus, S. (2019). The Evaluation of Word Embedding Models and Deep Learning Algorithms for Turkish Text Classification. 2019 4th International Conference on Computer Science and Engineering (UBMK). https://doi:10.1109/ubmk.2019.8907027

Lenc, L.,& Král, P. (2017). Two-level neural network for multi-label document classification. International Conference on Artificial Neural Networks, Springer,10614, 368–375. https://doi.org/10.1007/978-3-319-68612-7_42

Lin, R., Fu, C., Mao, C., Wei, J., & Li, J. (2018). Academic news text classification model based on attention mechanism and RCNN. CCF Conference on Computer Supported Cooperative Work and Social Computing, Springer, 917, 507–516. https://doi.org/10.1007/978-981-13-3044-5_38

Liu, J., Wu, Y. Chang, W., Yang, Y. (2017). Deep learning for extreme multi-label text classification. Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. hpp://dx.doi.org/10.1145/3077136.3080834

Liu, G. & Guo, J. (2019). Bidirectional LSTM with attention mechanism and convolutional layer for text classification. Neurocomputing S0925-2312(19)30106-7

https://doi.org/10.1016/j.neucom.2019.01.078

Peng, H., Li, J., He, Y., Liu, Y., Bao, M., Wang, L., Soug, Y., & Yang, Q. (2018). Large-scale hierarchical text classification with recursively regularized deep graph-cnn. Proceedings of the 2018 world wide web conference. https://doi.org/10.1145/3178876.3186005

Premchander, K., Sarma, S. S. V. N., Vaishali, K., Reddy, P. V., Anjaneyulu, M., & Nagaprasad, S. (2018). WordNet-based text categorization using convolutional neural networks. Proceedings of International Conference on Recent Advancement on Computer and Communication, Springer. https://doi.org/10.1007/978-981-10-8198-9_25

Rebala, G., A. Ravi, et al. (2019). Learning Models. An Introduction to Machine Learning, Springer: 19-23.

Rebala, G., A. Ravi, et al. (2019). Machine Learning Definition and Basics. An Introduction to Machine Learning, Springer: 1-17.

Semberecki, P. & Maciejewski, P. (2017). Deep learning methods for subject text classification of articles. 2017 Federated Conference on Computer Science and Information Systems (FedCSIS), IEEE.,11, 357–360. https://doi.org/10.15439/2017F414

Wang, L., Chen, Q., Sun, Q., & Jiba, Y. (2018). Research on Short Text Classification Method Based on Convolution Neural Network. International Conference on Intelligent and Interactive Systems and Applications, Springer, 885, 397–403. https://doi.org/10.1007/978-3-030-02804-6_53

Wang, T., Liu,L., Liu, N.,Zhang,H., Zhang, L., Feng, S. (2020). A multi-label text classification method via dynamic semantic representation model and deep neural network. Applied Intelligence 50(8): 2339-2351. https://doi.org/10.1007/s10489-020-01680-w

Wu, H., Li, D., & Cheng, M. (2019). Chinese Text Classification Based on Character-Level CNN and SVM. School of Computer and Communication Engineering, 986, 227–238. https://doi.org/10.1007/978-981-13-6473-0_20

Zheng, J., Cai, F., Chen, W., Feng ,C., Chen, H. (2018). Hierarchical neural representation for document classification. Cognitive Computation11 (2)317-327. https://doi.org/10.1007/s12559-018-9621-6

Zhong, B., Xing, X., Love, P., Wang, X., & Luo, H. (2019). "Convolutional neural network: Deep learning-based classification of building quality problems." Advanced Engineering Informatics 40, 46-57. https://doi.org/10.1016/j.aei.2019.02.009

Zulgarnain, M., Ghazali, R., Ghouse, M.G., Mushtag, M. F. (2019). Efficient Processing of GRU Based on Word Embeddingfor Text Classification . International Journal On Informatics Visualization, 3(4), 377-383.


Refbacks

  • There are currently no refbacks.