Acta Informatica Pragensia 2022, 11(1), 15-35 | DOI: 10.18267/j.aip.1675879

Image-based Product Recommendation Method for E-commerce Applications Using Convolutional Neural Networks

Pegah Malekpour Alamdari ORCID...1, Nima Jafari Navimipour ORCID...2,3, Mehdi Hosseinzadeh ORCID...4, Ali Asghar Safaei ORCID...5, Aso Darwesh ORCID...6
1 Department of Computer Engineering, Qeshm Branch, Islamic Azad University, Qeshm 7953163135, Iran
2 Department of Computer Engineering, Tabriz Branch, Islamic Azad University, Tabriz 5157944533, Iran
3 Future Technology Research Center, National Yunlin University of Science and Technology, Douliou 64002, Taiwan
4 Pattern Recognition and Machine Learning Lab, Gachon University, 1342 Seongnamdaero, Sujeonggu, Seongnam 13120, Republic of Korea
5 Department of Medical Informatics, Faculty of Medical Sciences, Tarbiat Modares University, Tehran 14115-111, Iran
6 Information Technology Department, University of Human Development, Sulaimaniyah 0778-6, Iraq

Recommender systems (RS) are designed to eliminate the information overload problem in today's e-commerce platforms and other data-centric online services. They help users explore and exploit the system's information environment utilizing implicit and explicit data from internal e-commerce systems and user interactions. Today's product catalogues include pictures to provide visual detail at a glance. This approach can effectively convert potential buyers into customers. Since most e-commerce stores use product images to promote, arouse users' visual desires and encourage them to buy products, this paper develops an image-based RS using deep learning techniques. To perform the research, we use five convolutional neural network (CNN) models to extract the features of the products' images. Then, the system uses the features to calculate the similarity between images. The selected CNN models are VGG16, VGG19, ResNet50, Inception V3 and Xception. We also analysed four versions of the MovieLens dataset to demonstrate the accuracy improvement of the recommendations, including 100k, 1M, 10M and 20M. Results of the experiment showed a significant increase in accuracy compared with traditional approaches. Also, we express many related open issues including use of multiple images per item, different similarity metrics, other CNN models, and the hybridization of image-based and different RS techniques for future studies. This method also provides more accurate product recommendations on e-commerce platforms than traditional methods.

Keywords: Image-based recommender systems; Recommender systems; E-commerce; Deep learning; Convolutional neural network.

Received: October 5, 2021; Revised: December 2, 2021; Accepted: December 2, 2021; Prepublished online: December 2, 2021; Published: March 13, 2022  Show citation

ACS AIP APA ASA Harvard Chicago Chicago Notes IEEE ISO690 MLA NLM Turabian Vancouver
Malekpour Alamdari, P., Navimipour, N.J., Hosseinzadeh, M., Asghar Safaei, A., & Darwesh, A. (2022). Image-based Product Recommendation Method for E-commerce Applications Using Convolutional Neural Networks. Acta Informatica Pragensia, 11(1), 15-35. doi: 10.18267/j.aip.167
Download citation
  • BibTeX (.bib)
  • Bookends (.ris)
  • EasyBib (.ris)
  • EndNote (.enw)
  • EndNote 8 (.xml)
  • ISI WoS (.isi)
  • Medlars (.medlars)
  • Mendeley (.ris)
  • MODS (.xml)
  • Papers (.ris)
  • RefWorks (.txt)
  • RefManager (.ris)
  • RIS (.ris)
  • MS Word (.xml)
  • Zotero (.ris)
    • Open full article

    References

    1. Adomavicius, G., & Tuzhilin, A. (2005). Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 17(6), 734-749. https://doi.org/10.1109/tkde.2005.99 Go to original source...
    2. Al-Shamri, M. Y. H. (2016). User profiling approaches for demographic recommender systems. Knowledge-Based Systems, 100, 175-187. https://doi.org/10.1016/j.knosys.2016.03.006 Go to original source...
    3. Alamdari, P. M., Navimipour, N. J., Hosseinzadeh, M., Safaei, A. A., & Darwesh, A. (2020). A Systematic Study on the Recommender Systems in the E-Commerce. IEEE Access, 8, 115694-115716. https://doi.org/10.1109/access.2020.3002803 Go to original source...
    4. Alyari, F., & Jafari Navimipour, N. (2018). Recommender systems. Kybernetes, 47(5), 985-1017. https://doi.org/10.1108/k-06-2017-0196 Go to original source...
    5. Burke, R. (2000). Knowledge-based recommender systems. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.21.6029&rep=rep1&type=pdf
    6. Burke, R. (2002). Hybrid recommender systems: Survey and experiments. User Modeling and User-Adapted Interaction, 12(4), 331-370. https://doi.org/10.1023/a:1021240730564 Go to original source...
    7. Chen, L., Yang, F., & Yang, H. (2017). Image-based product recommendation system with convolutional neural networks. Stanford University. http://cs231n.stanford.edu/reports/2017/pdfs/105.pdf
    8. Chu, W.-T., & Tsai, Y.-L. (2017). A hybrid recommendation system considering visual information for predicting favorite restaurants. World Wide Web, 20(6), 1313-1331. https://doi.org/10.1007/s11280-017-0437-1 Go to original source...
    9. Da'u, A., & Salim, N. (2019). Sentiment-Aware Deep Recommender System With Neural Attention Networks. IEEE Access, 7, 45472-45484. https://doi.org/10.1109/access.2019.2907729 Go to original source...
    10. Deshpande, M., & Karypis, G. (2004). Item-based top-N recommendation algorithms. ACM Transactions on Information Systems, 22(1), 143-177. https://doi.org/10.1145/963770.963776 Go to original source...
    11. Elkahky, A. M., Song, Y., & He, X. (2015). A Multi-View Deep Learning Approach for Cross Domain User Modeling in Recommendation Systems. In WWW '15: Proceedings of the 24th International Conference on World Wide Web (pp. 278-288). ACM. https://doi.org/10.1145/2736277.2741667 Go to original source...
    12. Hazratzadeh, S., & Jafari Navimipour, N. (2016). Colleague recommender system in the Expert Cloud using features matrix. Kybernetes, 45(9), 1342-1357. https://doi.org/10.1108/k-08-2015-0221 Go to original source...
    13. He, M., Zhang, S., & Meng, Q. (2019). Learning to Style-Aware Bayesian Personalized Ranking for Visual Recommendation. IEEE Access, 7, 14198-14205. https://doi.org/10.1109/access.2019.2892984 Go to original source...
    14. He, R., & Mcauley, J. (2016). Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering. In WWW '16: Proceedings of the 25th International Conference on World Wide Web (pp. 507-517). ACM. https://doi.org/10.1145/2872427.2883037 Go to original source...
    15. He, R., & McAuley, J. (2016). VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16). AAAI. Go to original source...
    16. Huang, P.-S., He, X., Gao, J., Deng, L., Acero, A., & Heck, L. (2013). Learning deep structured semantic models for web search using clickthrough data. In Proceedings of the 22nd ACM International Conference on Conference on Information & Knowledge Management. ACM. https://doi.org/10.1145/2505515.2505665 Go to original source...
    17. Huang, Y., Wang, N., Zhang, H., & Wang, J. (2019). A novel product recommendation model consolidating price, trust and online reviews. Kybernetes, 48(6), 1355-1372. https://doi.org/10.1108/k-03-2018-0143 Go to original source...
    18. Zhou, H.-h., Ltu, Y.-j., Zhang, W.-q., & Xie, J.-y. (2004). A Survey of Recommender System Applied in E-commerce. Application Research of Computers, 1, no. 3. http://en.cnki.com.cn/Article_en/CJFDTotal-JSYJ200401003.htm
    19. Leng, Y., Lu, Q., & Liang, C. (2016). A collaborative filtering similarity measure based on potential field. Kybernetes, 45(3), 434-445. https://doi.org/10.1108/k-10-2014-0212 Go to original source...
    20. Lops, P., de Gemmis, M., & Semeraro, G. (2011). Content-based Recommender Systems: State of the Art and Trends. In Recommender Systems Handbook (pp. 73-105). Springer. https://doi.org/10.1007/978-0-387-85820-3_3 Go to original source...
    21. Lu, Y., Gao, M., & Saga, R. (2021). Apparel Recommender System based on Bilateral image shape features. arXiv preprint arXiv:2105.01541. https://arxiv.org/abs/2105.01541
    22. Maros, A., Belém, F., Silva, R., Canuto, S., Almeida, J. M., & Gonçalves, M. A. (2019). Image Aesthetics and its Effects on Product Clicks in E-Commerce Search. In Proceedings of the SIGIR 2019 eCom workshop. CEUR-WS. http://ceur-ws.org/Vol-2410/paper25.pdf
    23. Mbelwa, H. (2021). Image-based poultry disease detection using deep convolutional neural network. Nelson Mandela African Institution of Science and Technology. https://dspace.nm-aist.ac.tz/handle/20.500.12479/1344
    24. Jafari Navimipour, N., Rahmani, A. M., Habibizad Navin, A., & Hosseinzadeh, M. (2015). Expert Cloud: A Cloud-based framework to share the knowledge and skills of human resources. Computers in Human Behavior, 46, 57-74. https://doi.org/10.1016/j.chb.2015.01.001 Go to original source...
    25. Ratnasingham, P. (1998). The importance of trust in electronic commerce. Internet Research, 8(4), 313-321. https://doi.org/10.1108/10662249810231050 Go to original source...
    26. Resnick, P., & Varian, H. R. (1997). Recommender systems. Communications of the ACM, 40(3), 56-58. https://doi.org/10.1145/245108.245121 Go to original source...
    27. Schafer J.B., Frankowski D., Herlocker J., & Sen S. (2007). Collaborative Filtering Recommender Systems. In Brusilovsky P., Kobsa A., Nejdl W. (eds.) The Adaptive Web (pp. 291-324). Springer. https://doi.org/10.1007/978-3-540-72079-9_9 Go to original source...
    28. Shankar, D., Narumanchi, S., Ananya, H., Kompalli, P., & Chaudhury, K. (2017). Deep learning based large scale visual recommendation and search for E-Commerce. arXiv preprint arXiv:1703.02344. https://arxiv.org/abs/1703.02344
    29. Sharif, S. H., Mahmazi, S., Jafari Navimipour, N., & Farid Aghdam, B. (2013). A Review on Search and Discovery Mechanisms in Social Networks. International Journal of Information Engineering and Electronic Business, 5(6), 64-73. https://doi.org/10.5815/ijieeb.2013.06.08 Go to original source...
    30. Shoja, B. M., & Tabrizi, N. (2019). Customer Reviews Analysis With Deep Neural Networks for E-Commerce Recommender Systems. IEEE Access, 7, 119121-119130. https://doi.org/10.1109/access.2019.2937518 Go to original source...
    31. Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. https://arxiv.org/abs/1409.1556
    32. Tuinhof, H., Pirker, C., & Haltmeier, M. (2019). Image-Based Fashion Product Recommendation with Deep Learning. In LOD 2018: Machine Learning, Optimization, and Data Science (pp. 472-481). Springer. https://doi.org/10.1007/978-3-030-13709-0_40 Go to original source...
    33. Turban, E., King, D., Lee J., & Viehland, D. (2002). Electronic commerce: A managerial perspective 2002. Prentice Hall.
    34. Vandecasteele, F., Vandenbroucke, K., Schuurman, D., & Verstockt, S. (2017). Spott: On-the-Spot e-Commerce for Television Using Deep Learning-Based Video Analysis Techniques. ACM Transactions on Multimedia Computing, Communications, and Applications, 13(3s), 1-16. https://doi.org/10.1145/3092834 Go to original source...
    35. Vasudevan, S., Chauhan, N., Sarobin, V., & Geetha, S. (2020). Image-Based Recommendation Engine Using VGG Model. In Advances in Communication and Computational Technology, Select Proceedings of ICACCT 2019 (pp. 257-265). Springer. https://doi.org/10.1007/978-981-15-5341-7_21 Go to original source...
    36. Wang, S., Wang, Y., Tang, J., Shu, K., Ranganath, S., & Liu, H. (2017). What Your Images Reveal: Exploiting Visual Contents for Point-of-Interest Recommendation. In WWW '17: Proceedings of the 26th International Conference on World Wide Web (pp. 391-400). ACM. https://doi.org/10.1145/3038912.3052638 Go to original source...
    37. Wigand, R. T. (1997). Electronic Commerce: Definition, Theory, and Context. The Information Society, 13(1), 1-16. https://doi.org/10.1080/019722497129241 Go to original source...
    38. Yu, W., Zhang, H., He, X., Chen, X., Xiong, L., & Qin, Z. (2018). Aesthetic-based Clothing Recommendation. In WWW '18: Proceedings of the 2018 World Wide Web Conference (pp. 649-658). ACM. https://doi.org/10.1145/3178876.3186146 Go to original source...

    This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits use, distribution, and reproduction in any medium, provided the original publication is properly cited. No use, distribution or reproduction is permitted which does not comply with these terms.


    Return to the content