AI-Based Data Quality Assurance for Business Intelligence and Decision Support Systems
DOI:
https://doi.org/10.63282/3050-9246.IJETCSIT-V6I2P103Keywords:
Data Quality, Artificial Intelligence, Machine Learning, Data Cleansing, Anomaly Detection, Data GovernanceAbstract
BI and DSS are valuable tools that help businesses make better decisions during the present information-oriented economy. However, these systems are highly dependent on the kind of data they receive and process. This means that one wrong piece of information collected creates wrong information, leading to wrong decisions and losses. Therefore, this paper reviews AI data quality assurance relevant to BI and DSS. The paper reviews data quality criteria, including accuracy, completeness, consistency, timeliness, and validity. Using such works as machine learning, deep learning, natural language processing, and knowledge graphs in data quality assurance is thoroughly studied. A new approach for data pre-processing has been developed, comprising anomaly detection, data cleansing, intelligent imputation, and semantic reconciliation to enhance data quality. Thus, we decided to conduct a literature review aimed at assessing the existing methodologies and identifying the deficits that require the use of AI. In order to address the data concerns in the implementation of training algorithms, the proposed methodology focuses on using supervised and unsupervised learning to identify the problems of data and rectify the same simultaneously. Some studies, based on experiments carried out for a BI system on a large-scale data set of a financial institution, showed better data quality and accuracy of decision-making. Moreover, we provide recommendations, drawbacks, and prospects for the application of the augmentation of AI in data quality assurance
Downloads
References
[1] Wang, R. Y., & Strong, D. M. (1996). Beyond accuracy: What data quality means to data consumers. Journal of Management Information Systems, 12(4), 5-33.
[2] Ballou, D. P., & Pazer, H. L. (1985). Modeling data and process quality in multi-input, multi-output information systems. Management Science, 31(2), 150-162.
[3] Batini, C., Cappiello, C., Francalanci, C., & Maurino, A. (2009). Methodologies for data quality assessment and improvement. ACM computing surveys (CSUR), 41(3), 1-52.
[4] Redman, T. C. (1997). Data quality for the information age. Artech House, Inc.
[5] Kimball, R., & Caserta, J. (2004). The data warehouse ETL toolkit. John Wiley & Sons.
[6] Shivisha Patel, AI in Business Intelligence, Vlink, online. https://vlinkinfo.com/blog/ai-in-business-intelligence/
[7] Olson, J. E. (2003). Data quality: the accuracy dimension. Elsevier.
[8] Rahm, E., & Do, H. H. (2000). Data cleaning: Problems and current approaches. IEEE Data Eng. Bull., 23(4), 3-13.
[9] English, L. P. (1999). Improving data warehouse and business information quality: methods for reducing costs and increasing profits. John Wiley & Sons, Inc.
[10] Kimball, R. (2004). Kimball design tip# 59: Surprising value of data profiling. Kimball Group, (59).
[11] Wu, L., Chen, Y., Shen, K., Guo, X., Gao, H., Li, S., ... & Long, B. (2023). Graph neural networks for natural language processing: A survey. Foundations and Trends® in Machine Learning, 16(2), 119-328.
[12] Naseer, S., Saleem, Y., Khalid, S., Bashir, M. K., Han, J., Iqbal, M. M., & Han, K. (2018). Enhanced network anomaly detection based on deep neural networks. IEEE Access, 6, 48231-48246.
[13] Ji, S., Pan, S., Cambria, E., Marttinen, P., & Yu, P. S. (2021). A survey on knowledge graphs: Representation, acquisition, and applications. IEEE transactions on neural networks and learning systems, 33(2), 494-514.
[14] Sasidharakarnavar, S. (2025). Revolutionizing Hr: Leveraging Workday Platform For Enhanced Workforce Management. International Journal of AI, BigData, Computational and Management Studies, 6(1), 98-105. https://doi.org/10.63282/3050-9416.IJAIBDCMS-V6I1P110
[15] Hogan, A., Blomqvist, E., Cochez, M., d’Amato, C., Melo, G. D., Gutierrez, C. & Zimmermann, A. (2021). Knowledge graphs. ACM Computing Surveys (Csur), 54(4), 1-37.
[16] How AI Is Transforming Data Quality Management, accel-data, online. https://www.acceldata.io/blog/how-ai-is-transforming-data-quality-management
[17] Peng, C., Xia, F., Naseriparsa, M., & Osborne, F. (2023). Knowledge graphs: Opportunities and challenges. Artificial Intelligence Review, 56(11), 13071-13102.
[18] Soori, M., Jough, F. K. G., Dastres, R., & Arezoo, B. (2024). AI-based decision support systems in Industry 4.0, A review. Journal of Economy and Technology.
[19] Felderer, M., & Ramler, R. (2021). Quality assurance for AI-based systems: Overview and challenges (introduction to interactive session). In Software Quality: Future Perspectives on Software Engineering Quality: 13th International Conference, SWQD 2021, Vienna, Austria, January 19–21, 2021, Proceedings 13 (pp. 33-42). Springer International Publishing.
[20] Catti, P., Freitas, A., Pereira, E., & Gonçalves, G. (2024). Data Analytics and AI for Quality Assurance in Manufacturing: Challenges. In Learning Factories of the Future: Proceedings of the 14th Conference on Learning Factories 2024, Volume 1 (Vol. 1, p. 205). Springer Nature.
[21] Sasidharakarnavar, S. (2025). Enhancing HR System Agility through Middleware Architecture. International Journal of AI, Big Data, Computational and Management Studies, 6(1), 89-97. https://doi.org/10.63282/3050-9416.IJAIBDCMS-V6I1P109
[22] Wang, C., Yang, Z., Li, Z. S., Damian, D., & Lo, D. (2024). Quality assurance for artificial intelligence: A study of industrial concerns, challenges, and best practices. arXiv preprint arXiv:2402.16391.
