Acquiring Knowledge from Data Analytics and Performance-Boosting on Multimedia Content

Abstract

In gaining meaningful and actionable insights from complex and diverse multimedia content, many studies have applied data analytics approaches—particularly data mining and machine learning—to uncover patterns, relationships, and hidden knowledge. This systematic literature review synthesizes 26 studies conducted over the past decade on acquiring knowledge from multimedia content using data analytics and performance-boosting techniques. Across domains such as social media, education, healthcare, e-commerce, and public safety, most works integrate text–image or audio–video pairs and increasingly adopt attention-based architectures and transformer models with early fusion strategies. To ensure comparability, each study’s evidence is recorded by considering the reported performance improvement over the authors’ baseline using the same dataset and evaluation metric. The most frequently used metrics include Accuracy, the F1-score (a harmonic mean of Precision and Recall), Precision, Recall, and the Area Under the Receiver Operating Characteristic Curve (AUC), which provides a threshold-independent measure of classification quality. The most common challenges identified include modality integration and alignment, data noise and quality, limitations of datasets and benchmarks, and domain shift, with fewer studies reporting class imbalance, computational cost, and interpretability or privacy issues. At the same time, promising opportunities emerge in the development of standardized multimodal benchmarks, efficient transformer-based and hybrid fusion pipelines, integration of external knowledge, domain-robust learning, and privacy-preserving explainable multimodal artificial intelligence. Overall, this review contributes a consolidated map of modalities, methods, and metrics, a performance-gain versus baseline table for quick comparability, a quantified challenge landscape, and a practical roadmap for guiding future research in multimodal sentiment analysis and related fields.