Overview
We propose a new multimodal large model paradigm: the Hypergraph Multimodal Large Language Model (HMLLM). It uses hypergraphs to model complex relationships among video elements, EEG signals, and eye-tracking data. Integrating modalities closes semantic gaps and improves reasoning and semantic analysis, allowing HMLLM to simulate different groups' varied responses to the same video elements.
Connected profiles
More Video generation products
Sources · 0Sources & provenance
English text is maintained separately from the original record. Consult the original source before citing a fact.
No source links are available for this profile.
Original source text
Names may use phonetic renderings when no English name is supplied. The original wording is retained below.
我们提出了全新的多模态大模型的范式：超图多模态大语言模型（HMLLM），它利用超图构建视频元素、脑电信号和眼动追踪数据之间的复杂关系。通过整合不同模态的信息，HMLLM弥合了丰富模态间的语义差距，增强了其逻辑推理和语义分析的能力。使得HMLLM可以模拟不同人群观看到同一段视频元素时的异质反应。