mm-arxiv.github.io - Multimodal ArXiv

home > domains > mm-arxiv.github.io

Description: Vision-Language Feedback

multimodal chatbot (10)

Example domain paragraphs

arXiv Dataset(ArxivCap) Dataset(ArxivQA) Model(Coming soon) Abstract Large vision-language models (LVLMs), exemplified by GPT-4V, excel across diverse tasks involving concrete images from natural scenes. However, their ability to interpret abstract figures, such as geometry shapes and scientific plots, remains limited due to a scarcity of training datasets in scientific domains. To fill this gap, we introduce Multimodal ArXiv, consisting of ArXivCap and ArXivQA, for enhancing LVLMs scientific comprehension.

Comparison with previous scientific figure datasets. Our ArXivCap is the largest captioning dataset and our ArXivQA is the only QA dataset that covers a wide range of domains from real papers.

Overview of our dataset curation process. Starting from the ArXiv paper source files, we ensure the paper quality by selecting papers according to publication records. Figure and caption pairs are extracted and then cleaned according to manually designed rules. ArXivQA is generated by prompting GPT-4V with a curated template.

Links to mm-arxiv.github.io (3)

• tobiaslee.top Stay Hungry,Stay Foolish.
• lilei-nlp.github.io About - Lei Li’s Homepage
• ultra-editing.github.io UltraEdit