SciMKG is a large-scale multimodal educational knowledge graph (MEKG) covering text, images, videos, and audio for K-12 science education. It is automatically constructed using a novel LLM-powered pipeline for concept extraction and multimodal alignment.

• Four modalities covered: text, image, video, audio
• 1,356 knowledge points
• 34,630 multimodal concepts
• 403,400 triples
• 10,527 images · 10,425 videos · 34,630 audios

SciMKG is built using an Extraction–Verification–Integration–Augmentation (EVIA) pipeline:

1. Extraction
   Use multiple LLMs to extract K–12 science concepts from MOOC subtitles.
2. Verification
   Apply self-feedback (SELF-REFINE) to prune ambiguous or irrelevant concepts.
3. Integration
   Use self-consistency voting to merge multiple LLM outputs.
4. Augmentation
   Expand concepts through ConceptNet & Wikipedia; generate rewritten text and audio.
5. Multimodal Alignment
   Align images, videos, and audio to concepts using multimodal LLMs (e.g., GPT-4o, Gemini).

This pipeline ensures robustness, high precision, and semantic consistency across modalities.

pip install scimkg

import scimkg


kg = scimkg.path("your-video.mp4", "your-pdf.pdf")

kg.set_llm_config(
    api_key="your-api-key-here",
    base_url="https://api.your-llm-provider.com/v1",
    model="your-model-name"
)

kg.set_pdf_api_config(
    # default https://your-string-pdf(127.0.0.1:3000)/api/v1/misc/extract-images
    api_url="your-pdf-api-url-here"
)

triples = kg.triples() 

rdf = kg.rdf()

print(rdf)

SciMKG enables:

• Multimodal educational question answering
• Multimodal question generation
• Cross-modal knowledge retrieval
• Intelligent tutoring systems
• Science education agents
• Curriculum-level analytics

Project Lead: [email protected]

Dataset: https://zenodo.org/records/17578391

Demo: https://demo.scimkg.lutong.space/

Web Page: https://scimkg.lutong.space/

[2026.11.11] 🌸 We deploy the wiki page for SciMKG !

[2026.11.11] 🐱 We deploy the new Demo page for SciMKG !

If you use SciMKG or our construction framework, please cite:

@article{Lu_Wang_Zhou_Guan_Bai_Du_2026,
  title        = {SciMKG: A Multimodal Knowledge Graph for Science Education with Text, Image, Video and Audio},
  volume       = {40},
  url          = {https://ojs.aaai.org/index.php/AAAI/article/view/38574},
  doi          = {10.1609/aaai.v40i18.38574},
  number       = {18},
  journal      = {Proceedings of the AAAI Conference on Artificial Intelligence},
  author       = {Lu, Tong and Wang, Zhichun and Zhou, Yaoyu and Guan, Yiming and Bai, Zhiyong and Du, Junsheng},
  year         = {2026},
  month        = {Mar.},
  pages        = {15466-15474}
}