Decoding the Web of Belief

“There are no facts, only interpretations.”

— Friedrich Nietzsche

I had a simple question during a walk the other day:

"If you could create a physical map of all conspiracy theories and how they are related, what would it look like?"

This entire project is an effort to build this map. Drawing on findings from my own published research, recent advances in large language models and the discovery of a massive archive of Reddit data, I decided to try and make this map myself.

This project will be technical in nature, covering the following topics:

• Data Engineering
• Natural Language Processing
• Large Language Models
• Knowledge Graphs (named entity recognition and relation extraction)
• Basic Analytics

I will discuss end-to-end how the knowledge graph was built. In conjunction, I will write a shorter, non-technical blog post exploring the findings from the project.

Note - Due to the significant compute resources required for the analytics, this project will likely continue through 2025.