Building a Second Brain with Gemini CLI + Obsidian — The LLM Wiki Approach

I’ve spent a lot of time on this blog writing about coding agents and agent harnesses. But there’s another side of AI tooling that’s been on my mind: personal knowledge management.

Andrej Karpathy’s LLM Wiki concept — a markdown-based personal knowledge base that an AI agent maintains and queries — has been influential in this space. I’ve been running my own version using Claude Code and Obsidian, but a recent video by Everyday with Captain showed something I hadn’t tried: building it with Gemini CLI and integrating kepano’s official Obsidian AI skill.

The video is in Thai, but the setup is universal. Here’s what I learned.

What is an LLM Wiki?

The idea comes from Karpathy’s gist: instead of forcing your notes into a rigid structure, you maintain a folder of markdown files and let an AI agent handle the ingestion, organization, and retrieval.

The workflow is:

1. Ingest — drop raw content (articles, transcripts, notes) into a raw/ folder
2. Process — the AI reads each file, extracts key takeaways, and writes structured markdown into a wiki/ folder
3. Query — ask the AI questions; it searches the wiki and returns answers with context
4. Organize — periodically, the AI restructures and cross-links the content

The magic is that the AI does the categorization and linking automatically. You just supply the raw material.

Why Gemini CLI Instead of Claude Code?

The video walks through the exact same LLM Wiki setup that was shown in a previous tutorial using Claude Code, but this time with Gemini CLI. The difference is significant:

• Gemini CLI is free — no API key required for basic use, just a Google account
• It runs through VS Code’s terminal — familiar workflow for most developers
• It has native file system permissions — approve or deny file reads/writes per session
• The Gemini 2.5 Flash model handles markdown processing and structured output well

The setup is straightforward:

1. Install VS Code and Obsidian
2. Create an Obsidian vault (the video uses gemini-demo)
3. Open the vault folder in VS Code
4. Install Gemini CLI via npm: npm install -g @google/gemini-cli
5. Install the Gemini CLI Companion VS Code extension
6. Log in with your Google account

Once the terminal session starts with gemini, you’re ready.

Setting Up the LLM Wiki

The key step is giving Gemini CLI Karpathy’s LLM Wiki prompt. You paste the entire concept as your first message:

“You are my wiki agent. Help me implement this LLM Wiki setup. Create the following structure: geminie.md (schema), index.md, log.md, following the details I’ve provided. This is going to be my second brain. Guide me step by step and show me how to do the first ingest.”

The agent then:

1. Reads the instructions and sets up the folder structure
2. Creates raw/ for incoming content and wiki/ for processed notes
3. Generates gemini.md (the schema file defining metadata structure), index.md (auto-updated table of contents), and log.md (ingestion history)
4. Runs the first ingest on the LLM Wiki concept itself as a demonstration

The result is a living Obsidian vault where every file is a structured markdown note with metadata, tags, and cross-links — viewable in Obsidian’s graph view.

The Obsidian AI Skill

The standout feature of this video is the integration of kepano’s Obsidian Skills — an official AI skill released by the CEO of Obsidian himself.

kepano/obsidian-skills is an MCP server and skill pack that gives AI agents native understanding of Obsidian’s markdown format, Canvas files, and vault structure. Instead of treating your notes as plain markdown, the AI understands:

• Wiki links ([[page]]) as connections between concepts
• Canvas files as visual knowledge maps
• Obsidian-flavored markdown (callouts, metadata, tags)
• Vault structure — where files belong and how they relate

Installing it is as simple as giving Gemini CLI the GitHub link:

“Take this skill from https://github.com/kepano/obsidian-skills and install it.”

The agent clones the repo, reads the skill definitions, and integrates them into its workflow. From that point on, when it ingests content, it:

• Creates properly formatted Obsidian notes with YAML frontmatter
• Generates visual Canvas files for concept mapping
• Uses wiki links to connect related ideas
• Organizes files into subfolders (entities, concepts, sources) for manual browsing

The Ingest Demo

The video demonstrates the full cycle with a real article:

1. Use the Obsidian Web Clipper browser extension to save a page (Flipbook.page) directly into the raw/ folder
2. Tell Gemini CLI: “I’ve added a file about Flipbook. Please ingest it.”
3. The agent reads the article, extracts key takeaways, and writes a structured note into wiki/
4. It also generates a Canvas visualization — a visual diagram explaining the concept
5. When the wiki folder gets cluttered, ask: “Help organize the wiki folder — maybe split into entities, concepts, and sources subfolders”

The Canvas generation is particularly impressive. The AI creates an Obsidian Canvas file (.canvas JSON) with nodes and arrows connecting the main concept to related tools, features, and ideas — essentially auto-generating a mind map of the ingested content.

Why This Matters

I’ve been running a similar setup with my second-brain project, where an LLM agent maintains a knowledge base autonomously. The key difference with this approach is:

1. Zero code required. The video targets non-developers who use Obsidian. Install three tools, paste a prompt, and you’re running.
2. Obsidian-native output. Because the AI understands Obsidian’s format (via the skill), the generated notes are first-class citizens in the vault. They show up in graph view, respond to backlinks, and work with Obsidian’s search.
3. Free tier. Gemini CLI’s free tier is generous enough for personal knowledge base use. No API costs.
4. Visual by default. The Canvas generation means you get both structured notes and visual maps — the best of both worlds.

The combination of Karpathy’s LLM Wiki structure, Google’s Gemini CLI, and Obsidian’s knowledge management creates a personal AI knowledge system that’s genuinely useful. It’s not a demo — it’s something you can use daily to process articles, meeting notes, research, and ideas into a searchable, interconnected knowledge base.

If you’re interested in setting this up, the video walks through every step visually, and the resource page has all the links collected in one place.