AI Knowledge Base Builder — Turn Any Content Into a Searchable Knowledge Base

Tired of digging through 20 browser tabs to find what you need?

You write code, bookmark pages, save documents. But when you need to find something specific, you either forget where it was or you have to Google it all over again.

AI Knowledge Base Builder fixes that. Point it at a URL, a folder, or a file. It automatically extracts the content, builds a semantic search index, and gives you a searchable knowledge base with both CLI and Web UI. Fully offline.

How It Works:

URL / File / Directory → Extract Text → TF-IDF Index → Cosine Similarity Search → CLI + Web UI

Features:

- Web Crawling: Extract clean content from any URL, ads and nav removed automatically

- File Support: Works with .py, .md, .txt, .html, .json, .csv, .yaml

- Directory Scan: Import entire projects with a single command

- Semantic Search: TF-IDF + cosine similarity, understands meaning not just keywords

- Built-in Web UI: Zero-config browser interface at localhost:8080

- JSON Export: Portable and backup-friendly, also works as RAG knowledge base

- Pipe Support: cat file.txt | python kb_builder.py build

- Auto Stats: View source count, chunk count at a glance

Who Is This For:

- Developers: Index your codebase, search for patterns and docs in seconds

- Researchers: Crawl papers and articles, build a personal research database

- Content Creators: Collect reference material, search across hundreds of sources

- AI Enthusiasts: Use as a local knowledge base for RAG pipelines

- Students: Organize course materials, search key concepts instantly

Quick Start:

 pip install -r requirements.txt

 python kb_builder.py build --source https://example.com

 python kb_builder.py build --source ./my_project/

 python kb_builder.py search "machine learning"

 python kb_builder.py serve

Zero Risk:

- Fully Offline: No API key, no GPU, no cloud, no internet required

- MIT License: Use, modify, distribute commercially with no restrictions

- Cross-Platform: Windows, macOS, Linux, Python 3.8+

- Lightweight: Only 5 dependencies, well-documented ~400 line codebase

FAQ:

Q: Do I need an API key?

A: No. Everything runs offline, zero external calls.

Q: Can I use it for commercial projects?

A: Yes. MIT license, free to use, modify, and distribute.

Q: Can it handle large codebases?

A: Yes. Supports thousands of files, auto-chunked for performance.

Q: Does it have a GUI?

A: Yes. Run "python kb_builder.py serve" and open your browser.