Skip to content

Self-Hosted AI Tools Evaluation

Status: Research notes. Tools evaluated for potential internal use and client projects.

Use Cases

  1. Internal knowledge base Q&A - Chat with our docs (oracle.minnova.io)
  2. Client RAG solutions - Document chat for client projects
  3. ChatOps - AI assistant in Slack/Zulip
  4. Code assistance - Already using Claude Code

RAG / Document Chat Tools

Second brain with RAG. Has built-in Slack bot.

  • GitHub: https://github.com/QuivrHQ/quivr
  • Slack bot: https://github.com/QuivrHQ/slackbot
  • Features: Any files, any LLM (Ollama, OpenAI, Anthropic), PGVector/Faiss
  • Deployment: Docker

Khoj

AI second brain. Simpler than Quivr.

  • GitHub: https://github.com/khoj-ai/khoj
  • Features: Markdown, PDF, Word, Notion, web pages
  • Access: Browser, Desktop, WhatsApp, Obsidian
  • Deployment: Docker

AnythingLLM

All-in-one RAG with workspaces and agents.

  • GitHub: https://github.com/Mintplex-Labs/anything-llm
  • Features: Multi-user, roles, agents, 10-min auto-sync (desktop)
  • Best for: Drop in docs, chat with them
  • Deployment: Docker, Desktop app

Danswer/Onyx

Enterprise Q&A with 40+ connectors.

  • GitHub: https://github.com/onyx-dot-app/onyx
  • Features: GitHub (issues/PRs), Confluence, Google Drive, web crawl
  • Best for: Teams with many data sources
  • Deployment: Docker Compose

Chat UIs (ChatGPT-like)

Open WebUI

Most popular self-hosted ChatGPT alternative.

  • GitHub: https://github.com/open-webui/open-webui
  • Features: Ollama integration, built-in RAG, OIDC support
  • Deployment: Docker, Helm chart

LibreChat

Multi-provider chat UI with OAuth.

  • GitHub: https://github.com/danny-avila/LibreChat
  • Features: OpenAI, Anthropic, Ollama, conversation branching
  • Deployment: Docker

AI Workflow Builders

Flowise

Visual LangChain builder for RAG apps.

  • GitHub: https://github.com/FlowiseAI/Flowise
  • Website: https://flowiseai.com
  • Best for: No-code AI workflow building

Langflow

Similar to Flowise, slightly faster on large docs.

  • GitHub: https://github.com/langflow-ai/langflow
  • Best for: Visual LLM app building

Dify

Full AI platform with prompt IDE, RAG, agents.

  • GitHub: https://github.com/langgenius/dify
  • Best for: Complete AI development platform

n8n

Workflow automation with AI nodes.

  • GitHub: https://github.com/n8n-io/n8n
  • Features: 400+ integrations, Ollama support
  • Best for: Automation + AI together

LLM Runners

Ollama

Run LLMs locally. Most popular option.

  • Website: https://ollama.ai
  • Models: Llama 3.2, Mistral, Qwen, etc.
  • Usage: ollama run llama3.2

LocalAI

OpenAI-compatible API for local models.

  • GitHub: https://github.com/mudler/LocalAI
  • Best for: Drop-in OpenAI API replacement

Recommendations

Use Case Tool
Chat with our docs + Zulip bot Quivr
Simple internal ChatGPT Open WebUI + Ollama
Client RAG projects Build custom (see rag-system-internal-docs.md) or Flowise
Workflow automation n8n

Decision Factors

Self-hosted vs API (OpenAI/Anthropic)

Self-hosted API
Data stays on your server Better model quality
No per-token cost Zero maintenance
Offline access Works immediately
Learning experience Simpler

Verdict: For day-to-day work, APIs are fine. Self-host when: - Client requires on-prem - Building RAG to understand the stack - Bulk processing (cost savings)

Next Steps

  1. Set up Zulip for team chat
  2. Evaluate Quivr with Zulip integration
  3. Consider building custom RAG (Elixir) for client demos

Resources