LLM Connection Guide

Novellis is designed to be flexible. You can connect it to local AI models for maximum privacy, or high-performance cloud models for deeper narrative analysis.

All connections in Novellis use the OpenAI-compatible API standard. This means you can use almost any modern LLM provider by providing their Base URL and an API Key.

Model Categories

To optimize for both speed and intelligence, Novellis splits LLM tasks into four categories.

CategoryDescriptionPerformance Note
ThinkingDeep analysis, story arc building, and character growth tracking.Use high-intelligence models.
InstructCharacter extraction, descriptions, and natural chat.Use balanced, instruction-following models.
PlannerA fast model for orchestrating data retrieval and query routing.Use fast, low-latency models.
EmbeddingSpecialized for converting text into searchable vector data.Requires dedicated embedding models.
tip

Flexible Mapping: It is perfectly fine to use the same model (e.g., gpt-4o-mini) for Thinking, Instruct, and Planner categories if you prefer simplicity.

Provider Setup

1. OpenAI

The industry standard for reliability and performance.

  • Base URL: https://api.openai.com/v1
  • Recommended Models:
    • Chat/Reasoning: gpt-4o-mini
    • Embedding: text-embedding-3-small
  • Documentation: OpenAI API Reference

2. Gemini (Google)

Exceptional reasoning with very low latency.

  • Base URL: https://generativelanguage.googleapis.com/v1beta/openai
  • Recommended Models:
    • Chat/Reasoning: gemini-2.5-flash-lite (or latest available)
    • Embedding: text-embedding-004
  • Documentation: Google AI Studio Gemini API Docs

3. Local (Ollama / LM Studio)

Run models entirely on your own hardware for maximum privacy.

  • Base URL:
    • Ollama: http://localhost:11434/v1
    • LM Studio: http://localhost:1234/v1
  • Recommended Models:
    • Chat/Reasoning: qwen3:8b
    • Embedding: qwen3-embedding:4b
  • Documentation: Ollama.com | LMStudio.ai

How to Configure

In Novellis, navigate to Settings > AI & Connectivity.

  1. Base URL: Enter your provider's API endpoint (e.g., https://api.openai.com/v1 or http://localhost:11434/v1).
  2. API Key: Required only for cloud providers. Leave blank for local models like Ollama.
  3. Model Names: Enter the specific model IDs exactly as defined by your provider.
important

Privacy First: Your API Keys are stored locally on your machine. They are never sent to our servers or any third party other than the LLM provider you've configured.