AI Features in Open Source

In Open Source, AI features work through separate AI servers that the administrator deploys:

  • gramax/docportal-ai — for AI search on the documentation portal;

  • gramax/editor-ai — for AI editing in the editor.

Both need an OpenAI API-compatible model server: cloud (OpenAI, DeepSeek) or local (LM Studio, Llama.cpp, vLLM).

How it works

  1. Catalogs on the portal are split into chunks. An embedding model converts them into vectors, which are stored in the Qdrant vector database.

  2. A reader enters a query on the portal, and Gramax sends it to the gramax/docportal-ai AI server.

  3. The server searches the database for matches, and a chat model generates an answer based on the found articles.

One docportal-ai server can serve several portals.

Setup

  1. Deploy docportal-ai following the guide on Docker Hub. Example docker-compose.yml:

    version: "3.8"
services:
  db:
    image: qdrant/qdrant
    environment:
      - QDRANT__SERVICE__MAX_REQUEST_SIZE_MB=512
    ports:
      - 3006:6333
    volumes:
      - server-ai-qdrant-storage:/qdrant/storage

  docportal-ai:
    image: gramax/docportal-ai
    environment:
      - VECTORDB__TYPE=qdrant
      - VECTORDB__HOST=http://db:6333

      - EMBEDDING__TYPE=openai
      - EMBEDDING__MODEL=text-embedding-3-small
      - EMBEDDING__DIMENSIONS=1536
      - EMBEDDING__APIKEY=<your API key>

      - CHAT__TYPE=openai
      - CHAT__MODEL=gpt-4o
      - CHAT__APIKEY=<your API key>

      - AUTH__ADMIN__TOKEN=<token for Gramax>
    ports:
      - 3005:3005
    volumes:
      - server-ai-api-storage:/app/storage
      - ./config.yaml:/app/config/config.yaml
    depends_on:
      - db

volumes:
  server-ai-qdrant-storage:
    external: false
  server-ai-api-storage:
    external: false

  2. In docker-compose.yaml of the documentation portal, add the variables:

    • AI_SERVER_URL — URL of the docportal-ai service, without a trailing /.

    • AI_TOKEN — the token from AUTH__ADMIN__TOKEN.

    • AI_INSTANCE_NAME — unique identifier of the portal.

  3. Restart the portal.

Streaming Search Responses

For streaming to work correctly, you need to properly configure a proxy that will proxy requests to the LLM service.

Example for Angie

upstream vm-ai-gramax {
    zone http:vm-ai-gramax 1m;
    server <ip>:<port>;
}

server {
    listen <ip>:80;
    server_name ai-server.gramax www.ai-server.gramax;
    return 301 https://$server_name$request_uri;
}

server {
    listen <ip>:443 ssl;
    http2 on;
    server_name ai-server.gramax www.ai-server.gramax;
    status_zone https:ai-server.gramax;

    include /etc/angie/http.d/common/ssl-gram-ax.conf;
    include /etc/angie/http.d/common/noindex_robots.conf;
    include /etc/angie/http.d/common/error_pages.conf;

    location / {
        include /etc/angie/http.d/common/reverse-proxy.conf;
        client_max_body_size 500m;
        proxy_http_version 1.1;
        proxy_pass http://vm-ai-gramax;
        proxy_buffering off;
        proxy_cache off;
        proxy_read_timeout 3600s;
        proxy_send_timeout 3600s;
        tcp_nodelay on;
    }
}

AI editing

The app sends text or audio to the gramax/editor-ai AI server, the server passes the request to the model and returns the result to the editor.

  1. Deploy editor-ai following the guide on Docker Hub. Minimal docker-compose.yaml:

    version: "3.8"
services:
  editor-ai:
    image: gramax/editor-ai
    environment:
      - CHAT__TYPE=openai
      - CHAT__MODEL=gpt-4o
      - CHAT__APIKEY=<your API key>
      - AUTH__ADMIN__TOKEN=<token for Gramax>
    ports:
      - 3007:3005
    volumes:
      - ./config.yaml:/app/config/config.yaml

    Inside the container the service listens on port 3005, like docportal-ai. The external port in the example is 3007 so that both servers can run on one host: port 3005 is taken by docportal-ai and 3006 by Qdrant.

  2. Give users the server URL and the token from AUTH__ADMIN__TOKEN. They enter them in space settings — see AI editing.