How to Deploy a Self‑Hosted XMPP Server with Prosody, CoTURN and Docker – A Complete Guide

You can deploy a fully‑featured, secure XMPP server by running Prosody in Docker, securing it with Let’s Encrypt TLS, enabling OMEMO encryption, and adding a CoTURN TURN/STUN service for reliable voice and video calls.

Why Self‑Host Your XMPP Server?

Federated messaging gives you control over data, eliminates vendor lock‑in, and ensures interoperability with any XMPP‑compatible client. For organizations that value privacy, compliance, or simply want to tailor features (file sharing, push notifications, AI‑enhanced bots), a self‑hosted XMPP server is the ideal foundation.

Prosody is a lightweight, extensible XMPP server written in Lua, and Docker makes its deployment repeatable and portable. Pairing it with a TURN/STUN CoTURN service unlocks high‑quality voice/video calls even behind NATs.

Below you’ll find a concise, MECE‑structured guide that covers everything from DNS SRV records to advanced security hardening, plus ideas on extending your XMPP ecosystem with AI services from UBOS homepage.

XMPP, Prosody, and the Power of Self‑Hosting

XMPP (Extensible Messaging and Presence Protocol) has been the backbone of real‑time communication since 1999. Its federated nature means each server can talk to any other server without a central authority.

• Open standards – works with UBOS templates for quick start and countless clients.
• End‑to‑end encryption via OMEMO, ensuring even the server cannot read messages.
• Extensible modules – add file sharing, push notifications, or AI bots.

Prosody’s modular architecture lets you enable only the features you need, keeping the footprint small and the attack surface minimal.

What the Original Guide Covers

The reference article (original guide) walks through a complete Docker‑Compose setup, DNS SRV configuration, Let’s Encrypt TLS, essential Prosody modules (carbons, smacks, mam, OMEMO), security hardening, firewall rules, and TURN/STUN server configuration with CoTURN.

Key take‑aways:

1. Docker‑Compose isolates Prosody and CoTURN, simplifying upgrades.
2. DNS SRV records (_xmpp-client._tcp and _xmpp-server._tcp) enable client discovery and federation.
3. Let’s Encrypt provides free, automated TLS certificates without exposing port 80.
4. Modules like carbons, smacks, mam, and omemo deliver multi‑device sync, reliable delivery, message archiving, and end‑to‑end encryption.
5. Security – enforce TLS, disable anonymous registration, and lock down firewall ports.
6. CoTURN supplies TURN/STUN for seamless WebRTC‑based voice/video calls.

Benefits & Real‑World Use‑Cases

Deploying a self‑hosted XMPP server unlocks several scenarios that are hard to achieve with consumer‑grade messengers.

Step‑by‑Step Highlights & Best Practices

1. Prepare the Host

Ensure Docker Engine (≥ 20.10) and Docker‑Compose are installed. A fresh Ubuntu 22.04 LTS VM with at least 2 GB RAM is sufficient for modest loads.

2. Obtain TLS Certificates

Use Certbot with the DNS‑01 challenge (Cloudflare, Route53, etc.) so port 80 stays closed. Example command (adjust paths):

docker run --rm \
  -v $HOME/docker/xmpp/certs:/etc/letsencrypt \
  -v $HOME/docker/xmpp/cloudflare.ini:/etc/cloudflare.ini:ro \
  certbot/dns-cloudflare certonly \
  --dns-cloudflare \
  --dns-cloudflare-credentials /etc/cloudflare.ini \
  -d xmpp.example.com

Set proper permissions (Prosody runs as prosody user) and schedule a monthly renewal cron.

3. Configure DNS SRV Records

In your DNS provider, add:

• _xmpp-client._tcp.example.com. 0 5 5222 xmpp.example.com.
• _xmpp-server._tcp.example.com. 0 5 5269 xmpp.example.com.

Also create an A record for xmpp.example.com and optional CNAME records for upload and conference subdomains.

4. Docker‑Compose File

Save the following as docker-compose.yml in a dedicated folder:

version: '3.8'
services:
  prosody:
    image: prosodyim/prosody:13.0
    container_name: xmpp
    restart: unless-stopped
    ports:
      - "5222:5222"
      - "5269:5269"
    volumes:
      - prosody-data:/var/lib/prosody
      - ./prosody.cfg.lua:/etc/prosody/prosody.cfg.lua:ro
      - ./certs/live/xmpp.example.com/fullchain.pem:/etc/prosody/certs/xmpp.example.com.crt:ro
      - ./certs/live/xmpp.example.com/privkey.pem:/etc/prosody/certs/xmpp.example.com.key:ro
  coturn:
    image: coturn/coturn:latest
    container_name: coturn
    restart: unless-stopped
    network_mode: host
    volumes:
      - ./turnserver.conf:/etc/coturn/turnserver.conf:ro
volumes:
  prosody-data:

5. Prosody Configuration (prosody.cfg.lua)

Key sections (trimmed for brevity):

modules_enabled = {
  "roster", "saslauth", "tls", "dialback", "disco",
  "posix", "ping", "register", "time", "uptime",
  "version", "blocklist", "carbons", "smacks",
  "mam", "vcard_legacy", "pep", "bookmarks",
  "admin_shell"
}
c2s_require_encryption = true
s2s_require_encryption = true
s2s_secure_auth = true
authentication = "internal_hashed"
allow_registration = false

VirtualHost "xmpp.example.com"
ssl = {
  key = "/etc/prosody/certs/xmpp.example.com.key";
  certificate = "/etc/prosody/certs/xmpp.example.com.crt";
}
Component "conference.xmpp.example.com" "muc"
Component "upload.xmpp.example.com" "http_file_share"
turn_external_host = "xmpp.example.com"
turn_external_port = 3478
turn_external_secret = "YOUR_SECRET_HERE"

6. CoTURN Configuration (turnserver.conf)

Store this file alongside docker-compose.yml:

listening-port=3478
tls-listening-port=5349
min-port=49152
max-port=49200
realm=xmpp.example.com
use-auth-secret
static-auth-secret=YOUR_SECRET_HERE
no-multicast-peers
no-cli
no-tlsv1
no-tlsv1_1
log-file=stdout
external-ip=YOUR_PUBLIC_IP/YOUR_PRIVATE_IP

7. Firewall & Network

Open only the required ports:

sudo ufw allow 5222 comment 'XMPP client'
sudo ufw allow 5269 comment 'XMPP federation'
sudo ufw allow 3478 comment 'TURN/STUN UDP'
sudo ufw allow 5349 comment 'TURN/STUN TLS'
sudo ufw allow 49152:49200/udp comment 'TURN relay ports'

8. Create User Accounts

Since registration is disabled, add users manually:

docker exec -it xmpp prosodyctl adduser alice@xmpp.example.com

9. Verify the Setup

Run Prosody’s built‑in diagnostics:

docker exec xmpp prosodyctl check

Fix any DNS, TLS, or module warnings. Optionally test with the XMPP Compliance Tester.

10. Extend with AI Services

UBOS makes it trivial to attach AI capabilities to your XMPP environment. For example:

• Deploy the AI Chatbot template as a microservice that listens on XMPP messages and replies using OpenAI ChatGPT integration.
• Use Chroma DB integration to store conversation embeddings for semantic search.
• Enable voice interactions with ElevenLabs AI voice integration, turning chat messages into spoken replies.

These extensions turn a plain messaging server into a full‑featured AI‑augmented communication hub.

Architecture Diagram

The diagram below (generated by UBOS) visualizes the relationship between Prosody, CoTURN, reverse proxy, and optional AI modules.

Next Steps & Resources

Ready to accelerate your communication stack? Explore UBOS’s ecosystem for ready‑made components and templates that complement your XMPP deployment.

• Start with the UBOS platform overview to understand how low‑code services integrate.
• Check out the AI Email Marketing template for cross‑channel outreach.
• Leverage the AI Video Generator to create onboarding videos for new chat users.
• Explore the AI Survey Generator to collect feedback from your community.
• For startups, the UBOS for startups page outlines pricing and support.
• SMBs can benefit from UBOS solutions for SMBs that include managed hosting.
• Enterprises looking for scale should review the Enterprise AI platform by UBOS.
• Build custom workflows with the Workflow automation studio.
• Experiment with the Web app editor on UBOS to prototype admin dashboards.
• Review UBOS pricing plans to find a tier that matches your usage.
• See real‑world implementations in the UBOS portfolio examples.
• Kick‑start development with UBOS templates for quick start, such as the AI Article Copywriter or the AI SEO Analyzer.

By combining a hardened XMPP core with UBOS’s AI extensions, you gain a future‑proof communication platform that scales from a single‑person testbed to enterprise‑wide collaboration.