Skip to main content

dqlitepy

Python bindings for the dqlite distributed SQLite engine. dqlitepy provides a fully replicated, fault-tolerant SQLite database with Raft consensus, packaged as a self-contained Python library with no external dependencies.

Features​

  • 🔄 Distributed SQLite: Replicate your SQLite database across multiple nodes
  • 🛡️ High Availability: Automatic leader election and failover with Raft consensus
  • 🐍 Pythonic API: Simple, intuitive interface for node and cluster management
  • 🔗 SQLAlchemy Support: Full ORM integration with correct column mapping
  • 📦 DB-API 2.0: PEP 249 compliant interface for database compatibility
  • 🚀 Self-Contained: No system dependencies - everything bundled in the wheel
  • 🔒 ACID Transactions: Strong consistency guarantees across the cluster

Why dqlitepy?​

dqlitepy combines the simplicity of SQLite with the reliability of distributed systems:

  • No separate database server - Embed directly in your Python application
  • Strong consistency - All writes go through Raft consensus
  • Automatic replication - Data synchronized across all nodes
  • Simple deployment - Just a Python package, no infrastructure to manage

Quick Start​

Install from wheel:

pip install dqlitepy-0.2.0-py3-none-any.whl

Single Node Example​

from dqlitepy import Node
from pathlib import Path

# Create and start a node
node = Node(
    address="127.0.0.1:9001",
    data_dir=Path("/tmp/dqlite-data")
)
node.start()

# Open a database and execute queries
node.open_db("myapp.db")
node.exec("CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)")
node.exec("INSERT INTO users (name) VALUES ('Alice')")

# Query data
results = node.query("SELECT * FROM users")
print(results)  # [{'id': 1, 'name': 'Alice'}]

node.stop()

Cluster Example​

from dqlitepy import Node
from pathlib import Path

# Start first node (bootstrap)
node1 = Node(
    address="172.20.0.11:9001",
    data_dir=Path("/data/node1"),
    cluster=["172.20.0.11:9001", "172.20.0.12:9001", "172.20.0.13:9001"]
)
node1.start()

# Start second node (joins cluster)
node2 = Node(
    address="172.20.0.12:9001",
    data_dir=Path("/data/node2"),
    cluster=["172.20.0.11:9001", "172.20.0.12:9001", "172.20.0.13:9001"]
)
node2.start()

# Write to any node, read from any node
node1.open_db("myapp.db")
node1.exec("CREATE TABLE products (id INTEGER PRIMARY KEY, name TEXT)")
node1.exec("INSERT INTO products (name) VALUES ('Widget')")

# Data is automatically replicated
node2.open_db("myapp.db")
results = node2.query("SELECT * FROM products")
print(results)  # [{'id': 1, 'name': 'Widget'}]

SQLAlchemy Integration​

from sqlalchemy import create_engine, Column, Integer, String
from sqlalchemy.orm import declarative_base, Session
from dqlitepy import Node
from dqlitepy.sqlalchemy import register_dqlite_node

Base = declarative_base()

class User(Base):
    __tablename__ = 'users'
    id = Column(Integer, primary_key=True)
    name = Column(String(50))
    email = Column(String(100))

# Start node and register with SQLAlchemy
node = Node("127.0.0.1:9001", "/tmp/dqlite")
node.start()
register_dqlite_node(node, node_name='default')

# Use standard SQLAlchemy ORM
engine = create_engine('dqlite:///myapp.db')
Base.metadata.create_all(engine)

with Session(engine) as session:
    user = User(name='Alice', email='alice@example.com')
    session.add(user)
    session.commit()

What is dqlite?​

dqlite is a distributed SQLite engine that provides:

  • Raft Consensus: Leader election and log replication
  • SQLite Compatibility: Standard SQLite SQL syntax and features
  • No Split-Brain: Strong consistency with majority quorum
  • Automatic Failover: New leader elected if current leader fails
  • Battle-Tested: Used in production by Canonical (LXD, MicroCloud)

Next Steps​