MiniMe Technical Whitepaper

Abstract

Knowledge workers generate thousands of fragmented signals daily — active windows, browser tabs, code commits, meetings, and search queries — scattered across dozens of applications. Existing solutions either require manual effort (time trackers), violate privacy (cloud surveillance tools), or lack personal context (generic LLM assistants).

MiniMe introduces a local-first activity intelligence platform that passively captures device telemetry, enriches it into a Personal Knowledge Graph (PKG), and exposes a natural-language interface backed by a retrieval-augmented generation (RAG) pipeline. All raw data remains on-device by default. Cloud sync, when enabled, transmits only AES-256-GCM ciphertext that the server cannot read. This paper documents the system architecture, data model, storage choices, enrichment pipeline, and privacy guarantees.

1. System Architecture

MiniMe is composed of four decoupled layers:

1.1 Sensor Layer (Desktop Daemon)

A lightweight background process built with Tauri and Rust. The daemon polls active window state at 1Hz using:

• macOS: Accessibility API (CGWindowListCopyWindowInfo)
• Windows: Win32 User32 API (GetForegroundWindow)
• Linux: X11 via xdotool or D-Bus on Wayland

The daemon buffers events in memory and flushes batches to the local backend every 10 seconds. CPU usage stays below 0.5% on Apple M-series hardware. RAM footprint is under 40MB.

Browser activity is captured by separate Manifest V3 (Chrome, Edge) and Manifest V2 (Firefox) extensions. Each extension tracks tab URL, domain, page title, and dwell time. Data is batched locally and POSTed to /api/v1/activities/batch on an interval.

1.2 Backend Service

A FastAPI application (Python 3.10+) running at localhost:8000. Key responsibilities:

• Receives and validates activity batches from the daemon and extensions
• Writes normalized records to PostgreSQL (relational store, via SQLAlchemy)
• Schedules enrichment jobs via a background task queue
• Serves the REST API consumed by the Next.js dashboard
• Streams AI chat responses via Server-Sent Events (SSE)

JWT-based authentication is enforced on all endpoints. Tokens are issued at login and refreshed automatically. The dashboard connects via localhost:3000 → backend at localhost:8000.

1.3 Storage Layer

MiniMe uses three storage engines, each chosen for a specific access pattern:

• PostgreSQL: Normalized relational store for activities, users, goals, settings, and billing state. Enables complex time-series queries and aggregations (focus score, time-by-domain breakdowns).
• Qdrant: Vector database for activity embeddings. Each enriched activity record is embedded using a sentence transformer and stored as a dense vector. Powers the RAG retrieval step in AI chat — user queries are embedded and matched against the activity corpus via approximate nearest-neighbor search.
• Neo4j: Graph database for the Personal Knowledge Graph. Stores entities (Person, Project, Skill, Concept, Organization, Artifact, Event, Interaction) as nodes and temporal relationships as edges with weights derived from co-occurrence frequency.

1.4 AI Enrichment Pipeline

Raw activity records are sparse — a window title like "layout.tsx - minime-backend — VS Code" carries implicit semantics (TypeScript file, project name, code editor) that must be extracted to be useful.

The enrichment pipeline runs as a background job on a configurable interval (default: every 5 minutes during idle periods). Steps:

1. Entity extraction: A structured prompt extracts entities (project name, file type, skill domain, application category) from the raw window title and URL. This runs against the backend API (OpenAI-compatible endpoint by default; configurable to self-hosted Ollama for full local operation).
2. Embedding: The enriched activity text is embedded using all-MiniLM-L6-v2 and stored in Qdrant.
3. Graph update: Extracted entities and their relationships are upserted into Neo4j. Edge weights are incremented on each co-occurrence.
4. Analytics aggregation: Daily focus scores, skill time distributions, and wellness indicators are recomputed and cached in PostgreSQL.

2. The Personal Knowledge Graph (PKG)

The PKG is MiniMe's primary output artifact. It transforms a flat activity log into a queryable semantic network.

Node types: Person, Project, Skill, Concept, Organization, Artifact, Event, Interaction

Example edges:

(User)-[:WORKED_ON {hours: 4.2}]->(Project: "minime-backend")
(User)-[:USED_SKILL {count: 28}]->(Skill: "Python")
(Project: "minime-backend")-[:REQUIRES]->(Skill: "FastAPI")
(User)-[:ATTENDED {date: "2026-03-04"}]->(Event: "Sprint Planning")

Edges carry temporal metadata so the graph reflects not just what the user does but when and how often. A Cypher query like MATCH (u)-[:USED_SKILL]->(s) WHERE s.name ="Rust" RETURN count(*) returns a precise count of activities involving Rust over any time window.

The graph dashboard in the web app renders the PKG using a force-directed layout, with node size proportional to edge count (activity frequency) and color encoding entity type.

3. Privacy and Security Model

The core invariant: raw activity data never leaves the user's machine without their explicit action.

3.1 Local-only mode (Free plan)

All data stays in the local PostgreSQL and Qdrant instances. The backend runs entirely on localhost. No outbound network calls for data storage. AI enrichment can run via a locally hosted Ollama instance to eliminate external API calls entirely.

3.2 Cloud Sync (Pro plan)

When cloud sync is enabled, the client serializes and encrypts the relevant data subset using AES-256-GCM before transmission. The encryption key is derived from the user's account credentials using PBKDF2-HMAC-SHA256 with a per-user salt. The MiniMe cloud backend receives and stores opaque ciphertext blobs in Google Drive via the Drive API — the server has no access to the encryption key and cannot construct any analytics from the stored data.

3.3 Access control

All backend API endpoints enforce JWT bearer authentication. Tokens expire after 24 hours. The local PostgreSQL instance is accessible only to the running backend process — no external network exposure. The Qdrant and Neo4j instances listen on loopback addresses only.

4. Measured Performance

Measured on an Apple M3 Pro (16GB RAM) running macOS 15:

• Sensor daemon CPU: < 0.5% (steady state)
• Sensor daemon RAM: < 40MB
• Activity batch write latency (PostgreSQL): < 5ms
• Qdrant embedding insert: < 20ms per record
• AI chat first token latency (backend API): < 2s
• Dashboard initial load: < 1.2s (Next.js static export)

5. Dashboard and Integrations

The web dashboard (Next.js 14, React 18, Tailwind CSS) runs at localhost:3000 and connects to the backend API via JWT-authenticated fetch. It ships with 14 built-in views:

Activity Timeline · Knowledge Graph · AI Chat · Productivity Analytics · Skills · Goals · Wellness · Collaboration · Career · Weekly Digest · Tasks · Enrichment · Billing · Settings

Three platform integrations are currently active:

• GitHub: Commit metadata, repository activity
• Google: Calendar events, Gmail thread metadata (subject, sender — no body)
• Notion: Page access patterns (which pages, when) — not page content

6. Roadmap

Planned additions (not yet shipped):

• Safari browser extension (MV3 implementation in progress)
• Wearable data ingest (Apple Health, Oura) for wellness correlation
• Local-only LLM mode with full Ollama integration for offline AI chat
• Enterprise team aggregate analytics (anonymized, opt-in)
• Obsidian and Logseq PKM sync