01
EdTech PlatformLive

Tutorly-AF Academy.

Full learning platform for multi-country curricula, launching in Zambia. Expo mobile app for students, a Tauri desktop studio for tutors, a GraphQL backend, and a Rust media pipeline — with live classes on self-hosted LiveKit, an AI tutor with voice and RAG, end-to-end encrypted chat, and a non-destructive video editor.

// Overview

Tutorly connects students with tutors through courses, live classrooms, 1:1 voice and video calls, an AI tutor, quizzes, study plans, Shorts, podcasts, scientific articles, and communities. It is built for multi-country curricula (country → education system → grade level → institution), with Zambia as the launch market and purchasing-power pricing.

I designed and built the whole system: an Expo app for students on iOS, Android, and web; a Tauri desktop studio where tutors teach, record, and edit; a Bun + Hono server exposing GraphQL as its primary API; and a Rust worker that turns uploads into streamable, data-friendly media.

Much of the engineering is shaped by the market — mobile data is expensive, devices are modest, many learners are minors, and many study above the official age for their grade — so the platform is built to be cheap to stream, safe by default, and welcoming rather than gatekeeping.

01

AI Tutor

  • Gemini-powered tutor that streams answers over SSE and accepts text, images, documents, and voice notes.
  • Real-time voice tutoring: the server bridges the app’s WebSocket to a Gemini Live session, so the API key never reaches the client and guardrails, metering, and safeguarding still apply.
  • Tool calling with 19 tools — search the curriculum, build a practice quiz, grade work, generate a diagram, review past mistakes, save notes, book a live session, enrol in a course, set a study reminder, and more.
  • Tool schemas never accept a user ID; the server binds the authenticated student itself, closing the most common IDOR hole in tool-calling apps.
  • Answers are grounded in published courses via a pgvector RAG index (HNSW) and in lesson transcripts, so the tutor can answer from what a teacher actually said.
  • Three-layer guardrails: a cheap Flash-Lite pre-classifier, a constrained main call, and an output check — including detection of requests to do graded work.
  • Safeguarding: if a child signals self-harm, the tutor keeps talking supportively while a durable escalation reaches a human, with regional crisis resources.
  • Cost control that makes “unlimited AI” safe on a K99/month plan: rolling, tutoring-aware history compaction, prompt caching, daily caps, a per-period cost ceiling, and Gemini → Nova provider failover.
02

Live Classes, Calls & Radio

  • Group classes and 1:1 sessions on self-hosted LiveKit, metered separately per plan because their costs differ.
  • Session recording through LiveKit Egress, with replays for students who missed the class.
  • Native phone-style calls: CallKit on iOS via VoIP push, FCM on Android, and the tutor’s desktop app rings over WebSockets.
  • Live radio: audio-only broadcasts that automatically become podcast episodes — recorded, loudness-normalised, streamed, and published, with the tutor notified.
  • Session requests, tutor availability, and scheduled reminders.
03

Rust Media Pipeline

  • A standalone Rust worker driven by a pg-boss queue and Postgres NOTIFY, running FFmpeg and writing results back to object storage.
  • Adaptive HLS from 1080p down to an always-present 360p rung for mobile data — never upscaled, portrait stays portrait.
  • Listen-only audio for learners on data bundles and compressed offline downloads for paid plans.
  • whisper.cpp transcription produces captions, word-level timings, and searchable transcripts.
  • Signed HLS playback out of a private bucket, with the signature in the path so native players that can’t send headers still work.
  • Speaker tracking for vertical Shorts: a ~1 MB UltraFace ONNX model compiled into the binary finds the tutor’s face and builds a smooth “patient camera operator” crop path from landscape lessons.
  • Audio processing for podcasts: loudness levelling, silence detection, and waveforms.
04

Tutor Desktop Studio

  • Tauri 2 app for tutors: teach live, author courses and quizzes, manage students, track earnings, and moderate communities.
  • Non-destructive video editor: the timeline is a JSON document previewed live and rendered server-side; text layers are rasterised by the app so the render matches the preview exactly.
  • Edit a video by editing its transcript — deleting words cuts the matching source ranges, as a single undo step.
  • An AI editing assistant proposes cuts that are applied against the current edit, even if it changed since the proposal.
  • Shorts builder: pick moments from a lesson and get 9:16 clips with 2–3 word captions sized for phones.
  • Podcast audio editor: trim, cut pauses and “um”s, add intro/outro and a music bed, fade, and level loudness — the original is always restorable, and chapters and listeners’ positions move with the edit.
  • Brand kits (logo, colours, font, intro/outro), including automatic removal of a logo’s solid background with an edge flood fill.
  • Crash-safe recording saved to IndexedDB every second, access tokens in the OS keychain, and signed auto-updates.
  • Student insights: study time, course progress, and quiz performance per learner.
05

Privacy & Safety

  • End-to-end encrypted chat with NaCl keys: the server stores only public keys and opaque message envelopes.
  • PIN-protected key recovery using scrypt, with server-enforced brute-force lockout.
  • Rich messages — equations, code, questions, assignments — that keep everything the user wrote inside the encrypted body.
  • Guardian consent for minors, and a “guide, don’t gate” age policy so over-age learners (common in Zambia) are never turned away.
  • Blocking, reporting, and moderation queues across chat, communities, and comments.
06

Learning & Content

  • Courses with video, interactive, and text lessons, bookmarks, notes, and comments.
  • Quizzes with feedback, AI-generated practice quizzes, study plans, progress tracking, and streaks.
  • Shorts, podcasts with scheduled episodes and subscriptions, and topic communities with events.
  • Scientific articles with a block editor, co-authors, moderation, and PubMed/PMC import via NCBI E-utilities, rate-limited to stay within NCBI policy.
  • Offline-first mobile app with a persisted Apollo cache and downloadable lessons, plus a 3D onboarding built with React Three Fiber.
07

Business & Growth

  • Purchasing-power pricing (K99 locally vs $19 in USD markets), with trials, entitlements frozen at purchase, and allowances that reset on the billing anniversary.
  • New markets are a data change, not a deploy — currencies for South Africa, Botswana, and Malawi are already seeded.
  • WhatsApp support bot that links a number to an account, answers from that account, books or enrols only after a confirm tap, and hands schoolwork to the in-app tutor.
  • Notifications over Expo push, email, and SMS (eSMS Africa), with per-user preferences.
  • Object storage on S3 with automatic Cloudflare R2 failover and an optional mirror mode.
08

Architecture

  • Layered routes/resolvers → controllers → services → Prisma, so GraphQL and REST share the same business logic.
  • About 5,000 lines of GraphQL schema across 29 domains; typed operations on mobile with gql.tada and live updates on desktop with graphql-ws.
  • Native Bun WebSockets for chat, calls, live sessions, and voice tutoring, with Redis pub/sub.
  • Local JWT plus Auth0 with just-in-time user provisioning.
  • Deployed on Fly.io and an Oracle VPS, with LiveKit, Egress, and the media worker in containers.