AEYEZ: photograph documents, catch deadlines
Problem
Official letters, invoices, contracts and receipts arrive on paper, and the important information sits in a few fields: sender, amount, deadline. I wanted an app that turns a photo into structured data and deadlines. Users should decide which AI provider sees their documents, down to a local model.
My role
I set the idea and scope, signed off the design and implementation plans and reviewed the code.
Approach
A pnpm monorepo with an Expo app (React Native, Android and web), a thin Next.js API server on Vercel and a landing page. Documents and pages are stored in SQLite on the device, API keys in SecureStore. For each task (analysis, chat, translation, folder suggestion) users pick provider and model; the client sends that choice as headers, optionally with their own key.
The analysis calls generateObject from the Vercel AI SDK with a zod schema: document type, ten subtypes, typed fields such as amount, date or IBAN, and deadlines with a priority. For OpenRouter it goes through text instead: extract the JSON from the reply, normalise field names, then validate against the same schema. The app talks to Ollama and Hugging Face directly, without going through the server.
Architecture
Capture
Camera, gallery or PDF in the Expo app, several pages per document
Local storage
SQLite on the device, keys in SecureStore
AIClient
Picks provider and model per task. Local providers directly, cloud via the backend
Next.js API on Vercel
4 routes: analyze, chat, translate, suggest-folders. Headers x-ai-provider, x-ai-model, x-ai-key
Structured output
generateObject with a zod schema; for OpenRouter text plus normalisation plus zod.parse
App views
Documents, deadlines, chat with documents, settings
Decision
One schema as the contract for every provider
Every provider returns JSON slightly differently, and not all of them support structured output. Instead of maintaining one parser per provider, the zod schema is the single source of truth: generateObject where possible, otherwise text, normalisation and analysisSchema.parse. Whatever fails the schema never reaches the app. The cost: the schema is strict, so one misnamed category fails the whole analysis instead of a single field.
Code excerpt
const analysisSchema = z.object({
type: z.enum(['document', 'image', 'other']),
subtype: z.enum(['receipt', 'invoice', 'letter', 'gov_letter', 'form', 'contract', 'medical', 'id_document', 'certificate', 'bank']).nullable(),
confidence: z.number().min(0).max(1),
language: z.string().nullable(),
needs_translation: z.boolean(),
title: z.string().nullable(),
vendor: z.string().nullable(),
date: z.string().nullable(),
message: z.string(),
items: z.array(z.object({
item_key: z.string(),
item_value: z.string(),
value_type: z.enum(['string', 'number', 'currency', 'date', 'iban', 'boolean']),
category: z.enum(['line_item', 'meta', 'address', 'deadline', 'party', 'financial']),
confidence: z.number(),
})),
deadlines: z.array(z.object({
label: z.string(),
date: z.string(),
priority: z.enum(['low', 'normal', 'high', 'critical']),
sender_email: z.string().nullable(),
})),
});Results
- 197commits between 8 Mar and 14 Apr 2026
- 10document subtypes in the schema, from receipt to official letter
- 7selectable providers: 5 cloud, 2 local
- 4API routes in the backend
What I learned
The server falls back to its own keys when the user sends none, the API has no login, and CORS is set to *. Before any public test I would have needed auth and a rate limit first. I also went wide too early: a module system with purchase tracking and a scraper. I removed the module registry and the scraper in April, more than 1,100 lines. And there are no automated tests, even though the normalisation of model responses needed them most.