ENGINEERING NOTES

How Wakr checks you got out of bed without a frame leaving the phone.

Wakr’s alarm keeps ringing until the phone confirms a real action, like squats, brushing your teeth, or finding the object you photographed. These are the rules behind those checks, the numbers we tuned, and what we got wrong.

Wakr is an iPhone alarm with no snooze button and no stop action. It rings until you finish a mission you picked the night before, and the phone decides whether you did. That puts a camera in a bedroom at 6am, judging someone who is half asleep and would very much like it to say yes.

Two constraints shaped everything here. No frame leaves the phone: nothing is uploaded and nothing is written to disk. And the decision comes from plain rules over detector output, kept apart from the detectors, so the same evidence always gets the same answer and every rule can be unit-tested.

The pipeline, frame by frame

Wakr is a Flutter app. The camera plugin streams frames at its medium preset, which is 640×480 on iPhone, in BGRA. Each frame crosses a platform channel to a small native plugin that runs Apple’s Vision framework and sends back points, labels, and numbers, never pixels. The mission rules then run in Dart.

Frames are dropped, not queued. If a frame arrives while the previous one is still being analysed, it is thrown away. A queue would only make the app judge where you were a second ago.

The native side does not copy. CVPixelBufferCreateWithBytes wraps the bytes Flutter already holds for the length of one analysis, and nothing keeps the buffer after it. Vision runs on one serial queue at user-initiated priority.

Each mission runs only the requests it needs. A brushing check runs face, hand, and body-pose requests. A barcode check runs only the barcode request. Find bright light, Spot the sky, and Spot a plant run no Vision request at all: their numbers come from about 1,800 evenly spaced pixel samples per frame, read in Dart.

Time between frames is capped at half a second. Most rules add up seconds of evidence, so a stalled pipeline that suddenly delivers a frame must not be credited with the whole gap.

Why Vision, not ML Kit, on iPhone

The first iOS builds used Google’s ML Kit for pose and text recognition, which the Android build still uses for pose. Its iOS pods alone added about 50 MB to the app. Vision ships with the operating system, so pose detection on iPhone moved to it in September 2026.

The move surfaced a bug worth knowing about if you combine the Flutter camera plugin with Vision. On iOS the plugin sets the capture connection’s video orientation, so buffers arrive upright, but it still reports a sensor orientation of 90 degrees. Passing that rotation on made Vision analyse every frame turned on its side, where the body model found nobody. Wakr now tells Vision that iOS frames are already upright.

Text recognition went at the same time. Write your top priority used to wait until OCR had read five characters. It now looks for a pen, a hand, and a fingertip that moves, because it only needs to know you are writing, not what you wrote. Wakr no longer reads your note at all.

One pose from three detectors

Vision’s body-pose model is trained on whole bodies. A face mission gives it something else: a head and a hand at arm’s length, often with no shoulders in shot. The model mostly gives up on that framing.

So the pose the rules see is assembled from three requests. VNDetectFaceLandmarksRequest supplies the nose, eyes, and mouth corners of the largest face in frame. VNDetectHumanHandPoseRequest, allowed two hands, supplies wrists and index fingertips. VNDetectHumanBodyPoseRequest fills in whatever else is visible. Vision reports normalised points with the origin at the lower left; the plugin converts them to upright pixels with y growing downward, which is what the rules are written against.

Rules measure distance in body units, not pixels, so they behave the same at one metre or three. The unit is shoulder width when both shoulders are seen with confidence, otherwise six times the distance between the eyes, otherwise eight times the width of the mouth.

The rules, mission by mission

Every rule follows one principle written into the verification engine: a single frame never completes a mission. Evidence has to persist for a set time or form a complete movement cycle. The values below are for the default Normal difficulty.

MissionSignalPasses when
Brush teethFace and hand landmarksA wrist stays within one body unit of the mouth for 5 seconds in total.
Drink waterFace and hand landmarksThe same hand-at-mouth rule, for 5 seconds in total.
Wash faceFace and hand landmarksA wrist within 1.3 units of the mouth is also moving, for 5 seconds.
20 jumping jacksBody poseWrists cross the shoulder line and back, past a 0.15-unit dead band. With ankles in view, feet must be wider than 1.25 shoulder widths when open and narrower than 1.05 when closed.
15 squatsBody poseHips sink 0.32 torso lengths below the highest point of the last 4 seconds, then return to within 0.12.
Push-ups for 10 secondsBody poseWith the hips in view, the torso stays near level, and the shoulders or nose travel 0.12 units up and down within each 1.5-second window.
Find bright lightPixel samplesMean brightness reaches 55%, or 5% of samples are blown-out highlights, for 3 seconds.
Spot the skyPixel samplesThe same brightness test, held for 5 seconds without a gap.
Spot a plantPixel samplesAt least 8% of samples are clearly green, held for 5 seconds without a gap.
Write your top priorityLabels and hand landmarksA pen, a hand, and a moving fingertip are seen together for 3 seconds.
Leave your bedroomLabels and poseA bed is in view and nobody is, for 3 seconds.
Scan your codeBarcodeThe exact saved payload is read and held for 0.6 seconds.
Find your objectFeature printThe view stays within 0.62 of the saved print for 1.5 seconds.
Make your bedFeature printThe view is within 0.62 of your made-bed print and at least 0.05 closer to it than to the bed as it looked when the mission opened, for 1.5 seconds.
Hunt down 2 objectsImage labelsOne of the named object’s labels scores 0.25 or more for 1 second, for each of 2 objects.
Read 2 phrases aloudOn-device speechAt least three quarters of each phrase’s words are heard in order, for 2 phrases.

Brushing and drinking ask for no movement. A toothbrush and a glass are not things the detectors reliably see, so the check verifies the hand holding them: at the mouth, for longer than a passing gesture. An early version also required motion. A brushing stroke is a few millimetres at the wrist and drinking is a hold, so it mostly stalled people doing exactly what was asked.

Jumping jacks fall back to time. Counting a repetition needs a frame at both extremes, and pose inference is slower than a jump, so the frame that would close a cycle is often never analysed. Time spent in the open position earns progress too, at 0.6 seconds per repetition. It carries the same feet check, so with ankles in shot, waving your arms earns nothing. A phone on a nightstand rarely sees ankles, and there the arm cycle counts on its own.

Squats follow you. Standing is the highest the hips have been in the last 4 seconds, so the baseline adjusts when you shuffle closer or farther away, and sitting in a chair cannot pass for a squat for longer than that window. Depth is measured in torso lengths, because the torso barely changes length through a squat: hips at parallel sit about two thirds of one below standing.

Push-ups are timed, not counted. A phone propped on the floor sees a large face, a foreshortened shoulder line, and usually no hips. The check rides whatever it can see, the shoulders when they are scored and the nose otherwise, each tracked separately so a detector flickering between them never looks like movement. Shoulders are accepted at a lower confidence than elsewhere, because the body model is unsure of a horizontal body, and a push-up is nothing but one.

Brightness, and the auto-exposure lesson

Brightness is plain arithmetic: luma from the Rec. 709 weights, 0.2126 red, 0.7152 green, and 0.0722 blue, averaged over the samples. The first version of Find bright light passed at an average of 62%, and it failed at the first bright window. Auto-exposure pulls every scene back toward mid-grey, so daylight and a lamp-lit bedroom average out to similar numbers.

What auto-exposure cannot hide is the highlights it blows out. The check now also passes when 5% of samples reach 235 out of 255, and that is what separates daylight from a dim room.

Spot a plant counts samples where green clearly beats red and blue. These are deliberately coarse measurements. Spot the sky does not recognise sky and Spot a plant does not recognise plants: a bright enough lamp or a green T-shirt will pass. They exist to get you out of bed and to a window, and they are cheap enough to run on every frame.

Matching objects, beds, and codes

Find your object asks you to photograph something, such as the coffee machine, the night before, then show it to the camera in the morning. At setup, Wakr averages the VNGenerateImageFeaturePrintRequest output of three frames into one reference vector. In the morning, each frame’s print is compared with it by the Euclidean distance between unit-length vectors.

The pass mark of 0.62 came from calibration. Re-framed, rotated, dimmed, warm-lit, and closer views of one object scored about 0.34 typically and 0.56 for nine views in ten. Different objects started around 0.78. Two near-identical subjects, two roses, came as close as 0.42, which is why a match must also hold for 1.5 seconds rather than relying on the bar alone.

The request is pinned to revision 1. Prints from different revisions cannot be compared, so without the pin an iOS update could quietly invalidate every saved reference.

Make your bed reuses feature prints but measures a direction of change, because a made bed and the same bed slept in are one object in one room, and a plain match would pass either. When the mission opens, Wakr averages the first clear views of the bed, with nobody in it, into a baseline. A frame passes only when it is within 0.62 of the made-bed print you saved and at least 0.05 closer to it than to that baseline. A bed that was already made when the mission opened cannot pass this way.

Hunt down objects needs no reference at all. It names everyday objects and runs VNClassifyImageRequest, whole-frame classification rather than object detection, until one of the named object’s labels scores 0.25 or more for a second. That bar was set from desk photos, where an object filling a good part of the frame scores well above it and clutter at the edges mostly below.

What Wakr keeps is the vector, as raw floats in the app’s local settings, never the photo. ML Kit on Android has no equivalent of feature prints, so Find your object and Make your bed are iPhone-only. Scan your code is simpler: it stores the payload of the barcode or QR code you saved, and setup needs three identical reads in a row so a misread cannot become the reference.

Steps without the camera

Walk 100 steps reads the hardware step counter through Core Motion’s CMPedometer. The first reading after the mission starts becomes the baseline and only increases count, so steps taken before the alarm cannot be reused.

If the hardware counter stays silent through 3 seconds of detected motion, an accelerometer step detector takes over, and only one source ever counts. It registers a step at 1.35 m/s² of user acceleration, re-arms below 0.55, and ignores peaks less than 280 ms apart. The first version latched: a phone in a pocket never returns fully to rest, so the detector counted one step and waited for a reset that never came. It now re-arms after any peak held for 600 ms.

Speech stays on the device too

Read 2 phrases aloud uses Apple’s Speech framework with requiresOnDeviceRecognition set. Where a language has no on-device recognizer, the plugin refuses to start rather than fall back to Apple’s servers.

A phrase passes when at least three quarters of its words are heard in order, scored as the longest common subsequence of normalised words, with each Chinese or Japanese character counted as a word. Demanding every word would fail people who did read the line, because recognizers mishear the odd word, spell a name another way, or write a number as digits. A listening turn ends after 8 seconds of silence or 40 seconds at most, and two turns that hear nothing offer the backup mission.

When a check cannot work, the alarm still has to end

A mission during a ringing alarm cannot be dismissed, so a pipeline that produces nothing has to say so rather than sit at 0%. If 12 seconds pass without one analysable frame, the screen reports that the camera has no usable view. If 75 seconds pass without progress, it offers a backup: shaking the phone for 30 seconds, read from the raw accelerometer, which every phone has and no platform puts behind a permission.

Spot the sky offers the backup after 20 seconds instead. Its usual failure is not a struggling detector but an alarm set before sunrise, and no amount of waiting fixes that.

What we got wrong

Nearly every verification bug that reached a real morning was the same bug: a rule stricter than the movement it was checking. Requiring both hands while washing failed because a hand raised to the face routinely hides the other one. Requiring both wrists to agree on a jumping jack failed because one arm usually lags. Requiring ankles before any rule ran failed on every phone propped at bed height, though no rule needed them. And some builds report landmark confidence as a flat zero, so a confidence cutoff threw away the whole skeleton.

Each one left someone genuinely doing the mission, with an alarm ringing, watching progress sit at zero. That set the priority for every rule since: at 6am a false negative costs far more than a false positive. A person standing at the sink with a toothbrush is awake, and the check only needs to be harder to fake than it is to do.

The honest limits

These checks are built to make staying in bed more effort than getting up, not to be impossible to fool. Brushing, drinking, and Leave your bedroom require no movement, so a still image held up long enough could in principle pass them. Spot the sky and Spot a plant measure brightness and colour, not what is in the picture. The shake backup can be done lying down.

Wakr’s answer is a second check rather than a stricter detector. Five minutes after you finish, it asks whether you are still up. If nobody answers within three minutes, the alarm rings again, and that alarm ends the same way: with a mission. The AlarmKit write-up covers how that alarm is delivered.

Privacy, stated precisely.

Nothing in the app takes a picture or writes a frame to a file. A frame exists in memory for one analysis and is released. Speech for Read aloud is recognised on the phone. The only camera-derived data kept are the reference vectors for Find your object and Make your bed and the payload text for Scan your code, all in the app’s local settings on the phone, which are included in your device backups like any app’s settings. The parts of Wakr that use the network never receive frames or feature prints.

Sources

  1. VNDetectHumanBodyPoseRequest, Apple Developer Documentation, accessed October 21, 2026.
  2. VNDetectHumanHandPoseRequest, Apple Developer Documentation, accessed October 21, 2026.
  3. VNDetectFaceLandmarksRequest, Apple Developer Documentation, accessed October 21, 2026.
  4. VNGenerateImageFeaturePrintRequest, Apple Developer Documentation, accessed October 21, 2026.
  5. CMPedometer, Apple Developer Documentation, accessed October 21, 2026.

The one-line version

Wakr verifies missions with narrow, testable rules over on-device detector output, tuned so a person who is really up passes, and keeps nothing from the camera except feature vectors and a barcode’s text.

Building something similar and want to compare notes, or spotted a mistake here? Write to Ian.

READY WHEN YOU ARE

Make your next alarm a first step.

Choose a mission and start building mornings that move forward.

Download Wakr on the App StoreAvailable now on the App Store. Android is in development.