Notes from the workshop

Your folder names were the training data

The archive this app was built on is 314,391 files: about 223,000 photos and 91,000 videos, sitting on a 6 TB external drive. It's one family's entire visual record, and like a lot of families, they went through two very different eras of keeping it.

From 2004 to 2016, every event got its own folder, named by hand, the day after it happened: 2012-04 — the spring trip, that kind of thing. Roughly 600 of these. Then, sometime around 2016, the folders stopped. What replaced them was the modern default: giant, flat phone-backup dumps, thousands of files named IMG_4471.HEIC, grouped by nothing except the date range a backup happened to run. (If your recent years are still trapped on the phone itself, the iPhone offload guide covers getting them onto the drive.)

The obvious plan for a tool like this is to auto-detect events and auto-generate names for the whole archive, named era and dumped era alike. The thing I didn't expect going in was that the named era wasn't just nicer to look at — it was a labeled dataset, sitting there for free, twelve years deep.

Ground truth, not a guess

Six hundred human-drawn boundaries around what counts as "one event" is exactly the kind of thing you'd normally pay for. This family had already done the work of deciding, event by event, where one moment ends and the next begins. So instead of picking an event-clustering approach off a shelf and hoping it felt right, I treated the named folders as ground truth for how this particular family splits time into events, and tuned against them directly.

The clustering approach itself is deliberately simple: a time-gap pass, grouping photos into the same event whenever the gap between consecutive timestamps stays under a threshold. The only real knob is where you set that gap. I measured it the boring, correct way — pairwise co-membership agreement. For every pair of photos in the named era, check whether the human folder split agrees with the algorithm's split (both together, or both apart), and score the fraction that agree. Sweep the threshold, plot the curve, take the peak.

314,391
files in the founding archive
~600
hand-named event folders, 2004–2016
F1 0.93
at a 24-hour gap threshold

The peak landed at a 24-hour gap, with an F1 of 0.93 against the named folders. That's not a benchmark from a paper — it's a number tuned against one family's own twelve years of judgment calls about where their events begin and end. Which is exactly the point: the "right" clustering threshold isn't universal, it's a property of how a specific family lives. This family apparently doesn't split a two-day trip into two events just because they slept somewhere in between. A 24-hour gap captures that. A different family's named folders might peak somewhere else entirely, and the app is built to re-measure per archive rather than assume.

Once events cluster the way the family already clusters them, naming is the next problem, and the same trick applies again: the 600 named folders aren't just boundaries, they're a style guide. 2012-04 — the spring trip tells you the family's convention is date prefix, em dash, lowercase, short and specific rather than generic ("the spring trip," not "Family Outing"). Auto-naming for the un-named decade learns that voice instead of imposing a generic one.

The wrinkles that would have quietly corrupted everything

Three things came up while building this that are worth writing down, because none of them are visible until you go looking, and all three would have silently degraded the timeline if I'd trusted the obvious source of truth.

Camera clocks lie, and folder names don't. In the named era, camera clocks were sometimes a year off — not wildly, just wrong in the way a camera clock gets wrong when nobody resets it after a battery change or a trip across a date line. The fix was to stop trusting EXIF as the default truth for dating and instead let the human-typed folder date prefix win whenever the two disagree. The person who typed 2012-04 into a folder name the week it happened is a better witness than a camera's internal clock that hasn't been checked in years.

Digitized media carries the wrong date twice. Old camcorder tapes, digitized later, are worse: the file's own timestamp metadata reflects when the digitization happened, not when the footage was shot. For this class of file, EXIF isn't just occasionally wrong, it's wrong in a specific, structural way. The fix is the same shape as the camera-clock fix, one level up: for scanned and digitized media, the year folder the family filed it under overrides the file's own timestamp, because the folder is describing the memory and the timestamp is describing the scanning session.

Trips need a definition of "home" that isn't fooled by enthusiasm. To detect a trip as a trip — a departure from normal life — the app needs to know where "normal life" was, and that shifts over 22 years as a family moves. The naive approach, count photos per location, fails in an obvious way once you think about it: a five-day vacation generates far more photos than five ordinary days at home, so a location a family visited once for a week could easily out-vote the house they actually lived in for six months. The fix is a home model that votes per half-year using distinct days present at a location, not photo counts. A place only counts toward "home" by being lived in on many different days, not photographed a lot on a few of them. That single substitution — days instead of photos — is what lets a trip get correctly recognized as a departure from home instead of quietly rewriting where home was.

The named years didn't just get labels. They got to define, in this family's own hand, what an event even is.

None of this generalizes as "here's the correct threshold, ship it." What generalizes is the method: if part of an archive was hand-labeled by the people who lived it, that part isn't just nicer, it's ground truth, and it's worth the extra step of measuring against it rather than assuming a default is close enough. The other twenty thousand files in a typical family archive were never going to get a maintainer's careful attention. But twelve years of one usually can.

← Back to all posts