Calf identity

Paper in preparation

Which calf is which, across a week

Records for each animal need every animal to keep its name from one day to the next. We track four dairy calves with one overhead camera for seven days, and give each calf its identity back every morning after the camera has been off all night.

Overhead view of a straw pen with four calves, each covered by a coloured mask with a number.
The pen on 2024-04-22 at 17:18. Each colour is one calf.
7 daysfour calves, one identity each
6 of 6nights with every calf recovered
99.27 %of labelled time correct on the held-out day
22 Mparameters in the embedding model, as good as 6.7 B

The week

A tracker holds an identity only while it can follow an animal from frame to frame. On this farm that chain breaks every night.

So identity has to be rebuilt from appearance six times, without one wrong morning spoiling the rest of the week.

Timeline of nine recording days by hour. Each day has a recording bar from about 05:45 to 23:50, with a gap each night. 04-20 and 04-21 have no recording. Labels give each day's role.
What was recorded and what each day is used for.

How it works

Pipeline from overhead video through the tracker to anonymous segments, then crops and embeddings, then the offline identity layer with an anchor pool, a colour filter and mask refinement.

Track first, name later

SAM 3 with a distilled backbone from FarmSAM follows the calves and writes anonymous segments, one mask per calf per frame. An RF-DETR detector re-seeds it when a calf is lost. The tracker never decides who a calf is.

Identity in short windows

Masked crops are embedded with DINOv3 ViT-S/16. Every 3 minutes the identity layer compares them with verified crops from the training days and solves one joint assignment for the four calves. When the evidence is thin it writes "?" rather than guess.

Clean masks

Where a mask breaks into fragments, the detector repairs it. A colour check sets aside anything that is clearly not a black and white calf. Neither step changes identity.

Results

Identity

Times are calf-minutes: one calf for one minute. 04-22 and 04-23 were used for validation; 04-24 to 04-28 were held out.

DayLabelled time correctTrusted calf-minutes correctLeft as "?"
04-24, held-out test99.27 %93.0 % of 3,825.55.2 %
04-22, validation99.97 %98.45 % of 2,557.90.02 %
04-23, validation99.89 %98.1 % of 4,127.50.09 %

A veterinarian watched 04-25 to 04-28 at 50× speed and found no wrongly assigned identity, apart from one calf that took about 20 minutes after dawn on 04-28 to get its correct label.

Every night

NightResultChecked against
04-22 to 04-234 of 4 calvesVerified identity chain
04-23 to 04-244 of 4 calvesExpert notes
04-24 to 04-25CorrectExpert review
04-25 to 04-26CorrectExpert review
04-26 to 04-27CorrectExpert review
04-27 to 04-28Correct from 06:00Expert review

Mask refinement

DaysFragmented masks, slot-hours beforeAfter
04-222.810.37
04-233.871.52
04-24 to 04-2824.563.90

On 04-24 to 04-28 every fragment episode of 10 minutes or more was repaired (19 before, 0 after).

A small embedding model is enough

DINOv3 backboneParametersPer-crop rank-1Re-ID in 5-min windows
ViT-S/1622 M88.0 %98.8 %
ViT-B/1686 M86.2 %98.6 %
ViT-L/16303 M87.1 %98.9 %
ViT-7B/166,716 M85.5 %98.6 %

Scope. One pen, four black and white calves, one camera and one week. Identity is decided offline, and tracking runs at about 3.75 fps on an A10 GPU. Only 04-24 has numeric identity ground truth, and no frame has pixel-level mask ground truth. The tables are also in the repository as CSV.

Videos

Videos run at 50× speed, one frame every 2 s of real time, with the clock in each frame.

The week, day by day

One video per day, from the first light after the night. Each colour is one calf. While people are in the pen, the picture is mosaicked, and the feed alley beyond the rail is mosaicked throughout.

04-24, held-out test, from 05:48. Mosaicked for 91 minutes in all, around the times people were in the pen.

Mask refinement

04-22, 16:41 to 17:48: tracked masks before (left, or top on a phone) and after (right, or bottom) refinement. Each colour is one calf. At 17:18 the green mask covers part of its calf before and the whole calf after.

Teacher and student

04-23, 18:00 to midnight, from daylight into infrared night, frame for frame. Left (top on a phone): the tracker with SAM 3's own backbone. Right (bottom): the same tracker with the distilled student backbone, run with its mask repair on. Here the colours are tracker slots, not calves. From about 23:30 the teacher loses calves; the student keeps all four.

Masks

Per-frame masks are on Hugging Face, one gzipped JSON file per video clip, at 5 fps.

SetDaysWhat it is
Generated04-22 to 04-28Final refined masks, with rules that map tracker slots to calves over the day.
Verified04-18, 04-19, 04-22, 04-23, 04-24Masks of an earlier pass whose identity labels people checked or reviewed. The dataset card says how, day by day.

Reading a mask

import gzip, json
from pycocotools import mask as mask_utils

with gzip.open("masks_FarmA_R1_ch01-20240425-054428-061038.json.gz", "rt") as f:
    clip = json.load(f)

frame = clip["frames"][0]              # keys 0-3 are tracker slots, not calves
rle = frame["masks"]["0"]              # None if that slot has no mask
h, w = rle["size"]
binary = mask_utils.decode(mask_utils.frPyObjects(rle, h, w))   # h x w, uint8

Slots become calves through relabel_week_<day>.json. See the dataset card for the layout, and examples/decode_masks.py for a decoder that needs only NumPy.

The video is camera FarmA_R1 of the Calf Play Behavior Dataset (doi:10.57967/hf/10912), and each mask file carries the name of its clip there.

Licences

ItemTerms
CodeMIT
Masks and labelsSAM License, because they come from SAM 3 and a student distilled from it
SAM 3SAM License, gated
Calf student and calf detectorSAM License, at Calf-Identity-Tracking-Weights
DINOv3 ViT-S/16DINOv3 License
RF-DETR (library)Apache-2.0

Citation

The paper is in preparation. Until it is out, please cite this repository and the two works below.

@misc{calf_identity_week,
  title  = {Which calf is which, across a week},
  author = {Yang, Haiyu},
  year   = {2026},
  url    = {https://github.com/Sonam525/Re-id-for-tracking}
}