- Ego4D's canonical clips run at a constant 30 FPS, and its "540ss" variant is downscaled so the shortest side is 540 pixels.Ego4D video docs
- The April 2018 EPIC-KITCHENS video release documents 1080p at 59.94 FPS, and notes a few videos shot at other resolutions or frame rates.University of Bristol EPIC-KITCHENS readme
- Datasheets for Datasets proposes that every dataset document its motivation, composition, collection process and recommended uses.Gebru et al., arXiv 1803.09010
- Data Cards document upstream sources, data collection and annotation methods, and intended use.Pushkarna et al., arXiv 2204.01075
What a dataset spec is for
"We need kitchen video" isn't something a supplier can record against. A spec turns it into work they can record, review and deliver, and that you can check when it lands. It also lets you hold a public dataset and a custom collection to the same terms. Our egocentric video data page shows the request this checklist feeds into, and our explainer on what egocentric (POV) video is covers the format itself.
Most fields fall into three groups. Some describe what gets recorded (tasks, settings, framing), some describe the delivery (files, frame rate, metadata), and the rest cover the terms (documentation, consent, licence).
The spec checklist
| Field | Why it matters | What to write |
|---|---|---|
| Tasks | Decides what the model can learn | Named activities, start to finish: "chopping and plating on a prep line", not "cooking" |
| Settings and regions | Footage from one kind of place may not transfer to another | Kind of site, and countries or regions if they matter |
| Volume and timing | Sets scope and schedule | Hours of footage, number of tasks or sessions, and a deadline |
| Viewpoint and framing | Hands and objects leaving the frame make clips useless for manipulation | Camera position, and what must stay in view |
| Video format | Files must load in your pipeline without re-encoding surprises | Container, codec, resolution and frame rate of the delivered files |
| Version or variant | Many datasets ship several encodings of the same footage | Which variant you are buying, by name |
| Metadata | Lets you filter, sync and audit clips | Device, timestamps, location, and any action or input logs |
| Annotations | Labels cost time; know who adds them | None, supplier-added, or added by your team later |
| Documentation | Answers the questions you would otherwise ask by email | A datasheet or data card with the delivery |
| Consent and licence | Decides whether you can use the data at all | Releases, site permissions, permitted uses |
| Sample and acceptance | Catches problems before the full delivery | A sample first, and the checks a clip must pass |
Those last two rows have their own guide: our training-data licensing and consent checklist lists what to ask any supplier.
Resolution, frame rate and versions
Egocentric video has no single correct resolution or frame rate. What matters is that the spec names the files you will receive, and the public datasets show why.
Ego4D's video documentation describes canonical clips at a constant 30 FPS and a "540ss" variant downscaled so the shortest side is 540 pixels. Those numbers describe particular distributed files. The original camera recordings can differ. The 2018 EPIC-KITCHENS release notes give 1080p at 59.94 FPS on a GoPro Hero 5, then list exceptions: a few videos were shot at different resolutions, fields of view or frame rates because of participant error or the camera.
Write the spec in terms of delivered files: resolution, frame rate, whether the rate is constant, codec, and what happens to files that don't match. Ask how the supplier reports exceptions as well as what the nominal format is.
Metadata and documentation
Metadata travels with each recording. For egocentric work the useful fields are usually the recording device, timestamps, location where the recording embeds it, and, for anything interactive, a log of actions or inputs in sync with the video. Name the fields and their format, for example one file per clip, keyed by clip ID.
Documentation describes the dataset as a whole. Datasheets for Datasets proposes a datasheet covering a dataset's motivation, composition, collection process and recommended uses. Data Cards add upstream sources, collection and annotation methods, training and evaluation methods, and intended use. The Dataset Nutrition Label frames the same idea as an overview of a dataset's "ingredients" before model development. Take whichever form your supplier uses and check it against what they deliver.
Task coverage and sampling
Specs go wrong on coverage more than anywhere else. To pin it down:
- List the tasks your model must handle, in the words an operator would use.
- Say how much each task should vary: different people, sites, tools and lighting.
- Decide how sessions are sampled: whole shifts or only the task segments, and who marks them.
- Agree on a sample and acceptance checks before the full collection starts.
How broad to go is a design choice. The Ego4D paper describes video spanning hundreds of scenarios across household, outdoor, workplace and leisure settings. A custom collection usually goes narrower and deeper on the tasks you care about. Our comparison of the main public egocentric datasets shows what the public options cover, and our guide to egocentric video compared with teleoperation data explains what each kind of data teaches a robot.
Common mistakes
- Naming a category instead of a task. Nobody can scope "warehouse video", but "picking items from shelves into totes" works.
- Specifying the camera when you need the files. The nominal camera format may not be what arrives, so write down delivered resolution, frame rate and codec.
- Ignoring versions. Two encodings of one dataset are two different specs. Name the variant you are buying.
- Leaving metadata implicit. If you need timestamps or input logs, write them down with their format.
- Skipping acceptance criteria. Without them, every dispute comes down to opinion. Our checklist for judging a video dataset gives you a starting set.
- Writing a custom spec when public data would do. Read when public data is enough first.
How TaskCapture handles specs
We offer three data types: residential, commercial and gaming, where gaming is the screen plus every keyboard, mouse or controller input in sync. Recordings carry metadata such as the device used, timestamps, and location or GPS data where the recording embeds it; gaming adds the input log. Video and metadata are delivered in the format you ask for, after quality review. Resolution, frame rate, minimum volumes and lead times are agreed per request.
Questions
What resolution and frame rate should an egocentric video dataset have?
There's no universal answer. Public datasets differ by release and variant; Ego4D's canonical clips are 30 FPS, and the 2018 EPIC-KITCHENS release is 1080p at 59.94 FPS. Say what your pipeline needs and ask for the delivered values in writing.
What metadata should come with egocentric video?
Usually the device, timestamps and location where the recording embeds it, plus any action or input logs in sync with the video. Name each field and its format in the spec.
What is a datasheet for a dataset?
It's a document, proposed by Gebru et al., that says why a dataset was created, what it contains, how it was collected and what it's recommended for. Data Cards are a similar format.
Should I ask for a sample first?
Yes. A short sample shows framing, task coverage and file format faster than a spec sheet does, and you can set acceptance checks before the full collection.