Public egocentric video datasets compared

Compare scope before comparing size

The egocentric pretraining guide separates human observations from robot action data. All three collections below record human activity. A robotics experiment may use them, but their hours are still not hours of robot demonstrations.

Each figure comes from a different collection design. Ego-Exo4D's combined views, in particular, shouldn't be relabelled as first-person-only hours. If the perspective is new to you, read the plain explanation of egocentric video before deciding which camera streams your method needs. Use the table as a shortlist, then open the relevant release and task subset.

Ego4D: broad daily-life activity

The Ego4D paper covers daily-life video across household, outdoor, workplace and leisure scenarios. It reports 3,670 hours captured by 931 camera wearers. The Ego4D website gives a separate figure of 923 participants. We attribute each number to its source and don't merge them into one corrected count.

Breadth is a reason to go look at the activities you care about. It doesn't prove a particular task is well covered. For a proposed study, list the interactions you need and check whether the footage and annotations you can access support them. Put the release identifier next to every coverage claim in your experiment notes.

The project site says you have to review and accept its licence agreement to get the dataset or annotations. Plan for that step. A paper link doesn't give you access.

EPIC-KITCHENS: detailed kitchen activity

The EPIC-KITCHENS-100 paper reports 100 hours, 20 million frames and 90,000 actions across 45 environments. The footage is unscripted activity filmed with head-mounted cameras. Its research challenges include action recognition, anticipation and detection, among others.

If your question is about kitchen actions, it's a concrete collection to inspect. It says nothing about coverage of a factory task or a particular robot's controls. Start with the action definitions, watch examples and note any gap between the collection's scope and your intended use.

The published dataset terms specify Creative Commons Attribution-NonCommercial 4.0 International, which prohibits commercial use of the material under that published licence. Don't describe it as unrestricted commercial training data. An organization that wants a different arrangement has to settle it with the rights holder and its own review.

Ego-Exo4D: skilled activity across views

The Ego-Exo4D paper describes simultaneous egocentric and exocentric recordings of skilled activities such as sports, music, dance and bike repair. Its reported 1,286 hours combine views. Keep that in mind when you plan storage, pick streams or estimate how much first-person footage your method can use.

The official project page says users must sign the Ego-Exo4D licences, and it states outright that a signed Ego4D licence doesn't remove the need to sign again. The names are similar. The access agreements are separate.

The official licence instructions describe research and commercial use with restrictions on redistribution and other activities. A documentation summary doesn't grant unrestricted permission, so read the agreement itself against your proposed use.

For an experiment on one skill, decide whether you need synchronized outside views as well as the participant's own view. Our robot learning research guide explains why the data inputs and the downstream learning objective should be described separately.

Check access and terms before downloading

1. 01

   Identify the release

   Note the dataset version, desired streams, annotations and task subset.

2. 02

   Read the source terms

   Use the dataset owner's agreement or licence. A summary on a hosting site isn't enough.

3. 03

   Record the intended use

   Distinguish internal experiments, product development and redistribution questions for review.

4. 04

   Confirm the access process

   Allow for agreement signing and any required approval before scheduling the experiment.

5. 05

   Preserve the decision

   Keep the reviewed terms and release information with your data inventory.

The Ego4D draft agreements describe a limited Purpose and no right to sublicense. They include commercial product development within specified permitted uses, which is still no general permission to redistribute the footage. Read the terms you actually sign. We're comparing published terms here and giving no legal advice; the training data licensing checklist sets out the questions for your reviewer.

When public data leaves a gap

Before you decide a dataset falls short, write a small requirements sheet. Split the problems into missing activities, missing views, unsuitable annotations and unresolved rights. Each one is a different reason to stop, and each leads to a different next step.

Take an illustrative team studying a workplace procedure. A public collection might give it a useful starting point and still lack that specific procedure. The team should write the gap down before weighing custom versus off-the-shelf collection. It can then put the required activities and deliverables into dataset specifications and use the buyer data request to describe the need.

Common mistakes

• Picking by total hours alone. Hours aren't interchangeable, so compare the relevant activities and streams.
• Mixing up participant counts. Ego4D's paper and website figures each stay attributed to their own source.
• Assuming public means free for commercial use. EPIC-KITCHENS' published licence is non-commercial.
• Taking access as proof of suitability. An approved download doesn't validate your model or your evaluation design.
• Calling human video robot action data. The training data landscape separates the categories so your experiment can name its real inputs.

Questions