- Training is the stage in which a model learns settings that reduce its error, based on its training data.NIST CSRC glossary, "training stage"
- In supervised learning, a model learns to predict labels or answers that are often created by people.NIST CSRC glossary, "supervised learning"
- Not all training data is labeled by hand: self-supervised methods work out their labels from the data itself.NIST CSRC glossary, "self-supervised learning"
- Video of people doing everyday tasks is one kind of training data; the Ego4D research dataset holds 3,670 hours of it.Ego4D paper, arXiv 2110.07058
How does a model learn from examples?
Picture training data as a huge pile of worked examples. A model starts out guessing badly. During training it checks its guesses against the examples, measures how far off it was and adjusts its internal settings so it's a little less wrong next time. The US National Institute of Standards and Technology (NIST) describes the training stage as the stage "in which a model learns parameters that minimize its error against an objective function based on training data." Doing this over and over can make the model perform better, and how much better depends on the data and the training method. Our guide for collectors explains one way people get paid to supply examples like these.
NIST's definition of machine learning is careful about the outcome: systems "learn from data with the goal of improving accuracy." Better accuracy is the aim, and nobody promises it. Train a model on narrow or messy examples and it will come out narrow or messy too, so the quality of the data matters a great deal.
Chatbots are only one use. Translation, image recognition and robotics all learn from training data in the same way. When researchers introduced the Transformer, the design behind many language models, they tested it on English parsing "both with large and limited training data", as their original paper puts it.
Labeled and unlabeled data
There are two broad ways a model can learn from examples.
- With labeled data, each example comes with the answer: a photo is tagged "cat", or a chatbot reply is rated "helpful". NIST defines supervised learning as learning to predict "explicit (often human-generated) labels or output values for data." Those human-made labels are where a lot of data work comes from.
- With unlabeled data, the examples come without answers and the model learns patterns from the data itself. In self-supervised learning, NIST explains, the true labels are "automatically inferred from the training data", which makes very large datasets usable.
- Many projects use both: a large unlabeled collection and a smaller labeled one.
Some AI training data was labeled by a person, but much of it is raw material the model learns from directly.
Kinds of AI training data
| Kind of data | What it looks like | What a model can learn from it |
|---|---|---|
| Text | Articles, questions and answers, rated chatbot replies | Language, answering questions, writing |
| Images | Photos with or without tags | Recognizing objects, scenes and faces |
| Video | Recordings of people, places and tasks | How actions unfold over time |
| Actions | Recorded keyboard, mouse or controller inputs, or robot movements, kept in sync with video | What to do next in a game or a physical task |
Why first-person video matters
Video filmed from a person's own point of view, called egocentric or first-person video, shows what the hands are doing, which tools they use and in what order. That's close to what a robot or an assistant has to understand about a task. Our explainer on what egocentric video is covers the term in more depth.
Research teams have built large public collections of it. The Ego4D paper describes 3,670 hours of daily-life video "captured by 931 unique camera wearers from 74 worldwide locations and 9 different countries." Robotics researchers have used such video too: the R3M study pre-trained a visual representation "using the Ego4D human video dataset" for robot manipulation. Human video is one input among several, and on its own it doesn't give a robot a finished skill. For the details and the limits, read how robots learn from human video and our overview of robot learning research.
Where people fit in
Most training data has human work behind it, and that work comes in three broad kinds:
- Labelers tag images, mark objects in video or transcribe audio so the examples carry the right answers.
- Raters and writers compare or grade chatbot answers, or write example answers, usually at a computer.
- In recording work, you do a real task while it's recorded, such as cooking with a head camera on, or you play a game while the screen and inputs are captured.
Each kind needs different skills and happens in a different place. Our side-by-side guide to data collection compared with annotation jobs explains the differences. For what recording involves day to day, see the tips for recording first-person video.
Common mistakes and misconceptions
- Thinking all training data is labeled by people. Supervised learning often uses human labels, but self-supervised learning infers labels from the data itself.
- Expecting more data to always mean a better model. NIST frames improved accuracy as a goal, and data that is narrow, mislabeled or poorly documented can teach the wrong lesson.
- Believing only tech experts do data work. Recording work relies on knowing an everyday task well. You don't need to know how models are built.
- Trusting every AI training job offer. The name is popular with scammers, so learn how to spot AI training job scams before you apply anywhere.
How TaskCapture fits in
TaskCapture collects three types of first-person data: residential, commercial and gaming. Gaming means the screen plus every keyboard, mouse or controller input in sync, with no camera, voice, chat or gamer handles. We collect to order: a buyer makes a request, we match collectors who do those tasks, they record, and the footage goes to the buyer after review. Buyers may use it only to develop, train, evaluate and improve AI and machine-learning models.
There is no fee to apply. No coding or AI background is needed. Pay ranges from USD 5–15 per hour of accepted footage; it varies by project and is not guaranteed. Collectors work as independent contractors under the Data Collector Agreement, and the Privacy Policy sets out the consent rules. To see what you wear and record, read how recording work works.
Questions
What is AI training data, in one sentence?
It's the set of examples, such as text, images, video or recorded actions, that an AI model learns from by adjusting itself to make fewer mistakes on them.
Is all AI training data labeled by humans?
No. Supervised learning uses labels that are often created by people, but self-supervised learning infers its labels automatically from the data.
Why do AI teams want video of everyday tasks?
Video of people doing real tasks shows how actions unfold: the hands, the tools and the order of the steps. Research datasets such as Ego4D hold thousands of hours of it.
Do I need technical skills to help create training data?
It depends on the work. Some labeling and writing jobs ask for subject expertise; recording work asks you to do a task you already know while following the project's instructions.