Data Card

Key application for use

Human-centric computer vision, machine learning fairness and robustness testing

Intended use cases (e..g, the CV tasks one can evaluate against)

FHIBE is an evaluation dataset collected with the following tasks in mind: human pose estimation, face and body detection, face and body alignment, face and body parsing, face verification and image editing. The dataset was created to evaluate fairness and robustness across dimensions, such as data subject attributes, instruments and environments. Except for the development of tools designed only to assess fairness and mitigate biases, FHIBE is strictly an evaluation dataset and may not be used as training data for an AI system.  

Primary Data Type

Images 

Nature of content

• Diverse, crowdsourced digital photographs of 1 or 2 people, conducting actions in real-world environments, under various environmental conditions and camera devices, across the world.
  
• In addition to the primary dataset, there are two derivative face datasets: an unaligned face-cropped dataset and an aligned face-cropped dataset.
  

Dataset Characteristics

• Total number of images: 10,318
  
• Total number of data subjects: 1,981
  
• Maximum number of images/subject: 10
  
• Total dataset size
  
  • Compressed: 182.5 GB
    
  • Full resolution: 532.2 GB
    

License

FHIBE is owned by Sony AI who distribute the dataset under applicable terms of use and a custom license agreement. The dataset use is restricted to evaluation, with the one exception that it can be used for training bias detection or mitigation methods, for both research and commercial purposes.

Data collection sources

Crowdsourced data collection via third party vendors

Data selection criteria

Data vendors were presented with a set of diversity specifications that the collected images (in the aggregate) had to meet. These diversity specifications included attributes such as age, ancestry, pronouns, head and body poses, subject-object interactions, as well as various environment and camera conditions. Data subjects were recruited and tasked to submit images to meet the specifications as closely as possible. All appropriate rights and consents to the data had to be provided in writing from eligible data subjects (e.g., minimum age of majority, English proficient).

Sampling Methods, data distribution

The dataset contains many demographic attributes for image subjects. FHIBE was specified to have a diverse distribution across a wide variety of attributes, including age, pronouns, and ancestry. FHIBE is approximately balanced on pronouns and apparent skin color, though lighter skin colors are under-represented. Similarly, African and Asian ancestries, and younger age groups are overrepresented, while older age groups are underrepresented.

Validation methods

Validated by automated and manual means. Manual checks were done for:

• Systematic errors on self-reported attributes
  
• Correctness of labels of observable attributes
  
• Removal of PII, IP and objectionable content
  
• Validation of consent forms