Generate Alt Text by AI Assist Model Card

Model Release Date: May 2024
Model Version: v1.0
Model Type: Large Language Model (GPT-4 Turbo via OpenAI API)
Applicable Platform: Sprout Social

What It Does & Why It Matters

The Generate Alt Text by AI Assist feature analyzes uploaded images and generates descriptive alternative text (alt text), which is text that conveys the content and purpose of an image for screen readers and improves accessibility, for images uploaded to select social media platforms (Facebook, X, LinkedIn) through Sprout Social. This feature addresses the challenge of low alt text usage due to the time-intensive and sometimes complex nature of manual creation of captions, as well as a user’s lack of familiarity with accessibility best practices. By providing AI-optimized alt text with a single click, the feature aims to enhance the accessibility of social media content, ensuring visually impaired users can effectively engage with images.

Example Input:

Example Output: Six colorful smoothies garnished with various toppings, such as berries, mint, and spices, presented in clear glasses on a dark surface.

Intended Use Cases

• Generating initial alt text suggestions for social media images.
• Assisting users who lack the time or expertise to write alt text manually.
• Providing a baseline for users to review and further refine for accuracy and context.
• Encouraging wider adoption of alt text to improve social media accessibility.

Out-of-scope uses: 

• Writing alt text without human oversight and validation
• Medical applications or other life-critical decisions
• Identity recognition of all individuals
• Any other uses not included in the intended use case

Factors & Limitations

• During development, this feature was evaluated on English outputs only. Performance may vary in other languages. 
• The feature is designed to avoid mentioning gender, skin tone, race, or emphasizing physical disabilities, although we cannot guarantee that generated text will comply with those constraints. 
• The feature aims to output verbatim image text, although the model may occasionally omit, rephrase, or reorder text that appears in the image. 
• The quality of the output is dependent on the underlying capabilities of the third-party model and the clarity/resolution of the input images. 
• There is a dependency on the availability and performance of the OpenAI API.

Users are advised to review and edit the generated text to ensure accuracy and appropriateness for their specific context. 

Evaluation Metrics

Performance is evaluated on quantitative metrics, including feature adoption rate by users and AI-generated alt text acceptance rate.

Training Data

This feature uses an out-of-the-box GPT model that was not trained nor fine-tuned. However, during evaluation, it did require a set of images to develop a robust prompt. More information about this dataset is available below under ‘Evaluation Data’. 

Evaluation Data

The model was evaluated on a manually curated set of images to ensure coverage of a wide set of test cases. Special emphasis was placed on images that have been cited by customers and the research community as situations where models frequently display bias, diverge from human values, or are incapable of accurately describing an image.

Risks & Ethical Considerations

The development process included consultation with accessibility experts from the blind and low vision community. The dataset used for evaluation was human-curated to represent a wide range of image content, including an intentionally diverse set of images of people. The evaluation data received multiple rounds of human annotation and evaluation from accessibility experts and a team of internal specialists.

Significant effort was dedicated to prompt engineering to mitigate potential biases related to descriptions of people. The model is intentionally constrained to avoid subjective interpretations and descriptions of perceived gender, skin tone, race, and age to minimize the risk of harmful, offensive, or misleading outputs. User feedback and ongoing monitoring are in place to identify and address any potential issues. We welcome user feedback to continue improving the experience, and users can share feedback by submitting a request via our Support Center.