Meta Launches Tool to Detect Bias in AI Computer Vision Systems
The tool evaluates the performance of computer vision models across various characteristics, notably perceived gender and skin tone.
The tool evaluates the performance of computer vision models across various characteristics, notably perceived gender and skin tone.
Meta has launched a novel AI tool called FACET (FAirness in Computer Vision EvaluaTion) aimed at identifying racial and gender biases within computer vision systems. This move comes in response to the growing concerns over the systematic biases that many computer vision models exhibit against women and people of color.
Dataset Behind FACET
Meta's tool, named FACET (FAirness in Computer Vision EvaluaTion), evaluates the performance of computer vision models across various characteristics, notably perceived gender and skin tone. The tool is grounded on a dataset comprising over 30,000 images that feature 50,000 individuals. These images have been meticulously tagged by experts across diverse categories.
The tool's design allows it to answer intricate questions, such as whether a model is more adept at identifying skateboarders when they are perceived as male or if it differentiates between individuals with light and dark skin tones. Furthermore, it can determine if these biases are amplified based on hair type, for instance, curly versus straight hair.
Open-Sourcing DINOv2
In addition to FACET, Meta has also made its DINOv2 computer vision model available under the Apache 2 open-source license, which permits commercial utilization. This move is expected to foster innovation and collaboration within the computer vision community.
In the official announcement from Meta, the company highlighted the challenges of benchmarking fairness in computer vision. They stated, “The risk of mislabeling is real, and the people who use these AI systems may have a better or worse experience based not on the complexity of the task itself, but rather on their demographics.” Chloe Bakalar, Meta's chief ethicist, emphasized the company's commitment to advancing AI systems responsibly. She mentioned, “We want to continue advancing AI systems while acknowledging and addressing potentially harmful impacts of that technological progress on historically marginalized and underrepresented communities.”
Editorial: Meta is Now an AI Company
Many tech companies have pivoted this year to integrate AI into their services. Microsoft has led the way with the broadest adoption of AI, bringing tools to all its major services. However, no company has drastically changed course as much as Meta. It is worth remembering Meta is, in a sense, a rebranded Facebook… it is a social media company.
Or at least it was until Meta formed as a separate parent entity of which Facebook and other social platforms such as WhatsApp and Instagram are part of. That move was done for two reasons: to put daylight between the company's overall plans and the decreasing reputation of Facebook, and to emphasize the Meta's move into the Metaverse.
It is right there in the name, Meta is a company focused on developing the metaverse. However, that was in 2021 when the metaverse seemed like the next big thing on the tech horizon. While AI has been at the forefront of peoples imagination of a technology-infused future, its rapid rise over the last year was not as easily predictable.
Meta's all-in approach to the metaverse was already on the rocks before AI became mainstream. The company spent billions of dollars pushing its metaverse goals, with little in return to show. So much so that CEO Mark Zuckerberg was on the ropes and the future was open to debate.
AI came at the right time for Meta and for the second time in less than three years, the company has gone through a massive pivot. Zuckerberg and other execs may argue that Meta remains oriented towards the metaverse, but it is clear the focus is now on AI. Meta is now an AI company.
All-in on AI and Competing with OpenAI, Microsoft, and Google
Meta is in an increasingly competitive market where big tech companies are jostling for dominance in AI. Microsoft has delivered a slew of products including Bing Chat, Microsoft 365 Copilot, GitHub Copilot, and more. Using a partnership with OpenAI, Microsoft leverages the GPT-4 large language model to help drive its AI products.
Speaking of OpenAI, the company's ChatGPT chatbot paved the way from the mainstreaming of generative AI products. Microsoft and OpenAI caught Google off guard with their swift action, prompting Mountain View to rush out the ever-improving Bard chatbot. Google is also developing Gemini as its new generative AI platform.
Then there is Meta, which has responded to Microsoft and OpenAI's charge with a range of its own AI products:
An Open Approach: Democratizing AI or Fooling the Crowd?
Meta envisions that generative AI can empower creators and businesses globally, but it also recognizes the necessity of transparency and user control as the technology advances.
One of the principles of Meta's AI strategy is openness, both for users and in terms of access. Providing users the option to opt out of sharing information is a crucial step towards democratizing AI. However, there may be skepticism that Meta's move is a token gesture that will not be enforced.
Meta has a notorious track record of violating user data and privacy when it was Facebook. A rebranding strategy is not sufficient, users demand concrete changes in how Meta safeguards their rights.
Meta also claims that its AI is open, giving developers access to create their own solutions. However, “open” is a vague term and it seems Meta's definition is not comprehensive. For instance, the company admits that some data is intentionally withheld. There was also a recent study that revealed that Meta and OpenAI ChatGPT are not as open as they profess.
The study, conducted by AI experts from Radboud University in Nijmegen, Netherlands, demonstrates that some of the most powerful AI LLMs are inaccessible to the public, because the code that trained them is not shared.
The study names OpenAI and Meta as the most closed LLM makers and argues that this harms the AI community. It calls for more honesty and openness from companies, so that others can benefit from their work and improve it.