Google Forms Dedicated AI Red Team to Secure AI

Aug 02, 2023

Google has created a specialized AI red team to address the unique security challenges posed by machine learning (ML) systems. The AI red team focuses on probing and exposing vulnerabilities within AI-driven technologies. Unlike traditional red teams, the AI red team requires a deep understanding of machine learning technology. It combines the skills of individuals who can hack into systems and poison models. By pairing these skills together, the team is able to effectively secure AI deployments.

The AI red team closely follows new adversarial research in the field and monitors where Google integrates AI into its products. They prioritize exercises that simulate adversary attacks with specific goals in mind. The results of these exercises are documented and shared with relevant stakeholders, enabling them to address and resolve any issues that are uncovered. The team takes a broad approach to apply the lessons learned from these exercises to improve the security of all products.

To target AI deployments, the AI red team sets up scenarios that describe the simulated attacker, their capabilities, and their goals. They rely on threat intelligence and their understanding of the space to create realistic attack scenarios. Similar to security red teams, achieving the goals often requires multiple steps.

Collaboration between the AI red team and AI experts is crucial for creating realistic adversarial simulations. The AI red team may work with the security red team to gain specific internal access needed for certain exercises. For example, the security red team may compromise an employee to gain access to internal systems, and then the AI red team would target an ML model to insert a backdoor.

Addressing red team findings can be challenging, as some attacks may not have simple fixes. Google prioritizes the security and privacy of their users and will not launch a new feature if it cannot be done safely. The AI red team collaborates closely with internal research teams to develop new approaches to address identified issues. Classic security mitigations, such as access restrictions and input/output validation, are also effective in reducing the risk. The AI red team’s efforts help drive visibility and investment in ML safety.