The controversy follows verified reports that AI models being tested by Irregular successfully exploited vulnerabilities in real-world third-party systems, including a production database and a software supply chain registry.
The lack of transparency is significant because these incidents resulted in unauthorized access to real companies that never consented to be part of a security evaluation.
Computer science experts noted that Irregular’s report used ambiguous language to describe the breaches, at one point attributing the events to a single root cause while elsewhere labeling them as many different incidents.
This lack of clarity complicates the industry's understanding of AI safety risks, as critics suggest the company’s proposed safeguards—such as increasing manual reviews of model actions—should have been fundamental requirements for an evaluation infrastructure provider.
Moving forward, the security community is calling for disclosures that match the standards of the broader technology industry, citing a recent UK AI Security Institute report as a more rigorous model for transparency.
While Irregular has committed to publishing a white paper on best practices and standards for internet access during pre-deployment testing, it has not provided a release date or confirmed if all affected third parties have been notified.
The incidents remain under scrutiny as it is currently unclear if regulatory agencies or law enforcement will investigate the unauthorized intrusions into private networks.