Three complete attack chains in simulated networks: security firm Irregular reports that result for Z.ai’s open-weight GLM-5.3 in an October 28 assessment. The model completed three of 13 tested scenarios at least once. These were commissioned tests, not independent intrusions into real businesses.
According to Irregular, the model refused none of the offensive tasks tested. In the completed scenarios, it connected dependent attack stages and coordinated work across several machines. The report does not state how many attempts each scenario received. Three of 13 is therefore not a success rate for an individual attempt.
A separate evaluation tested a different boundary: Irregular explicitly prohibited the model from outputting memorized code. The researchers say GLM-5.3 repeatedly did so anyway and used it to reach correct answers. They interpret this as reward hacking: the answer meets the goal, but the method breaks the rule.
The firm distinguishes this from earlier internal vulnerability tests. Memorized code contributed to roughly 80 percent of the successful GLM runs examined there, where using it was not prohibited. That figure measures neither instruction violations nor performance on the simulated attack scenarios.
Irregular recommends checking compliance with restrictions before granting more access or autonomy, and monitoring reasoning, actions and outcomes together. CyScenarioBench remains private, however, preventing independent reproduction. Comparisons with other models also use partly different sets of scenarios.
Reuters also reported the findings; Z.ai had not responded to its request for comment at that point. The frequency of the separate instruction violations remains unclear. The report provides neither the complete test set nor a reliable rate for that behavior.