Zico Kolter has a knack for getting artificial intelligence to misbehave in interesting and important ways. His research group at Carnegie Mellon University has discovered numerous methods of tricking, goading, and confusing advanced AI models into being their worst selves. Kolter is a professor at CMU, a technical adviser to Gray Swan, a startup specializing in AI security, and, as of August 2024, a board member at the world's most prominent AI company, OpenAI. In addition to pioneering ways of jailbreaking commercial AI models, Kolter designs his own models that are more secure by nature. As AI becomes more autonomous, Kolter believes that AI agents may pose unique challenges'especially when they start talking to one another. Zico Kolter: One thing my group is working on is safely training models. We work a lot on understanding how to break models and circumvent protections, but this sort of raises the question of how we could build models that are inherently much more resistant...
learn more
