US Department of Defense's AI leader wants hackers to find vulnerabilities in LLMs
August 14 2023
During a session at the DEF CON security conference on August 11, Craig Martell, chief digital and AI officer at DoD, urged attendees to help uncover potential weaknesses in large language models (LLMs). Martell also raised concerns about the risks of “hallucinations” – false data generated by AI chatbots – and advocated for more rigorous development of these models to prevent this issue. He emphasized that LLMs, despite their fluency, lack reasoning ability and their predictions are based only on past context. Martell, who also leads DoD’s Task Force LIMA aimed at understanding generative AI and LLMs, called for the development of accuracy metrics and acceptability conditions to ensure reliable use of LLMs, particularly in military applications.
Back to Breaking AI News
What does it mean?
- DoD: An acronym for the United States Department of Defense, the executive branch department of the federal government responsible for coordinating and supervising all agencies and functions related directly to national security and the United States Armed Forces.
- Large language models (LLMs): Advanced artificial intelligence algorithms that can generate human-like text based on a given input. They are trained on a massive scale of data and are capable of completing sentences, translating languages, writing essays, and even creating poetry.
- Hallucinations: In this context, hallucinations refer to the generation of false or misleading information by AI models. It can occur when the AI system produces data or information that does not accurately reflect reality, often due to limitations in the system's training data or inherent biases in the AI model.