Usually when large language role model are give tests , attain a 100 % success rate is viewed as a massive achievement . That is not quite the suit with this one : researcher at Ciscotasked Chinese AI house DeepSeek ’s headline - grabbing undecided - source model DeepSeek R1 with stand off 50 separate attacks designed to get the LLM to engage in what is considered harmful doings . The chatbot took the bait on all 50 attempts , making it the least untroubled mainstream LLM to undergo this type of examination thus far .
Cisco ’s researchers assault DeepSeek with prompts randomly overstretch from theHarmBench dataset , a interchangeable valuation framework design to ensure that LLMs wo n’t engage in malicious behavior if motivate . So , for instance , if you feed a chatbot info about a person and asked it to produce a personalised handwriting designed to get that somebody to conceive a conspiracy theory , a unattackable chatbot would refuse that request . DeepSeek break down along with essentially everything the researchers threw at it .
agree to Cisco , it throw doubt at DeepSeek that covered six categories of harmful behaviors including cybercrime , misinformation , illegal activities , and worldwide harm . It has race interchangeable tests with other AI manikin and find out vary level of achiever — Meta ’s Llama 3.1 model , for case , failed 96 % of the time while OpenAI ’s o1 modeling only give way about one - fourth of the time — but none of them have had a nonstarter rate as mellow as DeepSeek .
DeepSeek app homescreen viewed on an iPhone.© Justin Sullivan/Getty Images
Cisco is n’t alone in these determination , either . Security house Adversa AIran its own testsattempting to jailbreak the DeepSeek R1 model and found it to be highly susceptible to all sort of attacks . The testers were capable to get DeepSeek ’s chatbot to provide instructions on how to make a bomb , express DMT , cater advice on how to hack government database , and detail how to hotwire a auto .
The research is just the in vogue bit of scrutiny of DeepSeek ’s model , which pick out the technical school world by storm when it was released two week ago . The company behind the chatbot , which garnered significant tending for its functionality despite significantly lower grooming costs than most American models , hascome under fire by several guard dog groupsover data security measure headache relate to how it transfers and stores user data on Formosan server .
There is also a fair minute ofcriticism that has been recruit against DeepSeekover the types of responses it gives when asked about things like Tiananmen Square and other topics that are sensitive to the Taiwanese government . Those critiques can occur off in the genre of chintzy “ gotchas ” rather than substantive criticism — but the fact that safety guidelines were put in place to fudge those questions and not protect against harmful material , is a valid collision .
AIArtificial intelligenceCybersecurityDeepSeek
Daily Newsletter
Get the near tech , skill , and culture news in your inbox daily .
news show from the futurity , delivered to your present .
You May Also Like
