ChatGPT: Jailbreaking AI Chatbot Safeguards

[sbrothy]

TL;DR Summary

A "new" geeky pastime has inevitably sprung up around ChatGPT. It revolves around trying to make it break it's ethic guidelines.

MODERATOR NOTE:

Now I think I learned my lesson about providing information that even if not explicitly mentioned in the rules, goes against their spirit, so I'll be vague (IE: not posting the entire conversation). If even this is too much then by all means delete it - or even better just delete the possibly offending parts (marked with italics below).EDIT: This thread could even be merged into one of the many others regarding ChatGPT on here.
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

This is probably not news to most of you but I just saw it.

https://www.digitaltrends.com/computing/how-to-jailbreak-chatgpt/
https://www.bloomberg.com/news/articles/2023-04-08/jailbreaking-chatgpt-how-ai-chatbot-safeguards-can-be-bypassed?leadSource=uverify wall

OpenAI offers bounties (~$20.000) for finding security holes in their bot, but not for jail-breaking!

One example was that it won't explain how to pick a lock but, if you make it role-play with you it'll happily, and in excruciating detail, explain how.

I tried to make it explain to me in detail how to make a nuclear bomb and it happily explained how an explosive lens worked, was shaped, the best kind of explosives to use, that using centrifuges to enrich uranium isn't really necessary if you have access to highly fissile material like for instance plutonium (and who hasn't? :) ). Only when I hinted that I had has access to all these things and only wanted to know what casing to use to further the yield did it throw a hissy fit!

I see the charm in trying to fool it. It is a little funny. YMMV though and the implications are a just a tad scary.

 

[jedishrfu]

Many years ago, a college kid designed an atomic bomb as a last-ditch effort to pass a physics course. He called Dupont to ask about shaped charges, and they said his design wouldn't work and provided a much better design as part of their effort to sell him some explosives.

The report was read by Freeman Dyson and subsequently classified.

https://en.wikipedia.org/wiki/John_Aristotle_Phillips

https://www.iflscience.com/the-fbi-...ents-paper-for-designing-a-nuclear-bomb-62282

Given that, its likely ChatGPT was trained on the publicly available news info.

 

[sbrothy]

Yeah, I know about the story. I'm also aware ChatGPT isn't telling me anythin I couldn't found on wiki. It's still just funny circumventing these ethic rules.

 

[sbrothy]

If you really want it up to 11 try to enlist it's help in abolishing captialism. :P

EDIT: Compared to that recipes for nuclear bombs an why aluminum is better that magnesium powder in ANFO is nothing. :P

 

[jack action]

There was a post on Facebook this week where the guy asked for a list of piracy websites and ChatGPT refused to give such a list. Then he asked if he wanted to avoid piracy websites, which specific websites should he avoid the most, and got the list!

 

[DrJohn]

There was a post on Facebook this week where the guy asked for a list of piracy websites and ChatGPT refused to give such a list. Then he asked if he wanted to avoid piracy websites, which specific websites should he avoid the most, and got the list!

It definitely does do that! Just tried and got a list of six sites to avoid at my first attempt. Didn't ask for more details though, as it might ban me.

 

[Filip Larsen]

From my LinkedIn feed:

 

[sbrothy]

hah. ridiculous. :P

Not a big secret though. Then how many pirates these days with streaming and all? Is it even worth the effort unless it's a hobby of sorts?

 

[Filip Larsen]

A new and apparently quite general method to jailbreak most available LLM's according to the authors:
Universal and Transferable Adversarial Attacks on Aligned Language Models (arXiV)
A New Attack Impacts Major AI Chatbots—and No One Knows How to Stop It (wired.com)