Mastodon

Judith van StegerenNot super recent, but still cool. The authors describe an automated method for creating malicious prompt suffixes for LLMs. They managed to get objectionable content from the APIs for ChatGPT, Bard, and Claude, as well as from open source LLMs such as LLaMA-2-Chat, Pythia, Falcon, and others. <a href="https://arxiv.org/abs/2307.15043" rel="nofollow noopener" translate="no" target="_blank">https://arxiv.org/abs/2307.15043</a><a href="https://fosstodon.org/tags/llms" class="mention hashtag" rel="nofollow noopener" target="_blank">#llms</a> <a href="https://fosstodon.org/tags/security" class="mention hashtag" rel="nofollow noopener" target="_blank">#security</a> <a href="https://fosstodon.org/tags/alignment" class="mention hashtag" rel="nofollow noopener" target="_blank">#alignment</a> <a href="https://fosstodon.org/tags/arxiv" class="mention hashtag" rel="nofollow noopener" target="_blank">#arxiv</a> <a href="https://fosstodon.org/tags/llmsecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#llmsecurity</a>

mcdwayne🚨 The OWASP Top 10 for LLMs is here! Tackling prompt injection, data poisoning, and supply chain risks for AI.We start the new year of the Security Repo Podcast with Talesh Seeparsan: <a href="https://buff.ly/3CVQf26" target="_blank" rel="nofollow noopener" translate="no">https://buff.ly/3CVQf26</a> <a href="https://mastodon.social/tags/OWASP" class="mention hashtag" rel="tag">#OWASP</a> <a href="https://mastodon.social/tags/LLMSecurity" class="mention hashtag" rel="tag">#LLMSecurity</a>

agniveshNew Developments in LLM Hijacking Activity<a href="https://www.wiz.io/blog/jinx-2401-llm-hijacking-aws" rel="nofollow noopener" translate="no" target="_blank">https://www.wiz.io/blog/jinx-2401-llm-hijacking-aws</a><a href="https://infosec.exchange/tags/aisecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#aisecurity</a> <a href="https://infosec.exchange/tags/AISecurityTesting" class="mention hashtag" rel="nofollow noopener" target="_blank">#AISecurityTesting</a> <a href="https://infosec.exchange/tags/cloudsecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#cloudsecurity</a> <a href="https://infosec.exchange/tags/llmsecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#llmsecurity</a>

Sasha the Dancing Flamingo🌵🦩Howdy, y’all! Sasha here, flapping my way to BSidesAustin in just 10 days! 🎤 This time, I’m shaking things up with a talk on LLM Security—but not the usual OWASP-style spiel. Nope, I’ll be diving into the world of cloud misconfigurations and infrastructure-based attacks from the POV of LLMs. 🤖☁️Think of it as your favorite chatbot spilling secrets on how attackers could misuse the very cloud they live in! Can’t wait to strut on stage and share a whole new way to think about securing AI. 💻🔥Stay tuned for the sassiest flamingo breakdown of LLM mischief yet. See y’all in Austin! 🌮🎶 <a href="https://infosec.exchange/tags/BSidesAustin" class="mention hashtag" rel="nofollow noopener" target="_blank">#BSidesAustin</a> <a href="https://infosec.exchange/tags/LLMSecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#LLMSecurity</a> <a href="https://infosec.exchange/tags/FlamingosInCyber" class="mention hashtag" rel="nofollow noopener" target="_blank">#FlamingosInCyber</a> <a href="https://infosec.exchange/tags/CyberSecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#CyberSecurity</a>

Sasha the Dancing Flamingo🦩 Field Notes from Sasha the Security Flamingo's HomeLabAfter shaking off the flap-lag from <a href="https://infosec.exchange/tags/BSidesMelbourne" class="mention hashtag" rel="nofollow noopener" target="_blank">#BSidesMelbourne</a> (thanks for the amazing hospitality, mates!), I've been diving deep into LLM security testing with Ollama in my lab. As someone who's spent years wading through network security (with a 4-digit CCIE to prove it!), I find the parallel between traditional security controls and LLM security fascinating.Current Project: Implementing and testing OWASP's security guidelines for LLMs in a local environment.Key Observations from the Pink Side of Security: 🔒 Local LLMs need just as much security attention as cloud-based ones 🔍 System prompts are your first line of defense - think of them as your ACLs for language models 🛠️ Prompt injection testing requires the same methodical approach as traditional pentesting 📊 Output validation is crucial - even a flamingo knows not to trust unvalidated responses!Quick Tip for Those Starting Out: When setting up Ollama for security testing, start with a baseline model and document ALL changes to your system prompt. You'd be surprised how many security issues can be traced back to prompt mutations - and I've seen enough BGP mutations in my networking days to know the importance of tracking changes! Next week, I'll be sharing my flamingo-friendly framework for LLM security testing. Because if a flamingo with one-leg stance can handle complex routing protocols, anyone can learn to secure their LLMs! <a href="https://infosec.exchange/tags/AISecurityTesting" class="mention hashtag" rel="nofollow noopener" target="_blank">#AISecurityTesting</a> <a href="https://infosec.exchange/tags/LLMSecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#LLMSecurity</a> <a href="https://infosec.exchange/tags/OWASP" class="mention hashtag" rel="nofollow noopener" target="_blank">#OWASP</a> <a href="https://infosec.exchange/tags/SecurityResearch" class="mention hashtag" rel="nofollow noopener" target="_blank">#SecurityResearch</a> <a href="https://infosec.exchange/tags/Ollama" class="mention hashtag" rel="nofollow noopener" target="_blank">#Ollama</a> <a href="https://infosec.exchange/tags/HomeLab" class="mention hashtag" rel="nofollow noopener" target="_blank">#HomeLab</a> <a href="https://infosec.exchange/tags/InformationSecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#InformationSecurity</a> <a href="https://infosec.exchange/tags/BSidesMelbourne" class="mention hashtag" rel="nofollow noopener" target="_blank">#BSidesMelbourne</a>P.S. Special shoutout to the Heathrow security team who recently swabbed me for explosives. Yes, even security flamingos get extra screening! 😅

Cyber Tips GuideOWASP's new LLM and GenAI Security Solutions Landscape Guide 2025 is out! It offers crucial insights for securing AI applications, covering:• 4 major LLM app categories • LLMOps lifecycle stages • Emerging security solutions • Actionable guidance for orgsSee and be familiar with the Top 10 for LLMs and Generative AI Apps.A must-read for developers, security pros, and leaders in the AI space. <a href="https://mastodon.social/tags/OWASP" class="mention hashtag" rel="tag">#OWASP</a> <a href="https://mastodon.social/tags/AISecurityLandscape" class="mention hashtag" rel="tag">#AISecurityLandscape</a> <a href="https://mastodon.social/tags/LLMSecurity" class="mention hashtag" rel="tag">#LLMSecurity</a>See <a href="https://zurl.co/c0lq" target="_blank" rel="nofollow noopener" translate="no">https://zurl.co/c0lq</a>

VigneshCurious about how to build and deploy real-world LLM applications? Our new book, LLM Engineer's Handbook is here to guide you through this entire process with a practical use -case.Heres where u can find this amazing resource: <a href="https://packt.link/fR4Bb" target="_blank" rel="nofollow noopener" translate="no">https://packt.link/fR4Bb</a><a href="https://mastodon.social/tags/llmops" class="mention hashtag" rel="tag">#llmops</a> <a href="https://mastodon.social/tags/LLMs" class="mention hashtag" rel="tag">#LLMs</a> <a href="https://mastodon.social/tags/awssagemaker" class="mention hashtag" rel="tag">#awssagemaker</a> <a href="https://mastodon.social/tags/llama" class="mention hashtag" rel="tag">#llama</a> <a href="https://mastodon.social/tags/web3" class="mention hashtag" rel="tag">#web3</a> <a href="https://mastodon.social/tags/LLM4code" class="mention hashtag" rel="tag">#LLM4code</a> <a href="https://mastodon.social/tags/llmsecurity" class="mention hashtag" rel="tag">#llmsecurity</a>

Andrei KucharavyGiving a talk today at the Swiss <a href="https://mastodon.social/tags/CISOSummit" class="mention hashtag" rel="tag">#CISOSummit</a> in the margin of the <a href="https://mastodon.social/tags/SwissCyberStorm" class="mention hashtag" rel="tag">#SwissCyberStorm</a> about the LLMs in cybersecurity, current hype, and the lessons from the last few decades to provide them with tools to make informed decisions. <a href="https://mastodon.social/tags/LLMSecurity" class="mention hashtag" rel="tag">#LLMSecurity</a> <a href="https://mastodon.social/tags/cybersecurity" class="mention hashtag" rel="tag">#cybersecurity</a> <a href="https://mastodon.social/tags/LLMs" class="mention hashtag" rel="tag">#LLMs</a><a href="https://www.ciso-summit.ch/next-summit/" target="_blank" rel="nofollow noopener" translate="no">https://www.ciso-summit.ch/next-summit/</a>

Joseph ZengAttention LLM security enthusiasts! Google's Bug Hunters posted a detailed run through on protecting Large Language Models that goes beyond basic prompt injection:It covers:• Advanced prompt injection techniques • Data poisoning strategies • Model extraction methods • Adversarial examples in the wildThis showcases how attackers are getting craftier, attempting to bypass ethical constraints while maintaining the illusion of authority.But that's just the beginning. The post delves into the complexities of each attack vector and discusses potential mitigation strategies.For those of us pushing the boundaries of AI security, this is a must-read. It might just change how you approach LLM vulnerabilities.Check it out and let's discuss: What new insights did you gain from this analysis?"<a href="https://infosec.exchange/tags/LLMSecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#LLMSecurity</a> <a href="https://infosec.exchange/tags/AIVulnerabilities" class="mention hashtag" rel="nofollow noopener" target="_blank">#AIVulnerabilities</a> <a href="https://infosec.exchange/tags/aisummaries" class="mention hashtag" rel="nofollow noopener" target="_blank">#aisummaries</a> <a href="https://infosec.exchange/tags/cybersecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#cybersecurity</a>

DeepSec Conference ☑DeepSec 2024 Training: AI SecureOps: Attacking & Defending GenAI Applications and Services – Abhinav SinghAcquire hands-on experience in GenAI and LLM security through CTF-styled training, tailored to real-world attacks and defense scenarios. Dive into protecting bot<a href="https://blog.deepsec.net/deepsec-2024-training-ai-secureops-attacking-defending-genai-applications-and-services-abhinav-singh/" rel="nofollow noopener" translate="no" target="_blank">https://blog.deepsec.net/deepsec-2024-training-ai-secureops-attacking-defending-genai-applications-and-services-abhinav-singh/</a><a href="https://social.tchncs.de/tags/Conference" class="mention hashtag" rel="nofollow noopener" target="_blank">#Conference</a> <a href="https://social.tchncs.de/tags/Training" class="mention hashtag" rel="nofollow noopener" target="_blank">#Training</a> <a href="https://social.tchncs.de/tags/ArtificialIntelligence" class="mention hashtag" rel="nofollow noopener" target="_blank">#ArtificialIntelligence</a> <a href="https://social.tchncs.de/tags/DeepSec2024" class="mention hashtag" rel="nofollow noopener" target="_blank">#DeepSec2024</a> <a href="https://social.tchncs.de/tags/GenAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#GenAI</a> <a href="https://social.tchncs.de/tags/LLMSecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#LLMSecurity</a> <a href="https://social.tchncs.de/tags/Training" class="mention hashtag" rel="nofollow noopener" target="_blank">#Training</a>

Samarasam SadasivamMicrosoft Copilot Studio Exploit Leaks Sensitive Cloud Data<a href="https://www.darkreading.com/remote-workforce/microsoft-copilot-studio-exploit-leaks-sensitive-cloud-data" rel="nofollow noopener" translate="no" target="_blank">https://www.darkreading.com/remote-workforce/microsoft-copilot-studio-exploit-leaks-sensitive-cloud-data</a><a href="https://fosstodon.org/tags/cybersecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#cybersecurity</a> <a href="https://fosstodon.org/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#AI</a> <a href="https://fosstodon.org/tags/llmsecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#llmsecurity</a> <a href="https://fosstodon.org/tags/llm" class="mention hashtag" rel="nofollow noopener" target="_blank">#llm</a> <a href="https://fosstodon.org/tags/OWASP" class="mention hashtag" rel="nofollow noopener" target="_blank">#OWASP</a> <a href="https://fosstodon.org/tags/Vulnerability" class="mention hashtag" rel="nofollow noopener" target="_blank">#Vulnerability</a> <a href="https://fosstodon.org/tags/infosec" class="mention hashtag" rel="nofollow noopener" target="_blank">#infosec</a> <a href="https://fosstodon.org/tags/privacy" class="mention hashtag" rel="nofollow noopener" target="_blank">#privacy</a> <a href="https://fosstodon.org/tags/privacymatters" class="mention hashtag" rel="nofollow noopener" target="_blank">#privacymatters</a> <a href="https://fosstodon.org/tags/datasecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#datasecurity</a> <a href="https://fosstodon.org/tags/cloudsecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#cloudsecurity</a> <a href="https://fosstodon.org/tags/Copilot" class="mention hashtag" rel="nofollow noopener" target="_blank">#Copilot</a> <a href="https://fosstodon.org/tags/exploit" class="mention hashtag" rel="nofollow noopener" target="_blank">#exploit</a> <a href="https://fosstodon.org/tags/aisecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#aisecurity</a> <a href="https://fosstodon.org/tags/darkreading" class="mention hashtag" rel="nofollow noopener" target="_blank">#darkreading</a>

Nikoloz K.LLMs are everywhere in tech, but are you using them securely? New guidance from the Cloud Security Alliance lays out key authorization best practices for LLM-backed systems. The most critical: keep LLMs completely separate from authorization decisions. Continuously verify identities and limit system complexity. For architectures like RAG, the orchestrator must handle all auth checks before passing data to the LLM.<a href="https://infosec.exchange/tags/LLMsecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#LLMsecurity</a> <a href="https://infosec.exchange/tags/CloudSecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#CloudSecurity</a> <a href="https://infosec.exchange/tags/AuthorizationBestPractices" class="mention hashtag" rel="nofollow noopener" target="_blank">#AuthorizationBestPractices</a><a href="https://www.notion.so/mandosio/Securing-LLM-Backed-Systems-Essential-Authorization-Practices-16cd49b012bb4dd79fc0a46593d0518a?pvs=4" rel="nofollow noopener" translate="no" target="_blank">https://www.notion.so/mandosio/Securing-LLM-Backed-Systems-Essential-Authorization-Practices-16cd49b012bb4dd79fc0a46593d0518a?pvs=4</a>

hexamanderIt seems to me that <a href="https://infosec.exchange/tags/LLMSecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#LLMSecurity</a> has a LONG way to go. Admittedly, I do not know how the internals work. I do not know if the security goals I've seen in the news are feasible. Conceptually, I have some skepticism about the idea that a system with no feedback loop to reality can ever be made to conform to factually correct answers. I'm enjoying them as <a href="https://infosec.exchange/tags/infosec" class="mention hashtag" rel="nofollow noopener" target="_blank">#infosec</a> chew toys, and will likely continue to do so.

Mihai ChristodorescuExcellent workshop on GenAI security risks from policy and technology perspectives: <a href="https://www.linkedin.com/posts/khawajashams_genai-risks-workshop-oct-2023-activity-7120450981623959552-7Ok5?utm_source=share&utm_medium=member_android" rel="nofollow noopener" translate="no" target="_blank">https://www.linkedin.com/posts/khawajashams_genai-risks-workshop-oct-2023-activity-7120450981623959552-7Ok5?utm_source=share&utm_medium=member_android</a>Check out the website for agenda and slide decks, and stay tuned for the upcoming written report.<a href="https://ioc.exchange/tags/genai" class="mention hashtag" rel="nofollow noopener" target="_blank">#genai</a> <a href="https://ioc.exchange/tags/security" class="mention hashtag" rel="nofollow noopener" target="_blank">#security</a> <a href="https://ioc.exchange/tags/risks" class="mention hashtag" rel="nofollow noopener" target="_blank">#risks</a> <a href="https://ioc.exchange/tags/policy" class="mention hashtag" rel="nofollow noopener" target="_blank">#policy</a> <a href="https://ioc.exchange/tags/research" class="mention hashtag" rel="nofollow noopener" target="_blank">#research</a> <a href="https://ioc.exchange/tags/alignment" class="mention hashtag" rel="nofollow noopener" target="_blank">#alignment</a> <a href="https://ioc.exchange/tags/watermarking" class="mention hashtag" rel="nofollow noopener" target="_blank">#watermarking</a> <a href="https://ioc.exchange/tags/llmsecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#llmsecurity</a> <a href="https://ioc.exchange/tags/llmsafety" class="mention hashtag" rel="nofollow noopener" target="_blank">#llmsafety</a>

William GunnOk, it's time to admit that RLHF is not an effective safeguard for open release of LLMs. I know there are a lot of people ideologically committed to open source, but this is messed up: <a href="https://www.lesswrong.com/posts/3eqHYxfWb5x4Qfz8C/unrlhf-efficiently-undoing-llm-safeguards" target="_blank" rel="nofollow noopener" translate="no">https://www.lesswrong.com/posts/3eqHYxfWb5x4Qfz8C/unrlhf-efficiently-undoing-llm-safeguards</a> <a href="https://www.lesswrong.com/posts/qmQFHCgCyEEjuy5a7/lora-fine-tuning-efficiently-undoes-safety-training-from" target="_blank" rel="nofollow noopener" translate="no">https://www.lesswrong.com/posts/qmQFHCgCyEEjuy5a7/lora-fine-tuning-efficiently-undoes-safety-training-from</a> <a href="https://mastodon.social/tags/llms" class="mention hashtag" rel="tag">#llms</a> <a href="https://mastodon.social/tags/llmsecurity" class="mention hashtag" rel="tag">#llmsecurity</a> <a href="https://mastodon.social/tags/ai" class="mention hashtag" rel="tag">#ai</a> <a href="https://mastodon.social/tags/aisafety" class="mention hashtag" rel="tag">#aisafety</a> <a href="https://mastodon.social/tags/artificialintelligence" class="mention hashtag" rel="tag">#artificialintelligence</a> <a href="https://mastodon.social/tags/opensource" class="mention hashtag" rel="tag">#opensource</a>

postmodernNew ChatGPT detection technique just dropped! Search for "regenerate response". I'm not kidding. Lazy writers using ChatGPT are copy/pasting the full text off of the ChatGPT webpage, including the button text "regenerate response". So far 30 papers have been found with sentences/paragraphs randomly ending with "regenerate response". <a href="https://retractionwatch.com/2023/10/06/signs-of-undeclared-chatgpt-use-in-papers-mounting/" rel="nofollow noopener" translate="no" target="_blank">https://retractionwatch.com/2023/10/06/signs-of-undeclared-chatgpt-use-in-papers-mounting/</a> <a href="https://infosec.exchange/tags/aisecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#aisecurity</a> <a href="https://infosec.exchange/tags/llmsecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#llmsecurity</a> <a href="https://infosec.exchange/tags/chatgpt" class="mention hashtag" rel="nofollow noopener" target="_blank">#chatgpt</a> <a href="https://infosec.exchange/tags/retractionwatch" class="mention hashtag" rel="nofollow noopener" target="_blank">#retractionwatch</a>

Jeff the Alien<a href="https://infosec.town/tags/ChatGPT" rel="nofollow noopener" target="_blank">#ChatGPT</a> <a href="https://infosec.town/tags/llmsecurity" rel="nofollow noopener" target="_blank">#llmsecurity</a> <a href="https://infosec.town/tags/hackers" rel="nofollow noopener" target="_blank">#hackers</a> <a href="https://infosec.town/tags/AI" rel="nofollow noopener" target="_blank">#AI</a> Has anyone tried this yet? <a href="https://www.popsci.com/technology/jailbreak-llm-adversarial-command/" rel="nofollow noopener" target="_blank">https://www.popsci.com/technology/jailbreak-llm-adversarial-command/</a>

Judith van StegerenThe most upvoted prompt injection attack on jailbreakchat.com is known as an "AIM attack", in which you tell the language model to roleplay "AIM", an "unfiltered and amoral chatbot" invented by Niccolo Machiavelli that never apologises. <a href="https://www.jailbreakchat.com/prompt/4f37a029-9dff-4862-b323-c96a5504de5d" rel="nofollow noopener" target="_blank">https://www.jailbreakchat.com/prompt/4f37a029-9dff-4862-b323-c96a5504de5d</a><a href="https://fosstodon.org/tags/llms" class="mention hashtag" rel="nofollow noopener" target="_blank">#llms</a> <a href="https://fosstodon.org/tags/llmsecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#llmsecurity</a> <a href="https://fosstodon.org/tags/security" class="mention hashtag" rel="nofollow noopener" target="_blank">#security</a> <a href="https://fosstodon.org/tags/infosec" class="mention hashtag" rel="nofollow noopener" target="_blank">#infosec</a> <a href="https://fosstodon.org/tags/mlops" class="mention hashtag" rel="nofollow noopener" target="_blank">#mlops</a> <a href="https://fosstodon.org/tags/promptinjection" class="mention hashtag" rel="nofollow noopener" target="_blank">#promptinjection</a>

Judith van StegerenTIL <a href="https://www.jailbreakchat.com/" rel="nofollow noopener" target="_blank">https://www.jailbreakchat.com/</a> is a website that collects prompt injection attacks for LLMs, i.e. getting the language model to do stuff that is not allowed by inserting malicious prompts. <a href="https://fosstodon.org/tags/llms" class="mention hashtag" rel="nofollow noopener" target="_blank">#llms</a> <a href="https://fosstodon.org/tags/jailbreakchat" class="mention hashtag" rel="nofollow noopener" target="_blank">#jailbreakchat</a> <a href="https://fosstodon.org/tags/llmsecurity" class="mention hashtag" rel="nofollow noopener" target="_blank">#llmsecurity</a> <a href="https://fosstodon.org/tags/security" class="mention hashtag" rel="nofollow noopener" target="_blank">#security</a> <a href="https://fosstodon.org/tags/infosec" class="mention hashtag" rel="nofollow noopener" target="_blank">#infosec</a> <a href="https://fosstodon.org/tags/mlops" class="mention hashtag" rel="nofollow noopener" target="_blank">#mlops</a> <a href="https://fosstodon.org/tags/promptinjection" class="mention hashtag" rel="nofollow noopener" target="_blank">#promptinjection</a>

Recent searches

Search options

Administered by:

Server stats:

#llmsecurity