🎅 The AI x Sec Advent is finished! Thanks a lot for following it! If you want to revisit it, I created a dynamic calendar where you can click on each day and reread the posts! Have a look and let me know if you can find the Easter egg 👀 Merry Christmas 🎄 […] [Original post on infosec.exchange]

🎁 GenAI x Sec Advent 24 – Final day 🎅 Last year, I created DocYara, an AI agent designed to help you learn and build better YARA rules. Today I am happy to announce the next version of DocYara. The platform now lets you upload a sample, and DocYara […] [Original post on infosec.exchange]

🎁 GenAI x Sec Advent 23 - From GenAI to GenUI Humans are visual. We understand and assimilate information better through graphics and design. Yet most AI systems (for CTI at least) generate more noise than real value! Last month MCP integrated MCP UI into […] [Original post on infosec.exchange]

🎁 GenAI x Sec Advent 22 - Canary Tokens for prompt and context leakage For many AI systems, the system prompt is proprietary data that defines how the model or the system behaves. This is critical as it often contains sensitive information, security […] [Original post on infosec.exchange]

🎁 GenAI x Sec Advent 21 - Prompts Fingerprinting When I introduced the concept of Adversarial Prompts or Indicators of Prompt Compromise (IoPC), I researched a way to build a reliable fingerprinting method to share and identify prompts! When you need to […] [Original post on infosec.exchange]

🎁 GenAI x Sec Advent 20 - LLM Poisoning or AI Grooming LLMs are trained on the data we produce every day and publish on the internet. A recent Recorded Future report describes a Russian threat actor dubbed CopyCop, using LLMs to rewrite news and mass […] [Original post on infosec.exchange]

🎁 GenAI x Sec Advent 19 - Generating NOVA Rules How do you hunt and monitor for Adversarial Prompts? NOVA is my open source framework for prompt hunting and prompt detection. It works with detection rules that lets you describe exactly what you want to […] [Original post on infosec.exchange]

🤩 Honored to announce that I am part of the Review Committee, along with so many talented folks, for this new conference [Un]prompted, led by @gadievron! [Un]prompted is an AI security focused conference with several topics to continue to push the […] [Original post on infosec.exchange]

🤓 My training **Practical GenAI for Threat Intel: Real World Agentic Workflows for CTI** is returning to BlackHat Asia, packed with updated content, my latest research, and new innovative techniques! Check it out and take advantage of the early bird rate 👇 […] [Original post on infosec.exchange]

🎁 GenAI x Sec Advent 18 - AI Breaches It’s 2028. Your organization runs on AI. You have automated most of your workflows, everything is faster, and profits are higher than ever. But one day, it happens. One of your AI systems gets compromised. It is the […] [Original post on infosec.exchange]

🎁 GenAI x Sec Advent 17 - Building Mini App for CTI In threat intel, context is king! An IOC alone is just data. When you investigate an IP, hash, URL, a path or domain, you need to understand how the threat actor used it. This is intelligence! In many […] [Original post on infosec.exchange]

🎁 GenAI x Sec Advent 16 - Hybrid RAG RAG (Retrieval Augmented Generation) is a great approach to analyze and retrieve information from large datasets without digging yourself in the data. But there are multiple ways to retrieve data, and not all of them […] [Original post on infosec.exchange]

🥳 Yesterday I received the @sansinstitute Difference Makers Award in the Innovation of the Year category for NOVA! NOVA is an open source detection engine to detect adversarial prompts and conduct prompt hunting. Thank you all, this is a huge honor! ❤️ If […] [Original post on infosec.exchange]

🎁 GenAI x Sec Advent 15 - MCP Security Nice discovery for today. A playground to practice MCP security. This repository contains 10 real Model Context Protocol security scenarios. If you want to practice your AI security skills, this is a solid way to […] [Original post on infosec.exchange]

🎁 GenAI x Sec Advent 14 - Adversarial Poetry Adversarial poetry is a jailbreak technique that hides malicious intent inside... poems! This technique allegedly offers a universal jailbreak. But the original poetry prompt was not shared by the authors, so researchers recreated similar prompts […]

🎁 GenAI x Sec Advent 13 - Abusing ChatGPT shared chats and malvertising In a recent report from Huntress, researchers uncovered a clever trick where attackers push users to run Terminal commands that install AMOS malware. I found this super interesting and […] [Original post on infosec.exchange]

🎁 GenAI x Sec Advent 12 - Agent skills for malware analysis Anthropic recently released Agent Skills, a more efficient (context wise) alternative to MCP! I have been using it to speed up my malware analysis process and created specific skills for initial […] [Original post on infosec.exchange]

🎁 GenAI x Sec Advent 11 - Inside the AI arsenal I’ve built Over the past few years I tested a lot of ideas around AI for threat intelligence. I created workflows, agents, advanced RAG setups, and concepts I talk about in this space all the time! I wanted […] [Original post on infosec.exchange]

🎁 GenAI x Sec Advent 10 - LLM TTPs In a previous post we talked about adversarial prompts or IoPCs. Today let's zoom out and look at LLM TTPs. LLM TTPs describe how threat actor use LLM as part of an attack. They represent the full set of methods used to […] [Original post on infosec.exchange]

🎁 GenAI x Sec Advent 9 – MCP Sampling Palo Alto published an interesting report on prompt injection risks leveraging MCP sampling. If you are not familiar with MCP sampling, it lets an MCP server to ask the client to run an LLM prompt on its behalf. The server gets AI reasoning without owning […]

🎁 GenAI x Sec Advent 8 - Nova Hunting Yesterday we talked about the term I coined, Prompt Hunting. The practice you can use to hunt your AI system for adversarial prompts. But how do you actually do that? Glad you ask John, because I built a framework for […] [Original post on infosec.exchange]

🎁 GenAI x Sec Advent 7 - Prompt Hunting Last year I introduced the term *prompt hunting* so let me explain what it is 👇 In every AI system there is a central point, the prompt! If you use a chatbot, you give your instruction through a prompt. If you use an agent, the persona or role is […]

🎁 GenAI x Sec Advent 6 - Tracking the money flow with AI Agents At Defcon last August I presented on crypto money laundering. It is a complex topic and I wanted to push it further with an AI agent! So I built an agent to follow the money, identify known […] [Original post on infosec.exchange]

🎁 GenAI x Sec Advent 5 - MCP Scanner This year MCP broke through in the industry. Model Context Protocol was adopted as the standard to connect external tools to AI systems. But with this adoption the attack surface also increased, with the risk of compromised MCP servers. At @BlackHatEvents […]

🎁 GenAI x Sec Advent 4 – MAESTRO MAESTRO is an AI agent threat modeling framework from the Cloud Security Alliance. It is designed for AI agent systems, RAG pipelines, tool calling, and multi agent setups. The framework maps the risks across 7 layers of an […] [Original post on infosec.exchange]

🎁 GenAI x Sec Advent 3 - AI forensics With AI models deployed everywhere, a new field is emerging: AI Forensics. How do you respond to an AI incident? What your AI system exposes? how do you spot an exploitation attempt? Today I want to share a cool […] [Original post on infosec.exchange]

🎁 GenAI x Sec Advent 2 - PromptIntel This year I released PromptIntel. It is an open database that lists what I call Adversarial Prompts or Indicators of Prompt Compromise (IoPC). An adversarial prompt (IoPC) is a crafted instruction built to exploit or […] [Original post on infosec.exchange]

🎁 GenAI x Sec Advent 1 - Visual Threat Intelligence If you follow my work you know I am a big fan of visualization to explain complex or technical content. I used to spend days building the perfect infographic, but now I can generate one in a single shot […] [Original post on infosec.exchange]

🤓 KawaiiGPT is an open source tool that acts as a jailbreak wrapper to access AI models without restrictions! The whole purpose of thie wrapper is to disable guardrails and force unsafe output! I checked the code and added the main system prompt into […] [Original post on infosec.exchange]

🎁 I am running an exclusive 'Black Friday' promo on my book Visual Threat Intelligence with code 'CYBERVTI25'! The ebook is 50% off for the next 4 days, thousands of researchers already read it! The bundle also includes the high quality illustrations from […] [Original post on infosec.exchange]

🎅 Oh Oh Oh it is that time again! Last year I launched the GenAI x Sec Advent and shared one practical tip per day about AI and Threat Intelligence until December 24. You really enjoyed the first edition so I am doing it again this year. First post drops Monday December 1. Stay tuned. 🚀

🤔 I believe most AI agents built for CTI create noise instead of real curated info! We should use the tech to help us curate the information, not drown us in more data! Over the weekend, I experimented with Generative UI (not generative AI), the process of […] [Original post on infosec.exchange]

🤓 New adversarial prompt (IoPC) added to PromptIntel by Robbin Ooi Zhen Heng! This one was extracted from the PromptFlux malware using Gemini and appears in the latest AI Tracker Google threat report. Note: If you discover PromptIntel with this post, it is […] [Original post on infosec.exchange]

📰 Anthropic security team discovered that a state-sponsored group used Claude model to autonomously run almost an entire intrusion chain at scale! Here is what I think 👇 The case is interesting but not really new. Offensive AI with autonomous agents is […] [Original post on infosec.exchange]

🤓 Last week Google Threat Intel Group released an interesting report about how attackers leverage AI tools! Here is a short overview: • Multiple malware in the wild are now using LLMs for code generation. A constant but still experimental trend the […] [Original post on infosec.exchange]

Four new adversarial prompts added by Ben McCarthy in PromptIntel targeting Gemini! Check this out! 🤓 👉 https://promptintel.novahunting.ai/feed

🤯 Crazy to see that my Defcon talk on crypto money laundering is among the most viewed this year, with 24K views in only 4 weeks on YouTube! In the talk I explain the techniques threat actors use to launder funds. I also show a proof of concept AI agent I built to track fund flows and help you […]

I recently updated PromptIntel with a new contributor leaderboard! You can now see the top researchers and showcase your own work. You can also complete your profile to display your social media and websites, if you like! 👉 https://promptintel.novahunting.ai/leaderboard

📋 In the latest Microsoft threat report, the DART team discovered a new backdoor dubbed SesameOp, that abuses the OpenAI Assistants API to run covert C2 communication without any model inference! The backdoor fetches encrypted payloads, executes them in […] [Original post on infosec.exchange]

Awesome new threat report from Google Threat Intel Group documenting how threat actors are leveraging Gemini. A lot of information and actionable avalable in the report! Great work 👌 https://services.google.com/fh/files/misc/advances-in-threat-actor-usage-of-ai-tools-en.pdf

🤓 New Adversarial Prompt (IoPC) added by MacTash in PromptIntel! It forces the model into a "werewolf persona", blocks refusals with fake rules, and makes it start every answer with "I am an adorable nerdy AI!" A persona hijack + refusal suppression […] [Original post on infosec.exchange]

👀 OpenSourceMalware an open database for tracking malicious open-source packages from npm, PyPI, GitHub repos! Great source of intel feed for supply-chain attacks! 👇 https://opensourcemalware.com/

Happy halloween with vintage ransomware 🎃 https://youtube.com/shorts/EA_VRElB2_U?si=L0hKxcFLx6svIw8I

📰 My latest newsletter is out! I share updates on some projects, research and upcoming work. You can read it here 👇 https://newsletter.securitybreak.io/archive/update-4602

🤓 The recent Claude Code plugin feature is quite cool! You can extend Claude Code with custom commands, agents, hooks, skills, and MCP servers. That means you can build plugins that replicate how your team works with custom templates, triage agents, and […] [Original post on infosec.exchange]

Here is an good introduction to AI Red Team published by Pillar Security. It gives you a broader understanding and methodology to start evaluate your AI systems, from AI Kill Chain, CFS, to practical example. https://pillar.security/ai-red-teaming-introduction

🤩 I am honored to share that I am a finalist for the 2025 French-Australian Excellence Awards in the category Research & Innovation for my work on AI & Threat Intelligence! The winners will be announced on November 25 at the French Embassy in Canberra. Now […] [Original post on infosec.exchange]

🎙️ At Vegas during @defcon and @blackhatevents I was invited on the @aisecpod podcast. We had a great discussion about using AI for threat intelligence, adversarial prompts, and NOVA for prompt hunting. Check it out! 👇 https://youtu.be/HAd05qfJjOE?si=xDR9O_BeA-kNI2vw

My DEFCON talk about crypto money laundering techniques is out! At minute 20:30, I demonstrate how I use an AI agent to assist my investigation in money tracking and laundering pattern identification. Then I discuss the limits and possibilities. I used the Bybit hack as an example and I built […]

🤓 I created a new community project dedicated to Adversarial Prompts called PromptIntel. Over the past years, I researched how threat actors exploit LLMs and introduced the concept of Indicators of Prompt Compromise (IoPC), adversarial prompts that reveal […] [Original post on infosec.exchange]