Security
The Open-Weight Dilemma: Mitigating AI Cyber Risks Without Killing Open Source
<p>Open-weight LLMs (like LLaMA, Mistral, and DeepSeek-R1) have triggered a "Cambrian explosion" of innovation, but they have also democratized offensive cyber capabilities. Recent evaluations, such as MITRE’s OCCULT framework, show that publicly available models can now achieve >90% success rates on offensive cyber knowledge tests, enabling targeted phishing, malware polymorphism, and vulnerability discovery at scale.</p>
<p>For the Open Source community, this presents an existential crisis. Traditional security models (API gating, monitoring, rate limiting) rely on centralized control, which vanishes the moment weights are published. Furthermore, emerging regulations like the EU AI Act risk imposing impossible compliance burdens on open model developers for downstream misuse they cannot control, such as post-market monitoring.</p>
<p>In this talk, Alfonso De Gregorio (Pwnshow) will deconstruct the "Mitigation Gap"—the technical reality that once a model is downloaded, safety filters can be trivially fine-tuned away. Drawing on his direct consultation work with the European Commission, he will explain how we can navigate this minefield. We will discuss:</p>
<p>1/ The Threat Reality: A look at tools like Xanthorox AI and DeepSeek-R1 to understand the actual offensive capabilities of current open weights, and the state of the art in offensive AI.</p>
<p>2/ The Policy Trap: Why "strict" interpretations of the EU AI Act could stifle open innovation, and the fight to shift liability to the modifier and deployer rather than the open-source developer.</p>
<p>3/ The Way Forward: Technical solutions for "Responsible Release" (Model Cards, capability evaluations) and the necessity of AI-enabled defenses to counterbalance the offensive drop in barrier-to-entry.</p>
<p>This session is for security practitioners and open-source advocates who want to ensure the future of AI remains open, while pragmatically addressing the security chaos it unleashes.</p>
Additional information
| Live Stream | https://live.fosdem.org/watch/ub5132 |
|---|---|
| Type | devroom |
| Language | English |
More sessions
| 1/31/26 |
<p>The world of SBOMs and software transparency artefacts - In-Toto attestations, VEX updates and much more - all mention digital signatures. But not with what and how we should validate these. One thing is for sure - we don't want to use the existing WebPKI. There are some interesting initiatives, like SigStore, but they do not solve all issues. It's time that we work on solving this problem and define a solution for digital signatures that is distributed, secure and trustworthy. This is a call ...
|
| 1/31/26 |
<ul> <li>The pace at which quantum computing is evolving right now, threats of <code>harvest-now-decrypt-later</code> becoming more relevant. The widely deployed classical cryptographic algorithms such as RSA and ECC face a real risk of being broken by quantum attacks, most notably through Shor’s algorithm. This looming threat makes the transition to Post-Quantum Cryptography (PQC) urgent, not as a future project, but as a present-day migration challenge. </li> <li>You may have questions ...
|
| 1/31/26 |
<p>Most container images in production are still unsigned, and even when signatures exist, they often provide no clear guarantee about where the artifact came from or what threat the signature is supposed to protect against. Supply-chain attacks exploit this gap and become an increasingly important issue when publishing or importing open-source software.</p> <p>This talk presents security capabilities in Docker and Moby BuildKit that address these issues. BuildKit executes all build steps in ...
|
| 1/31/26 |
<p>It is widely considered good practice to sign commits. But leveraging those signatures is hard. <a href="https://sequoia-pgp.gitlab.io/sequoia-git/">Sequoia git</a> is a system to authenticate changes to a VCS repository. A project embeds a signing policy in their git repository, which says who is allowed to add commits, make releases, and modify the policy. <a href="https://sequoia-pgp.gitlab.io/sequoia-git/man/sq-git-log.1.html"><code>sq-git log</code></a> can then authenticate a range of ...
|
| 1/31/26 |
<p>Endpoints are where most security incidents begin. Compromises often start with phishing, software vulnerabilities, or simple misconfigurations on individual laptops and servers. Modern security teams rely on endpoint telemetry for detection, investigation, and response. But for many engineers, this part of the stack remains opaque and difficult to reason about.</p> <p>This talk presents a practical, open-source blueprint for building an endpoint telemetry pipeline that engineers can actually ...
|
| 1/31/26 |
<p>HyperDbg is a modern, open-source hypervisor-based debugger supporting both user- and kernel-mode debugging. Operating at the hypervisor level, it bypasses OS debugging APIs and offers stealthy hooks, unlimited simulated debug registers, fine-grained memory monitoring, I/O debugging, and full execution control, enabling analysts to observe malware with far greater reliability than traditional debuggers.</p> <p>When it comes to debugger stealthiness and sandboxing, environment artifacts can ...
|
| 1/31/26 |
<p>This is a live tutorial of hacking against keyboards of all forms. Attacking the keyboard is the ultimate strategy to hijack a session before it is encrypted, capturing plaintext at the source and (often) in much simpler ways than those required to attack network protocols.</p> <p>In this session we explore available attack vectors against traditional keyboards, starting with plain old keyloggers. We then advance to “Van Eck Phreaking” style attacks against individual keystroke emanations ...
|
