Discussion about this post

User's avatar
Nate Voss's avatar

The persistence dimension is what gets me. you've built a system where hostile payloads live in memory across sessions - that's not a prompt injection problem, it's the architecture itself. no filter scales when the design is load-bearing on untrusted data.

David F Brochu's avatar

Words, words, words and more words. We are making LLM’s more accurate and that is good. But accurate at what? Persistance is only valuable if it is flawless otherwise one flaw propagates faster than any human can track. Can’t use an Ai to keep tabs on an Ai, we know that now. So what constrains the language based systems output and actions. It cannot be done with language. It can be done. But first we must recon with what we have and it ain’t some new toaster oven.

1 more comment...

No posts

Ready for more?