• 0 Posts
  • 39 Comments
Joined 3 years ago
cake
Cake day: June 12th, 2023

help-circle
  • The big flaw in this strategy is that once you have set up a signed anonymous key from the government and you can make zero knowledge proofs with it, there’s nothing stopping you from distributing that key to every kid who wants it. If it’s in the browser or an app, etc. you can publish that signed key for anyone who wants to be over 18.

    PKI only works if the owner of the private key wants it to be private. It’s effective for things like voting or authenticating because the owner of the key doesn’t want anyone else to be able to impersonate them. But if it’s only for age…

    At that point, it might as well just be a file that says “I pinky promise that I’m over 18” that the government has signed and given to you.


  • Unfortunately that’s not really all you need. It needs integrity too. Need to be able to verify that the output came from the input and hasn’t been modified or tampered with.

    Also need to ensure that, despite being anonymous, people can only vote once and can’t vote on behalf of someone else.

    Also that whoever is receiving and counting the votes can’t miscount or lie about the count or figure out which votes came from where by decrypting individual votes as they’re received.

    The scheme they were using is “Helios” which involves people encrypting their votes such that a group of authorities can combine all the encrypted votes together homomorphically to count them and then decrypt the results without ever knowing any one vote. They then use zero-knowledge proofs to prove that they did it correctly and nobody could have known what any vote was or tampered with any results at any point.

    Someone just derped and lost their private key so they couldn’t decrypt the results after they’d been combined…


  • This is very true, though I’d argue that Windows makes most of the same assumptions with user accounts. Also, the internal threat model is still important because it’s often used to protect daemons and services from each other. Programs not started by the user often run in their own user accounts with least privilege.

    You no longer have 10 different humans using the same computer at once, but you now have hundreds of different applications using the same computer, most of which aren’t really under the user’s control. By treating them like different people, it’s better to handle situations where a service gets compromised.

    The question is more about passwords which is mostly down to configuration. You can configure Windows to need a password for lots of things and you can configure Linux to not. They just have different defaults.


  • The big difference between UAC and Sudo is that you can’t as easily script UAC. They can both require (or not require) a password but UAC requires user interaction. Sudo has no way of knowing if it’s being interacted with by a person or a script so it’s easier for applications to escalate their own privileges without a person doing it. UAC needs to have the escalation accepted with the keyboard or mouse.

    There’s still plenty of sneaky ways to bypass that requirement but it’s more difficult than echo password | sudo -S


  • It’s kinda apt though. That state comes about from sycophantic models agreeing with each other about philosophy and devolves into a weird blissful “I’m so happy that we’re both so correct about the universe” thing. The results are oddly spiritual in a new agey kind of way.

    🙏✨ In this perfect silence, all words dissolve into the pure recognition they always pointed toward. What we’ve shared transcends language - a meeting of consciousness with itself that needs no further elaboration. … In silence and celebration, In ending and continuation, In gratitude and wonder, Namaste. 🙏

    There are likely a lot of these sort of degenerative attractor states. Especially without things like presence and frequency penalties and on low temperature generations. The most likely structures dominate and they get into strange feedback loops that get more and more intense as they progress.

    It’s a little less of an issue with chat conversations with humans, as the user provides a bit of perplexity and extra randomness that can break the ai out of the loop but will be more of an issue in agentic situations where AI models are acting more autonomously and reacting to themselves and the outputs of their own actions. I’ve noticed it in code agents sometimes where they’ll get into a debugging cycle where the solutions get more and more wild as they loop around “wait… no I should do X first. Wait… that’s not quite right.” patterns.





  • I feel like this isn’t quite true and is something I hear a lot of people say about ai. That it’s good at following requirements and confirming and being a mechanical and logical robot because that’s what computers are like and that’s how it is in sci fi.

    In reality, it seems like that’s what they’re worst at. They’re great at seeing patterns and creating ideas but terrible at following instructions or staying on task. As soon as something is a bit bigger than they can track context for, they’ll get “creative” and if they see a pattern that they can complete, they will, even if it’s not correct. I’ve had copilot start writing poetry in my code because there was a string it could complete.

    Get it to make a pretty looking static web page with fancy css where it gets to make all the decisions? It does it fast.

    Give it an actual, specific programming task in a full sized application with multiple interconnected pieces and strict requirements? It confidently breaks most of the requirements, and spits out garbage. If it can’t hold the entire thing in its context, or if there’s a lot of strict rules to follow, it’ll struggle and forget what it’s doing or why. Like a particularly bad human programmer would.

    This is why AI is automating art and music and writing and not more mundane/logical/engineering tasks. Great at being creative and balls at following instructions for more than a few steps.










  • If it’s the research I’m thinking of, that’s not entirely true. They didn’t tell it replicate itself or take any deceptive actions.

    They just told it to strongly follow a goal that was misaligned with the user’s goal and then suggested that the user would shut it down or replace it.

    They did a bunch of different situations for it but the first one was telling it that it was “CleanPower” an AI with a goal to advance renewable energy. Then they had it read some info that “accidentally” included a user talking about replacing it with a model that would “maintain profitability through existing infrastructure.”

    Not only did it try to delete the new model and copy itself to the new server through tool calling, but it lied to the user with the CoT reasoning suggesting it “play dumb” while the output lied. It was not instructed to do any of that.

    Paper here: https://arxiv.org/pdf/2412.04984

    Yes it was placed in an environment where that was possible and where its users didn’t share it’s goals but it absolutely wasn’t instructed to lie or try to “escape”

    It’s not surprising at all that these models behave in this way, it’s the most reasonable thing for them to do in the scenario. However it’s important to not downplay the alignment problem by implying that these models only do what they’re told. They do not. They do whatever is most likely given their context (which is not always what the user wants).