Some of the big weaknesses of running a self-hosted Zulip server for your community are:
- Your server admin can see DMs (or at least metadata, not sure if Zulip does E2E for DMs). The same is true for centralized services in theory, but unless you're a terrorist or a person of interest to a major government, it's extremely unlikely that a Discord employee will have an incentive to spy on your messages specifically. Your admin is likely part of your community and may know you personally, so the temptation is much, much higher.
- If the admin dies and nobody else has the keys to the kingdom, the server can go down at any point, and there's no way for users to reconstitute the network semi-automatically. Discord servers don't just go away unless somebody actively makes them to.
- It's much less secure in practice, it relies on your admin to always be on guard and constantly update their server to prevent vulnerabilities, either in Zulip or in the myriad of other self-hosted services running on it. One guy in his basement that goes on vacation once a year and has family responsibilities is far more likely to make mistakes than a team of trained cybersecurity professionals.
- Many Discord users are in 20+ servers. Anything that doesn't provide a one-click server joining experience (for users who already have an account on a different server) is nowhere near a Discord replacement.
- People want bots (for things like high-fidelity Youtube music streaming on voice channels), and those are mostly Discord-only.
- Anything open source will be worse at phishing and fraud / abuse prevention by definition, as many fraud-prevention approaches rely on the fraudster blindly guessing at what the code and ML models (do you even have ML models for this) are doing.
> it's extremely unlikely that a Discord employee will have an incentive to spy on your messages specifically
No, but history shows some unscrupulous staff members will always snoop, whether its just pure interest or something more nafarious like intent to sell on the black market. This makes the risk of your private data being leaked > 0, which should always be treated as a valid risk.
> If the admin dies and nobody else has the keys to the kingdom, the server can go down at any point
This is how infrastructure works, and supposed to work, besides the point that servers "die by themselves" which of course isn't true in reality. You decrease the bus factor if this is a problem for you.
> Discord servers don't just go away unless somebody actively makes them to
If all the sysadmins at Discord died and nobody else has the keys, exactly the same problem happens. Discord though surely have multiple backups of the keys and so on, something you too can do when you have your own infrastructure, so overall that argument feels almost dishonest, since you don't compare the two accurately.
> Anything open source will be worse at phishing and fraud / abuse prevention by definition
What? Completely orthogonal concerns, and if your main "fraud-prevention approaches" depend on security by obscurity, I'm not sure you should even attempt to be involved in those efforts, because that's not what the rest of the industry is going by a long mile.
> People want bots (for things like high-fidelity Youtube music streaming on voice channels), and those are mostly Discord-only.
Actually, the further I get in your comment, the more it seems like you don't actually understand what Zulip offers nor what the parent comment is about. Music streaming on voice channels? Completely outside the scope of Zulip...
----------
I think you have to understand the comment you're replying to a bit better, before attempting to lift Discord above Zulip. They're specifically talking about Zulip as an alternative "for managing the firehose of busy communities", not as a general replacement for every single Discord "server" out there. Yet you've responded to the comment as that's what they've been doing.
> This is how infrastructure works, and supposed to work
No, infrastructure doesn't have to work this way. This is a very old-school mentality.
Sign the content with a key that you control. Back up the content locally. And boom- your server is easily replaced. It only helps copy data around and performs certain conveniences.
I've been working on this full-time for a few years. If we succeed, we solve link rot (broken links) on the web.
Well, you're basically repeating what I'm saying, but with more detail. It's still what I true, "the one who holds the key holds the kingdom", just shifting it to the user rather than the admin. This is great, and works too, but doesn't make what I say less true.
I might be misremembering, but AFAIK, that kind of surveillance mostly worked because many companies didn't bother encrypting datacenter-to-datacenter traffic, thinking that those networks are trusted. That mistake has since been rectified though.
With almost everything going over TLS these days and HTTPS being the norm, even for server-to-server APIs, it's much harder to snoop on traffic without the collaboration of one of the endpoints, and the more companies you ask for that kind of collaboration, the higher your risk of an unhappy employee becoming a whistleblower.
That's also about US companies that can't refuse or can't bother to challenge that a dragnet is set up in their process.
ISPs themselves didn't save any data.
However, they gave interception rooms to the NSA (which is indeed technically not them).
Nowadays ISPs aren't the right scale to do it for the reasons you mentioned. But the USA lowkey moved the dragnet to the main datacenters with prism, then made it mandatory for all with the CLOUD act.
And if the threat is not coming from the USA, but some other country starts to ask Discord to BCC them the IDs of their citizens, we can do the odds on whether Discord will challenge it or not.
Now I want to ask Discord who is their third party provider ? Why don't they process IDs themselves ?
Unless you use Cloudflare (or roughly any other DDOS protection system), in which case you're letting those companies MITM all requests on purpose. Protected between you and Cloudflare by PFS and any other acronym you like.
I think the odds that Cloudflare hasn't been forced into data snooping by the government are approximately zero. It's the by far the biggest, juiciest target.
The "money goes to the repo part" is the problem here, as it incentivizes maintainers to refuse legitimate pull requests.
Crypto has a perfect way to burn money, just send it to a nonexistent address from where it can never be recovered. I guess the trad fi equivalent are charitable donations.
The real problem here is the amount of work necessary to make this viable. I bet Visa and Mastercard would look at you funny if your business had such a high rate of voluntary transaction reversals, not to mention all the potential contributors that have no access to Visa/MC (we do want to encourage the youth to become involved with Open Source). This basically means crypto, and crypto has its own set of problems, particularly around all the annoying KYC/AML that a normie has to get through to use it.
> I bet Visa and Mastercard would look at you funny if your business had such a high rate of voluntary transaction reversals
Plenty of businesses do the “your credit card will be charged $1 and then reversed” as a verification method that I don’t think it would be a major issue. I do wonder how much those companies are paying for that, though… I am guessing they lose some of that $1.
You can reduce the transactions with payment providers. Instead of money exchanging from contributor to maintainer, have a token exchange. Contributors fund tokens with real money, and pull requests cost and refund tokens. Like an escrow account. But the money never goes to the target system. There are no perverse incentives to steal tokens. If you get a reputation of not refunding tokens (which have no value to a maintainer), then contributors will dry up.
Probably just making it non refundable works almost as well (since time really is expended reading it), without the hassle of spinning up an intermediary layer blockchain.
> I bet Visa and Mastercard would look at you funny if your business had such a high rate of voluntary transaction reversals
…you might be right, but I do wonder if the situation would be different if “your business” was “Microsoft”. Obviously they would discuss this plan ahead of time.
> The "money goes to the repo part" is the problem here, as it incentivizes maintainers to refuse legitimate pull requests.
That's not true. The issue is that the system the comment you're replying to described is escrow. Escrow degenerates in the way that you describe. I explain it a bit more in this comment elsewhere on this post:
A straight up non-refundable participation payment does not have this issue, and creates a different set of incentives and a different economy, while there also exist escape hatches for free-of-charge contributions.
> The real problem here is the amount of work necessary to make this viable.
Not necessarily. This article mentions Tezos, which is capable of doing such things on-chain already:
> all the annoying KYC/AML that a normie has to get through to use it.
There are always escape hatches. If your code is so great that people will want to pull it, then you don't pay to push. If it's not really that great, then what are we talking about? Maybe it disincentivizes mid code being pushed. So be it.
You can make friends, you can make a name for yourself, you can make a fork that's very successful and upstream will want to pull it in, you can exert social pressure / marketing to get your code merged in. Lots of options that do not involve KYC/AML.
For everyone else, I'd say KYC/AML are a good idea because of the increasing amount of supply chain exploits being pushed out into repos. If pushing by randos is gated by KYC/AML, then there's at least some method of chasing the perps down and taking them to justice.
That's a win-win-win-win situation. Less mid code, less exploits, earnings for maintainers, AI slop blocked. Absolutely amazing.
This feels like the perfect game to add (screen reader) accessibility to.
Sadly, I don't think it can be done by us screen reader users, as the Godot editor UI is not really accessible (though they're apparently changing that in the latest version).
"cancel or allow" (which Microsoft still does) makes no sense, it just trains user to click "allow" every time. Users don't know what they should allow or not.
It makes a bit more sense on accounts that have a password set, as it requires you to confirm identity when introducing significant changes to the system (and this is something that Apple also does).
Gatekeeper is a different thing, it basically makes sure that the software you're trying to run has been pre-scanned for malware by a trusted party, similar to Windows's "smart screen" and Defender or APt's GPG keyring integration. It's a mechanism that is completely invisible to 99+% of users. If you see a Gatekeeper pop-up and the app in question is not mlaware, the developer is doing something very wrong.
3 reasons why Python is much better than JS for this IMO.
1. Large built-in standard library (CSV, sqlite3, xml/json, zipfile).
2. In Python, whatever the LLM is likely to do will probably work. In JS, you have the Node / Deno split, far too many libraries that do the same thing (XMLHTTPRequest / Axios / fetch), many mutually-incompatible import syntaxes (E.G. compare tsx versus Node's native ts execution), and features like top-level await (very important for small scripts, and something that an LLM is likely to use!), which only work if you pray three times on the day of the full moon.
3. Much better ecosystem for data processing (particularly csv/pandas), partially resulting from operator overloading being a thing.
> JSX/TSX, despite what React people might want you to believe, are not part of the language.
I think you misunderstood this. tsx in this context is/was a way to run typescript files locally without doing tsc yourself first, ie make them run like a script. You can just use Node now, but for a long time it couldn’t natively run typescript files.
The only limitation I run into using Node natively is you need to do import types as type imports, which I doubt would be an issue in practice for agents.
Yes, thank you for pointing that out. Forgot that there's a another thing named "tsx" out there.
I wouldn't call it running TS natively - what they're doing is either using an external tool, or just stripping types, so several things, like most notably enums, don't work by default.
I mean, that's more than enough for my use cases and I'm happy that the feature exists, but I don't think we'll ever see a native TypeScript engine. Would have been cool, though, considering JS engines define their own internal types anyway.
> Do you think there are 22 competing package managers in python because the package/import system "just works"?
There aren't; a large fraction of tools people mention in this context aren't actually package managers and don't try to be package managers. Sometimes people even conflate standards and config files with tools. It's really amazing how much FUD there is around it.
But more importantly, there is no such thing as "the package/import system". Packaging is one thing, and the language's import system is a completely different thing.
And none of that actually bears on the LLM's ability to choose libraries and figure out language syntax and APIs. For that matter, you don't have to let it set up the environment (or change your existing setup) if you don't want to.
What I find to be the most impressive part here is that it wrote the compiler without reference to the C specification and without architecture manuals at hand.
This vindicates the pro-AI censorship crowd I guess.
It definitely makes it clear what is expected of AI companies. Your users aren't responsible for what they use your model for, you are, so you'd better make sure your model can't ever be used for anything nefarious. If you can't do that without keeping the model closed and verifying everyone's identities... well, that's good for your profits I guess.
It's not really different from how we treat any other platform that can host CSAM. I guess the main difference is that it's being "made" instead of simply "distributed" here
Those images are generated from a training set, and it is already well known and reported that those training sets contain _real_ CSAM, real violence, real abuse. That "generated" face of a child is based on real images of real children.
Indeed, a Stanford study from a few years back showed that the image data sets used by essentially everybody contain CSAM.
Everybody else has teams building guardrails to mitigate this fundamental existential horror of these models. Musk fired all the safety people and decided to go all in on “adult” content.
> let's distinguish between generated images, however revolting, and actual child sexual abuse.
Can't because even before GenAI the "oh its generated in photoshop" or "they just look young" excuse was used successfully to allow a lot of people to walk free. the law was tightend in the early 2000s for precisely this reason
Yes they could have an uncensored model, but then they would need proper moderation and delete this kind of content instantly or ban users that produce it. Or don’t allow it in the first place.
It doesn’t matter how CSAM is produced, the only thing that matters is that is on the platform.
It matters whether they attempt to block it or encourage it. Musk encouraged it, until legal pressure hit, then moved it behind a paywall so it's harder to see evidence.
In the UK, you must take "reasonable" steps to remove illegal content.
This normally means some basic detection (ie fingerprinting which is widely used from a collaborative database) or if a user is consistently uploading said stuff, banning them.
Allowing a service that you run to continue to generate said illegal content, even after you publicly admit that you know its wrong, is not reasonable.
Nothing in common law is "concrete", thats kinda the point of it.
Judges can evolve and interpret as they see fit, and this evolution is case law.
This is why in the US the supreme court can effectively change the law by issuing a binding ruling. (see 2nd amendment meaning no gun laws, rather than as written, or the recent racial profiling issues)
No law is concrete. Murder is killing with intent to kill. What concrete test shows if someone intended to kill? They say you have intent to kill if a reasonable person would expect the actions you took would result in killing.
if you can be sued for billions because some overbearing body, with a very different ideology to yours, can deem your moderation/censorship rules to be "unreasonable" then what you do is err on the side of caution and allow nearly nothing
this is not compatible with that line of business - perhaps one of the reasons nothing is done in Europe these days
> this is not compatible with that line of business - perhaps one of the reasons nothing is done in Europe these days
Except for 40% of all Big Tech products and a vast industrial network of companies, and the safe airplane building and decent financial services that don't take 3% of everything, then yeah, I guess nothing is done in Europe these days.
And wait, wasn't most of Google's AI stuff acquired from a European country?
Honestly, while Europe has a lot of problems, this notion that many US people have that literally nothing happens there is wildly off-base.
Like, this is a function of fragmented capital markets rather than anything else. Ryanair would 100% have a market cap of 50bn+ if it had a US listing.
Anyway, market cap is a really really bad metric for basically anything. Like Walmart has a market cap of over 1tn now, do you think its business has materially changed since 2021 (when it was half that)?
Meta has basically doubled since then, and again, their business is basically the same as it was 5 years ago.
As another example, Stripe is valued at about 100bn, while both OpenAI and Anthropic are 3.5-5x that. Which ones would you rather put your money in?
(apologies I figured the OECD would have better data but this was the best I could find).
So, on average it would take a low-income person half the time to become upper income in Denmark versus the US. Is this a better metric? Certainly for a low income person with unlimited mobility options.
The vast majority of the EU is not common law, so "reasonable" in this instance is different.
What you describe already happens in the USA, that why MLB has that weird local TV blackout, why bad actors use copyright to take down content they don't like.
The reason why its so easy to do that is because companies must reasonably comply with copyright holder's requests.
Its the same with CSAM, distributing it doesn't have first amendment protection, knowingly distributing it is illegal. All reasonable steps should be taken to detect and remove CSAM from your systems to qualify for safe harbour.
> Its the same with CSAM, distributing it doesn't have first amendment protection, knowingly distributing it is illegal. All reasonable steps should be taken to detect and remove CSAM from your systems to qualify for safe harbour.
nice try, but nobody is distributing or hosting CSAM in the current conversation
people trying to trick a bot to post bikini pictures of preteens and blaming the platform for it is a ridiculous stretch to the concept of hosting CSAM, which really is a transparent attack to a perceived political opponent to push for a completely different model of the internet to the pre-existing one, a transition that is as obvious as is already advanced in Europe and most of the so-called Anglosphere
> The vast majority of the EU is not common law, so "reasonable" in this instance is different.
the vast majority of the EU is perhaps incompatible with any workable notion of free speech, so perhaps America will have to choose whether it's worth it to sanction them into submission, or cut them off at considerable economic loss
it's not a coincidence that next to nothing is built in Europe these days, the environment is one of fear and stifling regulation and if I were to actually release anything in either AI or social networks I'd do what most of my fellow Brits/Europoors do already, which is to either sell to America or flee this place before I get big enough to show up in the euro-borg's radar
> nice try, but nobody is distributing or hosting CSAM in the current conversation
multiple agencies (Ofcom, irish police IWF, and what ever the french regulator is) have detected CSAM.
You may disagree with that statement, but bear in mind the definition of CSAM in the UK is "depiction of a child" which means that if its of a child or entirely generated is not relevant. This was to stop people claiming that massive cache of child porn they had was photoshoped.
in the USA CSAM is equally vaguely defined, but the case law is different.
> EU is perhaps incompatible with any workable notion of free speech
I mean the ECHR definition is fairly robust. But given that first amendment protection has effectively ended in the USA (the president is currently threatening to take a comedian to court for making jokes, you know, like the twitter bomb threat person in the UK) its a bit rich really. The USA is not the bastion of free speech it once was.
> either sell to America or flee this place before I get big enough to show up in the euro-borg's radar
Mate, as someone whos sold a startup to the USA, its not about regulations its about cold hard fucking cash. All major companies comply with EU regs, and its not hard. they just bitch about them so that the USA doesn't put in basic data protection laws, so they can continue to be monopolies.
Pretty disturbing to me how many people _on here_ are cheering for this. I thought that at least here of all places, there might be some nuanced discussion on "ok, I see why people are emotional about this topic in particular, but it's worth stepping back and putting emotions aside for a minute to see if this is actually reasonable overall..." but besides your comment, I'm not seeing much of that.
There's pro-AI censorship and then there's pro-social media censorship. It was the X offices that were raided. X is a social media company. They would have been raided whether it was AI that created the CSAM or a bunch of X contractors generating it mechanical-turk style.
It's a bit of a leap to say that the model must be censored. SD and all the open image gen models are capable of all kinds of things, but nobody has gone after the open model trainers. They have gone after the companies making profits from providing services.
The for-profit part may or may not be a qualifier, but the architecture of a centralized service means they automatically become the scene of the crime -- either dissemination or storing of illegal material. Whereas if Stability creates a model, and others use their model locally, the relationship of Stability to the crime is ad-hoc. They aren't an accessory.
This is not about AI but about censorship of a nonaligned social network. It's been a developing current in EU. They have basically been foaming at the mouth at the platform since it got bought.
I could maybe see this argument if we were talking about raiding Stable Diffusion or Facebook or some other provider of local models. But the content at issue was generated not just by Twitter's AI model, but on their servers, integrated directly into their UI and hosted publicly on their platform. That makes them much more clearly culpable -- they're not just enabling this shit, they're creating it themselves on demand (and posting it directly to victims' public profiles).
And importantly, this is clearly published by Grok, rather than the user, so in this case (obviously this isn't the US) but if it was I'm not sure Section 230 would apply.
I think having guardrails on your AI to not be able to produce this stuff is good actually. Also, Elon encourages this behavior socially through his posts so yeah he should face consequences.
It's not because it could generate CSAM. It's because when they found out it could generate CSAM, they didn't try to prevent that, they advertised it. Actual knowledge is a required compenent of many crimes.
There's no crowds or sides. It's all manufactured divisions because some of those who can't or don't want to create the technology are determined to control it. So they'll get you mad about what they need to, to justify actions that increase their control.
It's the same playbook that is used again and again. For war, civil liberties crackdowns, lockdowns, COVID, etc, etc: 0) I want (1); start playbook: A) Something bad is here, B) You need to feel X + Panic about it, C) We are solving it via (1). Because you reacted at B, you will support C. Problem, reaction, solution. Gives the playmakers the (1) they want.
We all know this is going on. But I guess we like knowing someone is pulling the strings. We like being led and maybe even manipulated because perhaps in the familiar system (which yields the undeniable goods of our current way of life), there is safety and stability? How else to explain.
Maybe the need to be entertained with drama is a hackable side effect of stable societies populated by people who evolved as warriors, hunters and survivors.
Holding corporations accountable for their profit streams is "censorship?" I wish they'd stop passing models trained on internet conversations and hoarded data as fit for any purpose. The world does not need to boil oceans for hallucinating chat bots at this particular point in history.
What would be censorship is if those same companies then brigaded forums and interfered with conversations and votes in an effort to try to hide their greed and criminality.
Not that this would _ever_ happen on Hacker News. :|
Not knowing any better, and not having seen any of the alleged images, my default guess would be they used the exact same CSAM filtering pipeline already in place on X regardless of the origin of the submitted images.
They obviously didn’t really implement anything as you can find that content or involuntary nudes of other people, which is also an invasion of privacy, super easily
And the sign out front says "X-Ray camera photographs anyone naked — no age limits!"
And the camera is pointing out the window so you can use it on strangers walking by.
There is a point in law where you make something so easy to misuse that you become liable for the misuse.
In the USA they have "attractive nuisance", like building a kid's playground on top of a pit of snakes. That's so obviously a dumb idea that you become liable for the snake–bitten kids — you can't save yourself by arguing that you didn't give the kids permission to use the playground, that it's on private property, that the kids should have seen the snakes, or that it's legal to own snakes. No, you set up a situation where people were obviously going to get hurt and you become liable for the hurt.
If the camera reliably inserts racist filters and the ballpen would add hurtful words to whatever you write, indeed, let them up their legal insurance.
- Your server admin can see DMs (or at least metadata, not sure if Zulip does E2E for DMs). The same is true for centralized services in theory, but unless you're a terrorist or a person of interest to a major government, it's extremely unlikely that a Discord employee will have an incentive to spy on your messages specifically. Your admin is likely part of your community and may know you personally, so the temptation is much, much higher.
- If the admin dies and nobody else has the keys to the kingdom, the server can go down at any point, and there's no way for users to reconstitute the network semi-automatically. Discord servers don't just go away unless somebody actively makes them to.
- It's much less secure in practice, it relies on your admin to always be on guard and constantly update their server to prevent vulnerabilities, either in Zulip or in the myriad of other self-hosted services running on it. One guy in his basement that goes on vacation once a year and has family responsibilities is far more likely to make mistakes than a team of trained cybersecurity professionals.
- Many Discord users are in 20+ servers. Anything that doesn't provide a one-click server joining experience (for users who already have an account on a different server) is nowhere near a Discord replacement.
- People want bots (for things like high-fidelity Youtube music streaming on voice channels), and those are mostly Discord-only.
- Anything open source will be worse at phishing and fraud / abuse prevention by definition, as many fraud-prevention approaches rely on the fraudster blindly guessing at what the code and ML models (do you even have ML models for this) are doing.