I wish lemmy.ml (also unstable) or lemmy.world would hand out a (nearly) full copy of the database so we can get more analysis done on PostgreSQL performance behaviors. Remove the private comments and password /2fa/user, or whitelist only comments/posts/communities/person tables - but most everything else should already be public information that’s shared via the API or federation anyway. it’s the quantity, grouping, and the age of the data that’s hard to reproduce in testing. And knowledge of other federated servers, even data that may have been generated by older versions of Lemmy that new versions can’t reproduce.
It’s been over 60 days of constant PostgreSQL overload problems and last week Lemmy.ca made a clone of their database to study offline with AUTO_EXPLAIN which surfaced a major overload on new comments and posts related to site_aggregates counting (it was counting each new post/comment against every known server, not just the single database row for a server).
I have an account over on World too, and every major Lemmy server I use throws errors with casual usage. It’s been discouraging, I haven’t visited a website with this many errors in years. Today (Sunday) has actually been better than yesterday, but I do not see many new postings being created on lemmy.ml today.
subscribe to every community, and let federation load overwhelm your server.
Did that, takes lots of time to wait for the content to come in… and there is no backfill. Plus I suspect that the oldest servers (online for several years) have some migration/upgrade related data that isn’t being accounted for.
ok, I did know about that, just didn’t memorize the name. I’m assuming only private messages and user account info (email address) are the real concern in terms of exposure? It’s mostly a public posting thing, or not?
However… consider the… uh… “charter” for the instance:
Kolektiva is an anti-colonial anarchist collective that offers federated social media to anarchist collectives and individuals in the fediverse. For the social movements and liberation!
There’s some action to the anti-establishment and individuals on the site may have participated in violent activities.
They’ve got a peer tube instance too - https://kolektiva.media/ … and want to make a bet if any of the accounts on the mastodon instance are the same as the ones on the peer tube instance and if any of the videos on there are incriminating… and you’ve got the email address and IP address of the person logging into the account which can then be used to identify them.
The admins won’t sell us out to the feds is one thing. The admins won’t work on an unencrypted version of the database that exposes personal information (and get raided) is another.
Working on a backup of live data without sanitizing personal information first is a risk that every DBA at a big company lectures the programmers about.
I wish lemmy.ml (also unstable) or lemmy.world would hand out a (nearly) full copy of the database so we can get more analysis done on PostgreSQL performance behaviors. Remove the private comments and password /2fa/user, or whitelist only comments/posts/communities/person tables - but most everything else should already be public information that’s shared via the API or federation anyway. it’s the quantity, grouping, and the age of the data that’s hard to reproduce in testing. And knowledge of other federated servers, even data that may have been generated by older versions of Lemmy that new versions can’t reproduce.
It’s been over 60 days of constant PostgreSQL overload problems and last week Lemmy.ca made a clone of their database to study offline with AUTO_EXPLAIN which surfaced a major overload on new comments and posts related to site_aggregates counting (it was counting each new post/comment against every known server, not just the single database row for a server).
I have an account over on World too, and every major Lemmy server I use throws errors with casual usage. It’s been discouraging, I haven’t visited a website with this many errors in years. Today (Sunday) has actually been better than yesterday, but I do not see many new postings being created on lemmy.ml today.
[This comment has been deleted by an automated system]
Did that, takes lots of time to wait for the content to come in… and there is no backfill. Plus I suspect that the oldest servers (online for several years) have some migration/upgrade related data that isn’t being accounted for.
Wasn’t this how kolektiva.social got in trouble 😂
What is the story? never heard of it.
Unencrypted backup and an FBI raid.
https://kolektiva.social/@admin/110637031574056150 ( https://web.archive.org/web/20230701101423/https://kolektiva.social/@admin/110637031574056150 if you have trouble accessing it).
ok, I did know about that, just didn’t memorize the name. I’m assuming only private messages and user account info (email address) are the real concern in terms of exposure? It’s mostly a public posting thing, or not?
Email address, IP, and posting information.
However… consider the… uh… “charter” for the instance:
There’s some action to the anti-establishment and individuals on the site may have participated in violent activities.
They’ve got a peer tube instance too - https://kolektiva.media/ … and want to make a bet if any of the accounts on the mastodon instance are the same as the ones on the peer tube instance and if any of the videos on there are incriminating… and you’ve got the email address and IP address of the person logging into the account which can then be used to identify them.
The admins won’t sell us out to the feds is one thing. The admins won’t work on an unencrypted version of the database that exposes personal information (and get raided) is another.
Working on a backup of live data without sanitizing personal information first is a risk that every DBA at a big company lectures the programmers about.