toad.social

0 posts0 participants0 posts today

jonny (good kind)so hopefully the picture is coming into focus of federated archive instances monitoring sets of pages, being able to distribute periodic and targeted snapshots of them as ongoing automatic scrape tasks across volunteer machines, autoseeding via torrent feeds, and then webtorrent-powered snapshot perusal. <a href="https://neuromatch.social/tags/sciop" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#sciop</a> is just getting started baby.

jonny (good kind)here is a sneak preview of a very simple python package for creating and handling <code>.torrent</code> files.<a href="https://torrent-models.readthedocs.io/en/latest/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://torrent-models.readthedocs.io/en/latest/</a><a href="https://github.com/p2p-ld/torrent-models" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://github.com/p2p-ld/torrent-models</a><ul><li>Full v2 and hybrid torrent support. </li><li>model-based validation, editing, and parsing with modern python type annotations and metadata.</li><li>Reasonable to use object when loaded with kind serdes handling (not just a wrapper around a bytestring dict)</li><li>Fast enough to use with a tracker/web service (a surprising amount of the existing packages have O(n2) spots for no reason, this is ~dozens-thousands of times faster than <code>torf</code>).</li><li>TODO: implement the rest of the BEPs, add methods for partial rehashing, more testing.</li></ul>I was surprised this needed to be written since torrents are a relatively simple file format, but it did - none of the existing packages are suited for use as a library e.g. for tracker/client software and most can only do v1. I wrote this for sciop, where we want to be experimenting on the format, but also do stuff like dynamically modifying torrents to add trackers, webseeds, and other metadata. I'll do a full announcement after implementing the rest of the BEPs.<a href="https://neuromatch.social/tags/Bittorrent" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Bittorrent</a> <a href="https://neuromatch.social/tags/Python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Python</a> <a href="https://neuromatch.social/tags/Sciop" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Sciop</a> <a href="https://neuromatch.social/tags/NewWork" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#NewWork</a>

Dave Rahardja<a href="https://oldbytes.space/@pixel" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@pixel</a> As promised, here are detailed instructions on how to install the Deluge BitTorrent client and Autobrr RSS subscriber on a TrueNAS Scale device! These instructions are likely portable to other Docker-capable appliances as well.With this I have been able to cache and seed 620 GB of SciOp data so far. Go do the same and help archive important scientific research data!cc <a href="https://fedihum.org/@SafeguardingResearch" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@SafeguardingResearch</a> cc <a href="https://neuromatch.social/@jonny" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@jonny</a> <a href="https://sfba.social/tags/resist" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#resist</a> <a href="https://sfba.social/tags/archive" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#archive</a> <a href="https://sfba.social/tags/sciOp" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#sciOp</a> <a href="https://sfba.social/tags/docker" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#docker</a> <a href="https://sfba.social/tags/truenas" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#truenas</a> <a href="https://sfba.social/tags/torrents" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#torrents</a> <a href="https://www.humancode.us/2025/05/24/guide-to-deluge-autobrr-truenas.html" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://www.humancode.us/2025/05/24/guide-to-deluge-autobrr-truenas.html</a>

jonny (good kind)we took an afternoon to add a quote randomizer to <a href="https://neuromatch.social/tags/sciop" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#sciop</a> because we all have too many heroes to honor. to me, this project is in loving memory to all our friend Aaron Swartz, who would have been holding up the gray archives right alongside us, and who would have understood the difference between ensuring all people have access to information by putting them directly in contact with it vs. selling the promise of information access by renting a product to them.

Henrik SchönemannHeya lovely people 👋 For <a href="https://fedihum.org/tags/SciOp" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SciOp</a> we (<a href="https://fedihum.org/@SafeguardingResearch" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@SafeguardingResearch</a>) are looking for dev contributions. There are a couple of bite-/morsel-sized tasks, e.g.: - Make Publisher independent table <a href="https://codeberg.org/Safeguarding/sciop/issues/341" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://codeberg.org/Safeguarding/sciop/issues/341</a> - Improve SearchableMixin to support full text search on joined items <a href="https://codeberg.org/Safeguarding/sciop/issues/182" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://codeberg.org/Safeguarding/sciop/issues/182</a> - and more: <a href="https://codeberg.org/Safeguarding/sciop/projects/17078" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://codeberg.org/Safeguarding/sciop/projects/17078</a>The stack is <a href="https://fedihum.org/tags/python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#python</a> (fastapi) + htmx, <a href="https://fedihum.org/tags/sqlite" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#sqlite</a> (sqlmodel), "low-fi, standards-compliant html and almost no js"Be a part of <a href="https://fedihum.org/tags/SafeguardingResearch" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SafeguardingResearch</a>, join us!

Moritz Negwer<a href="https://fedihum.org/@SafeguardingResearch" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@SafeguardingResearch</a> 18) Repeat until your harddisk is full. Thank you for keeping a tiny sliver of humanity's collective knowledge from disappearing forever! <a href="https://mstdn.science/tags/sciop" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#sciop</a> <a href="https://mstdn.science/tags/RogueArchivism" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#RogueArchivism</a> <a href="https://mstdn.science/tags/defendresearch" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#defendresearch</a>

Moritz NegwerThe outright wrecking of the scientific infrastructure in the US under the current administration chilled me to my bone. Kept me doomscrolling. But I finally found a way to do something positive, in a small way: I dedicated some storage space and a bit of my internet connection to a decentralized backup of deleted research data. I am happy to report that <a href="https://fedihum.org/@SafeguardingResearch" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@SafeguardingResearch</a> 's torrent swarm now has ca 14 TB storage and a Gbit connection more. <a href="https://mstdn.science/tags/sciop" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#sciop</a> <a href="https://mstdn.science/tags/safeguardingresearch" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#safeguardingresearch</a> <a href="https://mstdn.science/tags/academicchatter" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#academicchatter</a>

Henrik SchönemannIn 3h I will be doing the first of semi-regular office hours / Q&A for <a href="https://fedihum.org/@SafeguardingResearch" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@SafeguardingResearch</a> <a href="https://safeguar.de" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://safeguar.de</a> <a href="https://sciop.net" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://sciop.net</a>Feel free to drop in!More info: <a href="https://fedihum.org/@SafeguardingResearch/114507714151685422" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://fedihum.org/@SafeguardingResearch/114507714151685422</a> <a href="https://fedihum.org/tags/SafeguardingResearch" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SafeguardingResearch</a> <a href="https://fedihum.org/tags/SciOp" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SciOp</a>

Henrik SchönemannBei <a href="https://fedihum.org/tags/SciOp" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SciOp</a> gibt es das Verfassungsschutzgutachten zur Einstufung der AfD als gesichert rechtsextremistisch jetzt auch: <a href="https://sciop.net/datasets/bfv-report-afd" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://sciop.net/datasets/bfv-report-afd</a>Torrent über Internet Archive, inklusive LinksammlungUpdate: Neuer torrent; infohash ist gleich, aber mehr tracker hinzugefügt <a href="https://sciop.net/docs/intro/bittorrent/#trackers" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://sciop.net/docs/intro/bittorrent/#trackers</a><a href="https://fedihum.org/tags/AfD" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AfD</a> <a href="https://fedihum.org/tags/AfDGutachten" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AfDGutachten</a>

jonny (good kind)𝒹𝑜𝓃𝓀𝓈(we have <a href="https://sciop.net/docs/quickstart/" rel="nofollow noopener noreferrer" target="_blank">more</a> of <a href="https://codeberg.org/Safeguarding/sciop/pulls/371" rel="nofollow noopener noreferrer" target="_blank">them</a>)<a href="https://neuromatch.social/tags/Sciop" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Sciop</a> <a href="https://neuromatch.social/tags/WorldPremiere" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#WorldPremiere</a> <a href="https://neuromatch.social/tags/MusicVideo" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#MusicVideo</a> <a href="https://neuromatch.social/tags/LiveEvent" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LiveEvent</a> <a href="https://neuromatch.social/tags/AttendanceInTheBillions" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AttendanceInTheBillions</a> <a href="https://neuromatch.social/tags/UncountableAttendance" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#UncountableAttendance</a> <a href="https://neuromatch.social/tags/AQuantityOfDocsThatShamesTheHeavens" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AQuantityOfDocsThatShamesTheHeavens</a> <a href="https://neuromatch.social/tags/IfIcarusHadActuallyMadeItToSpaceInsteadOfMeltingThosePunkAssWingsItWouldBeLikeTheseDocs" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#IfIcarusHadActuallyMadeItToSpaceInsteadOfMeltingThosePunkAssWingsItWouldBeLikeTheseDocs</a> <a href="https://neuromatch.social/tags/ActuallyMostlyStillUnfinished" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ActuallyMostlyStillUnfinished</a> <a href="https://neuromatch.social/tags/NotThatBigOfAChange" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#NotThatBigOfAChange</a> <a href="https://neuromatch.social/tags/DontClickAnyOtherLinks" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#DontClickAnyOtherLinks</a> <a href="https://neuromatch.social/tags/IfYouCantTakeMeAtMyPartiallyDocumentedYouDontDeserveMeAtMyFullyDocumented" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#IfYouCantTakeMeAtMyPartiallyDocumentedYouDontDeserveMeAtMyFullyDocumented</a>

Henrik Schönemann<a href="https://www.spacebar.news/data-hoarding-more-important-than-ever/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://www.spacebar.news/data-hoarding-more-important-than-ever/</a> <a href="https://mastodon.social/@thespacebar" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@thespacebar</a> "The hobby of data hoarding might be most closely associated with saving illegally-downloaded movies on mountains of hard drives, but archiving public media and information has always been a component as well. Both parts are now critical to the survival of our shared culture and information base." "Patch the holes. Seed a torrent."Specificly mentioning <a href="https://fedihum.org/tags/SciOp" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SciOp</a> ✊ <a href="https://sciop.net" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://sciop.net</a> <a href="https://fedihum.org/tags/SafeguardingResearch" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SafeguardingResearch</a> <a href="https://fedihum.org/@SafeguardingResearch" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@SafeguardingResearch</a> h/t <a href="https://akademienl.social/@chartgerink" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@chartgerink</a>

Luca HammerMy connection isn't saturated yet, but <a href="https://social.luca.run/tags/sciop" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#sciop</a> uses significant portions of it. Good.Graph shows combined usage for all 33 containers and 2 VMs on that server.

Luca HammerI joined the <a href="https://social.luca.run/tags/safeguardingresearch" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#safeguardingresearch</a> <a href="https://social.luca.run/tags/sciop" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#sciop</a> torrent swarm to keep datasets from USA science institutions easily available.Setup a qbittorrent Docker <a href="https://docs.linuxserver.io/images/docker-qbittorrent/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://docs.linuxserver.io/images/docker-qbittorrent/</a> Added dataset feeds by threat level (I went with extinct, takedown_issued and endangered) <a href="https://sciop.net/feeds" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://sciop.net/feeds</a> Done

jonny (good kind)congratulations <a href="https://neuromatch.social/tags/sciop" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#sciop</a> and <a href="https://fedihum.org/@SafeguardingResearch" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@SafeguardingResearch</a> heads, in something like a month we became a little more than a quarter of uploads on academictorrents by volume.

MottGSciOp - a website mentioned in Hacker News to download or seed torrents for erased U.S. government website info or datasets.<a href="https://sciop.net/uploads/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://sciop.net/uploads/</a><a href="https://researchbuzz.masto.host/tags/research" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#research</a> <a href="https://researchbuzz.masto.host/tags/SciOp" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SciOp</a> <a href="https://researchbuzz.masto.host/tags/science" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#science</a>

petersuberUpdate. "<a href="https://fediscience.org/tags/SciOp" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SciOp</a> is part of Safeguarding Research & Culture (<a href="https://fediscience.org/tags/SRC" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SRC</a>). The bits must flow: let us resurrect the ancient art of <a href="https://fediscience.org/tags/Bittorrent" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Bittorrent</a> to ensure that our cultural, intellectual and scientific heritage exists in multiple copies, in multiple places, and that no single entity or group of entities can make it all disappear." <a href="https://sciop.net/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://sciop.net/</a> <a href="https://fediscience.org/tags/Censorship" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Censorship</a> <a href="https://fediscience.org/tags/DefendResearch" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#DefendResearch</a> <a href="https://fediscience.org/tags/OpenSource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#OpenSource</a> <a href="https://fediscience.org/tags/Preservation" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Preservation</a> <a href="https://fediscience.org/tags/Takedowns" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Takedowns</a> <a href="https://fediscience.org/tags/Trump" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Trump</a> <a href="https://fediscience.org/tags/USPol" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#USPol</a> <a href="https://fediscience.org/tags/USPolitics" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#USPolitics</a>

jonny (good kind)<a href="https://fedihum.org/@SafeguardingResearch" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@SafeguardingResearch</a> <a href="https://neuromatch.social/tags/Sciop" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Sciop</a> roughly doubled in size today from one set of uploads, public beta opened a week ago. Let's go, time to boot up the dusty under desk archive armada.Graph from ww (cant find ur @ )

jonny (good kind)hehe <a href="https://neuromatch.social/tags/sciop" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#sciop</a> <a href="https://neuromatch.social/tags/has" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#has</a> <a href="https://neuromatch.social/tags/achieved" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#achieved</a> <a href="https://neuromatch.social/tags/bittorrent" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#bittorrent</a> <a href="https://neuromatch.social/tags/sentience" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#sentience</a>

Henrik SchönemannYou have heard of <a href="https://fedihum.org/tags/SafeguardingResearch" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SafeguardingResearch</a> and know a thing or two about coding / technical writing?We could use your help with <a href="https://fedihum.org/tags/SciOp" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SciOp</a> <a href="https://sciop.net" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://sciop.net</a> ; see issues here: <a href="https://codeberg.org/Safeguarding/sciop/issues" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://codeberg.org/Safeguarding/sciop/issues</a> the label "help wanted" means "stuff that might be nice for someone to claim as a project and is relatively self-contained", but all issues would benefit :)cc <a href="https://fedihum.org/@SafeguardingResearch" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@SafeguardingResearch</a>

Recent searches

Search options

Administered by:

Server stats:

#sciop