Follow

Okay, hypothetically, if somebody was looking for a name for a Free Software project that focuses on Web censorship circumvention, using multiple configurable non-standard in-browser delivery mechanisms, what would a good name be?

Asking for a friend.

· · Web · 13 · 8 · 9

Oh, is this related to my session at MozFest in 3 weeks, you ask?

Yes. Yes it is.

Well will you look at this! My session has a date and a time now: public.zenkit.com/i/2RH604FcHf

Is this related to the Web censorship circumvention project I mentioned before? Why yes, it is indeed, nice of you to ask!

Ok, so we now have a shortlist of contenders for the new name of our little browser-based censorship circumvention project.

Let's vote!

The vote will not be necessarily binding, but I want to see if my feel about these names is correct.

Thank you all for helping out and suggesting the names! Special shout-outs to @pelican3301 , @Shufei , @mjjzf and others who suggested some names.

Ok, been hacking on this thing for about 3h now, making good progress; I deserve a reward.

Swimming pool tiem.

Basically the whole day of hacking on the little decentralization project today. Good progress. This thing is becoming surprisingly usable.

I will announce the new name (it's going to either be SideWays or Samizdat, most probably) early next week, and full code will follow shortly.

Haven't had a project I was that excited about for a long while. I missed that feeling a lot!

Well that went fast! Rename complete, code a bit cleaned, stuff tested.

I give you Saмiзdaт:
git.occrp.org/libre/samizdat

Example deployment:
cdn.test.occrp.org/projects/sa

Go on, test in modern Firefox, Chromium, or Chrome.

There are g-glitches every now and then, but basically once the ServiceWorker kicks in you should see a "YES" there on the example page. When you do, you should also notice the favicon appearing. That means IPFS is working; there is no favicon on that server!

I need to add a *lot* of documentation, but in the meantime a short primer:

- Samizdat uses a ServiceWorker to handle all requests; in case there's an error, the ServiceWorker attempts to fetch the content using available plugins.
- currently there is only one plugin implemented; it uses Gun to resolve paths into IPFS addresses, and then fetches the content from IPFS
- there are two ways to push content to IPFS and update Gun addresses: server-side (in a CI/CD pipeline), or client-side

Pushing new content server-side uses an ipfs daemon, a Gun daemon, and samizdat-cli. Gun user credentials are configured in and read from the CI/CD environment. Example:
git.occrp.org/libre/samizdat/-

After pushing content to IPFS and new addresses to Gun, the pipeline *verifies* they got actually pushed out (not just locally cached). This also pre-loads the content on one of the large IPFS gateways, making it more immediately available to clients.

Not every website admin has access to CI/CD pipelines, and ability to run IPFS and Gun daemons, though!

So there is a way to push content directly from the browser:
git.occrp.org/libre/samizdat/b

This:
1. gets a list of all resources loaded from the same domain
2. fetches them and add them to IPFS, *verifying* they become available via a major gateway
3. pushes the new IPFS addresses for these resources to Gun, *verifying* this update propagated

You only need a Gun username and password.

Samizdat is also not beholden to a particular technology. We want to implement an IPNS+IPFS plugin, and a dat:// plugin at some point. And BitTorrent, why not!

Other kinds of plugins are also on the radar and possible. For example, a plugin that fetches the content from the Wayback Machine. Or from a static dump of the site somewhere on CloudFront.

Eventually, plugins will be chainable, and there will be a way to configure simple rules: "try Wayback first, and if it fails then hit Gun+IPFS"

Please note, Samizdat is a Proof-of-Concept currently. It *seems* to work, but it might also eat your dog, even if you don't have one!

We need to make it easier to deploy and easier to manage for website admins, and easier to understand what's going on for regular users ("This site seems blocked, but we're using magic to get it for you; it might take longer than expected though!").

Finally, yes we will move the code to some place more accessible (GitLab.com, most probably).

I would also like to make it more clear what Samizdat is, what it does, and how. If anyone wants to help with that, I'm all ears.

I'm in my own bubble, and something that seems clear to me might reasonably be expected to seem like a random string of buzzwords to most! :)

And yes, if you loaded the example site, I would love to hear if it worked for you, what browser you're using, etc.

For you Lynx users, I see you, you magnificent sages!

Samizdat will not work for non-JS enabled browsers, sadly. I'm not a fan of JS, and I'm not a fan of the so-called modern Web, but there's no way I see to attack the problem of Web censorship other than using the tools users already have.

We get ~400 hits daily on our Tor Hidden Service. It works, but not on a scale of nations...

@kaniini Ah! Dangm there already was a Telecomix project named Streisand in the same topic space: en.wikipedia.org/wiki/Telecomi

But StreisandCDN would perhaps work. Thanks!

@mjjzf oh, I like "SideWays" (as in, side ways of delivering content).

Also has a nice ring to it when describing how the whole censorship thing would suddenly go "sideways". ;)

@rysiek
> The vote will not be necessarily binding
See Boaty McBoatface for details...

@pelican3301 @Shufei @mjjzf

@rysiek Hemlock
Wycliffe
Samizdat
Subvert
VoxPop
Barlow
Rosnec
Gadfly
Odysseus
Horse
Zenger
Sunlight
Lux

@dredmorbius *now* you throw these at me! ;)

Samizdat is definitely a great name, damn! I wish I had thought of it myself! Will consider it (but won't republish the poll now).

Thanks!

@dredmorbius in fact, "samizdat" is even better if we decide to implement dat:// protocol as one of the plugins!

samizdat:// -> I like it very much.

@rysiek It gets around censorship right? Soooo.... Censoround?

@rysiek Testing on current FF on linux. Serviceworker loads (although on first load the ipfs dependency didn't load properly, might be my bad connection). But then the web socket connection to wss://nyc-2.bootstrap.libp2p.io/ipfs/ always gives 502 errors

@norwin that wss:// connection should not matter, in the background a number of connections should get established by JS-IPFS.

Good to hear about the ServiceWorker! Did you get a "YES" for the ServiceWorker question?

Did you get a commit ID for the ServiceWorker in the footer?

Did the favicon load?

/me bites his nails, in anticipation

@norwin Interesting! So ServiceWorker loads fine, but Gun or IPFS do not kick in.

Which browser? Chrome/Chromium handle 404 on favicon.ico weird when it happens the first time (so, before ServiceWorker loads) - they seem to cache the failure and not try again when SW is active. Might work tomorrow when internal cache expires or something.

Can you go to this link and tell me if you see the paths and IPFS hashes under "Gun user published:"?
cdn.test.occrp.org/projects/sa

@rysiek Now the favicon appeared, not sure what changed. Previously there was empty space for a favicon, that quickly got cleared on page load. Now it's there 🎉

I notice full utilization of one CPU core though when the page is loaded

On the debug page, all the paths & hashes are listed.

@norwin favicon -- aw yiss!!

CPU utilization: interesting. Doesn't seem to be CPU-heavy here, or maybe I did not notice. optimizations are probably needed, but also: JS-IPFS is alpha, as far as I understand, so I would expect a lot of optimizing going on there at some point. But good to know I need to look at that!

Debug page: great. that means Gun is working as expected. But we know that already with the favicon.

@rysiek wow, blocking the domain via /etc/hosts makes it feel much more impressive. great work!

@norwin thank you!

In fact, I am spending way too much time lately just blocking the domain and reloading the page over and over again, watching the console log scroll by. Very satisfying. ;)

@rysiek

Samizdat is self-published material secretly copied and passed hand-to-hand clandestinely. Sneakernet.

I copied and distributed samizdat of Medvedev when I was in high school.

en.wikipedia.org/wiki/Samizdat

@hhardy01 yes, that's precisely why we chose this name for the project that I've been rambling about in this thread. ;)

@hhardy01 ah, interesting idea. I would need to find an nntp JS library that works in a browser (all I've seen in a quick search right now are node.js libs).

But if there's a way to get it in a browser, totally doable!

@rysiek Woah. This stuff is hot baked news to lil old me. Congrats on this project!

What all do we exactly need client side to get this running?

@Shufei Thank you!

All you need is a modern browser. The whole idea is that a website user should not be required to have anything else.

Gun and IPFS have JS implementations, and that's what we're using.

Ideally, after you visit a Samizdat-enabled site *once*, and ServiceWorker gets cached and activated, any time you try to access the site it loads - even if it's blocked. Samizdat intercepts errors and pulls content from IPFS if HTTPS fails.

@rysiek Righto, interesting, I read your API sketch and am getting a sense of it. Never heard of any of these gizmos before. So far nyet on WebKit. I’ll tweak some things.

@Shufei "nyet on WebKit" meaning it didn't work, no ServiceWorker?

ServiceWorkers do not work in private/incognito/christmass shopping/porn mode. Happy to help debug though!

there should be a LOT of output in the console too!

@rysiek Haha, I suppose I have mine in “porn mode” most of the time. But yes, I tried several iOS variants (all walled garden) with and without js, and with Tor, and no dice on either service worker or gun. Loading up nicely, but not running. I’ll fire up my Linux jalopy and see what the Quasimodo web beasts I abuse in that dungeon say.

@Shufei I have not tested this at all on iOS devices. Currently only tested on Firefox, Chromium/Chrome on GNU/Linux, and on Firefox mobile on Android.

I wonder what the issue with iOS is, but at this stage I have no way of testing there.

@Shufei plus:

1. ServiceWorkers are weird, and you might need to close the tab and open it again for it to kick in and get activated.

2. Gun+IPFS will *only* kick in in case of errors. So you will get a "YES" there if `cdn.test.occrp.org` is blocked (I test it by blocking it in /etc/hosts on my machine, for example).

But I appreciate you testing stuff out!

@rysiek Righto, I’ll try to get you something more cogent. Nope, the SW is just twiddling thumbs. But in my admittedly inexpert gut, that points to an issue. iOS safari, etc. Silo tabs much more severely than even chrome, if memory serves. Dumping tabs may just kill the ServWorker...?

@Shufei killing a tab should not kill the ServiceWorker. In fact, browsers are expected to cache SWs and run them automagically next time you go to a site that uses one.

On Firefox it seems to help to load the page, then kill the tab, and then load the page in a new tab - and boom, SW kicks in, like magic.

I hope I can track this down and fix it soon.

@rysiek Nor I at the mome, with helpful intelligence at least...

It’s worth putting on the notes, I reckon. Many of your likely patrons in (insert country between Korea and Afghanistan) will likely use iOS, natch.

@Shufei oh absolutely, and in the long run this will be iOS-compatible. We're just not there yet.

@rysiek @Shufei on Firefox on Android, it said Service Workers no the first time but turned green after reloading.

@mayel @Shufei that's expected. ServiceWorker needs to be downloaded and cached first, so the first ever load of an SW-enabled site will never go through the SW.

Hopefully every next one will.

@rysiek My turn:
1) I have no idea what it is supposed to do.
2) Firefox 69 on Linux
3) I get yes at ServiceWorker, but it takes a while to show up.
4) I got favicon.
5) I have IPFS companion installed and configured to use IPFS node running on another machine. I don't know if it was used, though.

@etam
1. the site is just a preview. the library is meant to be deployed by website admins thus making the sites way harder to block, without requiring users to have anything more than a browser
2. ack
3. yup.
4. great; Gun+IPFS worked. :)
5. no, it probably wasn't, as far as I understand how JS-IPFS works.

Thanks for testing!

@rysiek
4. But it shows "Gun and IPFS used? NO"

@etam yeah, it's a bit confusing. that's an indicator just for index.html. the favicon had to be pulled using Gun+IPFS, since there is no favicon on cdn.test.occrp.org.

I have some plans to make the status display more verbose and clear. in fact, can't wait to start hacking on that.

you can block 'cdn.test.occrp.org` in /etc/hosts and reload the page. you should get "Gun and IPFS used? YES" then.

@rysiek After blocking cdn.test.occrp.org I got both YES. Still I don't know how to check if my IPFS node was used or not.

@rysiek github.com/ipfs-shipyard/ipfs- says:
"Your IPFS node is exposed as window.ipfs on every webpage. Websites can detect if window.ipfs exists and opt-in to use it instead of creating their own js-ipfs node."
See also github.com/ipfs-shipyard/ipfs-

Sign in to participate in the conversation
Mastodon

Server run by the main developers of the project 🐘 It is not focused on any particular niche interest - everyone is welcome as long as you follow our code of conduct!

<svg xmlns="http://www.w3.org/2000/svg"><symbol id="mastodon-svg-logo" viewBox="0 0 216.4144 232.00976"><path d="M107.86523 0C78.203984.2425 49.672422 3.4535937 33.044922 11.089844c0 0-32.97656262 14.752031-32.97656262 65.082031 0 11.525-.224375 25.306175.140625 39.919925 1.19750002 49.22 9.02375002 97.72843 54.53124962 109.77343 20.9825 5.55375 38.99711 6.71547 53.505856 5.91797 26.31125-1.45875 41.08203-9.38867 41.08203-9.38867l-.86914-19.08984s-18.80171 5.92758-39.91796 5.20508c-20.921254-.7175-43.006879-2.25516-46.390629-27.94141-.3125-2.25625-.46875-4.66938-.46875-7.20313 0 0 20.536953 5.0204 46.564449 6.21289 15.915.73001 30.8393-.93343 45.99805-2.74218 29.07-3.47125 54.38125-21.3818 57.5625-37.74805 5.0125-25.78125 4.59961-62.916015 4.59961-62.916015 0-50.33-32.97461-65.082031-32.97461-65.082031C166.80539 3.4535938 138.255.2425 108.59375 0h-.72852zM74.296875 39.326172c12.355 0 21.710234 4.749297 27.896485 14.248047l6.01367 10.080078 6.01563-10.080078c6.185-9.49875 15.54023-14.248047 27.89648-14.248047 10.6775 0 19.28156 3.753672 25.85156 11.076172 6.36875 7.3225 9.53907 17.218828 9.53907 29.673828v60.941408h-24.14454V81.869141c0-12.46875-5.24453-18.798829-15.73828-18.798829-11.6025 0-17.41797 7.508516-17.41797 22.353516v32.375002H96.207031V85.423828c0-14.845-5.815468-22.353515-17.417969-22.353516-10.49375 0-15.740234 6.330079-15.740234 18.798829v59.148439H38.904297V80.076172c0-12.455 3.171016-22.351328 9.541015-29.673828 6.568751-7.3225 15.172813-11.076172 25.851563-11.076172z" /></symbol></svg> <svg xmlns="http://www.w3.org/2000/svg"><symbol id="mastodon-svg-logo-full" viewBox="0 0 713.35878 175.8678"><path d="M160.55476 105.43125c-2.4125 12.40625-21.5975 25.9825-43.63375 28.61375-11.49125 1.3725-22.80375 2.63125-34.8675 2.07875-19.73-.90375-35.2975-4.71-35.2975-4.71 0 1.92125.11875 3.75.355 5.46 2.565 19.47 19.3075 20.6375 35.16625 21.18125 16.00625.5475 30.2575-3.9475 30.2575-3.9475l.65875 14.4725s-11.19625 6.01125-31.14 7.11625c-10.99875.605-24.65375-.27625-40.56-4.485C6.99851 162.08 1.06601 125.31.15851 88-.11899 76.9225.05226 66.47625.05226 57.74125c0-38.1525 24.99625-49.335 24.99625-49.335C37.65226 2.6175 59.27976.18375 81.76351 0h.5525c22.48375.18375 44.125 2.6175 56.72875 8.40625 0 0 24.99625 11.1825 24.99625 49.335 0 0 .3125 28.1475-3.48625 47.69" fill="#3088d4"/><path d="M34.65751 48.494c0-5.55375 4.5025-10.055 10.055-10.055 5.55375 0 10.055 4.50125 10.055 10.055 0 5.5525-4.50125 10.055-10.055 10.055-5.5525 0-10.055-4.5025-10.055-10.055M178.86476 60.69975v46.195h-18.30125v-44.8375c0-9.4525-3.9775-14.24875-11.9325-14.24875-8.79375 0-13.2025 5.69125-13.2025 16.94375V89.2935h-18.19375V64.75225c0-11.2525-4.40875-16.94375-13.2025-16.94375-7.955 0-11.9325 4.79625-11.9325 14.24875v44.8375H73.79851v-46.195c0-9.44125 2.40375-16.94375 7.2325-22.495 4.98-5.55 11.50125-8.395 19.595-8.395 9.36625 0 16.45875 3.59875 21.14625 10.79875l4.56 7.6425 4.55875-7.6425c4.68875-7.2 11.78-10.79875 21.1475-10.79875 8.09375 0 14.61375 2.845 19.59375 8.395 4.82875 5.55125 7.2325 13.05375 7.2325 22.495M241.91276 83.663625c3.77625-3.99 5.595-9.015 5.595-15.075 0-6.06-1.81875-11.085-5.595-14.9275-3.63625-3.99125-8.25375-5.91125-13.84875-5.91125-5.59625 0-10.2125 1.92-13.84875 5.91125-3.6375 3.8425-5.45625 8.8675-5.45625 14.9275 0 6.06 1.81875 11.085 5.45625 15.075 3.63625 3.8425 8.2525 5.76375 13.84875 5.76375 5.595 0 10.2125-1.92125 13.84875-5.76375m5.595-52.025h18.04625v73.9h-18.04625v-8.72125c-5.455 7.2425-13.01 10.79-22.80125 10.79-9.3725 0-17.34625-3.695-24.06125-11.23375-6.57375-7.5375-9.93125-16.84875-9.93125-27.785 0-10.78875 3.3575-20.10125 9.93125-27.63875 6.715-7.5375 14.68875-11.38 24.06125-11.38 9.79125 0 17.34625 3.5475 22.80125 10.78875v-8.72zM326.26951 67.258625c5.315 3.99 7.97375 9.60625 7.83375 16.7 0 7.53875-2.65875 13.45-8.11375 17.58875-5.45625 3.99125-12.03 6.06-20.00375 6.06-14.40875 0-24.20125-5.9125-29.3775-17.58875l15.66875-9.31c2.0975 6.35375 6.71375 9.60625 13.70875 9.60625 6.43375 0 9.6525-2.07 9.6525-6.35625 0-3.10375-4.1975-5.91125-12.73-8.1275-3.21875-.8875-5.87625-1.77375-7.97375-2.51375-2.9375-1.18125-5.455-2.5125-7.55375-4.1375-5.17625-3.99-7.83375-9.3125-7.83375-16.11 0-7.2425 2.5175-13.00625 7.55375-17.145 5.17625-4.28625 11.47-6.355 19.025-6.355 12.03 0 20.84375 5.1725 26.5775 15.66625l-15.38625 8.8675c-2.23875-5.02375-6.015-7.53625-11.19125-7.53625-5.45625 0-8.11375 2.06875-8.11375 6.05875 0 3.10375 4.19625 5.91125 12.73 8.12875 6.575 1.4775 11.75 3.695 15.5275 6.50375M383.626635 49.966125h-15.8075v30.7425c0 3.695 1.4 5.91125 4.0575 6.945 1.95875.74 5.875.8875 11.75.59125v17.29375c-12.16875 1.4775-20.9825.295-26.15875-3.69625-5.175-3.8425-7.69375-10.93625-7.69375-21.13375v-30.7425h-12.17v-18.3275h12.17v-14.9275l18.045-5.76375v20.69125h15.8075v18.3275zM441.124885 83.2205c3.6375-3.84375 5.455-8.72125 5.455-14.6325 0-5.91125-1.8175-10.78875-5.455-14.63125-3.6375-3.84375-8.11375-5.76375-13.57-5.76375-5.455 0-9.93125 1.92-13.56875 5.76375-3.4975 3.99-5.31625 8.8675-5.31625 14.63125 0 5.765 1.81875 10.6425 5.31625 14.6325 3.6375 3.8425 8.11375 5.76375 13.56875 5.76375 5.45625 0 9.9325-1.92125 13.57-5.76375m-39.86875 13.15375c-7.13375-7.5375-10.63125-16.70125-10.63125-27.78625 0-10.9375 3.4975-20.1 10.63125-27.6375 7.13375-7.5375 15.9475-11.38 26.29875-11.38 10.3525 0 19.165 3.8425 26.3 11.38 7.135 7.5375 10.77125 16.84875 10.77125 27.6375 0 10.9375-3.63625 20.24875-10.77125 27.78625-7.135 7.53875-15.8075 11.2325-26.3 11.2325-10.49125 0-19.165-3.69375-26.29875-11.2325M524.92126 83.663625c3.6375-3.99 5.455-9.015 5.455-15.075 0-6.06-1.8175-11.085-5.455-14.9275-3.63625-3.99125-8.25375-5.91125-13.84875-5.91125-5.59625 0-10.2125 1.92-13.98875 5.91125-3.63625 3.8425-5.45625 8.8675-5.45625 14.9275 0 6.06 1.82 11.085 5.45625 15.075 3.77625 3.8425 8.5325 5.76375 13.98875 5.76375 5.595 0 10.2125-1.92125 13.84875-5.76375m5.455-81.585h18.04625v103.46h-18.04625v-8.72125c-5.315 7.2425-12.87 10.79-22.66125 10.79-9.3725 0-17.485-3.695-24.2-11.23375-6.575-7.5375-9.9325-16.84875-9.9325-27.785 0-10.78875 3.3575-20.10125 9.9325-27.63875 6.715-7.5375 14.8275-11.38 24.2-11.38 9.79125 0 17.34625 3.5475 22.66125 10.78875v-38.28zM611.79626 83.2205c3.63625-3.84375 5.455-8.72125 5.455-14.6325 0-5.91125-1.81875-10.78875-5.455-14.63125-3.6375-3.84375-8.11375-5.76375-13.57-5.76375-5.455 0-9.9325 1.92-13.56875 5.76375-3.49875 3.99-5.31625 8.8675-5.31625 14.63125 0 5.765 1.8175 10.6425 5.31625 14.6325 3.63625 3.8425 8.11375 5.76375 13.56875 5.76375 5.45625 0 9.9325-1.92125 13.57-5.76375m-39.86875 13.15375c-7.135-7.5375-10.63125-16.70125-10.63125-27.78625 0-10.9375 3.49625-20.1 10.63125-27.6375 7.135-7.5375 15.9475-11.38 26.29875-11.38 10.3525 0 19.165 3.8425 26.3 11.38 7.135 7.5375 10.77125 16.84875 10.77125 27.6375 0 10.9375-3.63625 20.24875-10.77125 27.78625-7.135 7.53875-15.8075 11.2325-26.3 11.2325-10.49125 0-19.16375-3.69375-26.29875-11.2325M713.35876 60.163875v45.37375h-18.04625v-43.00875c0-4.8775-1.25875-8.5725-3.77625-11.38-2.37875-2.5125-5.73625-3.84375-10.0725-3.84375-10.2125 0-15.3875 6.06-15.3875 18.3275v39.905h-18.04625v-73.89875h18.04625v8.27625c4.33625-6.94625 11.19-10.345 20.84375-10.345 7.69375 0 13.98875 2.66 18.885 8.12875 5.035 5.46875 7.55375 12.85875 7.55375 22.465"/></symbol></svg>