Inverting the Web
We use search engines because the Web does not support accessing documents by anything other than URL. This puts a huge amount of control in the hands of the search engine company and those who control the DNS hierarchy.
Given that search engine companies can barely keep up with the constant barrage of attacks, commonly known as "SEO". intended to lower the quality of their results, a distributed inverted index seems like it would be impossible to build.
@freakazoid Shifting ground (and jumping back up this stack -- we've sorted the URL/URI bit):
What you suggest that's interesting to me is the notion of _self-description_ or _self-identity_ as an inherent document characteristic.
(Where a "document" is any fixed bag'o'bits: text, audio, image, video, data, code, binary, etc.)
Not metadata (name, path, URI).
*Maybe* a hash, though that's fragile.
What is _constant_ across formats?
@freakazoid So, for example:
I find a scanned-in book at the Internet Archive, I re-type the document myself (probably with typos) to create a Markdown source, and then generate PDF, ePub, and HTML formats.
What's the constant across these?
How could I, preferably programmatically, identify these as being the same, or at least, highly-related, documents?
MD5 / SHA-512 checksums will identify _files_, but not _relations between them_.
Can those relations be internalised intrinsically?
@freakazoid Or do you always have to maintain some external correspondence index which tells you that SOURCE.PDF was the basis for RETYPED.MD which then generated RETYPED.MD.ePub and RETYPED.MD.html, etc.
Something that will work across printed, re-typed, error/noise, whitespace variants. Maybe translations or worse.
Word vectors? A Makefile audit? Merkel trees, somehow?
@dredmorbius We have real world solutions for these problems in the form of notaries, court clerks, etc. I.e. (registered) witnesses. Trusted third parties, but they don't have to be a single party.
@dredmorbius In the RDF world I guess one doesn't sign the individual triple but the entire graph.
And it might make more sense to call these 4-tuples, because it's really "this person says that this object is related in this way to this other object".
@freakazoid So for 4-tuple:
4. Obect1-Object-2 relation
"Signed" means that the whole statement is then cryptographically signed, making it an authenticatable statement?
@freakazoid And, so:
Back to search and Web:
- The actual URL and path matter to the browser.
- They may matter to me. Some RoboSpam site ripping off my blog posts _might_ leave the content unchanged, but they're still scamming web traffic, ads revenue, or reputation, based on false pretences. I want to read my content from my blog, not SpamSite, even if text and hashes match.
Ad revenue is basically a way to use the web's (accidental) dynamicism as a monetization strategy. If monetization were based on permission to access, you'd save on hosting costs if you *only* gave permission & whoever happened to be around did the hosting (like serving password-protected items off bittorrent and selling the passwords).
An ISP startup I worked for back in '96 (InterNex, later acquired by Concentric which renamed itself to XO Communications using one of Internex's domains for customers) tried to make something like this. It was essentially DRM for arbitrary content that used a .exe wrapper that contacted a license server. I don't think they ever managed to even bring it to makret.
@freakazoid @enkiv2 @dredmorbius
Right, I'm imagining a world without piracy. (It turns out that if it's easier to pay, the first world will generally just pay, and piracy becomes limited to folks who wouldn't pay anyway.) What I'm describing is xanadu 'transcopyright' though -- but transcopyright in xusp, xsp, oxu, & xuc is based on one time pads for subdivision reasons so it doesn't save you any bytes.
@enkiv2 @dredmorbius If it's easy to pay people will pay, but then there's also a strong encouragement to put stuff that would otherwise have been free behind paywalls, like we see in app stores. I don't think "no piracy" is the goal we should be looking for. It's maximum value for humanity from creativity.
@enkiv2 @dredmorbius Or to put it another way my goal is not to make sure that people pay to consume content but to make it so that people can make awesome stuff. A fixed payment per person or per use is about the crudest way I can think of to accomplish that. If anything it dramatically limits the utility of creativity, because even though it's nearly costless for additional people to benefit from it, unless they can or will pay the fixed price, they get nothing.
@enkiv2 @dredmorbius Likewise, there's a barrier to paying *more*. Especially since payment happens up front, before the payer has any idea what utility they will derive from the content. Far better to pay after the fact on a sliding scale. Sure, some will exploit that, and I think our aversion for that is what makes us accept such a shitty solution to begin with. But I think creators would get far more with such a model, especially since it helps eliminate middlemen
@humanetech @alcinnz @enkiv2 @dredmorbius I think the way to make sure creators earn something is to have something like a UBI, or otherwise make it so that one doesn't have to earn anything to live a dignified, healthy, happy, productive life. I think the only minimum price that makes sense is zero, because a) that's the cost of an additional copy, and b) there are a huge number of people who will benefit from a work who can't pay.
@humanetech @alcinnz @enkiv2 @dredmorbius Remember, the value to humanity of a creative work comes from its consumption. The value to any given individual is the difference between the value to them of consuming the work and the value of what they have to pay for it. The reason we enable creators to capture some of the value they produce is to incentivize them to create more. But we want them to create works with the most value to others.
@humanetech @alcinnz @enkiv2 @dredmorbius We also want people to create derivative works, and we want works that encourage derivation, because that multiplies their value. Excessive financial incentives derived from limiting access tend to reduce the amount of derivation by others, and it can cause excessive derivation by the creator who owns the original work in an effort to extract maximum value with minimum effort.
@humanetech @alcinnz @enkiv2 @dredmorbius So I think the optimal scenario is to create a culture of paying for creative works not based on the value one individually derives from them, but the value one feels humanity derives from them. And of paying at whatever point in time they can, not just right before or right after consuming it. It can be like tithing to the church, where the church is a global decentralized Patreon.
@woozle @humanetech @alcinnz @enkiv2 @dredmorbius Unfortunately "forced charity" tends to make people ungenerous, with the result that public goods tend to get underfunded. It also crowds out voluntary charity due to what I have been calling the "I gave at the office" effect.
There is also the issue that in a democracy only popular things get funded unless there is a very strong culture of experimentation and openness.
Generosity isn't an issue when the contribution amount is scaled to each individual's surplus income.
...and I suspect that what the *majority* want would definitely include artistic endeavors, as long as basic needs were also being met.
From what I can tell, voluntary charity is all but useless, so no harm if it is supplanted by a more robust system.
@woozle @humanetech @alcinnz @enkiv2 @dredmorbius I don't mean generosity in the form of how much people contribute, but in what they vote to fund and how much, and what they choose to fund outside of their taxes. I don't think having the government decide what creative works get funding is a great idea. Or even ignoring the question of government, any system by which things only get funded if the majority decide it should.
Well, we disagree there. In making collective policy decisions, the only alternative to government (in the broadest sense of the word) that I'm aware of is markets, and we know where that leads.
Mind you, government itself needs to be restructured from the ground up before it could be trusted with anything important.
@woozle @humanetech @alcinnz @enkiv2 @dredmorbius Governments and markets suffer from the same kinds of problems. There's a whole field of study around that called public choice theory. Capital and power are just measured differently when you're talking about government.
Another way to put it is that you're always dealing with a market of some kind, so the question is really just how you try to structure the market.
@Jens @humanetech @dredmorbius @enkiv2 @alcinnz That's a really good point. I hadn't really been thinking about the value to the creator themselves of having others use their work. It especially gives an interesting perspective on Hollywood and the media since they have a lot of influence on our culture and politics directly through the works they produce.
Server run by the main developers of the project It is not focused on any particular niche interest - everyone is welcome as long as you follow our code of conduct!