ugh that sounds nasty. I glanced once at the site generator landscape and ran away screaming. I was hoping maybe something usable had emerged but... at least this one doesn't seem to be it.

@natecull @teleclimber silly question, but do you have preferences for config/markup/template/query/data/metadata formats?


@natecull @teleclimber plenty of opportunity to fuck up outside of choices for those, but i am curious , some people get pretty fussy about it.

@zensaiyuki @teleclimber

I don't actually much like any current markup/query/config formats, which is why I've been trying to brew my own (slightly extended Lisp S-expressions).

My problem with most markup formats is that they start out thinking 'oh it's okay, don't sweat the syntax, nobody will ever need to do $BIG_THING in it' and always, always, sometimes within six months, they have to do $BIG_THING in it.

Every markup format becomes a Turing-complete programming language. Law of nature.

@zensaiyuki @teleclimber

And even if the markup format ITSELF doesn't become a Turing-complete programming language, some poor soul will find themselves forced to encode a Turing-complete programming language OVER it.

And some syntax choices make that task pleasant, and some syntax choices make that task extremely unpleasant.

SGML/XML is an example of many wrong choices made at the syntax level. The proof is that if it hadn't made wrong choices, we'd be writing Javascript just as HTML tags.

@zensaiyuki @teleclimber

And then I'm looking at, eg, Wordpress Shortcodes, and Tiddlywiki Filter notation, and I go "Stop! Stop! You're writing Turing-complete query functions over an ill-defined markup language that never imagined you'd want to do this, but you do, because of course you do, so you just threw another layer of syntax over the top, and of course it's all going to go terribly wrong."

@natecull @teleclimber if it’s inevitable maybe there’s a way to design to expect it up front, but also somehow avoid requiring it tor the simple stuff. like a well designed escape hatch.

@natecull @teleclimber i guess you’re leaning toward the lispy approach. my thinking is along the lines of relying heavily in simple well understood formats and tools.

@zensaiyuki @teleclimber

But what do you do when your simple well understood format just can't quite cope with what you need to do?

Do you roll your own format at that point? Or do you try to look for another format that can cope?

@zensaiyuki @teleclimber

I mean, you can't get more simple than CSV, right?

Well, CSV doesn't work for some things. What if you have fields inside fields?

Well then JSON, right?

Ok but JSON doesn't work for some things. What if you need keys to be objects?

(at that point I guess we can look at microformats or protocols on top of JSON, because good or bad we're probably stuck with it for a century now, so that's sorta where my thinking is)

@natecull @teleclimber are we talking about statically generating html at the end of the process or something to replace html and browser tech? i agree that there’s annoying topological inconsistencies in the syntaxes of html/json/xml/csv. but in my experience, it’s inconsistencies in vocabularies and schemas that end up being more of a pain in the ass than the particular way the data is stored.

@natecull @teleclimber title, description,keywords are the only things i’ve learned i can rely on. everything else, even id, becomes dust.

@natecull @teleclimber that really rather depends on what the big complicated problem is. a concrete example I can think of off the top of my head is a hairy multi-page form with complex dependencies and nesting structures, like say, a section that requires the addresses of multiple people, with the address fields depending on which country and state is selected for each.

at that point a new language is probably justified, along with a simple way to integrate /link it with the existing stuff.

@natecull @teleclimber but it is tempting to at this point burn it all down and replace everything with the sophisticated new language that can do it all. that’s sorta what the react craze is about- “let’s use the steamroller for everything”.

@zensaiyuki @teleclimber

The big complicated problem, for me, is "integrate and connect data from *all the other data formats on my desktop* without loss of data." Because we have a big social problem if we can't connect up all our knowledge because we don't have a language that can express *connections between* knowledge.

At least that's where I want to be.

So what I want is a syntax that could encode at a minimum, eg, the semantics of all of: HTML, XML, JSON, CSV, SQL, RDF, Javascript.

@zensaiyuki @teleclimber

I guess I'm not really thinking about a site generator at this point but about "what went wrong with HTML, Javascript and PHP such that the web stack fractured into multiple languages for each layer of the stack".

I think the answer is more that "nothing went right in the first place" rather than "something went wrong". All these separate languages came from different communities and just got wodged together, which is okay, but also, kinda sucks now.

@zensaiyuki @teleclimber

I can see Node.js eating the server and, possibly, Javascript eating the browser (like, couldn't we just send JSON documents instead of HTML? But encoding Javascript functions into JSON is still very very sucky.) Then maybe WebAssembly might eat Javascript and allow us to use a slightly better encoding syntax instead of Javascript/JSON? I dunno exactly where WebAssembly is going though to be honest. It might be falling into binary/security hell and stuck there.

@natecull @teleclimber web assembly is useful but fundamentally it’s kinda just flash/java/silverlight/unity all over again. webassembly it’s way more secure than those, but sooner or later some client is going to ask me to make a critical change to a webassembly thing and all I’ll be able to do is shrug at them. in fact, this has already happened.

@natecull @teleclimber the conversation is always like “well, do you have the source code?”

“what’s source code?”

@zensaiyuki @teleclimber

ugh :(((

Who's making these WebAssembly modules? What are they being written in?

@zensaiyuki @teleclimber

I suppose my Utopian hope was that someone might make a WebAssembly module that implemented decent scripting language, or one slightly better than JavaScript, and then we just move to using that.

But it won't work like that, will it? The WebAssembly ecosystem is effectively just a bunch of binaries, so it'll be like CP/M and DOS all over again.

@natecull @teleclimber the problem that webassembly solved is people annoyed that anyone can read and edit their source

@zensaiyuki @teleclimber

And I'm annoyed with people being able to hide their source! So it all works out.

@natecull @teleclimber in practice a scripting language is possible to implement in webassembly, but it’s not the utopic vision anyone imagines. you still have the same shitty browser APIs, but with an extra loading screen piled on top while the runtime downloads- once for each and every website that uses it.

@zensaiyuki @teleclimber

Can WebAssembly modules not be cached ????

That seems like a very poor design decision if so.

Show newer

@natecull @teleclimber ah yes. that one. i have been chipping away at that one myself. i have a spreadsheet comparing the fields in each of the major message/blog formats, as a kind of attempt to see what such a system would need

@zensaiyuki @teleclimber

Interesting! How many formats do you have comparisons for?

@zensaiyuki @teleclimber

The Linked Data people seem to be trying their best to revive the idea of RDF and the Semantic Web, and they've picked JSON as the syntax, which is a pragmatic choice, but I think still will cripple them. But, it is pragmatic.

But still, I wouldn't want to try to use JSON-LD to write a *program* in. And if I can't write a program in it, then I probably can't write a database query or spreadsheet-cell function in it. And if I can't write those, it seems.. insufficient.

@natecull @teleclimber it does seem almost easier to just roll your own. a couple lines of bash and a makefile. a dash of perl. make your own and you can have your own opportunities to fuck up the design when you need to do $Big_Thing, in a codebase you understand cos you wrote it.

though i do admire the simplicity of a makefile and judicious application of hxincl

@natecull @teleclimber i wrote a static site generator the other day using nothing more than Make, a json file, mustache.php and jq. i can’t really name it because it’s just a simple composition of existing tools.


> I've been trying to brew my own [master data/coding language] (slightly extended Lisp S-expressions)

hey do you have a demo of this

What i've really wanted for a while is literally just....
"nodejs #tiddlywiki except the widgets, macros, plugins, and javascript modules [hell, filters now you mention it] have all been replaced by a consistent lisp syntax that provides the same APIs more flexibly"

i would surely find at least one way to put this language to the test

But is turing completeness a real problem here? What if your static site generator does not terminate? It's not like it affects the actual "runtime" (people visiting the result), and you can do a lot of stuff more cleanly with a proper programming language.
Of course, a compiler not terminating is bad, but here we're talking more about libraries to build your own custom compiler, which does not need to be held to the same high standards.
@zensaiyuki @teleclimber

@fgaz @natecull @teleclimber the problem comes from the sorta stuff the blog post is complaining about. sooner or later complexity is going to seep in somewhere, and it’s a choice about where you put that complexity. the filesystem, and subtleties in naming are the wrong choice. similar problems would arise from starting with a “simple” non turing complete markup/templating language and trying to force it to do something outside the original usecases that it wasn’t designed for

@fgaz @natecull @teleclimber a good example is the mustache templating language. really great for most things. but then you need a simple way of feeding in an array and getting a numbered list. or you just want to print out the keys and values of an object and it doesn’t provide a way to print keys. or you need to show a different bit of content depending on the contents of a value. mustache can’t do any of that, so you either put more logic in code outside of mustache, or you make handlebars

@fgaz @natecull @teleclimber and before you know it, handlebars 5.0 is a full featured programming language

@fgaz @natecull @teleclimber (i heart mustache and handlebars, but its limitations make a good example)

or you get fed up with the limitations of mustache and we just promiscuously mix html and javascript like people do with react.js. just mix all that shit together. the circle is complete.

@fgaz @natecull @teleclimber we’re now to the point where people are noticing how much of a pain it is to individually feed bits of configuration to each react component and are reinventing CSS but as a janky javascript framework that reads theme variables from a json file.

@fgaz @natecull @teleclimber of course this all leads to Gall’s Law, and its corrolary, Greenspun’s Tenth Rule

@fgaz @natecull @teleclimber i found this one: “Tesler’s Law” or “the law of conservation of complexity”, every system has an irreducible level of complexity, and the only choice is where to put it: on the users, the application developers or the platform developers.

@zensaiyuki @fgaz @teleclimber

I feel like there's maybe a rider to this:

if you divide a system into modules such that the complexity is in the correct modules, overall system complexity lowers. If you put the complexity in the wrong modules, overall system complexity rises (potentially exponentially if you got the module divisions super wrong).

ergo you can tell if you've decomposed a system correctly by how much non-essential complexity the system as a whole has.

@zensaiyuki @fgaz @teleclimber

this is a rule I just now made up so I have no idea if it holds in real life

@natecull @fgaz @teleclimber i recently read a thing that outlined a way to spotcheck the complexity of a system by expressing lines of communication between modules as a graph. the goal is you want the graph to be representable in 2D without any of the edges crossing over each other.

@natecull @fgaz @teleclimber and by “recently” i apparently mean some point in the last 6 years.

@zensaiyuki @fgaz @teleclimber

I think platform churn is a huge driver of complexity. Complexity that's 'essential' for the person having to solve the short-term problem of 'getting the system running on platform X, and then platform Y, and then platform Z'. but not really essential to the long-term problem of 'this just needs to run, somehow'.

I guess that means system design is increasingly a social problem. "What culture do I trust with my system? What are their demands on it, and me?"

@zensaiyuki @fgaz @teleclimber

It turns out that Silicon Valley Culture is a very high-maintenance culture for anyone for whom constant disruption is.... well, a disruption.

@zensaiyuki @fgaz @teleclimber

And unfortunately, Free/Open Source Culture didn't entirely make Silicon Valley Culture better. It mitigated some of the worst parts (the ability for a company to completely evaporate big chunks of infrastructure software with copyright). It didn't quite mitigate all the endless loops of trend-chasing and the dominance of big money with small ethics on what free code gets written. Maybe that part didn't even come from Silicon Valley but somewhere deeper.

@natecull @fgaz @teleclimber my biggest revelation has been that software is way more of a social thing than I ever imagined. which sucks because I only got into it because I thought i’d be able to get away with never talking to anyone.

@zensaiyuki @fgaz @teleclimber

Yes! I find software *terrifyingly* social now, what with git and automated testing. The social barriers to entry seem really high. Yes you can publish any code I guess but if you want to publish 'the right code' using 'the right libraries/frameworks/methodologies' in 'the right places' to be seen by 'the right people'.... it starts to feel more like the art scene? Very cliquey? Very fashion-conscious?

@zensaiyuki @fgaz @teleclimber

"You're using MongoDB? In 2020? Darling, that's a very interesting choice but... well what exactly is it that your piece is trying to *say*?"

"Well, I want to interrogate the intersubjectivity of the JSON paradigm by undercutting the expected conventional understanding and juxtaposing it with the Kubernetes environment, so of course I need to use a slightly dated database module but with echoes of the aspirations of the past/future holism..."


@natecull @zensaiyuki

i'd say one can get pretty far on coding or contributing without "talking to anybody" in terms of like, bothering to know anybody posting on a forge

i don't feel like i know anybody on the pinephone issue trackers nor do I look at the ubuntu touch forum like I probably should, and yet i can get involved in debugging the distros to push issues closer to closing / problems closer to fixing pretty easily

@natecull @zensaiyuki

the big problem is always capitalism, focusing of human effort, and the fact that effort / compensation won't get focused where people need it unless somebody out there is good enough at marketing and convincing to get people's heads out of their butts

@Valenoern @natecull there’s also the dimension of socialising with the marketing person, and the relationship of the software with the audience- who might not be developers, or even power users.

but then you have API surfaces, explicitly documented and implicit unintended APIs, each one a social contract.

@zensaiyuki Meanwhile in the industry, your legacy system module connection diagram looks like a circle network diagram of Metcalfe's Law.


@natecull @teleclimber @fgaz

@clacke @natecull @teleclimber @fgaz i’ve decided the worst possible outcome is projects like TiddlyWiki and Twine who appear to have not decided where to put the complexity by assuming anyone using their software is all 3: a user, an application develooer and a platform developer- and so to even use twine, or tiddlywiki, you must first know html, css, javascript, plus whatever wiki language, and all the details of how twine and tiddlywiki are implemented.

@zensaiyuki @clacke @teleclimber @fgaz

Darn, Twine too? I thought that was aimed at newbies.

I guess that's why I kinda like p5js, it's somewhat simple. Ish. it's still javascript though and to do text output, you still have to know about the DOM, which I don't really want to.

@natecull @clacke @teleclimber @fgaz i thought it was aimed at newbies too. but then I tried to use it and the first decision it asks me to make is which “skin” do i want, which sounds like i am picking a color scheme but actually includes the wiki dialect and which features are available. anything beyond plain text, including just an ordinary image, i needed to edit html to add in.

that’s the impression i got from the documentation which seemed to assume it was talking to twine devs

@natecull @clacke @teleclimber @fgaz i cannot for the life of me work out how it got a reputation for being newbie friendly.

@natecull @clacke @teleclimber @fgaz and you have to decide up front, projects aren’t convertable from one skin to another, you gotta start over, and learn the different wiki language.

@natecull @clacke @teleclimber @fgaz i got the impression that maybe it was twine 1 that was newbie friendly. when i looked at it, only early versions of twine 2 were available, and i couldn’t find a download link for twine 1. this was years ago so maybe they fixed it.

Show newer
Sign in to participate in the conversation

Server run by the main developers of the project 🐘 It is not focused on any particular niche interest - everyone is welcome as long as you follow our code of conduct!