Possible Ways to Extend Browsers

We already do "extend" browsers with things like "external viewers." But there's not a very good integration with the browser. Ideally those external viewers should be rendering in-place inside the document, and be working together with the browser, be tightly integrated with the browser and other parts...

#docfs #webfs #kfc



So, a solution is what's been touted under with name "component software". The basic idea is that, rather than buildig one single monolithic application that does everythig from day one, we should be building a framework or architecture that ican be dynamic in its ability to have functionality added or deleted on the fly.



@dredmorbius @eliotberriot

We have 'component software' already, that's what COM and .NET are.

We maybe need something more?

@natecull @dredmorbius @eliotberriot
Like, theoretically, sure. But, realistically, these things are still walled gardens in terms of the apps created with them.

@enkiv2 @natecull @dredmorbius @eliotberriot and the efforts to implement this type of interface for other kinds of software (i.e. OpenDoc, CORBA, XPCOM (sort of)) haven't been particularly successful.

I can't tell how much of the problem is technical and how much is political.

Like, component based applications are in a sense profoundly anti-corporate-capitalist in that nobody can reasonably expect to control the 'look and feel' of a user's actual experience or cockblock their competition on the user's machine.

The unix shell does composability between totally independently developed things well, on the other hand.

@natecull @dredmorbius @eliotberriot

@eliotberriot @dredmorbius @natecull @a_breakin_glass

(The web tried to do some of this with frames early on & there were many lawsuits.)

@enkiv2 @eliotberriot @dredmorbius @natecull the reason why unix shell composability doesn't work for GUIs and we have to rely on APIs is partially, again, the result of GUIs still obeying VERY old limits

@a_breakin_glass @enkiv2 @eliotberriot @natecull Could you expand on this thought?

What are the limits?

How did / do shell tools fail in them?

What if there were, I don't know, some lightweight application-wrapper that could be tossed around shell tools?

Or does that fail to address concerns of the GUI space?

(I've never designed GUI tools, and always found the space vaguely mystifying.)

@dredmorbius @a_breakin_glass @eliotberriot @natecull
GUI toolkits aren't really designed around composability. combining bits of one widget with bits of another means rewriting both, usually. widgets can't overlap, except special frame-type ones. each window is its own little domain. applications don't by default have the ability to embed each other's components. a non-technical end user certainly can't mash up two apps.


@enkiv2 @dredmorbius @a_breakin_glass @eliotberriot

Just for speculation, would a GUI widget be more composable if it were seen as a Unix-like pipe/filter mapping a stream of input events (with initial/accumulating state) to output events?

(Doing so would require a framework where an 'event' or a 'message' is itself an object; which I think Smalltalk can do but not many other OOP systems)

Or can GUI widgets fundamentally not be implemented as a stream/filter model at all?

@natecull @dredmorbius @a_breakin_glass @eliotberriot
I'm not sure how to conceptualize widgets as filters in a general sense but I'm not willing to claim it can't be done.

@enkiv2 @dredmorbius @a_breakin_glass @eliotberriot

I guess one difference is that a GUI widget would be a filter over at least TWO I/O streams, bound together.

Eg, a file or database view:

* input 1: updates or refresh events from database
* input 2: keyboard or mouse user control events
* output 1: updates or view selections to database
* output 2: graphical redraws to canvas, or events to lower-level component widgets

And state still needs to be allocated/deallocated for caching, etc.

@eliotberriot @a_breakin_glass @dredmorbius @enkiv2

This is where I'd *like* to think that something like functional reactive programming can simplify the usual GUI-widget object-and-callback mess. But I'm not sure the dust has settled from the various FRP paradigms figuring out what works, yet.

But I'd love it if, eg, something like Elm wasn't so narrow-focused to 'compiling to Javascript in a web page' and aimed a bit broader.

@enkiv2 @natecull @a_breakin_glass @eliotberriot Maybe I need a better definition of "widget" here. Are these controls? Outputs? Both?

(I really don't do GUI dev.)

@dredmorbius @enkiv2 @a_breakin_glass @eliotberriot

a widget is any kind of object in a GUI system that has a graphical state.

Part of the complexity of GUIs is that you have to 'hook up' your widgets to receive events from others. A bit like lots of wires in a circuit board. Very brittle, hard to repurpose.

@eliotberriot @a_breakin_glass @enkiv2 @dredmorbius

So a widget, to work, needs to know the name and type of all its partners, and care about their state. Vs a Unix filter that knows nothing about what's upstream or downstream the pipe.

@natecull @eliotberriot @a_breakin_glass @enkiv2 That was pretty much what I was thinking. A Unix filter is entirely source-agnostic. Feed it to /dev/stdin, or read it from /dev/sdtout, and you're good.

That's opposed to various system or library calls which have specific structures and datatypes that must be met.

Is there a way to bridge that divide? To make widgets that act as pipes or filters?

@natecull @eliotberriot @a_breakin_glass @enkiv2 I've been thinking about, effectively, widgets, viewers, and shells, in the context of docfs. How tightly integrated should all that be?

Check the repo for my recent edits, just pushed now.

I'd like docfs to work from any shell (including remote, no GUI, etc.). But for the full flavour, you'd want a terminal that can incorporate widgets, basically. How much of a good or bad idea /that/ is I'm not entirely sure.

But, see the tectronics term.


@enkiv2 @a_breakin_glass @eliotberriot @natecull It had the notion of a full graphical environment (though with pretty primitive graphics capabilities associated) that you could toggle to.

What if the terminal could sprout /at least a sufficiently capable GUI viewer/?

A terminal that could speak a subset of semantic HTML, present a document, format the general structure, offer navigation, multiple fonts, at least static graphics. Document-viewing-in-terminal.

Or something like that.


@enkiv2 @a_breakin_glass @eliotberriot @natecull One question is whether or not that's a shell with browser capabilities or a browser with shell capabilities, and I suppose it depends on which end of the gun you're looking at. The question may be moot, or not.

In this talk of widgets and knowing parters / callers / etc., what if you /don't/ need all that, and can just stick stdin, stdout, and stderr on things (or maybe "datain" or something like that, reserving stdin for command stream)?


@enkiv2 @a_breakin_glass @eliotberriot @natecull And there's enough /standard/ markup for the terminal to look at some document and say "OK, so this is how I've got to deal with this crud", and render appropriately.

Not render as in "this was what the author intended", but as in "this is good enough for the reader, here and now".

And then to figure out what the fuck to do with headers, footers, asides, links, embeds, A/V, citations, footnotes, tables, comments, trees, lists, etc.



@enkiv2 @a_breakin_glass @eliotberriot @natecull If we want interactivity (sorting or summarising data, popping up / closing asides, ToCs, etc.) then how that's handled.

Recognising that the "terminal" might be a TTS output / voice input. Or a touchscreen. a 4 cm display, or 200 cm display. Colour. eInk. Touch. Flat. Monochrome.

Anyhow: how do you build a CLI-GUI integrated concept around all of that.


@dredmorbius @enkiv2 @a_breakin_glass @eliotberriot

I think Tcl/Tk widgets take a somewhat similar approach?

I don't really know, I've never used Tk, but, I think they send text strings around.


And the whole Smalltalk-era MVC thing suggests we should cleanly separate model/view/control, but in practice we almost never do.

I feel like 'pipes of events' SHOULD be a good paradigm, but maybe needs a lot of 'tee' components... eg, a publish/subscribe protocol.

@eliotberriot @a_breakin_glass @enkiv2 @dredmorbius

and, eg, even Unix pipes have a lot of components that don't strictly fit the pipe model. So there's some fundamental work that hasn't been done if we need pipes to be a first class model for all-purpose data communication.

Eg: 'cat filename' puts a file's lines ON the pipe, but doesn't use a pipe to GET that data. It's not clear that there's a good pipeful mechanism for 'get a chunk of data', but a lot of GUI stuff is selecting data.

@dredmorbius @enkiv2 @a_breakin_glass @eliotberriot

So far the interface that's best managed to combine CLI and GUI interfaces is the 'file directory'.

I can take a directory, and I know it always has roughly similar semantics:

* it has a text path
* it's a list of items
* items have names, properties and contents
* A given window (text or GUI ) maps to a path and shows the contents
* I can 'navigate' around either mode

Can we use 'objects' instead of 'directories' and generalise this?

@eliotberriot @a_breakin_glass @enkiv2 @dredmorbius

We would need, perhaps, a way of creating a 'virtual directory' where some 'files' were links to other places in the tree.

Those links (symlinks in a filesystem, or object references in an objectsystem) might be functions, perhaps.

Just this one concept might get us universal pure-functional computation...?

Like what if 'clicking a button' literally transported you to another 'place in the object tree'?

Could that cover all interaction?

@dredmorbius @enkiv2 @a_breakin_glass @eliotberriot

but even if we couldn't get to full interaction (and maybe we can though)

wouldn't it be, say, AWESOME if, eg your mail client appeared as 'just a directory' and you could then

cd mail
cd inbox

In PowerShell, *you can already do this with some datatypes*. You can explore the Registry and WMI Database and Active Directory as if they were a tree of file-like 'places'

What if windows - and sub-window widgets - were directories?

@natecull @enkiv2 @a_breakin_glass @eliotberriot Pretty much.

See mh: en.wikipedia.org/wiki/MH_Messa

It's not a virtual-filesystem based mail access, entirely, though it's got elements of that. Jerry Peek is a major fan.

Console email clients such as mutt or pine also get pretty close. A problem is that there's so much shitty (content-wise) email kicked around. Straight text is massively easier to deal with.

Restructuring crap to text might help, also just saying "fuck you, no" to ugly shit.

@natecull @enkiv2 @a_breakin_glass @eliotberriot One view is to treat the directory as a thing addressed directly via the terminal.

Another is to have an application for talking to directories and files. Midnight Commander (based of Norton Commander IIRC) is an example. It _also_ has a bunch of virtual filesystems -- various archive and packaging formats, FTP, SSH, and more.

@natecull @eliotberriot @a_breakin_glass @enkiv2 This is where rather than fucking around with symlinks, we just kind of separate /data storage/ from /data access/. The "filesystem" is a route to viewing file-shaped shit. As with /proc or /sys.

Where, say, in /docfs, there actually /are/ serialised representations of text (or audio, images, video, etc.) those /do/ have a specific storage spot. (I've been putting my "books" in the "/stacks" conceptually). But you don't go there generally.


@enkiv2 @a_breakin_glass @eliotberriot @natecull Instead, /docfs itself is your interface to works, through /title, /author, /subject, /date, /project, /group, or other "trees". Which are virtual filesystem-like interfaces to specific works.

An /interface/ but not the direct storage repo itself.


@dredmorbius @enkiv2 @eliotberriot @natecull huh. I wonder if this sort of thing could be done with some ebook reading tools and FUSE...

@natecull @enkiv2 @a_breakin_glass @eliotberriot Hence: "What if the Web were filesystem accessible?"

(Or your documents archive.)

Via a virtual filesystem. And a shell, shell tools, and terminal that could do useful, document-centric things, with them.

@natecull @eliotberriot @a_breakin_glass @enkiv2 AFAIU cat _does_ direct stdin to the file itself.

The _other_ commands are the parameters fed to the program itself. In Linux most of those are accessed via /proc, specifically /proc/$$/

I think even at the syscall level (not my baliwick).

@natecull @enkiv2 @a_breakin_glass @eliotberriot I've had Tcl/Tk in mind, though I've also not used them. Pretty clunky toolkit, though also widely used.

Pro: Since it was used in a lot of movies, instant Hollywood panache!

@natecull @enkiv2 @a_breakin_glass @eliotberriot My main thought is that stdin, stdout, and stderr isn't quite enough.


Control in (user interface, "stdin")
Doc / data in (dataflow)
Status out (system response, "stdout")
Display out (visual)
Doc / data out (dataflow)
Errors out (system response, "stderr")

So that's three new pipelines?

What about talking to whatever the feeder / consumer of information is as well. In-band or out-of-band? Two more pipes?

@natecull @dredmorbius @eliotberriot @enkiv2 *cut to a computer that uses electromechanical tonewheels as a metaphor somehow*

@natecull @dredmorbius @a_breakin_glass @eliotberriot
in the context of tk, 'command' means arbitrary TCL code. (any widget can bind any code to the "button1-up" event -- i.e., click and release -- but button has a default binding that shows it as pressed and released and then calls a function the programmer has provided)

tk is pretty conventional as a gui toolkit, mostly notable for being lightweight.

@enkiv2 @natecull @a_breakin_glass @eliotberriot It took me a moment to realise you /weren't/ talking about draft editing 'TK' marks.

Tcl/Tk totally screws with that, I've just realised.


Medium complains whenever I write anything about Tk.

tcl/tk stands for Tool Command Language/Tool Kit.
@natecull @a_breakin_glass @eliotberriot

@enkiv2 @natecull @a_breakin_glass @eliotberriot Oh, I know. I'm just used to seeing a naked "tk" in the context of an edit mark.

Sign in to participate in the conversation

Server run by the main developers of the project 🐘 It is not focused on any particular niche interest - everyone is welcome as long as you follow our code of conduct!