mastodon.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
The original server operated by the Mastodon gGmbH non-profit

Administered by:

Server stats:

360K
active users

Boring technical question which neither the documentation nor ChatGPT can answer.

I want to automate screenshots using 's Webdriver with .

This works - but the non-logged in experience of Twitter forces dark-mode.

I want a light-mode screenshot.

I can't find a way to set SEC-CH-PREFERS-COLOR-SCHEME in Selenium. Nor can I find a light-mode toggle for non-logged in Twitter users.

Any clues gang?

(I want to automate replacing my Twitter embeds with a screenshot.)

Aha! There's a way to cheat!

The old Twitter embed API lets you manually specify colour scheme - and is easy to screenshot.

For example: platform.twitter.com/embed/Twe

Hopefully I can replace all the Twitter embeds on my website with images (and, yes, I will add alt text).

Thanks @samir for giving me the inspiration.

platform.twitter.comTwitter Embed

Any experts in the house?

All my screenshots are slightly fuzzy. Why?!?!

I'm using the Chromium webdriver. Grabbing the area with `element.screenshot_as_png`

Is it because I have fractional scaling? Wayland?

I have set `chrome_options.add_argument("--ozone-platform=wayland")` but to no avail.

This'll be a two-drink minimum problem, I reckon.

Terence Eden

OK, if you have lots of Twitter embeds in your website, I've written an easy(ish) way to replace them with static images & alt text.

github.com/edent/Tweet2Img

Run:
`python tweet2img.py 1234`

And you'll get a ready-to-paste piece of HTML with a data-encoded image.

`<a href="https:/ /twitter.com/edent/status/123"><img src="data:image/webp;base64,Ukl..." width="550" height="439" alt="text from tweet."/></a>`

It also saves the WebP and alt text if you need them.

GitHubGitHub - edent/Tweet2Img: Convert a public Tweet into an image & alt textConvert a public Tweet into an image & alt text. Contribute to edent/Tweet2Img development by creating an account on GitHub.

Someone complained that an image with decent alt text wasn't accessible.

So now I'm practising Spite-Driven-Development to create a semantic HTML embed.

Doesn't look *totally* accurate - but not bad for a dozen lines of CSS.

Spite-Driven-Development continues.

I now have a scrap of semantic HTML which adequately replicates an embedded Tweet.

TODO:

* Add images
* Add entities (hashtags, mentions, links)
* Add parent of thread
* Add quoted tweet
* Add ... whatever else

Should be able to hijack the embed API to get most metadata (no number or replies / retweets though).

Parents and Quotes are just <blockquotes> within <blockquotes>. So that was easy!

I think that's good enough for the basics.

This code will (eventually) convert all your embedded Tweets into proper semantic HTML without any tracking back to Twitter.

Rough code at:
github.com/edent/Tweet2Embed

OK gang! This is ready for you to test.

python tweet2html.py --css 1234

That will take a Twitter ID and return HTML and CSS for you to embed in your website with no calling back to Twitter.

Features:
🗣 Avatars inlined as Base64 WebP
📸 All attached photos inlined
🎥 Video poster inline, <video> to original source
🔗 Hyperlinks don't use t.co
#️⃣ Hashtags and @ mentions linked
🕰 Semantic time
♥ and 🗨 counts

Try it out at github.com/edent/Tweet2Embed

Feedback and pull requests very welcome!

Now also shows parents to replies, and full content of quotes.

And polls!

HTML validates!

Play with the code at github.com/edent/Tweet2Embed

I'll be doing this for Mastodon later - because I really don't like iFrame embeds!

OK! I *think* I've finished.

You can now "rescue" any embedded Tweet and recreate it as simple HTML - no tracking.

Includes:

🗣 Avatars inlined as WebP
📸 All attached photos inlined
🎥 Video poster inline, <video> to original mp4
🔗 Hyperlinks don't use t.co
#️⃣ Hashtags & @ mentions linked
🔄 Includes reply threads & quote Tweets
🕰 Semantic time
🔍 Schema.org metadata
🖼 Cards
📊 Polls
♥ , ♻ & 🗨 counts

One command. No API key needed.

Code at github.com/edent/Tweet2Embed

Feedback *very* much welcome!

GitHubGitHub - edent/Tweet2Embed: Convert a public Tweet into embedded semantic HTMLConvert a public Tweet into embedded semantic HTML - edent/Tweet2Embed

@Edent Lots of missing python packages for that.

For example, apt doesn't have python3-webdriver (or what I need for "from webdriver_manager.firefox import GeckoDriverManager").

Maybe publish a list of what use would need?

@derickr how do I go about doing that?

I think I just pip installed everything I needed. Is there a way to find out what? I've never properly published a Python package before.

@Edent I don't really know either. I think there usually is some kind of file listing them that you can install?

@derickr I've added a requirements.txt - hopefully that works!

@Edent That worked, after using virtual env, and installing xclip:

python3 -m venv /tmp/t2i
/tmp/t2i/bin/pip3 install -r requirements.txt
sudo apt install xclip
/tmp/t2i/bin/python3 tweet2img.py 1821809135920550280

@derickr woo!
I used the Wayland version - good to know it works with the x version.

@Edent @derickr I could probably pack this up into a self-contained docker image if that seems like something that would be useful

@Edent @derickr at the very least create a requirements.txt. pip has a command to list one. Of course that will have every package you’ve ever installed. Belt, braces and another belt option. For future ref, pipenv is a really good command to manage dependencies in an isolated environment.

@Edent @derickr Oh. I now see you have done this. In my defence, I’m on holiday, so a bit behind on the old socials!

@Edent That’s pretty cool! I could see this being super useful for static sites where you can just automate any tweet in Markdown on its own line

@Edent I mean, they use a million lines of JS to provide an embed, but there’s nothing stopping you using their same CSS so it’s indistinguishable

@sil true! But the problem is, their CSS relies on this DIV soup!

@Edent yikes. I applaud the effort to attempt to emulate their look with decent semantic HTML. Keep it up :)

@Edent @sil Possible unpopular question, wouldn't feeding the CSS and div soup to an LLM potentially give you a good starting base ?

@Edent @sil darn it, now I want to try that.

@Edent @sil

but I thinks your implementation looks better and more inline with your site's style.

@Edent @sil

ah, i seem to be missing the video...

@Edent suggestion (if possible) expand the short links?

@ben see previous posts - on the todo list.
The expanded URls are in the entities that are returned by the API - but I'm not sure the best way of converting them back to HTML.

@Edent
My first thought is show the URL if it's less than n characters, use short link text for the rest, maybe "🔗 1", "🔗 2" etc. (U+1F517)
@ben

@Edent any chance it could handle tweet threads in the future?

I have some /epic/ threads I want to archive, 100+ tweets chained together, most of which are themselves quote-tweets

This one for example (only properly visible if you are signed in to Twitter) twitter.com/simonw/status/1077

X (formerly Twitter)Simon Willison (@simonw) on XThere is so much great #SpiderVerse behind the scenes content on Twitter if you start following some of the artists who created the film

@simon Ooooh. Possibly. I used to do epic things with their threading API. Not sure if it's still possible - but I'll put it on the roadmap.

Next step is quotes and replies.

If I can get that working, then it *should* be possible.

@Edent my SpiderVerse thread would be an epic stress-test - many of the quote-tweets have embedded video, so could be hundreds of MBs if you were to fetch and base64-embed all of them!

@simon I used to fetch and render the *content* of Twitter threads, though not the linked or embedded media. Then the nodes were clickable so you could view the original.

It was never very good, but it served my purposes:

solipsys.co.uk/Chitter/Suggest

I have bigger examples.

When the threads become humungous, viewing a static tree is sub-optimal.

CC: @Edent

@ColinTheMathmo

Your chart is ready, and can be found here:

solipsys.co.uk/Chartodon/11293

Things may have changed since I started compiling that, and some things may have been inaccessible.

In particular, the very nature of the fediverse means some toots may never have made it to my instance, in which case I can't see them, and can't include them.

The chart will eventually be deleted, so if you'd like to keep it, make sure you download a copy.

@Edent @simon I ended up leaning on parsing of exported archives to handle threads, but it (naturally) chokes on threads with multiple participants.

@colby @eaton @Edent @simon mine is designed to only show posts from the originating user though, so multiple participants isn't something it's designed to handle

@darius yep, that's what @eaton was talking about.

Folks should really be exporting their archives on the regular. Have you considered setting something up for people e.g. that explains that Twitter lets you do this—and give them a place for them to dump it? For those that don't want to go whole-hog on the Mastodon/Fediverse thing, but are amenable to doing just a tiny bit to be proactive about having an escape plan.

@darius @colby @eaton @Edent I care about threads that were entirely me replying to myself, but those replies include quotes of other tweets that I want to capture as well

@simon @colby @eaton @Edent my assumption is I don't have a moral right to rehost other people's content I've quoted so I just render it as a URL

@darius @simon @eaton @Edent in the case of Hypothes.is (where you also run into this, as someone will with the fediverse, too), it's a non-issue because all contributors to the main/public stream are explicitly releasing their stuff under Creative Commons.

@darius @colby @eaton @Edent yeah that’s what’s held me back in the past too - hosting my own tweets is obviously fine, embedding full videos from quotes tweets much less so

I don’t want to republish those videos though, I want to stash a personal copy in case Twitter fully implodes some day

@simon @darius @colby @eaton
The free embed API gives you access to all the videos as MP4s if you want to archive them.
A few are geolocked (mostly commercial ones like the Olympics) but all the others that I've tried work fine.

@simon @darius @colby @Edent i’ve been noodling with that and testingways of slowly transitioning media to other sources loke vimeo/youtube/etc. but it’s no small thing; it sobered me up to realize how dependent we’ve all become on an invisibly free slurry of forgettable storage

@eaton @darius @colby @Edent I recently posted the first video on my blog that was hosted by me (an mp4 file in a video element) as opposed to hosted on YouTube or similar: simonwillison.net/2024/Jul/29/

It's in my S3 bucket along with all my other images, but I have to admit I do worry that some day decades into the future I may fail to pay my AWS bill and risk my S3 bucket blinking out of existence

simonwillison.netSAM 2: The next generation of Meta Segment Anything Model for videos and imagesSegment Anything is Meta AI's model for image segmentation: for any image or frame of video it can identify which shapes on the image represent different "objects" - things like …

@simon @darius @colby @Edent exactly. Would love to hear future updates about how that plays out !

@simon @eaton @darius @colby @Edent this actually happened to me. I wasn't using my aws account, but there was some stuff there costing me pennies per month. The credit card I used for the account expired and I didn't see the alert emails. When I finally tried to go back to the account, it was gone.

@Edent @thegrugq something for your newsletter? having the tweets directly in the mail like that would be awesome

@swars @Edent oooo. This is very close to exactly what I need.

Ideally I want to automatically create Substack posts from just tweet IDs. But this looks like a great start.

@Edent this is incredibly cool. It's it per tweet right now or can you embed a feed?

@snim2 per Tweet.
But, if that Tweet is a reply - it will get the parent as well.
There's no "conversation ID" in the free API, so it can't get full threads.
There's also no easy way to iterate over an entire feed.

@Edent this is very cool! If you don’t mind I might use it as inspiration to rebuild and replace my old, no longer working webapp tweet.cluster.fun/

TweetSVGTweetSVGGenerate an SVG for a given Tweet ID

@Marcus as per the terms of the MIT licence, you may do so only after sending me one million bitcons.

@Edent for you, I’ll give you two million bitcons

@Marcus 😘

Feel free to use it. A link back would be appreciated.

@Edent assuming I ever actually get the motivation to tackle it, you betcha!

@Marcus that deserves to come back to life. All tweets should be SVG

@Edent