Boring technical question which neither the documentation nor ChatGPT can answer.
I want to automate screenshots using #Python's #Selenium Webdriver with #Chrome.
This works - but the non-logged in experience of Twitter forces dark-mode.
I want a light-mode screenshot.
I can't find a way to set SEC-CH-PREFERS-COLOR-SCHEME in Selenium. Nor can I find a light-mode toggle for non-logged in Twitter users.
Any clues gang?
(I want to automate replacing my Twitter embeds with a screenshot.)
Aha! There's a way to cheat!
The old Twitter embed API lets you manually specify colour scheme - and is easy to screenshot.
Hopefully I can replace all the Twitter embeds on my website with images (and, yes, I will add alt text).
Thanks @samir for giving me the inspiration.
Any #Selenium experts in the house?
All my screenshots are slightly fuzzy. Why?!?!
I'm using the Chromium webdriver. Grabbing the area with `element.screenshot_as_png`
Is it because I have fractional scaling? Wayland?
I have set `chrome_options.add_argument("--ozone-platform=wayland")` but to no avail.
This'll be a two-drink minimum problem, I reckon.
OK, if you have lots of Twitter embeds in your website, I've written an easy(ish) way to replace them with static images & alt text.
https://github.com/edent/Tweet2Img
Run:
`python tweet2img.py 1234`
And you'll get a ready-to-paste piece of HTML with a data-encoded image.
`<a href="https:/ /twitter.com/edent/status/123"><img src="data:image/webp;base64,Ukl..." width="550" height="439" alt="text from tweet."/></a>`
It also saves the WebP and alt text if you need them.
Someone complained that an image with decent alt text wasn't accessible.
So now I'm practising Spite-Driven-Development to create a semantic HTML embed.
Doesn't look *totally* accurate - but not bad for a dozen lines of CSS.
Spite-Driven-Development continues.
I now have a scrap of semantic HTML which adequately replicates an embedded Tweet.
TODO:
* Add images
* Add entities (hashtags, mentions, links)
* Add parent of thread
* Add quoted tweet
* Add ... whatever else
Should be able to hijack the embed API to get most metadata (no number or replies / retweets though).
Parents and Quotes are just <blockquotes> within <blockquotes>. So that was easy!
I think that's good enough for the basics.
This code will (eventually) convert all your embedded Tweets into proper semantic HTML without any tracking back to Twitter.
Rough code at:
https://github.com/edent/Tweet2Embed
OK gang! This is ready for you to test.
python tweet2html.py --css 1234
That will take a Twitter ID and return HTML and CSS for you to embed in your website with no calling back to Twitter.
Features: Avatars inlined as Base64 WebP
Hyperlinks don't use t.co
Hashtags and @ mentions linked
Semantic time
and
counts
Try it out at https://github.com/edent/Tweet2Embed
Feedback and pull requests very welcome!
Now also shows parents to replies, and full content of quotes.
And polls!
HTML validates!
Play with the code at https://github.com/edent/Tweet2Embed
I'll be doing this for Mastodon later - because I really don't like iFrame embeds!
OK! I *think* I've finished.
You can now "rescue" any embedded Tweet and recreate it as simple HTML - no tracking.
Includes:
Avatars inlined as WebP
Hyperlinks don't use t.co
Hashtags & @ mentions linked
Includes reply threads & quote Tweets
Semantic time
Schema.org metadata
Cards
Polls
,
&
counts
One command. No API key needed.
Code at https://github.com/edent/Tweet2Embed
Feedback *very* much welcome!
I'm starting work on this Mastodon version of this tool.
Take an ID and return a static and embeddable piece of HTML.
But there's a problem with the #MastodonAPI.
One of my posts has this JSON representation - https://mastodon.social/@edent/112948887103547937.json
But there's no count of likes, replies, or reposts. Is that expected? Is there a simple way to get those counts without an API key?
I could screen-scrape from https://mastodon.social/@edent/112948887103547937/embed
But that feels kinda cludgy.
Another #MastodonAPI question.
Is there a good set of "reference posts"?
That is, a set of example toots which cover the gamut of what they can do.
One with an image, one with a content warning, one with poll etc.
Sort of like a reference suite for testing.
@Edent suggestion (if possible) expand the short links?
@ben see previous posts - on the todo list.
The expanded URls are in the entities that are returned by the API - but I'm not sure the best way of converting them back to HTML.
@Edent any chance it could handle tweet threads in the future?
I have some /epic/ threads I want to archive, 100+ tweets chained together, most of which are themselves quote-tweets
This one for example (only properly visible if you are signed in to Twitter) https://twitter.com/simonw/status/1077737871602110466
@simon Ooooh. Possibly. I used to do epic things with their threading API. Not sure if it's still possible - but I'll put it on the roadmap.
Next step is quotes and replies.
If I can get that working, then it *should* be possible.
@Edent my SpiderVerse thread would be an epic stress-test - many of the quote-tweets have embedded video, so could be hundreds of MBs if you were to fetch and base64-embed all of them!
@simon if you don't already know it, https://www.solipsys.co.uk/DiscDAG.html by @ColinTheMathmo
@simon I used to fetch and render the *content* of Twitter threads, though not the linked or embedded media. Then the nodes were clickable so you could view the original.
It was never very good, but it served my purposes:
https://www.solipsys.co.uk/Chitter/SuggestedReading.svg
I have bigger examples.
When the threads become humungous, viewing a static tree is sub-optimal.
CC: @Edent
Calling @Chartodon spine ...
Your chart is ready, and can be found here:
https://www.solipsys.co.uk/Chartodon/112939515109242805.svg
Things may have changed since I started compiling that, and some things may have been inaccessible.
In particular, the very nature of the fediverse means some toots may never have made it to my instance, in which case I can't see them, and can't include them.
The chart will eventually be deleted, so if you'd like to keep it, make sure you download a copy.
@eaton you may or may not already be familiar with @darius's <https://tinysubversions.com/twitter-archive/>.
@darius yep, that's what @eaton was talking about.
Folks should really be exporting their archives on the regular. Have you considered setting something up for people e.g. that explains that Twitter lets you do this—and give them a place for them to dump it? For those that don't want to go whole-hog on the Mastodon/Fediverse thing, but are amenable to doing just a tiny bit to be proactive about having an escape plan.
@eaton @darius @colby @Edent I recently posted the first video on my blog that was hosted by me (an mp4 file in a video element) as opposed to hosted on YouTube or similar: https://simonwillison.net/2024/Jul/29/sam-2/
It's in my S3 bucket along with all my other images, but I have to admit I do worry that some day decades into the future I may fail to pay my AWS bill and risk my S3 bucket blinking out of existence
@simon @eaton @darius @colby @Edent this actually happened to me. I wasn't using my aws account, but there was some stuff there costing me pennies per month. The credit card I used for the account expired and I didn't see the alert emails. When I finally tried to go back to the account, it was gone.
@Edent this is incredibly cool. It's it per tweet right now or can you embed a feed?
@snim2 per Tweet.
But, if that Tweet is a reply - it will get the parent as well.
There's no "conversation ID" in the free API, so it can't get full threads.
There's also no easy way to iterate over an entire feed.
@Edent this is very cool! If you don’t mind I might use it as inspiration to rebuild and replace my old, no longer working webapp https://tweet.cluster.fun/
@Marcus as per the terms of the MIT licence, you may do so only after sending me one million bitcons.
@Edent for you, I’ll give you two million bitcons
Feel free to use it. A link back would be appreciated.
@Edent assuming I ever actually get the motivation to tackle it, you betcha!
@fastfinge @anantagd
I've never made a browser extension before.
Do feel free to fork the code and build it though - it's all open source.
@Edent phenomenal work
@Edent awesome. Can you write a sweeper that cleans up WordPress blog entries?! What a superb idea.
@noplasticshower
No.
There are too many ways to store them in WP - shortcode, embed, oEmbed, etc.
You'll need to go through your posts manually.
However, it is open source - so fell free to write one if your up for the challenge.
@Edent I am not thinking about this as a thing for me (though I might use it) but as a thing to eradicate X tendrils all over the net. The very idea is appealing.
@Edent this is awesome! A mastodon version would also be cool, being able to get tweets and toots as fully standalone html is really nice for static sites!
@Tomhodson thanks! Mastodon is this weekend's project. Should be easier. I hope!
@Edent awesome. I'd love one for Mastodon Posts too!
@phocks working on it
@Edent hah this is cool!
I ended up doing this completely manually when I redid waasabi.org for posterity after the -takeover...
https://waasabi.org/news/2021-05-11-progress-update.html
@Edent I ran into this when building https://sam.pikesley.org/blog/2023/03/06/mastodon-powered-comments-in-astro/
I had kind of assumed that data would just be right there in the JSON, but of course it's not. I guess maybe this is some consequence of federation?
@Edent I use this endpoint on my blog comments widget:
@Edent There's something someone once said to me, which has stuck in my mind for a long while: "You've got too much taste to be a web developer."
@darkling I am too hungover to understand what you mean by that, sorry.
"How do I do this?"
"This way."
"But that's icky."
"Yes, but that's the way you do it on the web."
@Edent oh, I remember someone talking about that before, an app developer I think. Annoyingly I can't remember who, it was a while ago but I have a feeling something might exist.