Ned Letcher's Avatar

Ned Letcher

@ned.sh.bsky.social

Data science, AI/ML, analytics, visualisation. Naarm/Melbourne @thoughtworks, #dataBS, #Python, #NLP, #DuckDB, & assorted whimsical miscellania

3,358 Followers  |  9,542 Following  |  126 Posts  |  Joined: 22.06.2023  |  1.915

Latest posts by ned.sh on Bluesky

Screenshot of embedding atlas showing the embedding view on the left, a table at the bottom and charts on the right.

Screenshot of embedding atlas showing the embedding view on the left, a table at the bottom and charts on the right.

πŸš€ We've just open-sourced Embedding Atlas – a tool for exploring large embedding spaces through rich, interactive visualizations πŸ“Š.

01.08.2025 08:24 β€” πŸ‘ 120    πŸ” 34    πŸ’¬ 3    πŸ“Œ 3
Post image Post image Post image Post image

I’ve been working on an experimental, multiplayer, open-world AI chat system called Numinex.

1. Uses branching comment trees
2. Multiplayer/multimodel
3. Explicit context curation
4. Built on ATProto
5. Open observation of model behaviors

09.07.2025 00:02 β€” πŸ‘ 100    πŸ” 13    πŸ’¬ 2    πŸ“Œ 7
A nutty tweet from EPA Administrator Lee Zeldin (dated 10 July 2025) reading "Americans have questions about geoengineering and contrails. They expect honesty and transparency from their government when seeking answers. For years, people who asked questions in good faith were dismissed, even vilified by the media and their own government. This ends today." --> to which Rep. Don Beyer's official account tweets this reply (also dated 10 July 2025): "Some people have 'questions' about whether birds are real β€” will that be your next project? How much taxpayer money will you be spending on this?"

A nutty tweet from EPA Administrator Lee Zeldin (dated 10 July 2025) reading "Americans have questions about geoengineering and contrails. They expect honesty and transparency from their government when seeking answers. For years, people who asked questions in good faith were dismissed, even vilified by the media and their own government. This ends today." --> to which Rep. Don Beyer's official account tweets this reply (also dated 10 July 2025): "Some people have 'questions' about whether birds are real β€” will that be your next project? How much taxpayer money will you be spending on this?"

LOL, @beyer.house.gov ftw

10.07.2025 14:50 β€” πŸ‘ 5309    πŸ” 851    πŸ’¬ 167    πŸ“Œ 60

Anyone got a good alternative to Pocket as a read later / stash a copy of an article tool?

13.06.2025 23:13 β€” πŸ‘ 65    πŸ” 7    πŸ’¬ 44    πŸ“Œ 3
Preview
Events - THE AI CON Upcoming CTMF-GAIL Seminar (Edinburgh), June 20, 2025 Alex will do a fireside chat with Shannon Vallor, hosted by the University of Edinburgh’s Generative AI Laboratory. UTS (Sydney), June 25, 2025 Em...

The AI Con book tour is headed abroad! @alexhanna.bsky.social and I aren't traveling together but check out thecon.ai/events for our upcoming events, starting with Edinburgh (Alex), Sydney (Emily) and Melbourne (Emily)

13.06.2025 23:51 β€” πŸ‘ 70    πŸ” 17    πŸ’¬ 8    πŸ“Œ 3
Preview
From Data to Dialogue: Why Pure Objectivity in Analytics is a Philosophical Dead End Rethinking data visualization through the lens of phenomenology and hermeneutics In boardrooms around the world, a familiar refrain echoes: "Just show me the facts." "Let the data speak for itself.

A great post from Dr Marco Motta on the pursuit of objectivity in data storytelling #dataviz #dataBS. Marco is both an expert in data viz and has a PhD in philosophy - so he has some unique perspectives that are worth engaging with www.linkedin.com/pulse/from-d...

11.06.2025 21:26 β€” πŸ‘ 6    πŸ” 1    πŸ’¬ 0    πŸ“Œ 0

I have an opportunity to hire a staff scientist for my lab. Looking for someone with outstanding skillset in ML/statistics, genomics applications; interest in mentoring, strong publication record, PD experience required.

Email CV to me+cc my assistant (see 'contact' on my website). Ad to follow.

01.06.2025 15:33 β€” πŸ‘ 82    πŸ” 108    πŸ’¬ 3    πŸ“Œ 3

the peak moment of my career was co-inventing (with Roman Garnett) Bayesian optimisation, only to discover just before we submitted that someone had already invented it a few decades before

29.05.2025 12:25 β€” πŸ‘ 41    πŸ” 1    πŸ’¬ 2    πŸ“Œ 0

our priority is to invest in the systems that prevent your stuff from being covered in poop in the first place

25.05.2025 12:13 β€” πŸ‘ 248    πŸ” 12    πŸ’¬ 2    πŸ“Œ 7
Preview
Software Engineer Intern DuckDB Labs provides services around the DuckDB in-process OLAP data management system directly from its main developers.

DuckDB Labs is looking for software engineer interns:
duckdblabs.com/jobs/Softwar...

Apply until June 15 and join us in Amsterdam!

13.05.2025 12:27 β€” πŸ‘ 30    πŸ” 16    πŸ’¬ 0    πŸ“Œ 0

Extremely +1 to DuckDB!

05.04.2025 08:32 β€” πŸ‘ 8    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0
Preview
Welcome to the age of AI-generated economic collapse On AGI, AI-generated tariffs, and what really threatens the social order

After all the discourse over whether super-intelligent, ultra-powerful AI is coming, we now have a prime example of exactly how AI is massively disruptive, today: When it is used by lazy, reckless, and malicious people in power.

Welcome to the age of AI-generated economic collapse, everyone

04.04.2025 17:13 β€” πŸ‘ 825    πŸ” 261    πŸ’¬ 12    πŸ“Œ 24

Textual Web app in Electron 🀣

03.04.2025 15:06 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

What about distributing things like Dash and streamlit apps as electron apps? I imagine that could be a viable strategy

03.04.2025 12:09 β€” πŸ‘ 2    πŸ” 0    πŸ’¬ 2    πŸ“Œ 0

That's some bona fide data BS right there πŸ‘Œ

03.04.2025 06:54 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0
Preview
Technology Radar | Guide to technology landscape The Technology Radar is an opinionated guide to today's technology landscape. Read the latest here.

Thoughtworks have published their latest #TWTechRadar.

A few 'blips' (as they call them) of note in the data space:

🟒 Adopt: Data product thinking
🟒 Adopt: Trino
πŸ‘ Trial: Databricks Delta Live Tables
πŸ‘ Trial: @metabase.com
βœ‹ Hold: Reverse ETL

www.thoughtworks.com/radar

#dataBS

02.04.2025 10:13 β€” πŸ‘ 10    πŸ” 3    πŸ’¬ 0    πŸ“Œ 0

This is a thing that seems to keep recurring for me too. Any #databs peeps have any ideas?

02.04.2025 08:03 β€” πŸ‘ 3    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0

If you're using duckdb in a python script or jupyter notebook, you can run con.execute('CALL start_ui()') at any point, and the ui will pop right up in your web browser with the current database automatically available.

(I knew about the UI, but I had missed this trick!)

01.04.2025 06:28 β€” πŸ‘ 6    πŸ” 2    πŸ’¬ 0    πŸ“Œ 0
Post image

Are you training self-supervised/foundation models, and worried if they are learning good representations? We got you covered! πŸ’ͺ
πŸ¦–Introducing Reptrix, a #Python library to evaluate representation quality metrics for neural nets: github.com/BARL-SSL/rep...
πŸ§΅πŸ‘‡[1/6]
#DeepLearning

01.04.2025 18:24 β€” πŸ‘ 27    πŸ” 9    πŸ’¬ 3    πŸ“Œ 2

Haha well I'm sold then

31.03.2025 09:58 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0
Preview
Vector Technologies for AI: Extending Your Existing Data Stack - MotherDuck Blog Understand when to use a vector database and how it differs from vector search engines. | Reading time: 17 min read

My latest article explores vector databases, their differences from vector engines, and how to integrate them into your existing Data Engineering Landscape.

28.03.2025 13:53 β€” πŸ‘ 12    πŸ” 1    πŸ’¬ 1    πŸ“Œ 0

+1 to the suggestions of trying DuckDB to do the indexing and then querying. Will be one of the faster ways I would expect and it has great dplyr integration

30.03.2025 02:37 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0
Preview
Analysing Russian news via Telegram, processing them with open LLMs – tadadit.xyz Testing alternatives for investigating public discourse.

New post, in which I indulge in exploring categorisation of text with open LLMs, and along the way incidentally develop an R package to facilitate relevant workflows.

Analysing Russian news via Telegram, processing them with open LLMs - tadadit.xyz/posts/2025-0...

18.03.2025 14:52 β€” πŸ‘ 6    πŸ” 2    πŸ’¬ 1    πŸ“Œ 0

I feel like you're gonna get a lot of head nodding up until the Rust part. What would you say to dyed-in-the-wool Pythonistas like myself to sell them on Rust over Python for data stacks?

21.03.2025 02:17 β€” πŸ‘ 3    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0
Preview
Exporting Notebooks from DuckDB UI

🌟 The new @duckdb.org UI is awesome! But what about the all the SQL you write in it?

I cobbled together a script that dumps the SQL out of notebooks to files, from where I can then easily upload them to gist :

πŸ”— rmoff.net/2025/03/19/e...

#databs

19.03.2025 17:42 β€” πŸ‘ 20    πŸ” 5    πŸ’¬ 0    πŸ“Œ 0
Book Club | Jenna Jordan The first rule of Book Club is we have fun talking about data/information & philosophy

Book Club is tomorrow at 7pm ET

We’re discussing Data & Reality Chapter 5: Attributes

Join us for fun philosophical discussions about data modeling!

jennajordan.me/book-club/

#datasky #databs

20.03.2025 01:52 β€” πŸ‘ 4    πŸ” 1    πŸ’¬ 4    πŸ“Œ 0
Preview
Programs: Byte Into IT – 19 March 2025, Byte Into IT β€” Triple R 102.7FM, Melbourne Independent Radio An episode of Byte Into IT on 19 March 2025

I was interviewed by @attacus.net for last night's broadcast of Byte into IT on RRR Melbourne Independent Radio. This is such a great conversation about my new book! Honestly one of my favorites so far. Huge thanks to @attacus.net. Listen to the episode here. www.rrr.org.au/explore/prog...

20.03.2025 03:26 β€” πŸ‘ 19    πŸ” 3    πŸ’¬ 1    πŸ“Œ 0
Preview
The data domain is about to get so fooked by β€œAI” But not the way you would think

Sometimes some #DataBS thoughts just have to come out:

open.substack.com/pub/agiledat...

14.03.2025 03:49 β€” πŸ‘ 8    πŸ” 1    πŸ’¬ 2    πŸ“Œ 0

I've been thinking a lot about the rich and wonderful challenges and risks of integrating AI into organisational ecosystems... aaand now I'm adding exploding integrations to the pile. It feels kind of obvious in hindsight, but I definitely haven't thought about it as explicitly as you presented it.

15.03.2025 03:07 β€” πŸ‘ 2    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0

@ned.sh is following 20 prominent accounts