Xander Garbett's Avatar

Xander Garbett

@xandergarbett.bsky.social

SRE or O11y engineer depending on the hat.

35 Followers  |  249 Following  |  45 Posts  |  Joined: 11.09.2024  |  2.1057

Latest posts by xandergarbett.bsky.social on Bluesky

Temporal (Java Platform SE 8 )

Or the even terser:

elapsed := time.Since(start)

I will say, that Joda time (and Java 8 time) set the gold standard for me.

date = date.with(next(WEDNESDAY));

Is magic when you need it!

docs.oracle.com/javase/8/doc...

12.01.2026 15:01 β€” πŸ‘ 2    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0
Preview
Terminal-Bench A benchmark for terminal agents

Terminal bench feels kinda close in what you are describing: www.tbench.ai/news/announc...

I imagine that it would be a similar eval to the task structure, but with different skills to try the evals

08.01.2026 14:26 β€” πŸ‘ 2    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0

I saw @samwho.dev talking about their voice set up for coding, I think they were suggest Talon or MacWhisper for voice.

07.01.2026 15:56 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

Wow, that sounds horrible. Glad to hear it’s over, and looking forward to reading the retro.

18.12.2025 22:01 β€” πŸ‘ 2    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

Works well in an internal setting, but would probably need to make it into the OTel client libs to be seamless.

09.12.2025 13:47 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0

When I was doing internal o11y at ClickHouse, for logs we’d mainly rely on the insane compression + SharedMergeTree (aka, blob store backed) engine. Was pretty cheap and pretty compressible.

I reckon something like a query time join could handle this. But you’d need to have pretty opinionated logs

09.12.2025 13:47 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

(More thinking aloud, sorry for spam) I guess just some reservoir sampling would work well here. Just make the service have a size quota, rather than log number quota.

08.12.2025 18:38 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

Are you thinking head sampling in this case? My worry with tail sampling is always putting pressure on a few hot shards

08.12.2025 18:31 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

Probably too easy to make something degen with a few small changes and flood the blob store though.

08.12.2025 18:28 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0

Wonder if there’s any scope to upload some of the blobs directly to blob storage with some log collection smarts. So at the very least you don’t saturate the full pipeline e2e with very large payloads. If it’s content addressed storage you probably get most of the way there

08.12.2025 18:27 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 2    πŸ“Œ 0

PEBCAK is another great one. β€œProblem Exists Between Chair And Keyboard”

27.11.2025 12:19 β€” πŸ‘ 13    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0
Preview
Who’s right about AI: economists or technologists? Forecasting the impact of artificial intelligence has become fraught, with evangelists pitched against sceptics

on.ft.com/485QAfv Link to the article is you want.

11.11.2025 06:19 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

Literally the rest of the opinion piece is saying that this is unrealistic, given the chart is reporting data from somewhere else?

11.11.2025 06:17 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 2    πŸ“Œ 0

404 on the Jagex link btw

15.10.2025 08:35 β€” πŸ‘ 2    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

There is something about being able to _use_ your own product that simply makes it better. I think its one part of the reason iBlocks just did better.

04.10.2025 21:19 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0

I use GWR reasonably often, and do enjoy getting the emails from the system unchanged from 5 years ago. I can tell because the GWR logo is still the slightly pixelated one because they couldn’t get us the assets for go-live, and no one has complained!

04.10.2025 21:03 β€” πŸ‘ 3    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

I have very fond memories of working at iBlocks, a tiny company against Fujitsus and Wordlines behemoths, and just winning and delivering against their glacial pace. I think they still run all the data distribution for the industry (fares, routing, timetables via the dtd)

04.10.2025 20:56 β€” πŸ‘ 2    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0
Preview
Delay repay for Go-Ahead | Case study | Tracsis Go-Ahead approached iBlocks looking for a solution that could significantly reduce the cost of processing Delay Repay claims, reduce time from claim submission to award and flag fraudulent claims.

I’ll also say, that there are still TOC run call centres and customer care that needs to exist, despite the automation. Not everyone is familiar, so you need that assistance too. But the case study of how much money got saved is cool! tracsis-iblocks.com/case-studies...

04.10.2025 20:49 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

Yeah, there is some matching that makes the booking system line up, but there were a few failed pushes to make it unified that never made it to the end. Too much of the industry was split into parts run by vendors. Too much politicking by RDG and bigger players!

04.10.2025 20:45 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

So, really, it’s already been hived off to be a separate company! It was pretty much 3-4 people who built and ran the system, hitting 85-90% automation rate on the claims if I remember.

04.10.2025 20:35 β€” πŸ‘ 2    πŸ” 0    πŸ’¬ 2    πŸ“Œ 0

One of the magical things we could do was automatically link advance purchase tickets to the delayed trains. So part of the system automatically emails customers on delayed advance purchase trains

04.10.2025 20:22 β€” πŸ‘ 2    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

The delay repay bit of Avanti, GWR, SWR, Southern and ThamesLink (among others!) is all run by one company external to the TOCs. Used to be a tiny company called iBlocks who got bought by Tracisis. I helped build parts of that DR system!

04.10.2025 20:20 β€” πŸ‘ 4    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0
Preview
Incremental materialized view | ClickHouse Docs How to use incremental materialized views to speed up queries

ClickHouse does this (and has for some time!) clickhouse.com/docs/materia...

It scales pretty well, we handle 10s of millions of events per second in our clusters without really having to worry about it! And you can chain them to so A -> B -> C data flows can happen

25.08.2025 08:40 β€” πŸ‘ 2    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0

ClickHouse instances out there, which is perfect for dogfooding the product! We can find the issues that happen at scale and feed that into development.

21.06.2025 07:13 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

It’s a tricky balance, and we’re trying to shift things (slowly). The main problem is if you are used to being able to get onto any ClickHouse instance and do "select from query_log", that’s the expectation going forwards. There is a secondary benefit though, in that LogHouse is one of the larger

21.06.2025 07:13 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

sampling part, because the kubelet had often rotated the logs before we’d got a chance to read them.

20.06.2025 17:36 β€” πŸ‘ 2    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

ClickHouse stack frames meant we could identify code paths running hotter on deployed versions.

2. Once we had all the wide data going directly to CH via SysEx, we could drop the log levels down so we didn’t have to engage in a CPU arms race. As the blog mentions, we didn’t even get to the

20.06.2025 17:36 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

There’s two parts to it (IMO)

1. The system tables were definitely going through a lossy transformation of CH -> logs -> CH.
Having the full fidelity there gives us some pretty incredible visibility. Being able to slice a query pattern across all instances and have that be linked to the

20.06.2025 17:36 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0
Preview
Scaling our Observability platform beyond 100 Petabytes by embracing wide events and replacing OTel Read how we scaled our observability platform from 19PB to 100PB and 500 trillion rows by replacing OpenTelemetry with a native ClickHouse-to-ClickHouse pipeline, embracing wide events and cutting CPU...

Some fun things we’ve been up to! clickhouse.com/blog/scaling...

19.06.2025 18:13 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0

(Obviously biased that I think ClickHouse is the best store for this ;) )

09.05.2025 14:12 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

@xandergarbett is following 20 prominent accounts