Carter Sifferman

congrat

Surprising to see mention of Piranesi on here. Love that book, and Jonathan Strange and Mr Norrell.

Early on in my PhD I thought "wouldn't it be nice to do extrinsic calibration without a calibration target". After (very little) searching, I learned about SLAM.

Watching the markets is a fun way to take in the news - it's interesting seeing how they react to new events

Who will be inaugurated as President? Polymarket | This market will resolve to "Yes" if Donald J. Trump is inaugurated as President of the United States. Otherwise, this market will resolve to "N...

There's a market for that. 98.2%: polymarket.com/event/who-wi...

What's so bad about this? I think the proliferation of gambling is bad, but these markets are good at aggregating info and have been shown to provide accurate odds. Markets like this one seem valuable to e.g. people affected by the fire who are trying to get a good estimate of its duration.

On the other hand, some papers have shown that training on unrealistic synthetic data forces the NN to learn the essential features of the problem, e.g.: openaccess.thecvf.com/content_cvpr...

Realistic isn't always best, but having accurate g.t. is definitely important and a separate issue.

Thanks for making it! Seems like it helped a lot of people get connected. Let’s hope they actually visit the platform and it sticks 😊

Of course, others are doing excellent work as well (too many to fit in one post):

Few-view 3D reconstruction with high resolution sensors: weihan1.github.io/transientang...

Handling specular / mirror surfaces:
arxiv.org/abs/2209.03336

Detecting human pose:
arxiv.org/abs/2110.114...

We have one paper tackling the general 3D reconstruction problem:
cpsiff.github.io/towards_3d_v...

And more on specific applications of these sensors on robotics, which utilize histogram info (+ one in review, stay tuned):
cpsiff.github.io/using_a_dist...
cpsiff.github.io/unlocking_pr...

If we can figure out how to take full advantage of ToF histogram information, there's the potential for huge improvements on any inference task (recognition, detection, segmentation) and on 3D reconstruction.

In many applications, the peak of the histogram (which roughly encodes the average distance to the scene in the pixel) is the only information used. But this throws out most of the rich scene information they encode.

Image of AMS TMF8820 ToF sensor held between two fingers. The sensor is roughly the size of a grain of rice.

Sony IMX459 automotive LiDAR

There exist a wide range of sensors which capture this data, from tiny proximity sensors (which my research focuses on) to automotive LiDAR and benchtop lab-grade setups.

Animation showing 3D geometry on one side, and the resulting transient histogram on the other. As the geometry changes, the histogram changes accordingly.

The quantized version of this signal is called the "transient histogram" or sometimes just "ToF histogram".

When the per-pixel FoV is wide, this histogram encodes rich information about the scene, as shown in this awesome animation by my labmate Sacha Jungerman (wisionlab.com/people/sacha...).

Diagram demonstrating a ToF sensor with a wide field-of-view. The outgoing light pulse has one peak, but the returning light pulse has two peaks due to the two prominent depths in the imaged scene region.

Direct ToF sensors send out a pulse of light, and measure the time it takes for that light to bounce off the scene and return.

Recently, a new class of these sensors have emerged that measure the intensity of returning light over very short (pico-to-nanosecond) timescales.

Since I have a new disparate set of followers on here, let me tell you about an emerging modality that's important to my research: 🙌 Time-of-Flight Histograms 🙌

Figuring out how to best utilize this modality has implications for the future of robotics, remote sensing, and 3D vision

🧵thread below 🧵

I've noticed this same issue with methods for 3D human pose / hand pose estimation. The depth map and 2D projection look great, but when you use a depth camera and visualize the prediction alongside the point cloud it's way off in 3D.

Welcome to all new arrivals here on Bluesky! :) Here's a starter pack of people working on computer vision.
go.bsky.app/PkAKJu5