mdsumner

Australia’s gambling harm is likely underreported – and authorities are still failing to act New research shows Australian gambling harms, including suicide, may have been systematically under reported.

"a lot like a protection racket for the gambling industry" is a very polite way of describing current state of gov't action.
#Auspol

Australia’s gambling harm is likely underreported – and authorities are still failing to act
theconversation.com/australias-g...

#rstats what do folks do when converting from raster to terra with saved data objects? active bindings is a possibility

github.com/ropensci/gra...

interested in any perspectives, I don't want to change the entire way the package works (I'm not the maintainer, just toying with ideas)

New blog post! kylebarron.dev/blog/fast-py...

This dives into performance pitfalls of Python bindings, why projects like Lonboard are so fast, and why I care about Apache Arrow so much. The post is high level and you don't need to know C/C++/Rust to understand it!

aye to that

The solution was to not use coord_sf(). I need to remember to not use sf in any project unless absolutely necessary.

in short:
it’s complicated and we’re fucked

#climatecrisis

Make fossil fuel exporters pay for the damage they are causing, not ordinary Australians Climate change is making natural disasters like fires, floods, heatwaves and drought worse, costing Australians billions every year. Make fossil fuel producers pay a climate disaster levy to help pay...

Make the fossil fuel barons pay, not the tax payers
#auspol

Great story on ABC #4Corners now on how online gambling companies operate, the damage they do to the community & how they keep the major political parties in the pockets.

Optimal GeoParquet Partitioning Strategy My quest for one-size-fits-all format

Definitely depends what you're up to! People are definitely experimenting with serving features directly from Parquet files for visualization (e.g., medium.com/center-for-c... )

is Scott Morrison in jail yet?

Impressive to outdo John Howard for evil

This evening, President Trump is ordering 400 members of the Texas National Guard for deployments to Illinois, Oregon, and other locations within the United States. No officials from the federal government called me directly to discuss or coordinate.

Watch | Israeli occupation forces have released new drone footage from Gaza, revealing the mass destruction and the erasure of every means of life in the Strip.

Amsterdam, we salute you!
#Auspol

The claim seems widely overstated because in the end it is about selecting the partitioning in order to push as much computation into storage layer as possible. Selecting the partitioning is mostly about access pattern just like db indexes

Oh yes I'm using gdalwarp here.

User draws on wmts and it looks at a .vrt file to match which geotiffs it overlays. That way I don't have to display the tifs.

Thinking of adding more crs options for the output and choice between geotiff and jpg2000. Just prototyping this

Why GeoParquet Is A Poor Cloud Native Format · cloudnativegeo cloud-optimized-geospatial-formats-guide · Discussion #82 (Geo)Parquet is an extremely useful column oriented data format. When working with local, massive data sets having many millions of features, the performance of this format is second to none. Plent...

Found a rich seam here: github.com/cloudnativeg...

Got the gdal clip working. First time hearing about ALB, security groups, ECS, services, task definitions, cloudfront, a bunch of other bs. I think I can scale this (so says Claude 🤷‍♂️)

- FGB is great! Lookups with small ranges work well but full scans are much slower (particularly if zipped!)
- SedonaDB, DuckDB, and pyarrow with fsspec can all do partial Parquet reads from http (give geoarrow.org/data a try!)

I did a filter on the ibis example

I think you're talking about tooling problems

Nice! Could use gdalwarp (gdal raster reproject) to clip and write (and resize or change crs) in one step fwiw

(gdal raster reproject has a --bbox-crs arg, and this can be pipelined or written in json as a GDALG virtual dataset)

draw custom polygon over wmts imagery service in web mercator, converts to UTM zone 19 and clips the corresponding geotiff (4band, 15cm, 16bit) and delivers to user

Hacking together a Clip-Zip-Ship tool: draw custom polygon over wmts imagery service in web mercator, converts to UTM zone 19 coordinates and clips the corresponding geotiff (4band, 15cm, 16bit) and delivers to user.

Next: Select town as clipping boundary, give 100m buffer

(open issue in nanoparquet was the unfinished bit there)

right, you mean arrow open_dataset() won't open a URI?

I don't know why that is (or if it's an isolated problem, it's an open issue in )

gist.github.com/mdsumner/e55...

I don't see any issues with queries otherwise in ibis or GDAL

not sure what you mean otherwise

If you sozip that FGB, it becomes 2GB as well.

Bonus points. Try accessing that GeoParquet remotely on a simple web server. You can't. Not without a backend. Uncompressed FGB? No problem.

lgtm (it surely can take time for support to roll out for new facilities, and downstream noise always complicates the scene, but it's not inherently bound to a backend)

gist.github.com/mdsumner/e55...

I cannot say enough about DataFusion...in order to build an engine that considers spatial types at every level we needed to customize types, functions, optimizer rules, joins, Parquet pruning, and more. DataFusion not only made this possible but documented even the most obscure bits. So cool!

Interesting...I think of GeoParquet and GeoArrow as tech that allows more people to avoid the cloud (i.e., use your laptop to do things that everyone thought they needed "the cloud" for five years ago). When your 14 GB .fgb files become 2GB GeoParquets, you can fit a whole lot more of them locally!

my most charitable opinion is that I also don't love that basically all software engineering currently requires using bespoke tooling and renting clock time from 1 of 3 corps

but that does mean that everyone everywhere needs tools for dealing with those corp's tech... not just the hyperscalers

🚨 “They put black people in one van and immigrants an another… Trucks and military style vans were used to separate children from their parents.” THIS. IS. HAPPENING. People need to see this. We all need to see it. Nobody gets a pass here. Please share with your networks. 1/2

R-multiverse: a new way to publish R packages · Community Call Community Call about R-multiverse

You might also be interested in the R-multiverse built on top of R-universe.
ropensci.org/commcalls/r-...