EJD

I can't provide support if I'm the developer, and I don't have the admin privileges to access the data. Also, even if I had privileges, I'm not sure it is kosher to try to build data access for data behind a login in an open source project.

I'll still hold on to my code in case of them making this publicly available again. But for now, the fuels charts won't work.

Unfortunately for FireWxPy users, it looks like now you need government credentials to access FEMS, otherwise you get an HTTP 403:Forbidden error. Guess I'll have to remove the fuels charts from FireWxPy because they won't work anymore.

Totally not a wannabe-patriot anti-science death cult. @shawnmilrad.bsky.social

Had a great day mountain biking at Snow Valley, California. #cawx #mtb

Operational forecasting now feels like a chore to me lol.

It's amazing how much interests can change in 10 years. When I was going into my senior year in college 10 years ago, I had zero interest in computer programming and was soley interested in forecasting. Now I am much more passionate about computer programming than I am about forecasting.

A criminal doing business with a genocidal criminal regime. Birds of the same feather flock together.

Remember kids, Donald Trump build a Trump Tower in Baku and Ivanka had a no-show consultant job on the project before Donny took office the first time!

2024 was her chance. I think the Democratic Party needs someone new in 2028, especially after she lost to Donald Trump of all people in 2024. I want Andy Beshear 2028.

Summer-to-date temperature anomaly for Alaska. @alaskawx.bsky.social

It's always a good day when the Mets win and the Phillies lose! #LGM

I also want to learn C++ with PyBind11. I have seen some tutorials on it. But I also need to learn C++ since I've never wrote C++ code before...😂

One application I am thinking of using Cython is for performing calculations on raster data in PyClimo. It takes several minutes when calculating new fields for each point on the geotiff PRISM data across CONUS when using Python. I suspect Cython will significantly speed up the process.

When I was much newer to development a couple years ago, I would download the data eachtime I ran the script via NOMADS OPENDAP and sometimes it resulted in a 5-min ban from data access. The scanner will help newer developers who might not be fully aware of these things so that they avoid them.

I plan to use wxdata to power the FireWxPy data access once it is done.

One thing I am building into wxdata is a scanner for the purpose of being nice to the servers. The scanner will determine if the latest run is in and if the files locally on your PC are up to date to avoid downloading new data if you are creating a bunch of graphics from the same data at once.

I am also starting to teach myself Cython because I think it can be useful for certain applications when preprocessing data. I think it can be good for the calc module because Cython is a C-extension of Python and thus runs closer to the speed of C/C++ to optimize performance.

Downloading a single set of data (GEFS0P25 Ensemble Mean) takes about 2-4 minutes.

I suspect no because this is file transfering over the internet. I found the GEFS0P25/GEFS0P50 takes about an hour for all 30 members. GEFS0P50 Secondary Parameters takes closer to 2 hours given those files are much bigger in size.

When downloading all 30 GEFS members via HTTPS in Python, the function takes a couple hours to run and I want to speed that up. I have 1000MB/s internet. I suspect the answer is no but I'll ask anyway: Is this something that using a lower level language would speed up or no?

I also used urllib to download the files from NOMADS via HTTPS.

4) Use xarray open_mfdataset() in a for-loop going through each ensemble member folder. Set concat_dim='step'.
5) Create a ds_list of all concatenated xarray data arrays from each folder.
6) Then use ds = xr.concat(ds_list, dim='number')

Done!

🧵Nvm I figured out a solution. Here is my method:

1) Use os to create a folder for each ensemble member.
2) Download the .F### files and use os to move those files into the correct folder corresponding to the ensemble member number.
3) Use os to change the files from .F### to .GRIB2 [1/n]

Trying to think of ways to concatenate all the GEFS ensemble members using open_mfdataset() in xarray along both the 'number' (ensemble) and 'step'. Problem is I get errors if I try more than 1 dimension and if I set concat_dim='step' I lose all but 1 member and concat_dim='number' I lose steps.

It's not a cult. It's a religion.

This violates the Hatch Act on so many levels. Trump Cabinet: "Hey federal workers: Hatch Act for thee but not for me!"

I combine the files by converting the .f{forecast hour} to f{forecast hour}.grib2 and then combine all files with file_pattern=f"{model}/*.grib2"

I also combine all the files into one xarray data array which will make it easier for users. Finally, I built in a function for users who want to make a map spanning the globe that uses cartopy.util add_cyclic_point. I simplified the process for that with the function cyclic_point(ds, param).

This project is still very much in its infancy but I did test out the scanner on the GEFS0p25 on NOMADS and it worked. Downloaded the new files when they updated on NOMADS but didn't re-download current files. Now it's repeating this for everything else.