-
-
Notifications
You must be signed in to change notification settings - Fork 194
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Trying to spatially predict(type="se")$se and the output has many null values and what look like spatial offsets? #668
Comments
I had the same issue @KWB4484! I solved it by converting the raster to data.frame. But the bigger problem was that my raster is so big (more than 111 million cells) so I had to tile it and do this in parallel. While doing that realized the "se" calculating varies between the tiles!!!! |
It seems the length of the new data affects se calculation. |
Thanks for the suggestion @rvalavi! I'll give it a try. I'm assuming you can retain xy and convert it back to raster after calculating the standard errors? Fingers crossed I have enough memory. |
My pleasure, @KWB4484! The model fitting with ranger is fantastic with very little RAM requirement, but I wish the prediction would have been the same story. |
See also #136. Also consider quantile regression or the forestError package? |
@rvalavi Well I tried going the data.frame route but I'm in the same boat -- too big --
and given your experience with tiling I'll just quit now. Thanks for saving me time! Thanks @mnwright, I'll try the forestError package. |
In short, I've trained regression models for 20 ecological units across the US by mapping this function to nested training datasets for each unit. Example:
I then create a raster stack of covariates, I join ecological unit boundaries to the "EcoModels" tibble and essentially map a function where for each row the raster stack gets crop/masked using the ecological unit boundary, the model is applied to the raster stack and the output is written to disk.
This worked well for response predictions, but my computer ran out of memory when trying to get standard errors. Luckily, I was just given the opportunity to use a computer with a ridiculous amount of RAM and I got it to run....but the output had lots of null values and spatial chunks that look like some offset was applied.
I checked the raster stack and it appears fine. Is there something that I'm doing wrong or is this normal because of xyz? Thanks for any ideas you can provide for troubleshooting this.
Standard Errors
Response predictions
Units
Example SE for one unit
The text was updated successfully, but these errors were encountered: