Web Content Display

Web Content Display (CoastWatch User Forum)

The NOAA CoastWatch Forum provides information in several categories.  The information is relevant to CoastWatch products and software (https://coastwatch.noaa.gov)  and vary in technical details.  For questions and issues not addressed here,  start a thread in the appropriate category or contact the CoastWatch HelpDesk (coastwatch.info@noaa.gov).  Please note our Privacy Policy (https://www.noaa.gov/privacy.html) and that this is a public forum.  All information is voluntarily and anonymous submissions are accepted pending review prior to posting. 

Forums

Back

ERDDAP: SST for range of latitudes and times?

MS
Michael Soracco, modified 4 Years ago.

ERDDAP: SST for range of latitudes and times?

Youngling Posts: 40 Join Date: 8/11/16 Recent Posts

Hi,

I am trying to use the ERDDAP to get multiple dates for multiple latitudes. I have 10 island regions that I want to get yearly mean, min, and max SST for all the years that have satellite coverage. Using ERDDAP, when I specify time as y variable, I only get one latitude option.  Can you advise on how one would go about getting data for each region? Maybe this comes later in the course and is coded in R?

 

https://oceanwatch.pifsc.noaa.gov/erddap/griddap/CRW_sst_v1_0_monthly.graph?analysed_sst[(last)][(12.025):(21.025)][(143.525):(148.025)]&.draw=surface&.vars=longitude%7Clatitude%7Canalysed_sst&.colorBar=%7C%7C%7C24%7C31%7C&.bgColor=0xffffffff 

 

Thank you,

Eva

erddap regional data multiple latitudes or longitudes
MS
Michael Soracco, modified 4 Years ago.

RE: ERDDAP: SST for range of latitudes and times?

Youngling Posts: 40 Join Date: 8/11/16 Recent Posts

That’s because you are plotting the data - and ERDDAP can only visualize 2 dimensions, not 3. But you can download 3 dimensions which is what you want to do. From that page click the data access form link in the upper right to get: 

 
MS
Michael Soracco, modified 4 Years ago.

RE: ERDDAP: SST for range of latitudes and times?

Youngling Posts: 40 Join Date: 8/11/16 Recent Posts

Hi Eva,

Yes, we do go over something like this in the R tutorials, but as a start, you can click on "Data access form" under the dataset name, to the right.

This will give you more options for downloading a range of dates for a region.

To do what you describe though, it will probably be more practical in R. You can't compute statistics within the ERDDAP webpage, you'll need to download data and work with it in R.

 

Melanie

erddap data access form
MS
Michael Soracco, modified 4 Years ago.

RE: ERDDAP: SST for range of latitudes and times?

Youngling Posts: 40 Join Date: 8/11/16 Recent Posts

This is a common problem. What ERDDAP is set up to do visualizing-wise, is a single time point across space, or a time-series from a single point. 

What you're asking to do would be to do one download for each of your ten spatial bounding boxes across the whole time-series, then summarize it after.

Using monthly data (as you are) is very wise here, if you are just going to summarize by year afterward. 

 

Were I you, I'd do these by hand through the viewer, try downloading 10 different .csv files, and then load into R, and summarize by year. There are a number of ways to do this, and learning it is a superpower.

Easiest/most accessible is to use a pivot table in Excel. In R there are a few ways, either using something like aggregate (base R), or ddply (from plyr), or dplyr (group_by/summarize) if you're using the tidyverse set of functions. Let me know if you need help there...

 

Tom

 

Update:

One note of caution, whenever downloading "all years of a time-series" you can run into the dreaded "Proxy Error" by asking for too much data at once. If it happens, what you do is download your data in time sections, and add 'em up later. With monthly data over a smallish area, you should be fine. Just don't try to download the whole Pacific at once...

 

Tom

MS
Michael Soracco, modified 4 Years ago.

RE: ERDDAP: SST for range of latitudes and times?

Youngling Posts: 40 Join Date: 8/11/16 Recent Posts

Another option is to download one set of data that covers the whole area, ie all 10 islands, and then do the spatial subsetting in R so you don’t have to mess around with 10 different areas and files.  As with most things, there is more than one way to do it. Since the dateset does not have that high of a spacial resolution you should be able to get the whole area.  

 
Cara
r spatial subsetting
MS
Michael Soracco, modified 4 Years ago.

RE: ERDDAP: SST for range of latitudes and times?

Youngling Posts: 40 Join Date: 8/11/16 Recent Posts

Cara's totally right, that there's more than one way to get this to work, and repeating it 10 times can be a pain. 

If you're comfortable locally subsetting (i.e. in R), and your islands are close together, one big download is definitely the way to go.  

 

For me, I'm usually looking at tiny islands with huge areas of open ocean between, and downloading a whole area tends to bump up into 'problematic' amounts of data (i.e. Proxy Errors, etc), for data that's mostly useless to my analysis.

At ESD, we found island-scale makes a nice middle way for local subsetting. That is if you're after site-level data, download an island's worth of data, for each island, and locally subset your sites, but don't download an ocean's worth, even if you have to repeat it a finite number of times. Interestingly, 10 times is about when the pain of repeating downloads starts to match the pain of a single big download...

 

Anyway, lots of ways to move ahead.

 

Tom  

MS
Michael Soracco, modified 4 Years ago.

RE: ERDDAP: SST for range of latitudes and times?

Youngling Posts: 40 Join Date: 8/11/16 Recent Posts

Hi Eva,

 

OK, this is verging on "too many cooks", but I'll add my two cents. As Melanie pointed out, the R Xtractomatic examples cover what you want to do, including stats. If you want to do it sans Xtractomatic, then the approach Tom suggested is great, with one small possible change. You could download the data as 3D netCDF files (lat range, lon range, time range), rather than CSV. NetCDF is a binary format and therefore a much smaller and quicker download than CSV files. That way you can likely avoid any Proxy Errors. Once you download the netCDF, there are some great tools for running stats on arrays. I'm more familiar with Python (where xarray and numpy modules would work), but I am sure that something like R Tidyverse would work for you. Tom and Cara can help you with that. Or if you wanted to use Excel you could flatten the 3D arrays and bring the resulting 2D lists into Excel to do your analysis on

 

I hope this did more to help than to harm.

 

-Dale

r xtractomatic tidyverse 3d arrays netcdf