Web Access to RSIG Data

Researchers with sufficient programming experience can use a web service to obtain data from RSIG and Estuary Data Mapper (EDM) without having to use the RSIG or EDM applications.

End users who are interested in bulk data transfer may opt to use the web service directly, or incorporate it into an automated script.

You may also want to read more about web service queries and essential performance tips.

About the web service

The RSIG web service (rsigserver) is OGC-WCS/WMS-compliant.

The following links exit the site

Essential information about the web service is as follows:

  • The purpose of rsigserver is to stream (over a network) aggregated (across multiple source data files) subsets (by time-range, variables, longitude-latitude-layer domain) of data into client applications (without the need for first writing files to disk, as in ftp).
  • EMVL extends rsigserver each time a new data source is requested for RSIG.
  • All data are GMT-hourly.
  • Longitude, latitude, elevation (meters above MSL when available), and GMT timestamps are always included implicitly to facilitate geospatial-temporal data alignment.

Web service for RSIG data

Source Description Variables
EPA AQS AQS Data Mart ground stations. pm25, pm25_daily_average, ozone, ozone_8hour_average, ozone_daily_8hour_maximum
EPA Airnow AIRNow ground stations. pm25, ozone
EPA CMAQ CMAQ modeled gridded met and air-quality data. o3, ta, uwind, vwind, etc.
NESDIS GOES-BB Satellite-derived biomass burning ground sites. pm25, co, co2, ch4, n2o, nh3, nox, so2, tnmhc
EPA UVNet UV irradiance at sparse ground points. irradiance
EPA FAQSD Statistical fused model and site data exposure surface. 12km.pm25_daily_average, 12km.ozone_daily_8hour_maximum
NASA CALIPSO NASA CALIPSO satellite LIDAR profile data (half-orbit-long line on the ground with 583 vertical values per ground point). total_attenuated_backscatter_532, perpendicular_attenuated_backscatter_532, attenuated_backscatter_1064, etc.
NASA MODIS NASA MODIS satellite data. 2-D patches on the surface of the Earth from two satellites, Aqua and Terra. mod4.optical_depth_land_and_ocean, mod7.surface_temperature, mod7.surface_pressure, mod7.surface_elevation, mod7.total_ozone, mod6.cloud_optical_thickness, etc.

WCS Output data formats

Format Description
ASCII Tab-delimited spreadsheet. Big and slow! Only use for importing a tiny subset of data into a spreadsheet.
XDR Simple, predictively parsable, efficient, fast, portable binary. Recommended format for streaming data into applications.
NetCDF-COARDS COARDS conventions NetCDF binary. Must be written to a file and then opened with nc_open.
NetCDF-IOAPI For CMAQ data only: IOAPI-conventions NetCDF file.
Original Original (unsubsetted) satellite files (tarred and gzipped). Huge and slow!

WMS Output image formats

Format Description
image/png Portable Network Graphics image format for a single timestep.
image/mpg Portable movie format for a multiple timesteps.
image/tiff Geotiff-RGB geo-referenced images for multiple timesteps (zipped).
image/tiff32 Geotiff-float (32-bit) geo-referenced rasterized data for multiple timesteps (zipped). Note data is lossy due to rasterization resolution not matching data resolution. E.g., MODIS ground points at 10km spacing vs rasterized pixel spacing.
application/vnd.google-earth.kmz Google Earth format.

Command-line examples using cURL

To help craft WCS queries, you can use cURL, a command available on multiple platforms for transferring data. As summarized on the cURL home page :

curl is an open source command line tool and library for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMB, SMTP, SMTPS, Telnet and TFTP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, HTTP/2, cookies, user+password authentication (Basic, Plain, Digest, CRAM-MD5, NTLM, Negotiate and Kerberos), file transfer resume, proxy tunneling and more.... curl is free and open software that compiles and runs under a wide variety of operating systems....
The cURL manpage  contains thorough documentation of cURL and its numerous options. R developers can take advantage of the RCurl library  to serve as a wrapper for libcurl in R-based applications.
 
Although these examples use cURL, applications may use their own routines to "retrieve a web page." The RSIG3D districution includes cURL in the "bin" directory.
 

The easiest way to learn these WCS commands is to run RSIG3D and inspect the status window in the lower left hand corner after a data request is made. The exact WCS call used by the application is echoed to the status window, and can be used as a starting point for constructing custom WCS calls.

Note that in the examples below, the curl commands should all be entered on a single line.

List available datasets from the RSIG WCS server

curl --silent --retry 0 -L --tcp-nodelay --max-time 0 'https://ofmpub.epa.gov/rsig/rsigserver?SERVICE=wcs&VERSION=1.0.0&REQUEST=GetCapabilities'

Listing just the names of available variables from a script

curl --silent --retry 0 -L --tcp-nodelay --max-time 0 'https://ofmpub.epa.gov/rsig/rsigserver?SERVICE=wcs&VERSION=1.0.0&REQUEST=GetCapabilities' | grep '^            ' | tr '[<>]' ' ' | awk '{print $2}'

Getting ASCII format data from NESDIS

curl --silent --retry 0 -L --tcp-nodelay --max-time 0 'https://ofmpub.epa.gov/rsig/rsigserver?SERVICE=wcs&VERSION=1.0.0&REQUEST=GetCoverage&COVERAGE=nesdis.pm25&TIME=2007-08-26T20:00:00Z/2007-08-27T01:59:59Z&BBOX=-125,25,-88,55,0,0&FORMAT=ascii'

Getting XDR-format MODIS AOD data from NASA Goddard

curl --silent --retry 0 -L --tcp-nodelay --max-time 0 'https://ofmpub.epa.gov/rsig/rsigserver?SERVICE=wcs&VERSION=1.0.0&REQUEST=GetCoverage&COVERAGE=modis.mod4.optical_depth_land_and_ocean&TIME=2005-08-26T20:00:00Z/2005-08-29T01:59:59Z&BBOX=-90,30,-88,32,0,0&STRIDE=2&FORMAT=xdr&COMPRESS=1' | gzip -d > sample.xdr ; head -15 sample.xdr

Use the STRIDE=2 option only for visualization. (It skips data points.) For data analysis omit that option to retrieve all data points (within the specified BBOX). Also, compression/decompression is optional but recommended when streaming large amounts of data over long-distance networks.

Same as above but including MODIS cell corner points

curl --silent --retry 0 -L --tcp-nodelay --max-time 0 'https://ofmpub.epa.gov/rsig/rsigserver?SERVICE=wcs&VERSION=1.0.0&REQUEST=GetCoverage&COVERAGE=modis.mod4.optical_depth_land_and_ocean&TIME=2005-08-26T20:00:00Z/2005-08-29T01:59:59Z&BBOX=-90,30,-88,32,0,0&STRIDE=2&FORMAT=xdr&COMPRESS=1&CORNERS=1' | gzip -d > sample.xdr ; head -15 sample.xdr

The CORNERS=1 option will result in 8 additional variables: Longitude_SW, Longitude_SE, Longitude_NW, Longitude_NE, Latitude_SW, Latitude_SE, Latitude_NW, Latitude_NE which are the linearly interpolated/extrapolated corner points surrounding each MODIS ground point center to provide quadrillateral cells for each MODIS data point.

Getting CALIPSO satellite LIDAR data

curl --silent --retry 0 -L --tcp-nodelay --max-time 0 'https://ofmpub.epa.gov/rsig/rsigserver?SERVICE=wcs&VERSION=1.0.0&REQUEST=GetCoverage&COVERAGE=calipso.l1.total_attenuated_backscatter_532&TIME=2006-07-05T00:00:00Z/2006-07-05T23:59:59Z&BBOX=-125,35,-65,40,0,0&SPARSE=100&FORMAT=xdr&COMPRESS=1' | gzip -d > sample.xdr ; head -15 sample.xdr

Use the SPARSE=100 option only for visualization. (Experiment with different SPARSE values for a given BBOX.) For data analysis omit that option to retrieve all data points (within the specified BBOX).

Getting CMAQ wind data for the lowest grid layer (in IOAPI format)

curl --silent --retry 0 -L --tcp-nodelay --max-time 0 'https://ofmpub.epa.gov/rsig/rsigserver?SERVICE=wcs&VERSION=1.0.0&REQUEST=GetCoverage&COVERAGE=cmaq.equates.conus.metdot3d.wind&TIME=2005-08-28T20:00:00Z/2005-08-29T01:59:59Z&BBOX=-90,30,-88,32,1,1&FORMAT=netcdf-ioapi' > sample.nc ; ncdump sample.nc | more

To obtain data for all layers, omit the last two values of BBOX:
BBOX=-90,30,-88,32&
To obtain the first three layers:
BBOX=-90,30,-88,32,1,3&
Optional compression/decompression
COMPRESS=1&' | gzip -d > sample.nc
Regridding one day of GASP AOD data to layer one of a 12km East CMAQ grid in Lambert space (saved as NetCDF COARDS format)

curl --silent --retry 0 -L --tcp-nodelay --max-time 0 'https://ofmpub.epa.gov/rsig/rsigserver?SERVICE=wcs&VERSION=1.0.0&REQUEST=GetCoverage&COVERAGE=gasp.aod&TIME=2008-06-21T00:00:00Z/2008-06-21T23:59:59Z&BBOX=-125,24,-65,50,0,0&REGRID=weighted&LAMBERT=33,45,-97,40&ELLIPSOID=6370000,6370000&GRID=279,240,-1008000,-1620000,12000,12000&FORMAT=netcdf-coards&COMPRESS=1' | gzip -d > sample.nc ; ncdump sample.nc | more

Regridding one day of MODIS AOD data to layer one of a 12km CONUS CMAQ grid in Lambert space (saved as NetCDF M3IO format)

curl --silent --retry 0 -L --tcp-nodelay --max-time 0 'https://ofmpub.epa.gov/rsig/rsigserver?SERVICE=wcs&VERSION=1.0.0&REQUEST=GetCoverage&COVERAGE=modis.mod4.optical_depth_land_and_ocean&TIME=2008-06-21T00:00:00Z/2008-06-21T23:59:59Z&BBOX=-140,23,-60,60,0,0&REGRID=weighted&LAMBERT=33,45,-97,40&ELLIPSOID=6370000,6370000&GRID=459,299,-2556000,-1728000,12000,12000&FORMAT=netcdf-ioapi&COMPRESS=1' | gzip -d > sample.nc ; ncdump sample.nc | more

Regridding one day of CALIPSO L2 Extinction data to a 108km CMAQ Hemispheric Stereographic grid with sigma-pressure vertical coordinates (saved as NetCDF M3IO format)

curl --silent --retry 0 -L --tcp-nodelay --max-time 0 'https://ofmpub.epa.gov/rsig/rsigserver?SERVICE=wcs&VERSION=1.0.0&REQUEST=GetCoverage&COVERAGE=calipso.l2_05kmapro.extinction_coefficient_532&BBOX=-180,-18,180,90,0,16000.0&TIME=2006-07-03T00:00:00Z/2006-07-03T23:59:59Z&REGRID=weighted&STEREOGRAPHIC=-98.0,90.0,45.0&ELLIPSOID=6370000.0,6370000.0&GRID=187,187,-1.0098E7,-1.0098E7,108000.0,108000.0&LEVELS=35,7,5000.0,1.0,0.9975,0.995,0.99,0.985,0.98,0.97,0.96,0.95,0.94,0.93,0.92,0.91,0.9,0.88,0.86,0.84,0.82,0.8,0.77,0.74,0.7,0.65,0.6,0.55,0.5,0.45,0.4,0.35,0.3,0.25,0.2,0.15,0.1,0.05,0.0,9.81,287.04,50.0,275.0,100000.0&MINIMUM_CAD=20&MAXIMUM_UNCERTAINTY=99.0&FORMAT=netcdf-ioapi&COMPRESS=1' | gzip -d > sample.nc ; ncdump sample.nc | more

Stream a single timestep as a PNG

curl --silent --retry 0 -L --tcp-nodelay --max-time 0 'https://ofmpub.epa.gov/rsig/rsigserver?SERVICE=wms&VERSION=1.3.0 &REQUEST=GetMap&LAYERS=modis.mod4.optical_depth_land_and_ocean,aqs.pm25,cmaq.east.pm25,cmaq.east.metdot3d.uwind,cmaq.east.metdot3d.vwind& STYLES=maplines&CRS=CRS:84&BBOX=-95,25,-80,35 &TIME=2005-08-30T00:00:00Z&WIDTH=1024&HEIGHT=768&FORMAT=image/png'

Stream a range of timesteps as an MPEG animation

curl --silent --retry 0 -L --tcp-nodelay --max-time 0 'https://ofmpub.epa.gov/rsig/rsigserver?SERVICE=wms&VERSION=1.3.0 &REQUEST=GetMap&LAYERS=modis.mod4.optical_depth_land_and_ocean,aqs.pm25,cmaq.east.pm25,cmaq.east.metdot3d.uwind,cmaq.east.metdot3d.vwind& STYLES=maplines&CRS=CRS:84&BBOX=-95,25,-80,35 &TIME=2005-08-30T00:00:00Z/2005-08-30T23:59:59Z&WIDTH=1024&HEIGHT=768& FORMAT=image/mpg'

Stream a range of timesteps as a KMZ animation and view with Google Earth

curl --silent --retry 0 -L --tcp-nodelay --max-time 0 'https://ofmpub.epa.gov/rsig/rsigserver?SERVICE=wms&VERSION=1.3.0 &REQUEST=GetMap&LAYERS=modis.mod4.optical_depth_land_and_ocean,aqs.pm25,cmaq.east.pm25,cmaq.metdot3d.uwind,cmaq.east.metdot3d.vwind& STYLES=&CRS=CRS:84&BBOX=-95,25,-80,35 &TIME=2005-08-30T00:00:00Z/2005-08-30T23:59:59Z&WIDTH=1024&HEIGHT=768& FORMAT=application/vnd.google-earth.kmz' > example.kmz ; googleearth `pwd`/example.kmz &

Example Windows and UNIX scripts for automated downloads
 

Contact the RSIG team for a sample script that will automate downloads using cURL.

High-End Scientific Computing

Contact Us About High-End Scientific Computing
Contact Us to ask a question, provide feedback, or report a problem.
Last updated on February 2, 2024