--- title: "Introduction to the climate package" author: "Bartosz Czernecki, Arkadiusz Głogowski, Jakub Nowosad" date: "`r Sys.Date()`" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Introduction to the climate package} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r setup, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>" ) old = options(scipen = 999) ``` The goal of the **climate** R package is to automatize downloading of *in-situ* meteorological and hydrological data from publicly available repositories: - OGIMET [(ogimet.com)](http://ogimet.com/index.phtml.en) - up-to-date collection of SYNOP dataset - University of Wyoming - atmospheric vertical profiling data (http://weather.uwyo.edu/upperair/) - Polish Institute of Meteorology and Water Management - National Research Institute [(IMGW-PIB)](https://dane.imgw.pl/) - National Oceanic & Atmospheric Administration - Earth System Research Laboratories - Global Monitoring Laboratory [(NOAA)](https://gml.noaa.gov/ccgg/trends/) - National Centers for Environmental Information, National Oceanic & Atmospheric Administration - National Climatic Data Center - Integrated Surface Hourly (ISH) [(NOAA)](https://www.ncei.noaa.gov/pub/data/noaa/) ## Functions ### Meteorological data - **meteo_ogimet()** - Downloading hourly and daily meteorological data from the SYNOP stations available in the ogimet.com collection. Any meteorological (aka SYNOP) station working under the World Meteorological Organization (WMO) framework after year 2000 should be accessible. Two backends are available and selected automatically: raw **SYNOP decoding** (`source = "synop"`, default for `interval = "hourly"`) and **HTML scraping** (`source = "html"`, default for `interval = "daily"`). Country-level bulk downloads are supported via the `country_name` argument (SYNOP backend only). - **meteo_imgw()** - Downloading hourly, daily, and monthly meteorological data from the SYNOP/CLIMATE/PRECIP stations available in the danepubliczne.imgw.pl collection. It is a wrapper for `meteo_monthly()`, `meteo_daily()`, `meteo_hourly()` and `meteo_imgw_datastore()` which gives access from monthly to even 10-min datasets. IMGW archive outputs use short English parameter names. The original IMGW names are available as `label` attributes on the corresponding columns, for example `attr(df$t2m_mean_mon, "label")`. - **meteo_noaa_hourly()** - Downloading hourly NCEI/NOAA Integrated Surface Hourly (ISH) meteorological data - some stations have > 100 years of observations. - **meteo_noaa_co2()** - Downloading monthly CO2 measurements from Mauna Loa Observatory. - **sounding_wyoming()** - Downloading measurements of the vertical profile of atmosphere (aka rawinsonde data). - **synop_parser()** - Decoding raw FM-12 SYNOP meteorological messages into structured R lists or data frames. For a full walkthrough see the [SYNOP Messages vignette](https://bczernecki.github.io/climate/articles/synop_parser.html). ### Hydrological data - **hydro_imgw()** - Downloading daily and monthly hydrological data from stations available in the danepubliczne.imgw.pl collection. It is a wrapper for `hydro_monthly()` and `hydro_daily()`. IMGW archive outputs use short English parameter names. The original IMGW names are available as `label` attributes on the corresponding columns, for example `attr(h$Q, "label")`. - **hydro_imgw_datastore()** - Downloading hourly and sub-hourly hydrological data from the IMGW-PIB hydro telemetry stations. ### Auxiliary functions and datasets - **stations_ogimet()** - Downloading information about all stations available in the selected country in the Ogimet repository - **nearest_stations_ogimet()** - Downloading information about nearest stations to the selected point using the Ogimet repository - **nearest_stations_noaa()** - Downloading information about nearest stations to the selected point in the NOAA ISH meteorological repository - **nearest_stations_imgw()** - List of nearby meteorological or hydrological IMGW-PIB stations in Poland - **imgw_meteo_stations** - Built-in metadata from the IMGW-PIB repository for meteorological stations, their geographical coordinates, and ID numbers - **imgw_hydro_stations** - Built-in metadata from the IMGW-PIB repository for hydrological stations, their geographical coordinates, and ID numbers - **stations_meteo_imgw_telemetry** - Downloading complete and up-to-date coordinates for IMGW-PIB telemetry meteorological stations - **stations_hydro_imgw_telemetry** - Downloading complete and up-to-date coordinates for IMGW-PIB telemetry hydrological stations - **imgw_meteo_abbrev** - Dictionary explaining variables available for meteorological stations (from the IMGW-PIB repository) - **imgw_hydro_abbrev** - Dictionary explaining variables available for hydrological stations (from the IMGW-PIB repository) ## Examples Examples shows application of climate package with additional use of tools that help with processing the data to increase legible of downloaded data. - [dplyr](https://CRAN.R-project.org/package=dplyr) - [tidyr](https://CRAN.R-project.org/package=tidyr) ### Example 1 Finding a 50 nearest meteorological stations for a given coordinates in a given country(ies): ``` {r stations , eval=T, fig.width=7,fig.height=7, fig.fullwidth=TRUE} library(climate) ns = nearest_stations_ogimet(country = c("United Kingdom"), point = c(-3, 50), no_of_stations = 50, add_map = TRUE) ``` ``` {r stations-2, eval=T} if (is.data.frame(ns)) { knitr::kable(head(ns, 15)) } ``` ### Example 2 Summary of stations available in Ogimet repository for a selected country: ``` {r stations-3, eval=T, fig.width=7, fig.height=7, fig.fullwidth=T} library(climate) PL = stations_ogimet(country = "Poland", add_map = TRUE) if (is.data.frame(PL)) { knitr::kable(head(PL)) } ``` ### Example 3 Downlading hourly meteorological data from Svalbard (Norway) for year 2016 using NOAA service ```{r noaa_svalbard, include=FALSE} df = readRDS(system.file("extdata/vignettes/svalbard_noaa.rds", package = "climate")) ``` ``` {r windrose,eval=F} # downloading data with NOAA service: df = meteo_noaa_hourly(station = "010080-99999", year = 2016) # You can also download the same (but more granular) data with Ogimet.com (example for year 2016): # df = meteo_ogimet(interval = "hourly", # date = c("2016-01-01", "2016-12-31"), # station = "01008") ``` ``` {r noaa-kable,eval=T} knitr::kable(head(df)) ``` ### Example 4 Downloading atmospheric vertical profile (sounding) for Łeba, PL station: ```{r sonda-read, eval=T, include=F, echo=F} library(climate) data("profile_demo") df2 = profile_demo[[1]] colnames(df2)[c(1, 3:4)] = c("PRESS", "TEMP", "DEWPT") # changing column names ``` ```{r sonda, eval=F, include=T} profile_demo = sounding_wyoming(wmo_id = 12120, yy = 2000, mm = 3, dd = 23, hh = 0) df2 = profile_demo[[1]] colnames(df2)[c(1, 3:4)] = c("PRESS", "TEMP", "DEWPT") # changing column names ``` ```{r sonda2, echo=FALSE} knitr::kable(head(df2, 10), caption = "Exemplary data frame of sounding preprocessing") ``` ### Example 5 Preparing an annual summary of air temperature and precipitation using **dplyr** syntax for 10-years period (1991-2000) ```{r imgw_meteo, include=FALSE} df = readRDS(system.file("extdata/vignettes/leba_monthly.rds", package = "climate")) ``` ```{r imgw_meteo-2, eval=FALSE, include=TRUE} library(climate) df = meteo_imgw(interval = "monthly", rank = "synop", year = 1991:2000, station = "ŁEBA") # please note that sometimes 2 names are used for the same station in different years ``` ```{r imgw_meteo-3, fig.width=7, fig.height=7, fig.fullwidth=TRUE, error=TRUE, eval=TRUE, include=TRUE} suppressMessages(library(dplyr)) df2 = dplyr::select(df, station:t2m_mean_mon, rr_monthly) monthly_summary = df2 %>% dplyr::group_by(mm) %>% dplyr::summarise(tmax = mean(tmax_abs, na.rm = TRUE), tmin = mean(tmin_abs, na.rm = TRUE), tavg = mean(t2m_mean_mon, na.rm = TRUE), precip = sum(rr_monthly) / n_distinct(yy)) monthly_summary = as.data.frame(t(monthly_summary[, c(5, 2, 3, 4)])) monthly_summary = round(monthly_summary, 1) colnames(monthly_summary) = month.abb ``` ```{r imgw_meteo2, echo=FALSE, error=TRUE} knitr::kable(head(monthly_summary), caption = "Exemplary data frame of meteorological preprocessing.") ``` ### Example 6 Calculate the mean maximum value of the flow on the stations in each year with **dplyr**'s `summarise()`, and spread data by year using **tidyr**'s `spread()` to get the annual means of maximum flow in the consecutive columns. ```{r data, eval=TRUE, include=FALSE, echo=FALSE} h = readRDS(system.file("extdata/vignettes/hydro_monthly.rds", package = "climate")) ``` ```{r data-2, eval=FALSE, include=TRUE} library(climate) library(dplyr) library(tidyr) h = hydro_imgw(interval = "monthly", year = 2001:2002) ``` ```{r data-3, eval=TRUE, include=TRUE, echo=TRUE} knitr::kable(head(h)) ``` ```{r filtering, eval=TRUE, include=TRUE} h2 = h %>% dplyr::filter(idex == 3) %>% dplyr::select(id, station, X, Y, hyy, Q) %>% dplyr::group_by(hyy, id, station, X, Y) %>% dplyr::summarise(annual_mean_Q = round(mean(Q, na.rm = TRUE), 1)) %>% tidyr::pivot_wider(names_from = hyy, values_from = annual_mean_Q) knitr::kable(head(h2)) ``` ```{r filtering2, echo=FALSE, eval=FALSE} knitr::kable(head(h2), caption = "Exemplary data frame of hydrological preprocesssing.") ``` ### Example 7 Downloading monthly CO2 measurements from Mauna Loa Observatory and plotting the Keeling Curve: ```{r co2, eval=FALSE, include=TRUE} library(climate) library(ggplot2) co2 = meteo_noaa_co2() co2$date = ISOdate(co2$yy, co2$mm, 1) ggplot(co2, aes(date, co2_avg)) + geom_line() + geom_smooth() + theme_bw() + labs(title = "Carbon Dioxide (CO2)", subtitle = "Mauna Loa Observatory", x = "", y = "ppm") ``` ### Example 8 Decoding raw SYNOP meteorological messages with **synop_parser()**: ```{r synop_parser, eval=TRUE, include=TRUE} library(climate) synop_code = "AAXX 01004 88889 12782 61506 10094 20047 30111 40197 53007 60001 81541" # Decode a single message — returns a named list result = synop_parser(synop_code) result$air_temperature$value # 9.4°C result$wind_speed$value # 6 kt result$sea_level_pressure$value # 1019.7 hPa ``` ```{r parser-df, eval=TRUE, include=TRUE} # Return a tidy data frame with one row per message df_parser = synop_parser(synop_code, as_data_frame = TRUE) knitr::kable(df_parser[, 1:8]) ``` ## Acknowledgment Ogimet.com, University of Wyoming, and Institute of Meteorology and Water Management - National Research Institute (IMGW-PIB), National Oceanic & Atmospheric Administration (NOAA) - Earth System Research Laboratories - Global Monitoring Laboratory, Global Monitoring Division and Integrated Surface Hourly (NOAA ISH) are the sources of the data. ## Contribution Contributions to this package are welcome. The preferred method of contribution is through a GitHub pull request. Feel also free to contact us by creating [an issue](https://github.com/bczernecki/climate/issues). ## Citation To cite the `climate` package in publications, please use [this paper](https://doi.org/10.3390/su12010394): Czernecki, B.; Głogowski, A.; Nowosad, J. Climate: An R Package to Access Free In-Situ Meteorological and Hydrological Datasets for Environmental Assessment. Sustainability 2020, 12, 394. https://doi.org/10.3390/su12010394" LaTeX version can be obtained with: ``` library(climate) citation("climate") ``` ```{r setup_restore, include = FALSE} options(old) ```