Intro: cbsodata4

List of datasets (toc)

A list of datasets that are available can be loaded with with cbs4_get_datasets() (cbs4_get_toc() is an alias)

datasets <- cbs4_get_datasets()
head(datasets[,c("Identifier", "Title", "Modified")])

Using cbs4_search a list of tables can be found that contain desired search terms, e.g. “Diesel”:

datasets <- cbs4_search("Diesel")
head(datasets[,c("Identifier", "Title", "rel")])

Using an “Identifier” from cbs4_get_datasets information on the table can be retrieved with cbs4_get_metadata

meta_petrol <- cbs4_get_metadata("80416ned")
meta_petrol
#> cbs odatav4: '80416ned':
#> "Pompprijzen motorbrandstoffen; brandstofsoort, per dag"
#> dimensions: Perioden
#> For more info use 'str(x)' or 'names(x)' to find out its properties.

The meta object contains all metadata properties of cbsodata in the form of data.frames. Each data.frame describes properties of the CBS table: “Dimensions”, “MeasureCodes” and one ore more “<Dimension>Codes” describing the meta data of the borders of a SN table.

names(meta_petrol)
#> [1] "Dimensions"     "MeasureCodes"   "PeriodenGroups" "PeriodenCodes" 
#> [5] "Properties"
meta_petrol$MeasureCodes[, 1:3]

meta_petrol$Dimensions

# just 1 dimension with the following categories:
head(meta_petrol$PeriodenCodes)

Data retrieval

With cbs4_get_observations and cbs4_get_data data can be retrieved. By default this will be downloaded in a temporary directory, but this can be set explicitly with the argument download_dir.

cbs4_get_observations is the format in which the data is downloaded from Statistic Netherlands (CBS). It is in so-called long format. It contains one Measure column, describing the topics/variable, one Value column describing the statistical value, one or more Dimension columns and some extra columns with value specific metadata.

obs <- cbs4_get_observations("80416ned")
head(obs)

cbs4_get_data returns the data in so-called wide format in which each Measure has its own column. For many uses this is a more natural format. It is a pivoted version of cbs4_get_observations().

# same data, but pivoted
data <- cbs4_get_data("80416ned", name_measure_columns = FALSE)
head(data, 2)

By default the names of the columns are more readable with cbs4_get_data

# same data, but pivoted
data <- cbs4_get_data("80416ned")
head(data, 2)

obs <- cbs4_get_observations("80416ned")
obs <- cbs4_add_label_columns(obs)
head(obs)

data <- cbs4_get_data("80416ned")
data <- cbs4_add_label_columns(data)
head(data, 2)

obs <- cbs4_get_observations("80416ned")
obs <- cbs4_add_date_column(obs)
head(obs)

data <- cbs4_get_data("80416ned")
data <- cbs4_add_date_column(data)
head(data)

obs <- cbs4_get_observations("80416ned")
obs <- cbs4_add_unit_column(obs)
head(obs)

Filter data before download

It is possible to restrict the download using filter statements. This may shorten the download time considerably.

meta <- cbs4_get_metadata("60006")
tail(meta$PeriodenCodes)

meta$MeasureCodes[,c("Identifier","Title")]

obs <- cbs4_get_observations("60006"
                            , Measure = c("M003026","M003019")     # selection on Measures
                            , Perioden = c("2019KW04", "2020KW01") # selection on Perioden
                            )
cbs4_add_label_columns(obs)

data <- cbs4_get_data("60006"
                     , Measure = c("M003026","M003019")     # selection on Measures
                     , Perioden = contains("2019") # retrieve all periods with 2019
                     )
data

data <- cbs4_get_data("60006"
                     , Measure = c("M003026","M003019")         # selection on Measures
                     , Perioden = contains("2019") | "2020KW01" # retrieve all periods with 2019
                     )
data

  # supply your own odata 4 query
  cbs4_get_data("84287NED", query = "$filter=Perioden eq '2019MM12'")

Download data

Data and metadata of a table can also be downloaded explicitly by using cbs4_download. This can be an option if you don’t want to load the data into memory (which both cbs4_get_data and cbs4_get_observations do), but only store it on disk.

cbs4_download("60006", download_dir = "./60006") # will download data and metadata in csv format.

Other catalogs

CBS / Statistics Netherlands also offers collections of datasets that are not part of the main collections: so-called catalogs. These can be retrieved with cbs4_get_catalogs().

catalogs <- cbs4_get_catalogs() 
catalogs[,1:2]

Another options is to set the catalog argument in cbs4_get_datasets to NULL

ds <- cbs4_get_datasets()
nrow(ds)
#> [1] 1620

ds_all <- cbs4_get_datasets(catalog = NULL)
nrow(ds_all)
#> [1] 1623

ds_asd <- cbs4_get_datasets(catalog = "CBS-asd")
nrow(ds_asd)
#> [1] 3