What is YAML?

YAML is a markup language for encoding structured data, similar to XML or JSON, but more geared towards human readability (you may already be familiar with YAML since it is used in the header of R markdown documents). It has the advantage of being both very easy for humans to read and write, as well as machine parseable (because although it looks natural, it actually has strict syntactic rules), with support in many popular languages, including R. You can edit YAML files in RStudio (with syntax highlighting).

The data specification file

The way YAML works is best illustrated with an example. The current directory contains the file data_spec.yaml which contains the following:

dataset:     Xyz-pk.csv

labels:
  sex:       Sex
  race:      Race
  ethnic:    Ethnicity
  hv:        Health Status
  age:       Age (y)
  agecat:    Age Group
  wt:        Body Weight (kg)
  ht:        Height (cm)
  bmi:       BMI (kg/m²)
  bsa:       BSA (m²)
  alb:       Albumin (g/L)
  alp:       ALP (U/L)
  alt:       ALT (U/L)
  ast:       AST (U/L)
  bili:      Bilirubin (µmol/L)
  crcl:      CrCL (mL/min)
  fdarenal:  Renal Impairment, FDA Classification
  form:      Formulation
  fasted:    Fasting Status

categoricals:
  study:
    - 1: Xyz-hv-01
    - 2: Xyz-ri-02
    - 7: Xyz-ph3-07
  sex:
    - 0: Male
    - 1: Female
  race:
    - 1: White
    - 2: Black or African American
    - 3: Asian
    - 4: American Indian or Alaskan Native
    - 5: Native Hawaiian or Other Pacific Islander
    - 6: Multiple or Other
  ethnic:
    - 1: Not Hispanic or Latino
    - 2: Hispanic or Latino
    - -99: Not reported
  agecat:
    - 0: < 65 years
    - 1: "\u2265 65 years"
  hv:
    - 1: Healthy Subject
    - 0: Patient
  fdarenal:
    - 0: Normal
    - 1: Mild
    - 2: Moderate
    - 3: Severe
  form:
    - 1: Capsule
    - 2: Tablet
  fasted:
    - 0: Fed
    - 1: Fasted
    - -99: Unknown

The meaning of the file contents is intuitively clear. Indentation is used to denote hierarchical structure. Line breaks separate items from each other. Space are used to indent things, and other than that spaces are basically ignored (except inside strings).

Data structures come in 2 forms: sequential (i.e. lists) and named (i.e. dictionaries). For sequential data, each element is preceded by a dash and whitespace (don’t use tabs) (e.g. - item); thus, it looks the way one would write a list in a plain-text e-mail, for instance. Named data consists of key-value pairs, where a colon and whitespace (don’t use tabs) separate the key from the value (e.g. key: value); if the value is itself a nested structure, it can appear indented starting on the next line (same for list items). Primitive types (numbers, strings) are written the way one would write them naturally. In most cases, strings to not need to be quoted (but they can be); there are some exceptions though. Strings can contain Unicode symbols. For more details on the syntax, see the YAML documentation.

In the example above, the whole file encodes a named structure, with 3 top-level items: dataset, labels and categoricals. The dataset item contains a single string, the name of a .csv file that contains the data to which this meta-data is associated. The labels item contains another named structure: key-value pairs of column names and associated labels. The last item, categoricals, contains information on the coding of certain variables (i.e., variables that are really categorical but have been assigned numeric codes in the dataset). When the data is presented in a table, these variables should be translated back to their original descriptive identifiers. Nested within the categoricals item is another named structure. Here, the names correspond to columns in the dataset, and the values are lists, whereby each list item relates a (numeric) code to its (string) identifier.

(Note: currently this file needs to be written by hand, but in the future its generation could be partially or fully automated.)

Reading a dataset from its specification

With the data_spec.yaml file above, we can use the read_from_spec() function to read the data and have it augmented with the meta-data from the spec file:

# Read in the data from its 'spec'
dat <- read_from_spec("data_spec.yaml")

Note that we did not need to include the name of the data file in our script, since it is contained in the spec.

Before proceeding to describe the baseline characteristics of our study subjects, we need to make sure that each individual is only counted once. There is a convenience function for that:

# Filter the data, one row per ID
dat <- one_row_per_id(dat, "id")

Only columns that are invariant (and hence unambiguous) within each ID level are retained.

Here are six random rows of the resulting dataset:

dat[sample(1:nrow(dat), 6),]

##      id      study    sex              race                 ethnic
## 928 159 Xyz-ph3-07 Female             White     Hispanic or Latino
## 118  14  Xyz-hv-01   Male             White Not Hispanic or Latino
## 442  50  Xyz-ri-02 Female Multiple or Other Not Hispanic or Latino
## 764 118 Xyz-ph3-07 Female             White Not Hispanic or Latino
## 379  43  Xyz-ri-02 Female             Asian           Not reported
## 968 169 Xyz-ph3-07 Female             White Not Hispanic or Latino
##                  hv    form  fasted age        agecat       wt       ht
## 928         Patient  Tablet Unknown  40 < 65 years 69.11780 162.6218
## 118 Healthy Subject Capsule     Fed  32 < 65 years 94.09640 178.6365
## 442         Patient  Tablet  Fasted  79    = 65 years 46.29503 154.5257
## 764         Patient  Tablet Unknown  37 < 65 years 63.69094 168.2729
## 379         Patient  Tablet  Fasted  67    = 65 years 75.17084 156.0623
## 968         Patient  Tablet Unknown  69    = 65 years 51.20350 149.6104
##          bmi      bsa      alb       alp      alt      ast      bili     creat
## 928 26.13560 1.742997 41.39164  59.16552 20.20104 29.25946  5.831341  59.48539
## 118 29.48715 2.127224 42.39993 102.36809 29.20991 29.82011 15.139345 106.61792
## 442 19.38798 1.416590 37.22390 140.36920 11.99238 29.56350  8.078682 159.12966
## 764 22.49309 1.725678 41.66294  66.00227 10.89596 58.22654  6.864295  46.61096
## 379 30.86407 1.753188 36.73265 131.09500 17.25876 15.40761 10.866950 183.66983
## 968 22.87580 1.444328 35.25484  60.00767 23.62474 16.59708  8.345510  65.85870
##          crcl fdarenal
## 928 121.26020   Normal
## 118 117.02707   Normal
## 442  18.52046   Severe
## 764 146.88099   Normal
## 379  31.17979 Moderate
## 968  57.60809 Moderate

Note that the categorical variables (which were numeric in the original .csv file) have been translated to factors, with the appropriate textual labels, and in the desired order. Compared to the corresponding R code that would be needed to achieve this, the YAML specification is much cleaner and more concise. Similarly for the label attributes.

Note on preserving label attributes: in most cases, subsetting a data.frame results in the label attributes being stripped away. The function subsetp() (‘p’ for preserve) can be used to avoid this. (It is used internally in one_row_per_id(), for instance.)

Summary of Baseline Characteristics in the PK Population – Demographic

table1(~ sex + race + ethnic + hv + age + agecat + wt + ht + bmi + bsa | study, data=dat,
  footnote=abbrev_footnote("BMI", "BSA", "SD", "Min", "Max", "N"))

Summary of Baseline Characteristics in the PK Population – Laboratory Tests

table1(~ alb + alp + alt + ast + bili + crcl + fdarenal | study, data=dat,
  footnote=abbrev_footnote("ALP", "ALT", "AST", "CrCL", "FDA", "SD", "Min", "Max", "N"))

For continuous variables, the arithmetic mean, standard deviation, median, coefficient of variation, minimum and maximum are displayed on 3 lines. For categorical variable the frequency and percentage (within each column) are shown, one line per category. Missing values, if any, will also be shown as count and percent. The following rounding rules are applied:

• All percentages are rounded to 1 decimal place, with the exception of 0% and 100% which are simply shown as is.
• All other numbers are rounded to 3 significant digits. This includes numbers bigger than 1,000 (so 123,456 gets rounded to 123,000 for instance).
• A ‘5’ digit is always rounded up. This is NOT the usual convention in R (which follows the IEC 60559 standard), but is the standard in SAS and Excel, for instance.

Continuous and categorical variables can be mixed in the same table (note that age and age category are next to each other).

The above tables (including headings and footnotes) can be copied from the Chrome browser directly to a Word document (or Excel sheet). All formatting will be preserved (the one exception I have found is that in certain cases, superscripts or subscripts may be too small, so it might be necessary to reset the font site for the whole table to 10 pt). Pasting into PowerPoint works too, but does a less good a job at preserving the formatting, so it may be better to paste to a Word document first, and then copy that to PowerPoint (use the “Keep Source Formatting” option when pasting).

The complete example can be found on GitHub.