Skip to contents

Overview

Certara.ModelResults uses the flextable package to generate, customize, and report model diagnostic tables. Users can specify many table customization options in the GUI, including adding, removing, and renaming columns, formatting alignment, adding captions and footers, specifying the number of digits, and more.

For advanced table customizations, including combining tables and custom cell styling (e.g., font, face, alignment, etc.) you can use the flextable code generated from resultsUI() and add additional options from R.

Combine theta, omega, and sigma tables for single model

Multiple parameter estimate tables generated from resultsUI() can be easily combined into a single table. To begin, we’ll initialize resultsUI() with our list of xpdb objects using the example data from Certara.ModelResults::xpdb_NLME.

Next, we’ll tag the “Theta”, “Omega”, and “Sigma” tables for our model TwCpt_IVBolus_SelectedCovariateModel_FOCE_ELS.

After tagging the three tables, navigate to the Tagged tab in the Shiny GUI and download the associated R script. We will be extracting code from the following R script that has been downloaded from Model Results:

library(Certara.ModelResults)
library(Certara.Xpose.NLME)
library(xpose)
library(ggplot2)
library(dplyr)
library(tidyr)
library(magrittr)
library(flextable)

xpobj <- xpdb_NLME[["TwCpt_IVBolus_SelectedCovariateModel_FOCE_ELS"]]
table <- xpobj %>%
  get_prmNlme() %>%
  filter(type == "the") %>%
  select(-type, -diagonal, -n) %>%
  mutate(`rse%` = as.numeric(rse) * 100) %>%
  flextable(col_keys = c("name", "label", "value", "se", "rse%", "fixed", "2.5% CI", "97.5% CI")) %>%
  set_header_labels(values = list(name = "Name", label = "Label", value = "Value", se = "SE", `rse%` = "RSE%", fixed = "Fixed", `2.5% CI` = "2.5% CI", `97.5% CI` = "97.5% CI")) %>%
  set_caption(caption = "Table Theta") %>%
  add_footer_row(values = "Source: script.R", colwidths = 8L) %>%
  colformat_double(digits = 4L) %>%
  align(align = "left", part = "all") %>%
  autofit() %>%
  fontsize(size = 10, part = "all") %>%
  font(fontname = "Times New Roman", part = "all") %>%
  bold(part = "header")
table


xpobj <- xpdb_NLME[["TwCpt_IVBolus_SelectedCovariateModel_FOCE_ELS"]]
table <- xpobj %>%
  get_prmNlme() %>%
  filter(type == "ome") %>%
  select(-type) %>%
  mutate(`rse%` = as.numeric(rse) * 100) %>%
  left_join(get_eta_shk(xpobj), by = "label") %>%
  flextable(col_keys = c("name", "label", "value", "se", "rse%", "fixed", "diagonal", "2.5% CI", "97.5% CI", "shrinkage%")) %>%
  set_header_labels(values = list(name = "Name", label = "Label", value = "Value", se = "SE", `rse%` = "RSE%", fixed = "Fixed", diagonal = "Diagonal", `2.5% CI` = "2.5% CI", `97.5% CI` = "97.5% CI", `shrinkage%` = "Shrinkage%")) %>%
  set_caption(caption = "Table Omega") %>%
  add_footer_row(values = "Source: script.R", colwidths = 10L) %>%
  colformat_double(digits = 4L) %>%
  align(align = "left", part = "all") %>%
  autofit() %>%
  fontsize(size = 10, part = "all") %>%
  font(fontname = "Times New Roman", part = "all") %>%
  bold(part = "header")
table


xpobj <- xpdb_NLME[["TwCpt_IVBolus_SelectedCovariateModel_FOCE_ELS"]]
table <- xpobj %>%
  get_prmNlme() %>%
  filter(type == "sig") %>%
  select(-type) %>%
  mutate(`rse%` = as.numeric(rse) * 100) %>%
  left_join(get_eps_shk(xpobj), by = "label") %>%
  flextable(col_keys = c("name", "label", "value", "se", "rse%", "fixed", "2.5% CI", "97.5% CI", "shrinkage%")) %>%
  set_header_labels(values = list(name = "Name", label = "Label", value = "Value", se = "SE", `rse%` = "RSE%", fixed = "Fixed", `2.5% CI` = "2.5% CI", `97.5% CI` = "97.5% CI", `shrinkage%` = "Shrinkage%")) %>%
  set_caption(caption = "Table Sigma") %>%
  add_footer_row(values = "Source: script.R", colwidths = 9L) %>%
  colformat_double(digits = 4L) %>%
  align(align = "left", part = "all") %>%
  autofit() %>%
  fontsize(size = 10, part = "all") %>%
  font(fontname = "Times New Roman", part = "all") %>%
  bold(part = "header")
table

Let’s extract the first few lines of code in the above chunk that is used to create the data.frame for the resulting flextable. We’ll assign them new values named df_theta, df_omega, df_sigma.

Our df_theta does not include a shrinkage% column, so we must add it and assign NA values to it in order to combine the three data.frame using rbind. The flextable package will display NA values as blank cells, by default.

Finally, we will extract only the columns we’d like to use in the final table: "name", "label", "value", "2.5% CI", "97.5% CI", "rse%", "shrinkage%", "Type".

library(Certara.ModelResults)
library(Certara.Xpose.NLME)
library(xpose)
library(dplyr)
library(magrittr)
library(flextable)
#> Warning: package 'flextable' was built under R version 4.3.3


xpobj <- xpdb_NLME[["TwCpt_IVBolus_SelectedCovariateModel_FOCE_ELS"]]

cols <- c("name", "label", "value", "2.5% CI", "97.5% CI",  "rse%", "shrinkage%", "Type")

#Theta
df_theta <- xpobj %>%
  get_prmNlme() %>%
  filter(type == "the") %>%
  mutate(`rse%` = as.numeric(rse) * 100,
         `shrinkage%` = NA,
         Type = "Fixed Effects"
         )  %>%
  select(all_of(cols))

#Omega
df_omega <- xpobj %>%
  get_prmNlme() %>%
  filter(type == "ome") %>%
  mutate(`rse%` = as.numeric(rse) * 100) %>%
  left_join(get_eta_shk(xpobj), by = "label") %>%
  mutate(Type = "Random Effects") %>%
  select(all_of(cols))

#sigma
df_sigma <- xpobj %>%
  get_prmNlme() %>%
  filter(type == "sig") %>%
  select(-type) %>%
  mutate(`rse%` = as.numeric(rse) * 100) %>%
  left_join(get_eps_shk(xpobj), by = "label")  %>%
  mutate(Type = "Residual Error") %>%
  select(all_of(cols))

Next, we’ll combine the above data.frames using rbind().

df_combined <- rbind(df_theta, df_omega, df_sigma)

Generate flextable

The as_grouped_data() command from the flextable package allows us to add a grouping row to the table. We will use the Type column we added previously to group rows in our table. We will first execute our grouping function as_grouped_data() on our data.frame, then coerce to flextable using as_flextable(), and finally change the label names and table size formatting using set_header_labels() and autofit().

tbl_parameters <- as_grouped_data(df_combined, groups = "Type") %>% 
  as_flextable() %>% 
  set_header_labels(values = list(name = "Parameters", label = "Label", value = "Value", 
                                `rse%` = "RSE (%)", `2.5% CI` = "2.5% CI", `97.5% CI` = "97.5% CI", 
                                `shrinkage%` = "Shrinkage (%)")) %>%
  autofit()


tbl_parameters

Parameters

Label

Value

2.5% CI

97.5% CI

RSE (%)

Shrinkage (%)

Type: Fixed Effects

THETA(1)

tvV

16.6412000

14.78050000

18.5019000

5.63725

THETA(2)

tvCl

7.6954200

7.01558000

8.3752500

4.45393

THETA(3)

tvV2

41.3330000

39.23290000

43.4332000

2.56165

THETA(4)

tvCl2

14.1088000

12.04610000

16.1715000

7.37075

THETA(5)

dVdBodyWeight

1.3543100

0.89544400

1.8131700

17.08190

THETA(6)

dCldBodyWeight

2.2574900

1.87345000

2.6415400

8.57680

Type: Random Effects

OMEGA(1,1)

nV

0.0129749

-0.00485840

0.0308082

69.29410

37.2430

OMEGA(2,2)

nCl

0.0221510

0.00497179

0.0393302

39.10020

8.3480

OMEGA(3,3)

nCl2

0.0446894

-0.02480850

0.1141870

78.40360

27.2320

Type: Residual Error

SIGMA(1,1)

CEps

0.1589610

0.12204700

0.1958750

11.70760

13.1926

Customize font

Next, we will add bold face to our header row. To better differentiate the parameter types, let’s also add bold face to the Type row.


tbl_parameters <- tbl_parameters %>%
  bold(part = "header") %>%
  bold(i = c(1, 8, 12), j = 1) 

tbl_parameters

Parameters

Label

Value

2.5% CI

97.5% CI

RSE (%)

Shrinkage (%)

Type: Fixed Effects

THETA(1)

tvV

16.6412000

14.78050000

18.5019000

5.63725

THETA(2)

tvCl

7.6954200

7.01558000

8.3752500

4.45393

THETA(3)

tvV2

41.3330000

39.23290000

43.4332000

2.56165

THETA(4)

tvCl2

14.1088000

12.04610000

16.1715000

7.37075

THETA(5)

dVdBodyWeight

1.3543100

0.89544400

1.8131700

17.08190

THETA(6)

dCldBodyWeight

2.2574900

1.87345000

2.6415400

8.57680

Type: Random Effects

OMEGA(1,1)

nV

0.0129749

-0.00485840

0.0308082

69.29410

37.2430

OMEGA(2,2)

nCl

0.0221510

0.00497179

0.0393302

39.10020

8.3480

OMEGA(3,3)

nCl2

0.0446894

-0.02480850

0.1141870

78.40360

27.2320

Type: Residual Error

SIGMA(1,1)

CEps

0.1589610

0.12204700

0.1958750

11.70760

13.1926

 

Customize digits and NA

The flextable package will display NA values as blank by default. But what if we wanted to use the characters "--" instead of blanks? We can change how NA is displayed for the shrinkage% column in our example table. Since this column is of class numeric, we must use col_format_double(na_str = "--").

In our particular table, we are reporting rse% and shrinkage%, which means we are probably OK to use two decimals of precision here. We’ll specify four digits of precisions for all other double columns.


tbl_parameters <- tbl_parameters %>%
  colformat_double(digits = 4) %>% # apply to all columns first
  colformat_double(digits = 2, j = c("rse%", "shrinkage%"), na_str = "--") # special formatting for rse% and shrinkage%


tbl_parameters

Parameters

Label

Value

2.5% CI

97.5% CI

RSE (%)

Shrinkage (%)

Type: Fixed Effects

THETA(1)

tvV

16.6412

14.7805

18.5019

5.64

--

THETA(2)

tvCl

7.6954

7.0156

8.3752

4.45

--

THETA(3)

tvV2

41.3330

39.2329

43.4332

2.56

--

THETA(4)

tvCl2

14.1088

12.0461

16.1715

7.37

--

THETA(5)

dVdBodyWeight

1.3543

0.8954

1.8132

17.08

--

THETA(6)

dCldBodyWeight

2.2575

1.8735

2.6415

8.58

--

Type: Random Effects

OMEGA(1,1)

nV

0.0130

-0.0049

0.0308

69.29

37.24

OMEGA(2,2)

nCl

0.0222

0.0050

0.0393

39.10

8.35

OMEGA(3,3)

nCl2

0.0447

-0.0248

0.1142

78.40

27.23

Type: Residual Error

SIGMA(1,1)

CEps

0.1590

0.1220

0.1959

11.71

13.19

Customize alignment

We can implement custom alignment of the header, body, and footer in our flextable using the align() function. We will center alignment for our header and body, and also implement special left alignment for the Type rows in order to make the table more appealing.

tbl_parameters %>%
  align(align = "center", part = "header") %>%
  align(align = "center", part = "body") %>%
  align(align = "left", i = c(1, 8, 12))

Parameters

Label

Value

2.5% CI

97.5% CI

RSE (%)

Shrinkage (%)

Type: Fixed Effects

THETA(1)

tvV

16.6412

14.7805

18.5019

5.64

--

THETA(2)

tvCl

7.6954

7.0156

8.3752

4.45

--

THETA(3)

tvV2

41.3330

39.2329

43.4332

2.56

--

THETA(4)

tvCl2

14.1088

12.0461

16.1715

7.37

--

THETA(5)

dVdBodyWeight

1.3543

0.8954

1.8132

17.08

--

THETA(6)

dCldBodyWeight

2.2575

1.8735

2.6415

8.58

--

Type: Random Effects

OMEGA(1,1)

nV

0.0130

-0.0049

0.0308

69.29

37.24

OMEGA(2,2)

nCl

0.0222

0.0050

0.0393

39.10

8.35

OMEGA(3,3)

nCl2

0.0447

-0.0248

0.1142

78.40

27.23

Type: Residual Error

SIGMA(1,1)

CEps

0.1590

0.1220

0.1959

11.71

13.19

Combine overall tables from multiple models

“Overall” tables generated from resultsUI() can be easily combined into a single table to compare models. To begin, we’ll initialize resultsUI() with our list of xpdb objects using the example data from Certara.ModelResults::xpdb_NLME.

Next, we’ll tag the “Overall” tables for both of our model runs.

After tagging the two tables, navigate to the Tagged tab in the Shiny GUI and download the associated R script for your tagged diagnostics. We will be extracting code from the following R script that has been downloaded from the Shiny GUI:

library(Certara.ModelResults)
library(Certara.Xpose.NLME)
library(xpose)
library(ggplot2)
library(dplyr)
library(tidyr)
library(magrittr)
library(flextable)

xpobj <- xpdb_NLME[["TwCpt_IVBolus_FOCE_ELS"]]
table <- xpobj %>%
  get_overallNlme() %>%
  mutate(RetCode = as.integer(RetCode), nObs = as.integer(nObs), nSub = as.integer(nSub), nParm = as.integer(nParm)) %>%
  flextable(col_keys = c("RetCode", "Condition", "logLik", "-2LL", "AIC", "BIC", "nParm", "nObs", "nSub")) %>%
  set_header_labels(values = list(RetCode = "RetCode", Condition = "Condition", logLik = "LL", `-2LL` = "-2LL", AIC = "AIC", BIC = "BIC", nParm = "nParm", nObs = "nObs", nSub = "nSub")) %>%
  colformat_double(digits = 4L) %>%
  align(align = "left", part = "all") %>%
  set_table_properties(layout = "autofit") %>%
  autofit() %>%
  fontsize(size = 10, part = "all") %>%
  font(fontname = "Times New Roman", part = "all") %>%
  bold(part = "header")
table


xpobj <- xpdb_NLME[["TwCpt_IVBolus_SelectedCovariateModel_FOCE_ELS"]]
table <- xpobj %>%
  get_overallNlme() %>%
  mutate(RetCode = as.integer(RetCode), nObs = as.integer(nObs), nSub = as.integer(nSub), nParm = as.integer(nParm)) %>%
  flextable(col_keys = c("RetCode", "Condition", "logLik", "-2LL", "AIC", "BIC", "nParm", "nObs", "nSub")) %>%
  set_header_labels(values = list(RetCode = "RetCode", Condition = "Condition", logLik = "LL", `-2LL` = "-2LL", AIC = "AIC", BIC = "BIC", nParm = "nParm", nObs = "nObs", nSub = "nSub")) %>%
  colformat_double(digits = 4L) %>%
  align(align = "left", part = "all") %>%
  set_table_properties(layout = "autofit") %>%
  autofit() %>%
  fontsize(size = 10, part = "all") %>%
  font(fontname = "Times New Roman", part = "all") %>%
  bold(part = "header")
table

Just like in our previous example, we’ll extract code from the above chunk used to create the data.frames for our tables and assign them to objects df_overall_base_model and df_overall_cov_model. Let’s also add a new column ModelName to indicate the corresponding model name.

library(Certara.ModelResults)
library(Certara.Xpose.NLME)
library(xpose)
library(ggplot2)
library(dplyr)
library(tidyr)
#> Warning: package 'tidyr' was built under R version 4.3.3
#> 
#> Attaching package: 'tidyr'
#> The following object is masked from 'package:magrittr':
#> 
#>     extract
library(magrittr)
library(flextable)


xpobj <- xpdb_NLME[["TwCpt_IVBolus_FOCE_ELS"]]

df_overall_base_model <- get_overallNlme(xpobj) %>%
           mutate(RetCode = as.integer(RetCode), nObs = as.integer(nObs), nSub = as.integer(nSub), nParm = as.integer(nParm)) %>%
           #Add additional mutate to create ModelName column
           mutate(ModelName = "TwCpt_IVBolus_FOCE_ELS")

 

Now let’s do the same for our covariate model.

 

xpobj <- xpdb_NLME[["TwCpt_IVBolus_SelectedCovariateModel_FOCE_ELS"]]

df_overall_cov_model <- get_overallNlme(xpobj) %>%
           mutate(RetCode = as.integer(RetCode), nObs = as.integer(nObs), nSub = as.integer(nSub), nParm = as.integer(nParm)) %>%
           #Add additional mutate to create ModelName column
           mutate(ModelName = "TwCpt_IVBolus_SelectedCovariateModel_FOCE_ELS")

Next, we’ll combine our two tables using rbind()


df_overall_combined <- rbind(df_overall_base_model, df_overall_cov_model)

Generate flextable

We’re ready to convert our data.frame with the flextable() function and some additional flextable() styling options.


df_overall_combined <- df_overall_combined %>%
        flextable(col_keys = c("ModelName", "RetCode", "Condition", "-2LL",
      "AIC", "BIC", "nParm", "nObs", "nSub")) %>%
      set_header_labels(values = list(ModelName = "Model Name", RetCode = "Return Code", Condition = "Condition Number", 
                                        `-2LL` = "-2LL", AIC = "AIC", BIC = "BIC", nParm = "Number of Parameters", 
                                        nObs = "Number of Observations", nSub = "Number of Subjects")) %>%
      colformat_double(digits = 4L) %>%
      align(align = "left", part = "all") %>%
      autofit() %>%
      fontsize(size = 10, part = "all") %>%
      font(fontname = "Times New Roman", part = "all") %>%
      bold(part = "header")

df_overall_combined 

Model Name

Return Code

Condition Number

-2LL

AIC

BIC

Number of Parameters

Number of Observations

Number of Subjects

TwCpt_IVBolus_FOCE_ELS

1

4.8372

1,265.5900

1,281.5900

1,303.3380

8

112

16

TwCpt_IVBolus_SelectedCovariateModel_FOCE_ELS

1

19.8807

1,217.2920

1,237.2920

1,264.4770

10

112

16

Further customizing the flextable

Using the table object we created in the previous example, we can explore additional styling options from the flextable package.

If we want to apply decimal formatting to specific columns (e.g., format AIC and BIC to 2 decimal places, and Condition to 5 decimal places) it can be done using the example below.

df_overall_combined  %>%
  colformat_double(digits = 2, j = c("AIC", "BIC")) %>%
  colformat_double(digits = 5, j = "Condition")  

Model Name

Return Code

Condition Number

-2LL

AIC

BIC

Number of Parameters

Number of Observations

Number of Subjects

TwCpt_IVBolus_FOCE_ELS

1

4.83716

1,265.5900

1,281.59

1,303.34

8

112

16

TwCpt_IVBolus_SelectedCovariateModel_FOCE_ELS

1

19.88073

1,217.2920

1,237.29

1,264.48

10

112

16

We can also apply flextable options to individual cells, using the i argument to select the row, and the j argument to select the column.

Let’s remove decimal formatting specifically for the value of -2LL in our model TwCpt_IVBolus_FOCE_ELS.

df_overall_combined  %>%
  colformat_double(digits = 0, i = 1, j = "-2LL")

Model Name

Return Code

Condition Number

-2LL

AIC

BIC

Number of Parameters

Number of Observations

Number of Subjects

TwCpt_IVBolus_FOCE_ELS

1

4.8372

1,266

1,281.5900

1,303.3380

8

112

16

TwCpt_IVBolus_SelectedCovariateModel_FOCE_ELS

1

19.8807

1,217.2920

1,237.2920

1,264.4770

10

112

16

You can implement the same custom styling options for rows, columns, and individual cells across a variety of flextable functions, including: style(), align(), bg(), font(), fontsize(), italic(), bold(), color(), and highlight(). See flextable documentation for additional details.