Running RsNLME on Remote Servers • Certara.RsNLME

RsNLME package logo

This guide explains how to run Certara’s Certara.RsNLME R package on a remote server, allowing you to leverage more powerful computing resources for your pharmacometric modeling tasks.

To get started, you’ll need to set up both your local machine (the client) and the remote server. Let’s walk through the prerequisites for each.

Setting Up Your Local Machine (Client)

Client System Requirements

RsNLME remote execution is supported on the following operating systems:

Windows 10
Windows 11
Windows Server 2016, 2019, 2022
Linux CentOS 8/RHEL 8
Linux Ubuntu 22.04

Client Installation Prerequisites

NLME Engine and RsNLME Package: You need to have the NLME Engine and the Certara.RsNLME package installed on your local machine. Refer to the RsNLME Installation Guide for detailed instructions. You’ll need the Certara.RsNLME package and its dependencies. Other Certara R packages can be installed but aren’t strictly required for remote execution.

Client Licensing

Local License: You need a valid NLME license on your local machine. See the Authentication and Licensing section of the installation guide.
Automatic Remote Licensing: Certara.RsNLME will automatically configure the necessary licensing information on the remote server if it’s not already set up.

Setting Up the Remote Server

Server System Requirements

The remote server should meet the following requirements:

Operating System: Linux CentOS 8/RHEL 8 or Linux Ubuntu 22.04
openssh-server software: Required for secure remote connections.
OpenMPI library: Installed and configured if you plan to use MPI for parallel processing (optional).
Grid System (Optional): A grid system like SGE, TORQUE, LSF, or SLURM can be installed and configured for enhanced parallel execution.

Server Installation Prerequisites

Refer to the RsNLME Installation Guide, specifically the Installation of NLME Engine and Installation of R Packages sections.

Important: The only required R package on the remote server is Certara.NLME8 (it should be installed manually). You don’t need to install the full suite of RsNLME packages on the server.

To install Certara.NLME8 on the remote server, run the following command:

# Execute this code on the remote server
install.packages("Certara.NLME8", 
  repos = c("https://certara.jfrog.io/artifactory/certara-cran-release-public/", 
            "https://cloud.r-project.org"))

Server Licensing

RsNLME will automatically take care of configuring the necessary licenses on the remote server.

However, if you prefer to manage licenses manually, you can do so by following the instructions in the Authentication and Licensing section of the RsNLME Installation Guide.

Configuring Remote Hosts for Different Run Types

Simple Remote Host (No Parallelization)

A simple remote host is suitable for running models without parallelization (e.g., fitting a model with a small number of fixed effects or omega parameters).

The following R code demonstrates how to configure a simple remote host. You’ll need to modify the settings (e.g., paths, host name, user name) to match your environment before running it on your local machine.

# Example model
model <- emaxmodel(
  checkBaseline = TRUE,
  checkFractional = TRUE,
  checkInhibitory = TRUE,
  data = pkpdData,
  ID = "ID",
  C = "CObs",
  EObs = "EObs"
)

remoteSimpleHost <- hostParams(
  sharedDirectory = "/home/user/temp",                 # Directory on the remote server to store model files (must exist)
  installationDirectory = "/home/user/InstallDirNLME", # Directory where NLME Engine is installed on the remote server
  hostName = "remoteSimpleHost",                       # Internal name for the host (not used in computations)
  machineName = "192.168.1.1",                         # IP address or hostname of the remote server 
  hostType = "UBUNTU2204",                             # Remote server's operating system (UBUNTU2204 or RHEL8)
  numCores = 1,                                        # Number of cores to use (1 for no parallelization)
  isLocal = FALSE,                                     # Set to FALSE for remote execution
  rLocation = "/usr/local/bin",                        # Path to the R executable on the remote server
  userName = "user",                                   # Your username on the remote server
  privateKeyFile = "c:/private/pass.key",              # Path to your private SSH key file (optional, see below)
  parallelMethod = "None"                              # No parallelization method
)

# Note: Avoid using shortcuts like ~ (tilde) in remote paths

# Use the host for remote model fitting:
fitmodelResults <- fitmodel(model = model, hostPlatform = remoteSimpleHost)

Using SSH Keys:

The example above uses a private SSH key for authentication instead of a password. This is generally more secure. Consult your server administrator or refer to resources like PuTTYgen and the ssh package documentation for information on generating SSH keys.

How RsNLME Handles Remote Execution:

RsNLME will automatically:

Transfer the necessary files to the specified remote directory.
Report the status of iterations during the model run.
Download the results from the remote directory when the run is complete.
Load the result data frames into your R environment on your local machine.

Running in the Background:

If you set runInBackground = TRUE in the fitmodel() function (or other model execution functions), the results will be stored on the remote server but won’t be automatically returned to your local machine.

Remote Host with MPI Parallelization

To take advantage of OpenMPI for parallel processing, you can configure your remote host as follows. This example uses a shell script to set the PhoenixMPIDir64 environment variable on the remote server:

  # /home/user/MPIEnable.sh content
  export PhoenixMPIDir64=/lib64/openmpi/

By using a script, you don’t need to set PhoenixMPIDir64 in your .bashrc file.

MPI parallelization is ideal for fitting complex models with many thetas, omegas, or ODEs. It will parallelize the model fitting process by subject (except for the QRPEM engine, which parallelizes by samples).

# Example model
model <- pkemaxmodel(
  parameterization = "Clearance",
  data = pkpdData,
  Time = "Time",
  ID = "ID",
  A1 = "Dose",
  C1Obs = "CObs",
  EObs = "EObs"
)

remoteMPIHost <- hostParams(
  sharedDirectory = "/home/user/temp",                 # Directory on the remote server to store model files
  installationDirectory = "/home/user/InstallDirNLME", # Directory where NLME Engine is installed on the remote server
  hostName = "remoteMPIHost",                          # Internal name for the host
  machineName = "192.168.1.1",                         # IP address or hostname of the remote server
  hostType = "UBUNTU2204",                             # Remote server's operating system
  numCores = 16,                                       # Number of cores to use
  isLocal = FALSE,                                     # Set to FALSE for remote execution
  rLocation = "/usr/local/bin",                        # Path to the R executable on the remote server
  scriptPath = "/home/user/MPIEnable.sh",              # Path to the script that sets up the MPI environment
  userName = "user",                                   # Your username on the remote server
  privateKeyFile = "c:/private/pass.key",             # Path to your private SSH key file
  parallelMethod = "MPI"                               # Use MPI for parallelization
)

# Use the host for remote model fitting:
fitmodelResults <- fitmodel(model = model, 
                            hostPlatform = remoteMPIHost)

The results will be available as a list in the specified variable (fitmodelResults in this case) and downloaded to the model directory on your local machine.

Troubleshooting:

If you encounter errors and the console output isn’t helpful, check the NlmeRemote.LOG file on your local machine for more detailed information.

Multicore Remote Host (Parallel Jobs)

For tasks that can be run as multiple independent jobs in parallel, you can use a multicore remote host. This approach lets the operating system distribute the jobs across the available cores. Suitable run types include:

sortfit
shotgunSearch
stepwiseSearch
bootstrap

input_data <- pkData

# Add gender code column
input_data$GenderCode = 0
input_data$GenderCode[input_data$Gender == "male"] = 1

2CMTModel <-
  pkmodel(
    numCompartments = 2,
    data = input_data,
    ID = "Subject",
    Time = "Act_Time",
    A1 = "Amount",
    CObs = "Conc"
  )

2CMTModel <-
  addCovariate(
    2CMTModel,
    covariate = "Gender",
    type = "Categorical",
    effect = c("V2", "Cl2"),
    levels = c(0, 1),
    labels = c("Female", "Male")
  )

2CMTModel <-
  addCovariate(
    2CMTModel,
    covariate = "BodyWeight",
    direction = "Backward",
    center = "Mean",
    effect = c("V", "Cl")
  )

CovariateEffectNames <-
  listCovariateEffectNames(2CMTModel)
combinations <-
  combn(c("", CovariateEffectNames),
        length(CovariateEffectNames),
        simplify = FALSE)

scenarioNames <-
  lapply(combinations,
         function(x) {paste(x, collapse = " ")})

scenarios <-
  lapply(scenarioNames,
         function(x, CovariateEffectNames) {
           CovariateCombinations <- unlist(strsplit(x, " ", fixed = TRUE))
           scenarioIndex <-
             paste(which(CovariateEffectNames %in% CovariateCombinations,
                         arr.ind = TRUE),
                   collapse = ", ")
           NlmeScenario(trimws(x), scenarioIndex)
         },
         CovariateEffectNames)

remoteMULTICOREHost <- hostParams(
  sharedDirectory = "/home/user/temp",                 # Directory on the remote server to store model files
  installationDirectory = "/home/user/InstallDirNLME", # Directory where NLME Engine is installed on the remote server
  hostName = "remoteMULTICOREHost",                    # Internal name for the host
  machineName = "192.168.1.1",                         # IP address or hostname of the remote server
  hostType = "UBUNTU2204",                             # Remote server's operating system
  numCores = 16,                                       # Number of parallel jobs to run
  isLocal = FALSE,                                     # Set to FALSE for remote execution
  rLocation = "/usr/local/bin",                        # Path to the R executable on the remote server
  userName = "user",                                   # Your username on the remote server
  privateKeyFile = "c:/private/pass.key",             # Path to your private SSH key file
  parallelMethod = "Multicore"                         # Use multicore parallelization
)

# Use the host for remote model fitting:
res <-
  sortfit(2CMTModel,
          hostPlatform = remotemulticoreHost,
          sortColumns = SortColumns("Gender"),
          scenarios = scenarios)

Grid Remote Host (Job Scheduling)

If your remote server has a supported grid system (Torque, SGE, LSF or SLURM), you can submit jobs to the grid for execution. Specify the appropriate grid system using the parallelMethod argument in hostParams().

remoteSLURMHost <- hostParams(
  sharedDirectory = "/home/user/temp",                 # Directory on the remote server to store model files
  installationDirectory = "/home/user/InstallDirNLME", # Directory where NLME Engine is installed on the remote server
  hostName = "remoteSLURMHost",                        # Internal name for the host
  machineName = "192.168.1.2",                         # IP address or hostname of the remote server
  hostType = "UBUNTU2204",                             # Remote server's operating system
  numCores = 16,                                       # Number of nodes to use 
  isLocal = FALSE,                                     # Set to FALSE for remote execution
  rLocation = "/usr/local/bin",                        # Path to the R executable on the remote server
  scriptPath = "/home/user/SLURMEnable.sh",            # Path to a script to initialize the grid (if needed)
  userName = "user",                                   # Your username on the remote server
  privateKeyFile = "c:/private/pass.key",             # Path to your private SSH key file 
  parallelMethod = "SLURM"                             # Use SLURM for parallelization 
)

# Use the host for remote model fitting:
shotgunSearchResults <- shotgunSearch(model = 2CMTModel, 
                                      hostPlatform = remoteSLURMHost)

remoteSGEHost <- hostParams(
  sharedDirectory = "/home/user/temp",                 # Directory on the remote server to store model files
  installationDirectory = "/home/user/InstallDirNLME", # Directory where NLME Engine is installed on the remote server
  hostName = "remoteSGEHost",                          # Internal name for the host
  machineName = "192.168.1.3",                         # IP address or hostname of the remote server
  hostType = "UBUNTU2204",                             # Remote server's operating system
  numCores = 16,                                       # Number of nodes to use
  isLocal = FALSE,                                     # Set to FALSE for remote execution
  rLocation = "/usr/local/bin",                        # Path to the R executable on the remote server
  scriptPath = "/home/user/SGEEnable.sh",              # Path to a script to initialize the grid (if needed)
  userName = "user",                                   # Your username on the remote server
  privateKeyFile = "c:/private/pass.key",             # Path to your private SSH key file
  parallelMethod = "SGE"                               # Use SGE for parallelization 
)

stepwiseSearch(model = model,
               hostPlatform = remoteSGEHost)

Important: Don’t overload the grid by requesting more cores (numCores) than are available. This can lead to errors.

Grid Remote Host with MPI Parallelization (Combined Parallelization)

For very large datasets with many replicates that require significant processing time, you can combine grid-based job scheduling with MPI parallelization within each job (e.g., parallelizing by both subject and replicate). This is useful for tasks like bootstrap, covariate search, or sortfit.

# Parallelization by subjects and replicates (suitable for bootstrap, 
# covariate search, or sortfit)

remoteSLURMMPIHost <- hostParams(
  sharedDirectory = "/home/user/temp",                 # Directory on the remote server to store model files 
  installationDirectory = "/home/user/InstallDirNLME", # Directory where NLME Engine is installed on the remote server
  hostName = "remoteSLURMMPIHost",                     # Internal name for the host
  machineName = "192.168.1.3",                         # IP address or hostname of the remote server 
  hostType = "UBUNTU2204",                             # Remote server's operating system
  numCores = 16,                                       # Total number of cores available
  isLocal = FALSE,                                     # Set to FALSE for remote execution
  rLocation = "/usr/local/bin",                        # Path to the R executable on the remote server
  scriptPath = "/home/user/SLURM_MPIEnable.sh",        # Optional path to a script for grid initialization and
                                                       # setting PhoenixMPIDir64 (if not in .bash_profile)
  userName = "user",                                   # Your username on the remote server
  privateKeyFile = "c:/private/pass.key",             # Path to your private SSH key file
  parallelMethod = "SGE_MPI"                           # Use SLURM and MPI for parallelization
)

# Use the host for remote model fitting:
bootstrapResults <- bootstrap(model = model, 
                              hostPlatform = remoteSLURMMPIHost,
                              numReplicates = 1000)

Automatic Core Distribution:

RsNLME will attempt to find the optimal distribution of nodes and MPI cores per node, considering the number of subjects and jobs. The number of cores used for each replicate will be the smaller of these two values:

The total number of available cores divided by the number of replicates.
The number of unique subjects in the replicate divided by 3.

Examples:

Example 1: 300 cores available, 2000 replicates requested, 200 unique subjects. Each replicate would use 1 core (300/2000 < 1, 200/3 = 66, 1 < 66). Total cores used: 300.
Example 2: 1000 cores available, 200 replicates requested, 300 unique subjects. Each replicate would use 5 cores (1000/200 = 5, 300/3 = 100, 5 < 100). Total cores used: 1000.