Client Prerequisites

Client System Requirements

  • Windows 8, 10
  • Windows Server 2018, 2019
  • Linux CentOS7/RHEL7

Client Installation Prerequisites

Please see RsNLME Installation Guide.

Note: Command line interface requires installation of the Certara.RsNLME R package and its associated dependencies. Other R packages developed by Certara may be installed but are not required.

Client Licensing

For remote execution of built-in models via simple fit, Phoenix license server is sufficient. However for alternative execution types (covariate search, bootstrap, etc)., textual model building, and/or workflows via Shiny GUI, both Phoenix License server and local license are required.

Note: There is no additional costs to acquire the local license if you have already licensed RsNLME for remote execution.

Server Prerequisites

Server System Requirements

  • Linux CentOS7/RHEL7 with ssh server ready and port 22 opened
  • SGE or TORQUE or LSF grid (optional).

Server Installation Prerequisites

Please see RsNLME Installation Guide, Installation of NLME Engine and Installation of R Packages chapters.

Note: The Certara.NLME8 package, a dependency that is automatically installed with Certara.RsNLME, is the only package that is required on the remote host. The user may simply install this package without installing the entire suite of RsNLME packages on the remote host.

# the code should be executed on remote host
install.packages("Certara.NLME8", 
  repos = c("https://certara.jfrog.io/artifactory/certara-cran-release-public/", 
            "https://cloud.r-project.org"))

Server licensing

License required on the server side. Please see Licensing.

For remote runs, a floating license is preferred (PhoenixLicenseServer env.variable which points to the License Server should be specified in .bash_profile file or in initialization remote script).

Host to be used for remote runs

The following code should be modified before execution on the local host. It creates a host which will be used for connection.

# no parallelization; could be used for simple fit of the model
# with few fixed effects/omegas
remoteSimpleHost <- NlmeParallelHost(
  sharedDirectory = "/home/user/temp",                 # a directory where the model files are stored
  installationDirectory = "/home/user/InstallDirNLME", # a directory where NLME Engine is unzipped
  hostName = "remoteSimpleHost",                       # an internal name of the host, not used in computations
  machineName = "192.168.1.1",                         # network address used for ssh session
  hostType = "Linux",                                  # remote host OS; currently Linux only supported
  numCores = 1,                                        # number of cores
  isLocal = FALSE,                                     # FALSE since the remote run is used
  rLocation = "/usr/local/bin",                        # R path to be used for execution
  userAuthentication = 
    NlmeUserAuthentication(userName = "user",          # user account on the remote host
                           userPassword = "password"), # user password on the remote host
  parallelMethod = NlmeParallelMethod("none")          # no parallelization methods for the current host
)

# note that shortcuts as tilde(~) should be avoided in remote paths
# when ready, the host could be used for remote runs:
# fitmodelResults <- fitmodel(model = model, hostPlatform = remoteSimpleHost)

Next, we’ll modify the host to use openMPI installed on the system: in the next R script the host for remote run has a shell script link which specifies MPI Directory env.variable, but firstly look at the shell script content:

  # /home/user/MPIEnable.sh content
  export PhoenixMPIDir64=/lib64/openmpi/  

In such case it is not necessary to specify PhoenixMPIDir64 in .bash_profile.

# parallelization by subject IDs; could be used for simple fit of complex model

remoteMPIHost <- NlmeParallelHost(
  sharedDirectory = "/home/user/temp",                 # a directory where the model files are stored
  installationDirectory = "/home/user/InstallDirNLME", # a directory where NLME Engine is unzipped
  hostName = "remoteMPIHost",                          # an internal name of the host, not used in computations
  machineName = "192.168.1.1",                         # network address used for ssh session
  hostType = "Linux",                                  # remote host OS; currently Linux only supported
  numCores = 16,                                      # number of cores
  isLocal = FALSE,                                     # FALSE since the remote run is used
  rLocation = "/usr/local/bin",                        # R path to be used for execution
  scriptPath = "/home/user/MPIEnable.sh",              # path to the script to be sourced
  userAuthentication = 
    NlmeUserAuthentication(userName = "user",          # user account on the remote host
                           userPassword = "password"), # user password on the remote host
  parallelMethod = NlmeParallelMethod("LOCAL_MPI")     # MPI parallelization for the current host
)

# when ready, the host could be used for remote runs:
# fitmodelResults <- fitmodel(model = model, 
                              hostPlatform = remoteMPIHost)

After execution, the results will be available as a list in the variable given and downloaded in the model directory.

If something goes wrong and console output is not informative, useful information can be found in the local NlmeRemote.LOG file or in the remote NlmeRemote.LOG file (if for some reason it was not downloaded locally). The location of the remote file can be found in the beginning of the execution output.

If one of the supported grid systems is installed on the remote host (e.g.,"Torque", "SGE", "SF"), you may execute the job on the grid by specifying applicable option to NlmeParallelMethod() for the argument parallelMethod. See ?NlmeParallelMethod for available options.

# parallelization by replicates
# could be used for bootstrap, covariate search or sortfit

remoteTORQUEHost <- NlmeParallelHost(
  sharedDirectory = "/home/user/temp",                 # a directory where the model files are stored
  installationDirectory = "/home/user/InstallDirNLME", # a directory where NLME Engine is unzipped
  hostName = "remoteTORQUEHost",                       # an internal name of the host, not used in computations
  machineName = "192.168.1.2",                         # network address used for ssh session
  hostType = "Linux",                                  # remote host OS; currently Linux only supported
  numCores = 16,                                       # number of nodes to be used
  isLocal = FALSE,                                     # FALSE since the remote run is used
  rLocation = "/usr/local/bin",                        # R path to be used for execution
  scriptPath = "/home/user/TORQUEEnable.sh",           # path to the script - i.e., grid initialization
  userAuthentication = 
    NlmeUserAuthentication(userName = "user",          # user account on the remote host
                           privateKeyFile = "D:/private/TORQUEkey.key"), # keyfile in OpenSSH format
  parallelMethod = NlmeParallelMethod("TORQUE")        # TORQUE parallelization for the current host
)

# when ready, the host could be used for remote runs:
# sortfitResults <- sortfit(model = model, 
                            hostPlatform = remoteTORQUEHost,
                            sortColumns = c("Gender", "Sex", "Site"))

Note: Do not overload the Grid with numCores argument exceeding the number of nodes available for the current grid, otherwise uninformative errors may be given.

Note that the example above uses private key file authentication instead of password. Please refer to ssh package documentation for details.

If supported Grid system is not installed, the user may start the jobs in parallel using one node. In doing so, the user delegates to the system a job distribution process across all available cores.

# parallelization by replicates
# could be used for bootstrap, covariate search or sortfit

remoteMULTICOREHost <- NlmeParallelHost(
  sharedDirectory = "/home/user/temp",                 # a directory where the model files are stored
  installationDirectory = "/home/user/InstallDirNLME", # a directory where NLME Engine is unzipped
  hostName = "remoteMULTICOREHost",                    # an internal name of the host, not used in computations
  machineName = "192.168.1.2",                         # network address used for ssh session
  hostType = "Linux",                                  # remote host OS; currently Linux only supported
  numCores = 16,                                       # number of nodes to be used
  isLocal = FALSE,                                     # FALSE since the remote run is used
  rLocation = "/usr/local/bin",                        # R path to be used for execution
  userAuthentication = 
    NlmeUserAuthentication(userName = "user",          # user account on the remote host
      privateKeyFile = "D:/private/TORQUEkey.key"),    # keyfile in OpenSSH format
  parallelMethod = NlmeParallelMethod("MULTICORE")     # TORQUE parallelization for the current host
)

# when ready, the host could be used for remote runs:
# shotgunSearchResults <- shotgunSearch(model = model, 
                                        hostPlatform = remoteMULTICOREHost)

For large datasets with many replicates, where each replicate requires extended processing time, it is useful to start with by job/by subject parallelization simultaneously.

# parallelization by subjects and by replicates
# could be used for bootstrap, covariate search or sortfit

remoteSGEMPIHost <- NlmeParallelHost(
  sharedDirectory = "/home/user/temp",                 # a directory where the model files are stored
  installationDirectory = "/home/user/InstallDirNLME", # a directory where NLME Engine is unzipped
  hostName = "remoteSGEMPIHost",                       # an internal name of the host, not used in computations
  machineName = "192.168.1.3",                         # network address used for ssh session
  hostType = "Linux",                                  # remote host OS; currently Linux only supported
  numCores = 16,                                       # number of nodes to be used
  isLocal = FALSE,                                     # FALSE since the remote run is used
  rLocation = "/usr/local/bin",                        # R path to be used for execution
  scriptPath = "/home/user/SGE_MPIEnable.sh",          # path to the script - i.e., grid initialization
                                                       # and PhoenixMPIDir64 export (if not done in .bash_profile)
  userAuthentication = 
    NlmeUserAuthentication(userName = "user",          # user account on the remote host
      privateKeyFile = "D:/private/SGEEkey.key"),      # keyfile in OpenSSH format
  parallelMethod = NlmeParallelMethod("SGE_MPI")       # SGE and MPI parallelization for the current host
)

# when ready, the host could be used for remote runs:
# bootstrapResults <- bootstrap(model = model, 
                                hostPlatform = remoteSGEMPIHost)

RsNLME will try to find optimal nodes/MPI cores by node distribution taking into account the number of subjects in the jobs and the number of jobs.