---
title: "Importing NONMEM into rxode2"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Importing NONMEM into rxode2}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
library(rxode2)
setRxThreads(1L)
library(data.table)
setDTthreads(1L)

# the ADVAN5/7 matExp() examples below need matrix-exponential support in the
# installed rxode2; skip them gracefully on older versions
.hasMatExp <- exists("indLin", where=asNamespace("rxode2"), inherits=FALSE)

```

The goal of `nonmem2rx` is to convert a NONMEM control stream to
`rxode2` for easy clinical trial simulation in R.

Here is a quick example of a conversion:

```{r model}
library(nonmem2rx)

# First we need the location of the nonmem control stream Since we are running
# an example, we will use one of the built-in examples in `nonmem2rx`
resFile <- system.file("mods/cpt/runODE032.res", package="nonmem2rx")
# You can use a control stream or the listing (.lst or .res) file

mod <- nonmem2rx(resFile, save=FALSE, determineError=FALSE)
mod
```

## Setting up `nonmem2rx` for your model

Some common options that you may want to change when importing NONMEM
control stream are:

- The default NONMEM output extension; By default it is `.lst`.  You
  can set it to something else, like `.res`, using the following
  option: `options(nonmem2rx.lst=".res")`.

- Turn on [extended control
  stream](https://wfn.sourceforge.net/wfncs.htm#control_streams)
  support, as used by Wings for NONMEM. You can turn it on by
  `options(nonmem2rx.extended=TRUE)`

- `ADVAN5`/`ADVAN7` general linear models are translated to `rxode2`'s native
  matrix-exponential `matExp()` model by default. If you prefer ordinary
  differential equations, turn this off with `nonmem2rx(..., matexp=FALSE)` or
  `options(nonmem2rx.matexp=FALSE)` (see [General linear models
  (`ADVAN5`/`ADVAN7`)](#general-linear-models-advan5advan7) below).

You probably also want to change the name of parameters and
compartments. The easiest way to name the parameters whatever you want
is to pre-specify the names.  For example:

```{r prespecify}
mod <- nonmem2rx(system.file("mods/cpt/runODE032.ctl", package="nonmem2rx"), lst=".res", save=FALSE,
                 thetaNames=c("lcl", "lvc", "lq", "lvp", "prop.sd"),
                 etaNames=c("eta.cl", "eta.vc", "eta.q","eta.vp"),
                 cmtNames = c("central", "perip"))

mod
```

This checks the parameter names to make sure they are the same
length as the input names, if they are not, the model will skip parameter
renaming and keep the default translation names `theta#` and `eta#`.

As a note, `sigma` parameters are not currently renamed; So for the
following model (which grabs the parameter automatically labels to
generate variables), `sigma` is simply `eps#`.

```{r prespecifySigma}
mod <- nonmem2rx(system.file("Theopd.ctl", package="nonmem2rx"), save=FALSE)
mod
```

You can still rename however you wish, though, using model piping
(`rxRename()` or `dplyr::rename()` would both work):

```{r renameMod}
mod <- mod %>% rxRename(add.var=eps1)
mod
```

This model does not specify the residuals in a way that makes sense to
`nlmixr2`.  If you want, you can still [convert the `rxode2` model to
a nlmixr2 fit](https://nlmixr2.github.io/nonmem2rx/articles/convert-nlmixr2.html).

## Pointing `nonmem2rx` at the input data

To validate the translation, `nonmem2rx` reads the NONMEM input
dataset and re-runs the model to compare against the NONMEM output.
By default it finds the data from the `$DATA` record of the control
stream.  When you import a model from another system, though, that
path often does not match your machine, so you can override it with
the `inputData` argument.

`inputData` accepts `NULL` (the default -- find the data from the
`$DATA` record of the control stream), a **path**, or an **in-memory
`data.frame`**.

The path form is the simplest fix when the data is a plain CSV that
just lives somewhere else -- for example a folder or two higher up
than the control stream:

```{r inputDataPath, eval=FALSE}
mod <- nonmem2rx("runs/run1/run1.ctl",
                 inputData = "../../data.csv")
```

When the data is not a plain on-disk CSV -- it is already in memory,
came from a database, or needs a custom reader (fixed-width,
tab-delimited, decompressed on the fly) -- read it in yourself and
pass the resulting `data.frame`:

```{r inputDataFrame}
# read the data from wherever it really is
d <- read.csv(system.file("mods/cpt/Bolus_2CPT.csv", package="nonmem2rx"))

mod <- nonmem2rx(system.file("mods/cpt/runODE032.ctl", package="nonmem2rx"),
                 lst=".res", save=FALSE, inputData = d)
mod
```

Either way the control stream itself is left untouched.

One thing to get right: the supplied `data.frame` is interpreted
**positionally against `$INPUT`** -- the column *names* are discarded
and re-applied from the `$INPUT` record, and the usual `$INPUT` names,
`DROP`, `IGNORE`/`ACCEPT` filters and record subsetting are then
applied.  So read the data in the same column order NONMEM would see
it and do not reorder, insert, or drop columns beforehand.  If the CSV
has no header, use `read.csv(..., header = FALSE)`; a header row that
NONMEM skips via `IGNORE=@` is fine with a plain `read.csv()`, since
the header becomes the (discarded) names and the data rows still line
up with `$INPUT`.

## General linear models (`ADVAN5`/`ADVAN7`)

NONMEM's `ADVAN5` and `ADVAN7` are *general linear* models: a system of
compartments connected by rate constants (`K12`, `K21`, `K20`, ...) that
NONMEM solves with matrix exponentials.  By default `nonmem2rx` mirrors this
and translates them to `rxode2`'s native matrix-exponential `matExp()` model,
using `cmt()` declarations and `k_<from>_<to>` rate constants (with
`k_<from>_output` for elimination):

```{r advan5matexp, eval=.hasMatExp}
advan5 <- system.file("mods/advan5/advan5.ctl", package="nonmem2rx")

mod <- nonmem2rx(advan5, validate=FALSE, save=FALSE)
mod
```

If you prefer explicit ordinary differential equations, use `matexp=FALSE` to
translate the same linear system to `d/dt()` equations instead:

```{r advan5ode, eval=.hasMatExp}
mod <- nonmem2rx(advan5, validate=FALSE, save=FALSE, matexp=FALSE)
mod
```

Both models solve to the same predictions; the matrix-exponential form simply
lets `rxode2` use its matrix-exponential solver rather than a general ODE
integrator.  Only `ADVAN5`/`ADVAN7` models are affected.

## Technical details about reading NONMEM to rxode2

The key files to import are the NONMEM control stream (or related
file) and the NONMEM output (often with a `.lst` or `.res` extension).

The import process steps are below:

- Read in the nonmem control stream and convert the model to a
  `rxode2` ui function.

- Try to determine an endpoint/residual specification in the model (if
  possible), and convert to a fully qualified ui model that can be
  used in `nlmixr2` and `rxode2`. If it cannot be determined
  automatically, [you can manually fix this](https://nlmixr2.github.io/nonmem2rx/articles/convert-nlmixr2.html) and
  still convert to a `nlmixr2` object (if the data/estimates are
  available of course).

- If available, `nonmem2rx` will read the final parameter estimates
  and update the model.

- The converter will read in the nonmem input dataset, and search for
  the output files with `IPRED`, `PRED` and the `ETA` values. The
  translated `rxode2` model is run for the population parameters and
  the individual parameters.  This will then compare the results
  between `NONMEM` and `rxode2` to make sure the translation makes
  sense.  This only works when `nonmem2rx` has access to the input
  data and the output with the `IWRES`, `IPRED`, `PRED` and the `ETA`
  values.

- Converts the upper case NONMEM variables to lower case (can be
  turned off with `nonmem2rx(..., toLowerLhs=FALSE))`)

- Replaces the NONMEM theta / eta names with the label-based names
  like an extended control stream (can be turned off with
  `nonmem2rx(thetaNames=FALSE, etaNames=FALSE)`)

- Replaces the compartment names with the defined compartment names in
  the control stream (ie `COMP=(compartmenName)`)
