---
title: "teal_data reproducibility"
author: "NEST CoreDev"
output:
  rmarkdown::html_vignette:
    toc: true
vignette: >
  %\VignetteIndexEntry{teal_data reproducibility}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

# Reproducibility of `teal_data` objects

Reproducibility is a primary function of the `qenv` class, which `teal_data` inherits from.
Every data modification in a `teal_data` object is performed in an encapsulated environment, separate from the global environment.

It is important to note that the reproducibility of this object is limited only to the data-code relationship.
Other aspects such as the reliability of the data source, reproducibility of the R session (including package versions), and creation and use of objects from other environments (e.g. `.GlobalEnv`) cannot be verified properly by `teal_data`.
It is advisable to always begin analysis in a new session and run all code that pertains to the analysis within the `teal_data` object.

## Verification

### Verification status

Every `teal_data` object has a _verification status_, which is a statement of whether the contents of the `env` can be reproduced by `code`.
From this perspective, `teal_data` objects that are instantiated empty are _verified_ but ones instantiated with data and code are _unverified_ because the code need not be reproducible.
Obviously, `teal_data` objects instantiated with data only are _unverified_ as well.

When evaluating code in a `teal_data` object, the code that is stored is the same as the code that is executed, so it is reproducible by definition.
Therefore, evaluating code in a `teal_data` object _does not_ change its verification status.

The verification status is always printed when inspecting a `teal_data` object.
Also, when retrieving code, unverified objects add a warning to the code stating that it has not passed verification.

```{r, message=FALSE, error=TRUE}
library(teal.data)

data_empty <- teal_data()
data_empty # is verified
data_empty <- within(data_empty, i <- head(iris))
data_empty # remains verified

data_with_data <- teal_data(i = head(iris), code = "i <- head(iris)")
data_with_data # is unverified
data_with_data <- within(data_with_data, i$rand <- sample(nrow(i)))
data_with_data # remains unverified
cat(get_code(data_with_data)) # warning is prepended
```

### Verification process

In order to confirm that the code stored in `teal_data` exactly reproduces the contents of the environment, one must run the `verify()` function.
This causes the code to be evaluated and the results to be compared to the contents of the environment.
If the code executes without errors and the results are the same as the contents already present in the environment, the verification is successful and the object's state will be changed to _verified_.
Otherwise an error will be raised.

#### verified

```{r}
library(teal.data)

data <- data.frame(x = 11:20)
data$id <- seq_len(nrow(data))

data_right <- teal_data(
  data = data,
  code = quote({
    data <- data.frame(x = 11:20)
    data$id <- seq_len(nrow(data))
  })
)
(data_right_verified <- verify(data_right)) # returns verified object
```

#### unverified

```{r, message=FALSE, error=TRUE, purl=FALSE}
data_wrong <- teal_data(
  data = data,
  code = quote({
    data <- data.frame(x = 11:20)
  })
)
verify(data_wrong) # fails verification, raises error
```

## Retrieving code

The `get_code` function is used to retrieve the code stored in a `teal_data` object.
A simple `get_code(<teal_data>)` will return the entirety of the code but using the `datanames` argument allows for obtaining a subset of the code that only deals with some of the objects stored in `teal_data`.

```{r}
library(teal.data)

data <- within(teal_data(), {
  i <- iris
  m <- mtcars
  head(i)
})
cat(get_code(data)) # retrieve all code
cat(get_code(data, datanames = "i")) # retrieve code for `i`
```

Note that in when retrieving code for a specific dataset, the result is only the code used to _create_ that dataset, not code that _uses_ is.

## Tracking object dependencies

Calling `get_code` with `datanames` specified initiates an analysis of the stored code, in which object dependencies are automatically discovered.
If object `x` is created with an expression that uses object `y`, the lines that create object `y` must also be returned.
This is quite effective when objects are created by simple assignments like `x <- foo(y)`.
However, in rare cases discovering dependencies is impossible, _e.g._  when opening connections to databases or when objects are created by side effects (functions acting on their calling environment implicitly rather than returning a value that is then assigned).
In such cases the code author must manually tag code lines that are required for a dataset by adding a special comment to the lines: `# @linksto x` will cause the line to be included when retrieving code for `x`.

See `?get_code` for a detailed explanation and examples.