How to arrange training and testing datasets in R

How to arrange training and testing datasets in R, To divide a data frame into training and test sets for model construction in R, use the createDataPartition() function from the caret package.

The basic syntax used by this function is as follows:

createDataPartition(y, times = 1, p = 0.5, list = TRUE, …)

where:

y: vector of outcomes

times: number of partitions to create

p: percentage of data to use in the training set

list: whether or not to save the results in a list

The example below demonstrates how to use this function in practice.

PCA for Categorical Variables in R » finnstats

Example:- How to arrange training and testing datasets in R

Assume we have a data frame in R with 1,000 rows containing information about students’ study hours and their final exam scores:

Make this example replicable.

set.seed(123)

Let’s create a data frame

df <- data.frame(hours=runif(1000, min=0, max=10),
                 score=runif(1000, min=40, max=100))

Now we can view the head of the data frame

head(df)

     hours    score
1 7.8355588 64.49499
2 0.7643654 77.62753
3 8.9624691 51.64345
4 0.1915280 85.47124
5 8.6345563 93.20182
6 8.9055675 44.02384

Assume we want to fit a simple linear regression model that predicts final exam score based on hours studied.

Assume we want to train the model on 80% of the rows in the data frame and then test it on the remaining 20%.

The following code demonstrates how to split the data frame into training and testing sets using the caret package’s createDataPartition() function.

How to combine Multiple Plots in R » finnstats

library(caret)

Divide the data frame into training and testing sets.

train_indices <- createDataPartition(df$score, times=1, p=.8, list=FALSE)

Now ready to create a training set

dftrain <- df[train_indices , ]

Now we can create a testing set

dftest  <- df[-train_indices, ]

Let’s view the number of rows in each set

nrow(dftrain)
[1] 800

nrow(dftest)
[1] 200

As we can see, our training dataset has 800 rows, which accounts for 80% of the original dataset.

ave for average calculation in R » finnstats

Similarly, our test dataset has 200 rows, which represents 20% of the original dataset.

The first few rows of each set can also be seen:

Now we can view the head of training set

head(dftrain)

     hours    score
1 8.966972 55.93220
2 2.655087 71.84853
3 3.721239 81.09165
4 5.728534 62.99700
5 9.082078 97.29928
7 8.983897 42.34600

Let’s view the head of the testing set

head(dftest)

      hours    score
6  2.016819 47.10139
12 2.059746 96.67170
18 7.176185 92.61150
23 2.121425 89.17611
24 6.516738 50.47970
25 1.255551 90.58483

We can then use the training set to train the regression model and the testing set to evaluate its performance.

When to Use plotly? » finnstats