Random Forest Classification Model in R

Random Forest is one of the most popular supervised machine learning algorithms for classification problems. It combines the predictions of multiple decision trees to produce a model that is generally more accurate, stable, and resistant to overfitting than a single decision tree.

Random Forest is widely used in finance, healthcare, fraud detection, customer segmentation, marketing analytics, and many other predictive modeling applications because it performs well with both numerical and categorical data.

In this tutorial, you’ll learn how to build a Random Forest classification model in R, split data into training and testing sets, evaluate model performance, identify important features, and generate predictions.

What Is Random Forest Classification?

Random Forest Classification is an ensemble learning method that creates many decision trees using different bootstrap samples of the training data.

Each tree independently predicts the class label, and the final prediction is determined by majority voting.

Because the model averages predictions across many trees, it typically reduces variance and improves generalization compared with a single decision tree.

Why Use Random Forest?

Random Forest offers several advantages:

• High predictive accuracy
• Handles nonlinear relationships
• Robust to outliers and noise
• Works with large datasets
• Handles both numerical and categorical variables
• Estimates feature importance
• Resistant to overfitting compared with individual decision trees
• Requires relatively little data preprocessing

Random Forest Classification vs Regression

Random Forest can solve two different types of machine learning problems.

Examples of classification include:

• Spam vs Not Spam
• Fraud vs Legitimate
• Churn vs Retained
• Disease vs Healthy

Regression predicts continuous values such as sales, prices, or temperatures.

Example Dataset

Suppose we have a dataset containing six predictor variables:

• Attribute1
• Attribute2
• Attribute3
• Attribute4
• Attribute5
• Attribute6

The response variable is Likeability, which is converted into two classes:

• LOW
• HIGH

A simple approach is to classify observations based on the average Likeability score.

Likeability < Mean  → LOW

Likeability ≥ Mean → HIGH

Note: While median or business-defined thresholds are often preferred in practice, this example uses the mean as the cutoff for illustration.

Load the Dataset

Suppose the data are stored in an Excel file.

library(openxlsx)

data <- read.xlsx(
  "D:/rawdata.xlsx",
  sheet = "Sheet1"
)

Convert the Target Variable into Categories

Create a binary classification variable.

mean_like <- mean(data$Likeability)

data$Likeability <- ifelse(
  data$Likeability < mean_like,
  "LOW",
  "HIGH"
)

data$Likeability <- as.factor(data$Likeability)

The response variable must be stored as a factor for classification models in R.

Split the Data

Machine learning models should always be evaluated on data that were not used during training.

A simple split is:

• 80% Training
• 20% Testing

set.seed(123)

train_index <- sample(
  seq_len(nrow(data)),
  size = 0.8 * nrow(data)
)

train_data <- data[train_index, ]

test_data <- data[-train_index, ]

Using random sampling helps reduce bias and produces a more representative train-test split than selecting rows by position.

Build the Random Forest Model

Load the package.

library(randomForest)

Train the model.

rf_model <- randomForest(

  Likeability ~ .,

  data = train_data,

  ntree = 100,

  mtry = 3,

  importance = TRUE

)

Model Summary

Display the model.

print(rf_model)

Example output:

Type of random forest:
Classification

Number of trees:
100

Number of variables tried at each split:
3

OOB estimate of error rate:
6.67%

What Is the Out-of-Bag (OOB) Error?

Random Forest automatically estimates prediction error using Out-of-Bag (OOB) samples.

Each tree is trained using a bootstrap sample of the data, leaving roughly one-third of the observations out of that tree’s training set. These excluded observations are used to estimate prediction error without requiring a separate validation dataset.

Lower OOB error generally indicates better model performance.

Confusion Matrix

Example:

            Predicted

Actual     HIGH LOW

HIGH         10   2

LOW           2  46

The confusion matrix summarizes the model’s predictions.

It reports:

• True Positives
• True Negatives
• False Positives
• False Negatives

From this table, additional performance metrics can be calculated.

Model Accuracy

Accuracy is computed as:$$Accuracy=\frac{\text{Correct Predictions}} {\text{Total Predictions}}$$Accuracy=Total PredictionsCorrect Predictions​

For example,

Accuracy = 93%

Although accuracy is easy to interpret, it may be misleading for highly imbalanced datasets. In such cases, metrics such as precision, recall, F1-score, and ROC-AUC should also be considered.

Feature Importance

One major advantage of Random Forest is its ability to estimate variable importance.

Plot the importance.

varImpPlot(rf_model)

Or display numerical values.

importance(rf_model)

Variables with larger importance scores contribute more to the model’s predictions.

Feature Selection Using Boruta

The Boruta package identifies variables that are statistically important for prediction.

library(Boruta)

important_vars <- Boruta(

  Likeability ~ .,

  data = train_data

)

print(important_vars)

Example output:

Boruta performed 29 iterations.

Confirmed important:

Attribute1

Attribute2

Attribute3

Attribute4

Attribute5

Attribute6

Boruta compares each feature against randomized “shadow” variables to determine whether its importance is greater than expected by chance.

Make Predictions

Generate predictions for the test dataset.

predictions <- predict(

  rf_model,

  newdata = test_data

)

predictions

Evaluate Predictions

Compare predicted values with the true class labels.

table(

  Actual = test_data$Likeability,

  Predicted = predictions

)

You can also compute the overall accuracy.

mean(

  predictions == test_data$Likeability

)

Hyperparameters

Several parameters influence Random Forest performance.

Hyperparameter tuning can improve predictive performance.

Applications of Random Forest Classification

Random Forest is widely used in:

• Credit risk analysis
• Fraud detection
• Medical diagnosis
• Customer churn prediction
• Email spam filtering
• Image recognition
• Loan approval systems
• Manufacturing quality control
• Marketing segmentation
• Predictive maintenance

Common Mistakes

Avoid these common issues:

• Training and testing on the same data.
• Ignoring class imbalance.
• Using too few trees.
• Overinterpreting feature importance as causal.
• Evaluating performance using accuracy alone.
• Failing to tune model parameters for optimal performance.

Conclusion

Random Forest Classification is a powerful and versatile machine learning algorithm that delivers strong predictive performance across a wide range of applications. By combining the predictions of multiple decision trees, it reduces overfitting, handles complex nonlinear relationships, and provides useful measures of feature importance.

In R, the randomForest package makes it straightforward to build classification models, evaluate them using Out-of-Bag error and confusion matrices, and generate predictions for new data. To achieve the best results, use proper train-test splits, assess multiple evaluation metrics, and tune model parameters based on your specific dataset and problem.