In machine learning, building a model is only half of the engineering task.
The more important question is:
How well does the model generalize to unseen data, and is it reliable enough for the intended application?
A useful evaluation process should answer several questions:
The interpretation of a metric is often more important than the metric itself.
A model with an excellent training score may still fail in production.
| Scenario | Possible interpretation |
|---|---|
| Low training error + high testing error | Overfitting |
| High training error + high testing error | Underfitting |
| Similar train and test errors | Better generalization |
| Excellent metric but unrealistic behavior | Possible leakage, invalid features, or poor data representation |
In engineering systems, models should also be evaluated for:
Regression models predict continuous numerical values.
Examples include:
Common regression metrics include:
MAE measures the average absolute difference between actual and predicted values.
MAE = (1/n) × Σ |y_true - y_pred|
For example:
MAE = 2 °C
means that the predictions differ from the actual values by about 2 °C on average.
MAE is useful when:
MSE squares the prediction errors before averaging them.
MSE = (1/n) × Σ (y_true - y_pred)²
Because the errors are squared, large errors receive substantially greater weight.
MSE is useful when:
A model can have a reasonable MAE but a much larger MSE when a small number of predictions contain very large errors.
RMSE is the square root of MSE.
RMSE = √MSE
RMSE has the same units as the target while retaining greater sensitivity to large errors than MAE.
For example:
RMSE = 5 MPa
means that the typical error magnitude, with large errors receiving greater weight, is around 5 MPa.
The relationship between MAE and RMSE can provide useful information:
| Observation | Possible interpretation |
|---|---|
| RMSE close to MAE | Errors are relatively uniform |
| RMSE much larger than MAE | Large errors or outliers are present |
| Low MAE + high RMSE | Model performs well generally but fails badly in some regions |
| Both high | Overall prediction quality is poor |
For sensor systems, large RMSE relative to MAE may indicate:
R², or the coefficient of determination, measures how much of the target variance is explained relative to a baseline mean prediction.
R² = 1 - (SS_res / SS_tot)
A value of:
R² = 1
indicates a perfect fit on the evaluated data.
An R² of 0 means the model performs no better than predicting the mean target value under the standard definition.
R² can also be negative when the model performs worse than that baseline.
R² does not tell us the physical magnitude of the prediction error.
Therefore, a strong regression evaluation should usually report R² together with MAE and/or RMSE.
For example:
R² → How much variation is explained?
MAE → What is the typical absolute error?
RMSE → How severe are large errors?
A high R² can still coexist with:
Mean Absolute Percentage Error expresses prediction error as a percentage.
MAPE = (100/n) × Σ |(y_true - y_pred) / y_true|
For example:
Actual: [100, 200, 50]
Predicted: [ 90, 220, 40]
Percentage errors:
10%, 10%, 20%
MAPE = 13.33%
MAPE becomes problematic when actual values are:
For sensor data, this can be a major limitation.
When MAPE is unsuitable, alternatives include:
Use MAPE when the target is strictly positive and values are not close to zero.
Classification models predict categories or classes.
Examples include:
Common evaluation tools include:
A confusion matrix breaks predictions into four categories:
| Predicted Positive | Predicted Negative | |
|---|---|---|
| Actual Positive | True Positive (TP) | False Negative (FN) |
| Actual Negative | False Positive (FP) | True Negative (TN) |
These four quantities form the basis of many classification metrics.
Accuracy = (TP + TN) / (TP + TN + FP + FN)
Accuracy measures the proportion of all predictions that are correct.
However, accuracy can be misleading for imbalanced datasets.
Suppose:
99% → Normal
1% → Failure
A model that always predicts Normal achieves 99% accuracy while detecting no failures.
Therefore, accuracy should not be used alone for strongly imbalanced fault-detection problems.
Precision measures how many predicted positives are actually positive.
Precision = TP / (TP + FP)
High precision means relatively few false alarms.
Precision becomes important when false positives are costly, for example:
Recall measures how many actual positive cases are detected.
Recall = TP / (TP + FN)
High recall means relatively few missed positive cases.
Recall is particularly important when missed failures have serious consequences.
For example, in industrial fault detection:
Missed fault → equipment may continue operating
False alarm → inspection may be triggered
The relative cost of these two errors should determine how the decision threshold and evaluation metrics are selected.
Precision and recall usually trade off as the classification threshold changes.
| Application concern | Metric to emphasize |
|---|---|
| Avoid false alarms | Precision |
| Avoid missed failures | Recall |
| Both are important | F1 / Precision-Recall analysis |
The correct threshold is therefore not necessarily 0.5.
It should be selected according to the application, error costs, and operating requirements.
F1 is the harmonic mean of precision and recall.
F1 = 2 × (Precision × Recall) / (Precision + Recall)
F1 is useful when:
However, F1 is still a threshold-dependent metric and should not replace inspection of the confusion matrix and application-specific error costs.
The Receiver Operating Characteristic (ROC) curve examines the relationship between:
across classification thresholds.
ROC-AUC summarizes the ranking ability of a classifier across those thresholds.
A value close to:
AUC = 1.0
indicates strong separation on the evaluated data, while:
AUC = 0.5
corresponds to random ranking under the standard interpretation.
ROC-AUC can appear strong on highly imbalanced datasets even when the positive-class precision is poor.
For rare-event detection, Precision-Recall curves and PR-AUC can therefore be more informative.
Precision-Recall AUC summarizes the precision-recall trade-off across thresholds.
PR-AUC is particularly useful when:
For industrial anomaly or fault detection, PR-AUC can provide a more relevant view of rare positive-event performance than ROC-AUC alone.
There is no universally best metric.
The metric should reflect the real-world cost of prediction errors.
Task
↓
Understand error costs
↓
Choose relevant metrics
↓
Select operating threshold
↓
Validate on representative unseen data
For example:
| Requirement | Useful evaluation |
|---|---|
| General classification | Accuracy + confusion matrix |
| Rare fault detection | Recall + PR-AUC |
| Avoid false alarms | Precision + false-positive rate |
| Balance precision and recall | F1 |
| Safety-oriented detection | Recall + false-negative rate |
| Alarm system | Precision + recall + detection delay |
For anomaly detection, sample-level metrics may not tell the complete story.
It can also be useful to evaluate:
A reliable evaluation requires a clear separation between model development and final performance reporting.
A common workflow is:
Training set
↓
Learn model parameters
Validation set
↓
Model selection / hyperparameter tuning / threshold selection
Test set
↓
Final unbiased performance estimate
The test set should remain unseen during model development.
The exact terminology can vary between workflows, but the key principle is:
The dataset used for final performance reporting should not be used to repeatedly make modeling decisions.
Random train-test splitting is often inappropriate for sensor and time-series data.
Adjacent measurements can be highly correlated.
For example:
t1 t2 t3 t4 t5 t6 t7 t8 ...
A random split may place highly similar neighboring windows into both training and testing datasets.
The resulting score can be overly optimistic.
For forecasting:
Past data → Training
Later data → Validation
Future data → Test
For machine generalization:
Machines 1–6 → Training
Machine 7 → Validation
Machine 8 → Test
The split should reflect how the model will actually be used after deployment.
Data leakage occurs when information that should be unavailable during prediction enters the model-development process.
Common examples include:
Incorrect:
scaler.fit(X_all)
X_train = scaler.transform(X_train)
X_test = scaler.transform(X_test)
The scaler has already seen the test distribution.
Better:
scaler.fit(X_train)
X_train = scaler.transform(X_train)
X_test = scaler.transform(X_test)
A useful rule is:
Fit preprocessing, feature-selection, and model-development steps using training data only, then apply the learned transformation to validation/test data.
Cross-validation evaluates a model over multiple training/validation splits.
For example, k-fold cross-validation:
Fold 1 → Train / Validation
Fold 2 → Train / Validation
Fold 3 → Train / Validation
...
Fold k → Train / Validation
The validation metrics can then be summarized across folds.
Standard random k-fold cross-validation can still cause leakage in time-series data.
Time-aware validation methods should be used when temporal ordering matters.
For regression:
Residual = Actual - Predicted
Residual analysis can reveal problems that summary metrics hide.
Residuals should ideally be:
Therefore, numerical metrics should be complemented by visual and residual analysis whenever possible.
A high-bias model is usually too simple.
Typical symptoms:
A high-variance model fits the training data too closely.
Typical symptoms:
The objective is not simply to maximize training performance, but to obtain a model that generalizes reliably.
Model evaluation should also consider why a model makes its predictions.
Explainability is especially useful in engineering applications where domain experts need to understand model behavior.
Two useful perspectives are:
Why did the model make this particular prediction?
Which features generally influence the model’s predictions?
Common methods include:
Feature attribution does not automatically imply causality.
A feature can be strongly associated with a prediction without being the physical cause of the phenomenon.
Different models provide different approaches to feature importance.
Many scikit-learn tree models provide:
model.feature_importances_
This is generally based on impurity reduction.
Permutation importance is model-agnostic.
from sklearn.inspection import permutation_importance
result = permutation_importance(
model,
X_test,
y_test,
n_repeats=10,
random_state=42
)
importance = result.importances_mean
The idea is to shuffle one feature and measure how much model performance changes.
SHAP (SHapley Additive exPlanations) provides feature-attribution values for individual predictions and can also be aggregated for global analysis.
Conceptually:
Model prediction
↓
Contribution from feature 1
Contribution from feature 2
Contribution from feature 3
...
↓
Final prediction
For engineering models, feature-importance analysis can help identify:
Correlation is useful during exploratory data analysis and feature analysis.
Pearson correlation measures linear association:
r = Cov(X, Y) / (σX × σY)
It ranges from:
-1 → strong negative linear association
0 → no linear association
+1 → strong positive linear association
In pandas:
df.corr(numeric_only=True)
For a target:
df.corr(numeric_only=True)["target"].sort_values()
Spearman correlation uses ranks and is useful for monotonic relationships:
df.corr(method="spearman")
A feature can have low Pearson correlation with the target and still be highly predictive.
For example, a nonlinear relationship such as:
y = x²
can have low Pearson correlation around a symmetric range even though x strongly determines y.
Therefore:
Correlation is useful for understanding relationships and redundancy, but it should not be the only criterion for feature selection.
Correlation also does not imply causation.
Many machine-learning models, particularly neural networks, are trained by minimizing a loss function.
The general update rule is:
θ_new = θ_old - η ∇J(θ)
where:
θ = model parametersη = learning rateJ(θ) = loss function∇J(θ) = gradient of the loss with respect to the parametersThe gradient points toward the direction of greatest increase in the loss, so the optimizer moves in the opposite direction.
If the learning rate is:
| Method | Data per update |
|---|---|
| Batch gradient descent | Entire dataset |
| Stochastic gradient descent | One sample |
| Mini-batch gradient descent | Small batch |
Mini-batch optimization is commonly used for neural-network training.
These are related but different:
Backpropagation
↓
Calculate gradients using the chain rule
↓
Optimizer
↓
Update model parameters
Backpropagation calculates the gradients; an optimizer such as SGD or Adam uses those gradients to update the weights.
Decision trees and random forests do not normally train their splits using gradient descent. They use criteria such as Gini impurity, entropy/information gain, or variance reduction.
For an embedded AI system, predictive performance is only one part of evaluation.
A model may have excellent accuracy but still be unsuitable for deployment if it is too large or too slow.
Important deployment metrics include:
A practical evaluation therefore looks like:
Predictive performance
+
Robustness
+
Interpretability
+
Deployment constraints
↓
Deployment suitability
This is especially important for smart sensors and resource-constrained edge devices.
Neural networks can use different numerical representations.
Common examples include:
The distinction between training precision and inference precision is important.
A model can be trained using floating-point arithmetic and later deployed using lower-precision integer arithmetic.
INT8 is particularly common for edge inference because it can reduce:
The trade-off is possible degradation in predictive accuracy due to numerical approximation.
Quantization maps high-precision values to lower-precision representations.
Conceptually:
Floating-point value
↓
Quantize
↓
Lower-precision value
A simplified affine quantization relationship can be represented as:
q ≈ round(x / scale) + zero_point
x ≈ scale × (q - zero_point)
where:
x = floating-point valueq = quantized integer valuescale = scaling factorzero_point = offset used for the integer representationQuantization is not a model architecture. It is a numerical representation/deployment technique.
With post-training quantization, the model is trained normally in floating point and quantized after training.
The basic workflow is:
Training
↓
FP32 model
↓
Post-training quantization
↓
INT8 / lower-precision model
↓
Embedded inference
There is no quantization-aware retraining in the basic PTQ workflow.
A simple TensorFlow/Keras example is:
import tensorflow as tf
import numpy as np
# Training data
X = np.random.rand(1000, 4).astype(np.float32)
y = (X.sum(axis=1) > 2).astype(np.float32)
X_train, X_test = X[:800], X[800:]
y_train, y_test = y[:800], y[800:]
# Normal FP32 model
model = tf.keras.Sequential([
tf.keras.layers.Input(shape=(4,)),
tf.keras.layers.Dense(16, activation="relu"),
tf.keras.layers.Dense(8, activation="relu"),
tf.keras.layers.Dense(1, activation="sigmoid")
])
model.compile(
optimizer="adam",
loss="binary_crossentropy",
metrics=["accuracy"]
)
model.fit(
X_train,
y_train,
epochs=10,
batch_size=32,
verbose=0
)
# Post-training quantization
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [
tf.lite.Optimize.DEFAULT
]
tflite_model = converter.convert()
with open("model_quantized.tflite", "wb") as f:
f.write(tflite_model)
The important point is that the model is already trained before quantization.
For full integer quantization, representative input data can be supplied to calibrate activation ranges.
For example:
def representative_dataset():
for sample in X_train[:100]:
yield [sample.reshape(1, 4).astype(np.float32)]
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [
tf.lite.Optimize.DEFAULT
]
converter.representative_dataset = representative_dataset
converter.target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS_INT8
]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8
tflite_model = converter.convert()
The representative dataset should resemble the data that the deployed model will receive.
For sensor applications, this means using representative sensor measurements covering the expected operating range.
Quantization-aware training takes a different approach.
Instead of waiting until training is complete:
FP32 training
↓
quantization
QAT simulates quantization effects during training:
Training
↓
Fake quantization
↓
Model adapts to quantization effects
↓
QAT model
↓
INT8 deployment
The model generally continues to use floating-point arithmetic during training, while fake-quantization operations simulate the numerical effects of reduced precision.
This allows the model to learn parameters that are more tolerant of quantization error.
A conceptual example is:
FP32 weights
↓
Simulated quantization
↓
Simulated dequantization
↓
Forward pass
↓
Loss
↓
Backpropagation
↓
Weight update
The rounding operation itself is not normally differentiable, so QAT implementations use gradient approximations such as the straight-through estimator.
A simple Keras QAT workflow can be implemented using TensorFlow Model Optimization Toolkit:
import tensorflow as tf
import tensorflow_model_optimization as tfmot
import numpy as np
# Example data
X = np.random.rand(1000, 4).astype(np.float32)
y = (X.sum(axis=1) > 2).astype(np.float32)
X_train, X_test = X[:800], X[800:]
y_train, y_test = y[:800], y[800:]
# Train normal FP32 model
model = tf.keras.Sequential([
tf.keras.layers.Input(shape=(4,)),
tf.keras.layers.Dense(16, activation="relu"),
tf.keras.layers.Dense(8, activation="relu"),
tf.keras.layers.Dense(1, activation="sigmoid")
])
model.compile(
optimizer="adam",
loss="binary_crossentropy",
metrics=["accuracy"]
)
model.fit(
X_train,
y_train,
epochs=10,
batch_size=32,
verbose=0
)
# Apply quantization-aware training
quantize_model = tfmot.quantization.keras.quantize_model
qat_model = quantize_model(model)
qat_model.compile(
optimizer="adam",
loss="binary_crossentropy",
metrics=["accuracy"]
)
# Fine-tune with simulated quantization
qat_model.fit(
X_train,
y_train,
epochs=5,
batch_size=32,
verbose=0
)
# Convert to TFLite
converter = tf.lite.TFLiteConverter.from_keras_model(qat_model)
converter.optimizations = [
tf.lite.Optimize.DEFAULT
]
tflite_model = converter.convert()
with open("model_int8.tflite", "wb") as f:
f.write(tflite_model)
| Property | Post-Training Quantization | Quantization-Aware Training |
|---|---|---|
| Initial FP32 training | Yes | Yes |
| Quantization simulated during training | No | Yes |
| Additional training/fine-tuning | No | Yes |
| Implementation complexity | Lower | Higher |
| Deployment efficiency | High | High |
| Potential accuracy degradation | Can be higher | Often lower |
| Use when | PTQ accuracy is acceptable | PTQ causes unacceptable accuracy loss |
A practical workflow is:
Train FP32 model
↓
Evaluate FP32 model
↓
Try PTQ
↓
Evaluate quantized model
↓
Accuracy acceptable?
↙ ↘
YES NO
↓ ↓
Deploy Try QAT
↓
Evaluate again
↓
Deploy
The key engineering principle is:
Start with the simplest deployment method that satisfies the accuracy and hardware requirements.
Quantization should not be evaluated only by checking whether the model converted successfully.
Compare the original and quantized models using:
A useful comparison is:
FP32 INT8
Accuracy 96.2% 95.8%
Model size 4.0 MB 1.1 MB
Latency 12 ms 5 ms
RAM ... ...
Energy/inference ... ...
The goal is not simply to obtain the smallest model.
The goal is to find an acceptable trade-off between:
Accuracy
↕
Latency
↕
Memory
↕
Energy
A practical engineering workflow can be summarized as:
1. Define the application
↓
2. Define the cost of errors
↓
3. Choose appropriate predictive metrics
↓
4. Design leakage-safe train/validation/test splits
↓
5. Train candidate models
↓
6. Analyze errors and residuals
↓
7. Check robustness and generalization
↓
8. Check explainability and physical consistency
↓
9. Evaluate deployment constraints
↓
10. Quantize if required
↓
11. Re-evaluate predictive + deployment performance
↓
12. Validate on representative unseen data
This prevents model selection from becoming a simple exercise of choosing the highest score.
| Mistake | Consequence |
|---|---|
| Using only accuracy | Misleading results on imbalanced data |
| Using only R² | Hides physical error magnitude |
| Ignoring MAE/RMSE differences | Large errors may remain hidden |
| Using MAPE near zero | Extremely unstable percentages |
| Ignoring train-test comparison | Hidden overfitting |
| Randomly splitting time-series data | Leakage and optimistic scores |
| Fitting preprocessing on all data | Data leakage |
| Selecting features using test data | Biased final evaluation |
| Ignoring residual plots | Missed systematic errors |
| Ignoring class imbalance | Poor rare-event detection |
| Using ROC-AUC alone for rare events | Positive-class performance may be obscured |
| Assuming correlation means importance | Nonlinear relationships may be missed |
| Treating feature attribution as causality | Incorrect physical interpretation |
| Ignoring deployment constraints | Accurate but unusable model |
| Quantizing without re-evaluation | Unknown accuracy degradation |
| Assuming PTQ is always sufficient | Some models may require QAT |
In engineering applications, the model with the highest single metric is not automatically the most appropriate model.
A practical model should be:
For an embedded sensor system, the evaluation therefore extends beyond:
"How accurate is the model?"
to:
"Can the model make sufficiently accurate decisions,
reliably, quickly, efficiently, and safely
under the conditions in which it will actually operate?"
Model evaluation is not simply about obtaining a high score.
A strong evaluation process should reveal:
For edge and industrial AI, model development and model deployment should therefore be treated as one engineering workflow:
Data
↓
Model
↓
Evaluation
↓
Interpretability
↓
Robustness
↓
Optimization / Quantization
↓
Hardware Evaluation
↓
Deployment
A strong engineer does not only train models.
A strong engineer understands how, where, and why the model fails, and verifies that the final system remains reliable under realistic operating conditions.