Card fraud datasets have a specific problem that most tabular ML write-ups skip over. The data moves fast, the patterns change constantly, and the thing you want to predict (is this transaction fraudulent?) is not a clean binary when you need to act on it in practice.
I have been running a proof of concept to compare how classification and regression models perform on synthetically generated card transaction data. The dataset was 200,000 rows, built locally because the sensitive nature of real transaction data ruled out any cloud-based analysis. Compute was limited to what a workstation could handle.
The finding that stuck: logistic regression, framed as a probability scorer rather than a classifier, produced more operationally useful output than Random Forest or Gradient Boosted Trees for quick-turnaround fraud analysis on high-velocity data.
The data problem: four dimensions of difficulty
Card fraud data hits all four Vs hard.
Volume. Even a mid-tier card issuer processes millions of transactions per month. The 200K-row synthetic dataset was a scaled-down representation, but the statistical properties (class imbalance, feature distributions, temporal patterns) were modelled to reflect production-scale behaviour.
Velocity. Transactions arrive in real time. A model that takes minutes to score a batch is useless when the fraudulent card is being used at three different merchants in the same hour. Any viable approach needs to score individual transactions fast.
Variety. Transaction amount, merchant category, geographic location, time of day, card-present vs. card-not-present, transaction frequency. The feature space is wide and the interactions between features matter. Feature engineering on this kind of data is as much about derived signals (velocity of spend, merchant category deviation, geographic distance between consecutive transactions) as it is about the raw columns. A $5,000 transaction at 2am from a new merchant in a different country tells a different story than the same amount at a regular retailer during business hours.
Veracity. This is the one that makes fraud detection an evolving problem. Fraud patterns shift. What worked as a detection signal six months ago may be irrelevant now because fraudsters adapt. Ground truth is also messy: not every flagged transaction is investigated, and not every uninvestigated transaction is clean.
Tooling: SAS for extraction, R for analysis, Python for validation
The workflow reflects what is common in analytics teams right now.
SAS Enterprise Guide handled data extraction and initial profiling. It is the standard tool for accessing structured data sources in most enterprise environments, and it does that job well. I did not use SAS for modelling.
R with caret was the primary analysis environment. The caret package provided a consistent interface for training and evaluating models across different algorithms. randomForest for Random Forest, e1071 for SVM (explored but dropped early for performance reasons on this dataset size), glm for logistic regression, and gbm for gradient boosted trees. pROC and ROCR handled ROC curves and threshold analysis.
Python with scikit-learn served as a validation environment. The goal was to confirm that results held across implementations, not to pick a winner between languages. scikit-learn offers RandomForestClassifier, GradientBoostingClassifier, and LogisticRegression with broadly comparable interfaces. pandas and numpy for data handling. matplotlib for plots.
One practical observation: R’s caret and Python’s scikit-learn made it straightforward to run the same conceptual model in both environments with minimal code changes. The consistency of results between the two gave confidence that findings were model-driven, not implementation-driven.
Why it looks like classification but acts like regression
The obvious framing: transaction is fraudulent (1) or legitimate (0). Binary target. Classification problem. Pick a classifier, optimise for recall (because missing fraud is worse than false alarms), done.
That framing works when you have time to investigate every flagged transaction. It falls apart when the dataset is high-velocity and the operational need is triage, not verdict.
What I found more useful was treating the output as a probability score. Instead of asking ‘is this fraud?’, ask ‘how likely is this fraud?’ and rank transactions by that likelihood. A logistic regression model produces a calibrated probability between 0 and 1 for each transaction. That score becomes a prioritisation tool: the top 100 transactions by score get investigated first.
This is not a novel insight in the fraud detection literature. But it was a useful one to validate empirically, because the classification framing is the default in every discussion I have about this data.
Model comparison: the numbers
Three models, same 200K-row synthetic dataset, same 70/30 train-test split, same feature set. The test set contained approximately 60,000 transactions with a 2% fraud rate (roughly 1,200 fraudulent transactions).
Confusion matrix and performance metrics (test set):
| Metric | Random Forest | Gradient Boosted Trees | Logistic Regression |
|---|---|---|---|
| True Positives | 1,024 | 1,051 | 941 |
| True Negatives | 58,217 | 58,328 | 58,094 |
| False Positives | 583 | 472 | 706 |
| False Negatives | 176 | 149 | 259 |
| Accuracy | 98.74% | 98.97% | 98.39% |
| Precision | 63.7% | 69.0% | 57.1% |
| Recall | 85.3% | 87.6% | 78.4% |
| F1 Score | 72.9% | 77.2% | 66.1% |
| AUC-ROC | 0.943 | 0.961 | 0.912 |
| Error Rate | 1.26% | 1.03% | 1.61% |
The pattern: Random Forest and Gradient Boosted Trees produced higher accuracy and F1 scores. They were better classifiers by standard metrics. Logistic regression scored lower on hard classification metrics but produced better-calibrated probability outputs.
The practical difference showed up when I ranked test-set transactions by predicted fraud probability. Logistic regression’s top-scored transactions had a higher density of actual fraud cases in the top 1% and top 5% than the tree-based models. The tree models tended to cluster predictions around 0 and 1, making the probability scores less useful for ranking within the grey zone.
Scoring speed matters for high-velocity data
On the 200K-row dataset, logistic regression scored the full test set in a fraction of the time that Random Forest or GBT required. The model is a matrix multiplication. No tree traversal, no ensemble aggregation.
This mattered for the POC because the premise was high-velocity transaction scoring. In a production fraud system, the model has roughly two seconds before the card scheme decides for you. A model that produces better probability estimates and scores faster is the better operational fit, even if its classification accuracy is a few points lower.
Challenges and honest limitations
Class imbalance. Fraud is rare. In the synthetic dataset, fraudulent transactions represented roughly 2% of the total. This means a model that predicts ‘not fraud’ for every transaction achieves 98% accuracy. Standard accuracy is a misleading metric. Precision, recall, F1, and AUC-ROC tell the real story.
Synthetic data. The 200K-row dataset was generated locally to approximate the statistical properties of real card transaction data. Results on synthetic data are directional, not definitive. Real-world fraud data has noise, labelling inconsistencies, and temporal drift that synthetic data can only partially replicate.
Local compute constraints. The dataset was capped at 200,000 rows because all analysis ran on a local workstation. Sensitive data could not leave the local environment for cloud-based processing. Larger datasets would likely shift the relative performance of these models, particularly for the tree-based approaches which tend to improve with more data.
No hyperparameter exhaustion. This was a POC, not a Kaggle competition. Models were tuned using caret’s default grid search and cross-validation, not exhaustive hyperparameter optimisation. The goal was to compare model families, not to squeeze the last 0.1% of AUC out of each one.
Evolving patterns not tested. The synthetic dataset was static. Real fraud detection requires models that handle concept drift, where the relationship between features and the target changes over time. This POC did not test model retraining or drift detection.
What this proved
For quick fraud scoring on high-velocity card data, a logistic regression model that outputs calibrated probabilities per transaction was more useful than tree-based classifiers that optimise for hard accuracy. The probability score gives analysts a ranked queue to work through, and the model scores fast enough to keep up with transaction volume.
The classification framing is not wrong. But when the operational question is ‘which transactions should I look at first?’ rather than ‘is this fraud?’, regression-style probability output answers it more directly.