[GENAI-001] · Databricks · Live

BSA/AML Regulatory RAG

Area · GenAI Stack · LangChain · Gemini 2.0 · ChromaDB · BM25 Dataset · FFIEC BSA/AML Examination Manual Live

Overview

Architecture

GitHub

The Problem

"Our compliance team spends hours searching the FFIEC manual to answer internal policy questions. We need a faster and more reliable way to query regulatory documentation."

Every financial institution operating in the United States must comply with the Bank Secrecy Act and Anti Money Laundering regulations. The documentation is dense: FinCEN guidelines, FFIEC manuals, SAR filing requirements, and compliance teams spend significant time navigating it manually.

This project builds a retrieval augmented generation system that answers regulatory questions in plain English, with every response citing the exact source document, section, and page number. The system uses a hybrid retrieval approach and is configured to run deterministically at temperature 0.0.

What It Delivers

Query speed

Regulatory questions that take 20 to 40 minutes to answer manually are answered in under 10 seconds.

Source citation

Every answer references the exact document, section, and page. Compliance officers can verify the source without relying on the model alone.

Pluggable stack

The LLM, embedding model, and vector store are all configurable via a single .env file. No code changes required to swap any component.

Deterministic output

Temperature is fixed at 0.0. The same question asked twice returns the same answer, which is a requirement in any compliance context.

Pipeline Overview

Fig. 1 · End to end pipeline. All layers are configurable via config.py and .env.

Key Decisions

Temperature 0.0

Regulatory answers must be deterministic. Any temperature above zero introduces variability, and the same question can return different answers, which is not acceptable in a compliance context.

Hybrid retrieval

Regulatory documents use precise legal terms such as "structuring" and "SAR". Pure semantic search does not reliably match these. BM25 keyword search covers what embeddings miss. The union of both returns better results than either alone.

Chunk size 500

Regulatory text is dense. Larger chunks mix multiple obligations into a single unit, reducing retrieval precision. 500 characters keeps each chunk semantically focused without losing context.

Mandatory citation

The generation prompt instructs the model to cite the source page for every claim. This is enforced at the prompt level, not trusted to model behavior alone.

Repository

genai-001-regulatory-rag-bsa-aml github.com →

bsa-aml-rag/
├── README.md
├── requirements.txt
├── .env.example
├── config.py
├── src/
│ ├── __init__.py
│ ├── ingestion.py
│ ├── processing.py
│ ├── embedding.py
│ ├── retrieval.py
│ └── generation.py
├── notebooks/
│ └── quickstart.ipynb
└── data/
└── BsaAmlManualSectionsPackage.pdf

Stack: LangChain, Gemini 2.0 Flash, Gemini Embedding 001, ChromaDB, BM25 (rank_bm25). Python 3.11 or higher. Requires a Google AI Studio API key, free at aistudio.google.com.

[RISK-001] · Databricks · Live

Customer Churn Scoring

Area · Risk Stack · XGBoost · Optuna · SHAP Dataset · Bank Customer Churn (Kaggle) Live

Overview

Architecture

GitHub

The Problem

Banks lose customers every day without knowing who is at risk or why.

This is a production-grade pipeline for predicting customer churn in banking, with automated feature engineering, Bayesian hyperparameter optimization, and SHAP-based explainability. It scores every customer by churn probability, ranks them by risk, and explains the exact factors driving each prediction so retention teams know who to prioritize and what to address.

Built to work with any tabular churn dataset. Swap the CSV, adjust the target column, and the pipeline handles the rest from raw data to a scored, explainable model.

What It Delivers

Automatic feature engineering

PolynomialFeatures generates candidate interactions automatically, then RandomForest importance decides which ones matter — reusable across different churn datasets without code changes.

Bayesian optimization

Optuna finds strong hyperparameters in far fewer trials than exhaustive grid search. The search space lives in config/xgb_params.py, so tuning depth is a configuration change, not a code change.

Business-first metrics

KS statistic and LIFT by decile instead of raw accuracy, which is misleading on imbalanced churn data. LIFT tells the business exactly how much better than random the model performs at each decile.

Saved artifacts

Label encoders, scaler, polynomial transformer, and selected feature list are all saved during training, allowing new customers to be scored with the exact same transformations, without retraining.

Metrics Explained

AUC-ROC

Overall discrimination power. 0.87+ is strong for churn.

Gini

Derived from AUC. 0.70+ is considered very good in banking.

KS

Maximum separation between churner and non-churner distributions. 0.50+ is a strong model.

LIFT

How many times more churners are found in a decile compared to random selection. A decile 10 LIFT of 4x means targeting the top 10% of scores finds 4 times more churners than reaching out randomly.

Pipeline Overview

Fig. 1 · Raw data to scored, explainable model. Every step saves its output to data/, models/, or outputs/, so the pipeline can be run in full or resumed from any step.

Key Decisions

01 · EDA

Summary report and initial cleaning, producing churn_clean.parquet.

02 · Target Analysis

Distribution and class balance check before any modeling decision is made.

03 · Preprocessing

Label encoding and standard scaling applied consistently across train and inference.

04 · Feature Engineering

PolynomialFeatures generates candidate interactions and powers automatically.

05 · Feature Selection

RandomForest importance selects the top N features from the generated candidates.

06 · Training

XGBoost combined with Optuna Bayesian optimization for hyperparameter tuning.

07 · Evaluation

AUC-ROC, Gini, KS statistic, and LIFT curve by decile.

08 · Explainability

SHAP values for global importance and per-customer drivers.

Repository

risk-001-customer-churn-scoring github.com →

churn-scoring-pipeline/
├── README.md
├── requirements.txt
├── .gitignore
├── config/
│ └── xgb_params.py
├── src/
│ ├── __init__.py
│ ├── eda.py
│ ├── target_analysis.py
│ ├── preprocessing.py
│ ├── feature_engineering.py
│ ├── feature_selection.py
│ ├── training.py
│ ├── evaluation.py
│ └── explainability.py
├── data/
│ ├── bank_customer_churn.csv
│ ├── churn_clean.parquet
│ ├── churn_preprocessed.parquet
│ ├── churn_features.parquet
│ └── churn_selected.parquet
├── models/
│ ├── xgb_churn.pkl
│ ├── label_encoders.pkl
│ ├── scaler.pkl
│ ├── polynomial.pkl
│ └── selected_features.pkl
├── outputs/
│ ├── ks_curve.png
│ ├── lift_curve.png
│ ├── shap_summary.png
│ ├── shap_bar.png
│ └── shap_waterfall.png
└── notebooks/
├── quickstart.ipynb
└── test_predict.ipynb

Stack: XGBoost, Optuna, SHAP, scikit-learn. Python 3.13. Works with any tabular churn dataset with a binary target column and minimal configuration changes.

[CAUSAL-001] · Databricks · Live

Credit Limit Intervention Analysis

Area · Causal Inference Stack · DoWhy · EconML · Causal Forest Dataset · UCI Credit Card Default (Taiwan) Live

Overview

Architecture

GitHub

The Hypothesis

Risk teams commonly assume that reducing the credit limit of customers with a history of late payments lowers their risk of future default. Less exposure, less risk — the reasoning seems straightforward.

This project tests that assumption using causal inference, not just correlation. Built with DoWhy and EconML to separate correlation from causation — and to challenge a common assumption in credit risk management.

What the Data Shows

Controlling for 22 confounders — payment history, bill amounts, demographics — the analysis finds the opposite of the common assumption. A one standard deviation increase in credit limit (approximately NT$130,000) decreases the probability of default by 1.19 percentage points.

This is consistent with published research on the same dataset, which found a similar effect using a different estimation method. The likely mechanism is that a higher credit limit reduces utilization ratio, which reduces financial pressure and, in turn, default risk.

The effect is not uniform across customers. Segmenting by individual treatment effect reveals that customers with lower existing limits and higher baseline risk benefit the most from a limit increase, while a small subgroup shows the opposite pattern — for them, increasing the limit is associated with higher default risk.

Key Terms

ATE

Average Treatment Effect — the average causal impact of the treatment across the full population.

CATE

Conditional Average Treatment Effect — the causal impact estimated for a specific customer or segment, which can differ from the average.

Backdoor criterion

The condition under which controlling for a set of confounders is sufficient to identify a causal effect from observational data.

Refutation test

A robustness check that perturbs the model in a known way to confirm the estimated effect behaves as a real causal effect should.

Pipeline Overview

Fig. 1 · Naive correlation → causal graph → identification → effect estimation → refutation → heterogeneous effects.

Key Decisions

Causal question is declared, not discovered

Unlike the churn and RAG projects in this hub, this pipeline cannot fully automate itself from a new dataset. Treatment, outcome, and confounders require domain knowledge, made explicit in causal_config.py. Automating causal assumptions would risk producing confident but wrong conclusions.

Naive correlation reported first

The EDA step deliberately calculates the raw correlation between treatment and outcome before any adjustment, making the value of causal inference visible.

Effect reported at business scale

A raw ATE per one-unit change in a currency variable is not interpretable when that variable ranges in the hundreds of thousands. The pipeline reports the effect per meaningful increment and per standard deviation, matching how a finding would be communicated to a risk committee.

Refutation is not optional

Every causal estimate goes through three robustness checks: adding a random confounder, replacing the treatment with a placebo, and re-estimating on a data subset. A causal claim without refutation testing is a hypothesis that has not been stressed.

Heterogeneity as the main output

An average effect can hide the fact that a subgroup responds in the opposite direction. The Causal Forest step exists specifically to surface that nuance, enabling a segmented recommendation instead of a blanket policy change.

Repository

causal-001-credit-limit-intervention github.com →

causal-001-credit-limit-intervention/
├── README.md
├── requirements.txt
├── .gitignore
├── config/
│ └── causal_config.py
├── src/
│ ├── __init__.py
│ ├── causal_eda.py
│ ├── causal_graph.py
│ ├── identification.py
│ ├── effect_estimation.py
│ ├── refutation.py
│ └── heterogeneous_effects.py
├── data/
│ ├── UCI_Credit_Card.csv
│ └── causal_clean.parquet
├── outputs/
│ ├── causal_graph.png
│ ├── cate_distribution.png
│ └── segment_effects.png
└── notebooks/
└── quickstart.ipynb

Stack: DoWhy, EconML, pandas. Python 3.13. Uses the Default of Credit Card Clients dataset (Taiwan), publicly available on Kaggle / UCI Machine Learning Repository.

[FRAUD-001] · Benchmark · Live

Imbalanced Classification Benchmark

Area · Payments Stack · XGBoost · imbalanced-learn · SHAP Dataset · Credit Card Fraud Detection (ULB) Live

Overview

Architecture

GitHub

The Problem

Fraud detection, rare disease diagnosis, equipment failure prediction. Every one of these problems shares the same core challenge: the event that matters is rare, often under 1% of the data, and standard evaluation quietly fails without anyone noticing.

This project does not ship a single fraud model. It ships a benchmark: nine techniques for handling extreme class imbalance, tested on the same data, the same split, and the same metrics, using a real credit card fraud dataset where fraud represents 0.173% of transactions, roughly 1 in every 578.

What the Benchmark Found

The most expensive technique tested, SMOTE combined with Edited Nearest Neighbors, took about 15 minutes to run and generated hundreds of thousands of synthetic rows. It finished sixth out of nine techniques.

The winning technique, a moderated class weight adjustment, changes nothing about the training data and adds a single parameter to the model. It trains in the same time as a model with no imbalance treatment at all, and still beat every resampling method tested.

Key Terms

PR-AUC

Area under the Precision-Recall curve. The primary metric for this benchmark, since it stays meaningful under severe imbalance, unlike Accuracy or ROC-AUC.

Class weight

An algorithm level adjustment that penalizes errors on the minority class more heavily, without touching the training data.

SMOTE

Synthetic Minority Oversampling Technique — generates synthetic examples of the rare class by interpolating between existing ones.

Threshold tuning

Adjusting the probability cutoff used to convert a model's output into a decision, instead of changing the data or the algorithm.

Pipeline Overview

Imbalanced Classification Benchmark Architecture

Fig. 1 · EDA → baseline → resampling, algorithm-level, anomaly detection and threshold tuning tested in parallel → benchmark → explainability on the winning technique.

Key Decisions

PR-AUC over Accuracy and ROC-AUC

A model predicting the majority class 100% of the time scores above 99.8% accuracy on this dataset. ROC-AUC also looks deceptively strong under extreme imbalance. PR-AUC is far more sensitive to how the model performs on the rare class, which is the one that matters.

Test set is never resampled

Every resampling technique is applied only to the training data. The test set always reflects the real world distribution, so every technique is compared on equal, realistic footing.

Same base model across every technique

XGBoost with fixed hyperparameters is used for every technique tested. This isolates the effect of the imbalance treatment itself, rather than mixing in the effect of different model choices.

Threshold tuning tested as a first resort

It reuses the already trained baseline model and searches for a better decision boundary. It costs nothing extra to compute and is often overlooked in favor of more complex resampling.

Isolation Forest as a contrast, not a candidate

It represents the alternative of ignoring the label entirely. Its weak performance makes a concrete point: when labeled examples of the rare class exist, even a small number of them, using that signal outperforms ignoring it.

Repository

fraud-001-imbalanced-classification-benchmark github.com →

fraud-001-imbalanced-classification-benchmark/
├── README.md
├── requirements.txt
├── .gitignore
├── config/
│ └── imbalance_config.py
├── src/
│ ├── __init__.py
│ ├── eda.py
│ ├── target_analysis.py
│ ├── baseline.py
│ ├── resampling_techniques.py
│ ├── algorithm_techniques.py
│ ├── anomaly_detection.py
│ ├── threshold_tuning.py
│ ├── benchmark.py
│ └── explainability.py
├── data/
│ └── creditcard.csv
├── models/
│ └── baseline_model.pkl
├── outputs/
│ ├── benchmark_final.parquet
│ ├── shap_summary.png
│ └── confusion_matrix.png
└── notebooks/
└── quickstart.ipynb

Stack: XGBoost, imbalanced-learn, scikit-learn, SHAP. Python 3.13. Uses the Credit Card Fraud Detection dataset (ULB), publicly available on Kaggle. The dataset exceeds GitHub's file size limit and is downloaded separately, with instructions in the README.

[EXPD-001] · Pipeline · Live

A/B Test Design Pipeline

Area · Experimentation Stack · statsmodels · SciPy · scikit-learn Dataset · Bank Marketing (UCI) Live

Overview

Architecture

GitHub

The Problem

A business team wants to test something: a new offer, a different rate, a changed flow. They pull a customer list and split it in half. That split is usually where the test breaks.

Splitting randomly across the whole base can leave one group with more customers from a specific profession, region, or education level than the other. When the results come in, nobody can tell whether the difference came from the test or from the composition of the groups.

This pipeline handles the part that comes before the test runs. It takes a raw customer base and returns two balanced groups, ready to use, with the sample size backed by power analysis, stratified randomization keeping both groups comparable, and a statistical check proving the split was fair.

What It Delivers

Two group files

Control and treatment exported as CSV, ready to hand off to the business team without further processing.

Sample size

Power analysis determines the minimum number of customers per group needed to detect the declared effect, and reports whether the available population is enough.

Balance proof

Every stratification column goes through a t-test or chi-square test after randomization, with a pass or fail result per variable.

Plain text report

A summary written for the business team, with the numbers that matter and no statistical jargon left unexplained.

Key Terms

Stratification

Splitting proportionally within each segment, so both groups keep the same composition as the original population.

Statistical power

The probability of detecting a real effect when one exists. 80% is the standard convention.

Minimum detectable effect

The smallest change the test is designed to catch. Smaller effects require larger samples.

Pipeline Overview

Fig. 1 · Population profiling and stratification check, then sample size, randomization and balance validation, ending in the exported groups and the experiment registry.

Key Decisions

Stratified over simple random

A simple split can leave one group over-represented in a segment purely by chance. Stratifying by the declared columns guarantees proportional composition in both groups from the start, rather than hoping randomness works out.

Sample size calculated, not assumed

Power analysis based on the declared confidence level, statistical power, and minimum detectable effect. The pipeline reports whether the available population is enough, and by how much it falls short when it is not.

Balance validated statistically

Every stratification column goes through a t-test or chi-square test after randomization. The PCA scatter plot is a complementary visual confirmation, not the validation itself.

Numerical columns must be grouped

Declaring a numerical column like age creates one segment per unique value, fragmenting the population into hundreds of tiny groups. The pipeline detects this and warns before the split happens.

Every experiment recorded

Each run is saved to a local SQLite database with its parameters and results, so past experiments can be reviewed without rerunning anything.

Repository

expd-001-ab-test-design-pipeline github.com →

expd-001-ab-test-design-pipeline/
├── README.md
├── requirements.txt
├── .gitignore
├── config/
│ ├── __init__.py
│ └── ab_test_config.py
├── src/
│ ├── __init__.py
│ ├── eda.py
│ ├── stratification_check.py
│ ├── sample_size.py
│ ├── randomization.py
│ ├── balance_validation.py
│ ├── export.py
│ └── registry.py
├── data/
│ └── bank_marketing.csv
├── outputs/
│ ├── group_a.csv
│ ├── group_b.csv
│ ├── experiment_summary.txt
│ └── group_overlap.png
├── db/
│ └── experiments.db
└── notebooks/
└── quickstart.ipynb

Stack: statsmodels, SciPy, scikit-learn, pandas, matplotlib. Python 3.13. Uses the Bank Marketing dataset from a Portuguese banking institution, publicly available on Kaggle and originally from the UCI Machine Learning Repository.