Business context of CLV

Customer Lifetime Value (CLV) refers to the projected revenue a customer is expected to generate over a long-term period, typically one year or more. Accurately estimating CLV allows businesses to make more informed decisions around customer acquisition, retention strategies, personalized marketing, and long-term planning. The current CLV model, however, performs poorly when predicting the value of newly acquired customers, especially due to challenges in capturing seasonal behavior. This highlights the need for a dedicated model focused specifically on new customers.

Model development

Data

The Customer Lifetime Value (CLV) model is an enterprise-wide solution used across Expedia Group, leveraging data from multiple brands including Expedia, Hotels.com, Vrbo, Orbitz, Travelocity, Ebookers, Wotif, and CheapTickets, as well as various lines of business such as Stays, Flights, Packages, Cars, and Cruises. The model incorporates over 200 engineered features, grouped into two main categories: bookings and engagement.

Booking features are further divided into two subgroups. The first subgroup captures detailed information about the customer’s most recent booking, such as the country of sale, brand, line of business, booking platform (e.g., desktop, app), booking value, booking type (domestic or international), time since last booking, and the booking window. The second subgroup includes aggregated historical data at the customer level over various timeframes (e.g., last 3 months, last 12 months).

These features include metrics like total booking count, cumulative booking value, brand distribution, time since first booking, average interval between bookings, and average booking window, among others.

CatBoost was selected for the initial implementation of the model due to several key advantages: it efficiently handles categorical variables with many unique values, manages missing numerical data without requiring imputation, offers fast training performance, and captures complex non-linear interactions effectively.

To tailor the model to different customer behaviors, users were first segmented into five geographic regions. Within each region, they were further classified based on booking recency and frequency, resulting in 30 distinct customer segments. Additionally, to minimize the influence of extreme values during training, target values were clipped at the 99.9th percentile within each segment.

Model Evaluation

The evaluation of the CLV model centers on two key questions: how effectively the model distinguishes high-CLV customers from others, and how closely the predicted values align with actual outcomes. To assess differentiation, metrics such as the Lorenz curve and Gini coefficient are used. Prediction accuracy is evaluated through bias, RMSE, and calibration plots. Customers are randomly split into a training set (80%) and a test set (20%), with models trained on the former and evaluated on the latter. Performance metrics are calculated both in aggregate and across various customer segments and levels of detail. The evaluation results referenced are based on predictions from the latest production version of the model, with axis values omitted for confidentiality reasons.

The calibration plot in the above figure indicates that the CLV model produces well-calibrated predictions, as the data points closely follow the 45-degree reference line. To generate this plot, customers were sorted by their predicted CLV values in ascending order and divided into ten equal-sized groups (deciles). For each group, the average predicted CLV (x-axis) was plotted against the average actual CLV (y-axis). To further improve calibration accuracy, Isotonic Regression was applied, ensuring that the predicted values align more closely with observed outcomes across all deciles.

The above plot displays the performance of the new CLV model in capturing seasonal trends. The blue line indicates actual average gross profit (GP), while the red line represents the model’s predictions. As shown, the model effectively tracks the sharp seasonal spike and maintains close alignment with actual values across the distribution, an improvement over the previous model, which struggled to account for such fluctuations. Due to confidentiality agreements, I’m unable to share comparisons with the current production model or disclose specific values.

Challenges & Takeaways

Given the massive scale of the bookings dataset and the complexity of generating customer-level features, prototyping the CLV models required careful design of data pipelines to ensure they were both efficient and robust. Performance bottlenecks and memory constraints had to be addressed early on.

Key optimizations included removing high-cost features that added little value to the model, rewriting feature logic to avoid resource-heavy operations (such as calculating average booking intervals using booking range and count rather than sorting all dates), and tuning Databricks clusters to improve processing speed. These efforts led to a 10x improvement in performance.

Through this process, I gained valuable experience in building scalable pipelines, optimizing for compute efficiency, and thinking critically about trade-offs between complexity and impact in feature engineering.