All posts
2025
- June 4, 2025 - Combining Thompson Sampling with Neural Networks for Classification Tasks
- April 27, 2025 - Online Calculation of the Mean and Variance of Logistic Regression Weights for Thompson Sampling
- March 31, 2025 - Predicting Watch Time like YouTube via Weighted Logistic Regression
- March 19, 2025 - Comparing Online Linear Regression and Kalman Filter Approaches to Online Models Plus Derivations
2024
- November 14, 2024 - Learning RecSys through Papers Vol III- Mixed Negative Sampling + Odds and Ends
- October 28, 2024 - Learning RecSys through Papers Vol II- The How, What, and Why of In-Batch Negatives
- April 22, 2024 - Learning RecSys through Papers- Implementing a Candidate Generation Model
- April 6, 2024 - Calculating Statistical Power When Your Analysis Requires the Delta Method
- February 26, 2024 - CUPED with Multiple Covariates and A Simpler the Delta Method Calculation
- February 14, 2024 - Connections Between the Delta Method, OLS and CUPED, Illustrated
2018
- September 3, 2018 - Extending the Gaussian Mixture Approach for Fantasy Football Tiering
- June 21, 2018 - Using Ordinary Differential Equations To Design State of the Art Residual-Style Layers
- April 18, 2018 - Learning About Deep Reinforcement Learning (Slides)
- March 23, 2018 - Understanding Attention in Neural Networks Mathematically
- January 7, 2018 - Adversarial Dreaming with TensorFlow and Keras
2017
- November 18, 2017 - Hogwild!? Implementing Async SGD in Python
- October 15, 2017 - Covariate Shift, i.e. Why Prediction Quality Can Degrade In Production and How To Fix It
- August 27, 2017 - An Annotated Proof of Generative Adversarial Networks with Implementation Notes
- July 26, 2017 - A Tour of Gotchas When Implementing Deep Q Networks with Keras and OpenAi Gym
- June 18, 2017 - Visualizing the Learning of a Neural Network Geometrically
- March 19, 2017 - Dealing with Trends. Combine a Random Walk with a Tree-Based Model to Predict Time Series Data
- March 5, 2017 - (My Opinion of) Best Practices for a Data Scientist in Industry
- February 26, 2017 - Leveraging Factorization Machines for Wide Sparse Data and Supervised Visualization
2016
- December 27, 2016 - Bayesian Hierarchical Modeling Applied to Fantasy Football Projections for Increased Insight and Confidence
- September 5, 2016 - Making Fantasy Football Projections Via A Monte Carlo Simulation
- June 29, 2016 - Build Your Own Event-Based Backtester in Python
- May 30, 2016 - Parsing HTML Tables in Python with BeautifulSoup and pandas
- May 8, 2016 - Detect Communities in Your Steam Friends List with the Steam Web API and Graph Theory
- May 1, 2016 - Why Blurring an Image is Similar to Warming Your Coffee
- April 24, 2016 - On Solving Partial Differential Equations with Brownian Motion in Python
- April 17, 2016 - Eigen-vesting IV. Predicting Stock and Portfolio Returns With Bayesian Methods
- April 5, 2016 - Train a Neural Network to Play Black Jack with Q Learning
- April 1, 2016 - Making a Markov Chain Twitter Bot in Python
- March 30, 2016 - Eigen-vesting III. Random Matrix Filtering in Finance
- March 28, 2016 - Connect The Dots. Least Squares, Linear Regression, and Bayesian Regression
- March 26, 2016 - Don't Solve-- Simulate! Markov Chain Monte Carlo Methods with PyMC3.
- March 22, 2016 - Eigen-vesting II. Optimize Your Portfolio With Optimization
- March 18, 2016 - Eigen-vesting I. Linear Algebra Can Help You Choose Your Stock Portfolio
- March 14, 2016 - How to Use Math to Win at Fantasy Football With a Lineup Optimizer
- March 13, 2016 - Blogging with iPython using Jekyll
- March 12, 2016 - Jitter, Convolutional Neural Networks, and a Kaggle Framework
Machine Learning
- - Combining Thompson Sampling with Neural Networks for Classification Tasks, An implementation of a training and serving a neural network for classification tasks with Thompson sampling built-in.
- - Online Calculation of the Mean and Variance of Logistic Regression Weights for Thompson Sampling, Deriving the online update for logistic regression using a Kalman filter to yield a mean and variance for Thompson Sampling.
- - Predicting Watch Time like YouTube via Weighted Logistic Regression, An explanation of how YouTube uses weighted logistic regression to predict video watch times.
- - Comparing Online Linear Regression and Kalman Filter Approaches to Online Models Plus Derivations, A derivation and implementation of Online Linear Regression and Kalman filters to estimate the weights of a linear model online.
- - Learning RecSys through Papers Vol III- Mixed Negative Sampling + Odds and Ends, Another modern-ish implementation of the candidate generation step of a recommender system in PyTorch with a an implementation of Mixed Negative Sampling and a comparion to previous methods in this series of posts.
- - Learning RecSys through Papers Vol II- The How, What, and Why of In-Batch Negatives, Another modern-ish implementation of the candidate generation step of a recommender system in PyTorch with a sketch of the proof of the LogQ correction for in-batch negatives.
- - Learning RecSys through Papers- Implementing a Candidate Generation Model, A modern-ish implementation of the candidate generation step of the "Deep Neural Networks For YouTube Recommendations" by Covington et al. with a discussion of next steps from other papers.
- - Using Ordinary Differential Equations To Design State of the Art Residual-Style Layers, A walkthrough of the theory behind Residual layers with comments on current research.
- - Learning About Deep Reinforcement Learning (Slides), My slides from a talk for Data Philly on Deep Reinforcement Learning.
- - Understanding Attention in Neural Networks Mathematically, Explains the Attention Mechanism's equations and demonstrates them geometrically and probabilistically.
- - Adversarial Dreaming with TensorFlow and Keras, Details a loss function to perform adversarial dreaming in Python.
- - Hogwild!? Implementing Async SGD in Python, Explains the Hogwild! algorithm and walks through an implementation using the multiprocessing library.
- - Covariate Shift, i.e. Why Prediction Quality Can Degrade In Production and How To Fix It, Describes the Kullback-Leibler Importance Estimation Procedure and introduces a python package to use it.
- - An Annotated Proof of Generative Adversarial Networks with Implementation Notes, Exploring gotchas associated with coding your own DQN implementation based on Google DeepMind's Nature paper.
- - A Tour of Gotchas When Implementing Deep Q Networks with Keras and OpenAi Gym, Exploring gotchas associated with coding your own DQN implementation based on Google DeepMind's Nature paper.
- - Visualizing the Learning of a Neural Network Geometrically, Walking through how to visualize the training process of a neural network.
- - Dealing with Trends. Combine a Random Walk with a Tree-Based Model to Predict Time Series Data, Using statistics and machine learning for time series data.
- - (My Opinion of) Best Practices for a Data Scientist in Industry, Giving unsolicited advice to upcoming Data Scientists.
- - Leveraging Factorization Machines for Wide Sparse Data and Supervised Visualization, A nonlinear model and visualization approach great for sparse data.
- - Train a Neural Network to Play Black Jack with Q Learning, It turns out the traditional "stay at 15" is almost the best strategy.
- - Jitter, Convolutional Neural Networks, and a Kaggle Framework, A recipe for approaching Kaggle competitions.
Practical
- - Combining Thompson Sampling with Neural Networks for Classification Tasks, An implementation of a training and serving a neural network for classification tasks with Thompson sampling built-in.
- - Online Calculation of the Mean and Variance of Logistic Regression Weights for Thompson Sampling, Deriving the online update for logistic regression using a Kalman filter to yield a mean and variance for Thompson Sampling.
- - Predicting Watch Time like YouTube via Weighted Logistic Regression, An explanation of how YouTube uses weighted logistic regression to predict video watch times.
- - Comparing Online Linear Regression and Kalman Filter Approaches to Online Models Plus Derivations, A derivation and implementation of Online Linear Regression and Kalman filters to estimate the weights of a linear model online.
- - Learning RecSys through Papers Vol III- Mixed Negative Sampling + Odds and Ends, Another modern-ish implementation of the candidate generation step of a recommender system in PyTorch with a an implementation of Mixed Negative Sampling and a comparion to previous methods in this series of posts.
- - Learning RecSys through Papers Vol II- The How, What, and Why of In-Batch Negatives, Another modern-ish implementation of the candidate generation step of a recommender system in PyTorch with a sketch of the proof of the LogQ correction for in-batch negatives.
- - Learning RecSys through Papers- Implementing a Candidate Generation Model, A modern-ish implementation of the candidate generation step of the "Deep Neural Networks For YouTube Recommendations" by Covington et al. with a discussion of next steps from other papers.
- - Calculating Statistical Power When Your Analysis Requires the Delta Method, Demonstration of a statistical power calculation when the variable in question requires the use of the delta method-- i.e., it is a ratio metric.
- - CUPED with Multiple Covariates and A Simpler the Delta Method Calculation, Theoretical derivation of the formula for CUPED with multiple covariates and a calculation trick to make the delta method tractable for this situation.
- - Connections Between the Delta Method, OLS and CUPED, Illustrated, Empirical evidence of the equivalence of OLS and CUPED, plus special considerations for page-level metrics.
- - Extending the Gaussian Mixture Approach for Fantasy Football Tiering, An extention to the Gaussian Mixture approach to tiering Players
- - A Tour of Gotchas When Implementing Deep Q Networks with Keras and OpenAi Gym, Exploring gotchas associated with coding your own DQN implementation based on Google DeepMind's Nature paper.
- - (My Opinion of) Best Practices for a Data Scientist in Industry, Giving unsolicited advice to upcoming Data Scientists.
- - Bayesian Hierarchical Modeling Applied to Fantasy Football Projections for Increased Insight and Confidence, How to make fantasy football projections better with Bayesian techniques.
- - Making Fantasy Football Projections Via A Monte Carlo Simulation, How to make fantasy football projections with historic data and Monte Carlo techniques.
- - Build Your Own Event-Based Backtester in Python, Use multiprocessing to speed up your backtesting!
- - Parsing HTML Tables in Python with BeautifulSoup and pandas, How to use BeautifulSoup and pandas to grab data from the web.
- - Detect Communities in Your Steam Friends List with the Steam Web API and Graph Theory, Using discrete math to analyze your Steam friends.
- - Train a Neural Network to Play Black Jack with Q Learning, It turns out the traditional "stay at 15" is almost the best strategy.
- - Making a Markov Chain Twitter Bot in Python, For those of you who want to spam their followers with gibberish.
- - Eigen-vesting III. Random Matrix Filtering in Finance, Part three in a Series on How Math Fits in Modern Portfolio Theory
- - Don't Solve-- Simulate! Markov Chain Monte Carlo Methods with PyMC3., A lightning tour of PyMC3 and Bayesian inference to solve (somtimes frustrating or impossible) pen and paper problems.
- - Eigen-vesting II. Optimize Your Portfolio With Optimization, Part Two in a Series on How Math Fits in Modern Portfolio Theory
- - Eigen-vesting I. Linear Algebra Can Help You Choose Your Stock Portfolio, A Series on How Math Fits in Modern Portfolio Theory
- - How to Use Math to Win at Fantasy Football With a Lineup Optimizer, All the math in the world won't guarantee a win, but it could help.
- - Jitter, Convolutional Neural Networks, and a Kaggle Framework, A recipe for approaching Kaggle competitions.
Blog
- - Blogging with iPython using Jekyll, Converting code to blogs with iPython.
Math
- - Extending the Gaussian Mixture Approach for Fantasy Football Tiering, An extention to the Gaussian Mixture approach to tiering Players
- - Leveraging Factorization Machines for Wide Sparse Data and Supervised Visualization, A nonlinear model and visualization approach great for sparse data.
- - Bayesian Hierarchical Modeling Applied to Fantasy Football Projections for Increased Insight and Confidence, How to make fantasy football projections better with Bayesian techniques.
- - Making Fantasy Football Projections Via A Monte Carlo Simulation, How to make fantasy football projections with historic data and Monte Carlo techniques.
- - Why Blurring an Image is Similar to Warming Your Coffee, Showing the connection between the heat equation and the Gaussian blur.
- - On Solving Partial Differential Equations with Brownian Motion in Python, When random walks solve deterministic equations
- - Eigen-vesting IV. Predicting Stock and Portfolio Returns With Bayesian Methods, Part four in a Series on How Math Fits in Modern Portfolio Theory
- - Making a Markov Chain Twitter Bot in Python, For those of you who want to spam their followers with gibberish.
- - Eigen-vesting III. Random Matrix Filtering in Finance, Part three in a Series on How Math Fits in Modern Portfolio Theory
- - Connect The Dots. Least Squares, Linear Regression, and Bayesian Regression, Sometimes it helps to draw a line or two.
- - Don't Solve-- Simulate! Markov Chain Monte Carlo Methods with PyMC3., A lightning tour of PyMC3 and Bayesian inference to solve (somtimes frustrating or impossible) pen and paper problems.
- - Eigen-vesting II. Optimize Your Portfolio With Optimization, Part Two in a Series on How Math Fits in Modern Portfolio Theory
- - Eigen-vesting I. Linear Algebra Can Help You Choose Your Stock Portfolio, A Series on How Math Fits in Modern Portfolio Theory
- - How to Use Math to Win at Fantasy Football With a Lineup Optimizer, All the math in the world won't guarantee a win, but it could help.
Sports
- - Extending the Gaussian Mixture Approach for Fantasy Football Tiering, An extention to the Gaussian Mixture approach to tiering Players
- - Bayesian Hierarchical Modeling Applied to Fantasy Football Projections for Increased Insight and Confidence, How to make fantasy football projections better with Bayesian techniques.
- - Making Fantasy Football Projections Via A Monte Carlo Simulation, How to make fantasy football projections with historic data and Monte Carlo techniques.
- - How to Use Math to Win at Fantasy Football With a Lineup Optimizer, All the math in the world won't guarantee a win, but it could help.
Finance
- - Build Your Own Event-Based Backtester in Python, Use multiprocessing to speed up your backtesting!
- - Eigen-vesting IV. Predicting Stock and Portfolio Returns With Bayesian Methods, Part four in a Series on How Math Fits in Modern Portfolio Theory
- - Eigen-vesting III. Random Matrix Filtering in Finance, Part three in a Series on How Math Fits in Modern Portfolio Theory
- - Eigen-vesting II. Optimize Your Portfolio With Optimization, Part Two in a Series on How Math Fits in Modern Portfolio Theory
- - Eigen-vesting I. Linear Algebra Can Help You Choose Your Stock Portfolio, A Series on How Math Fits in Modern Portfolio Theory
Theory
- - Using Ordinary Differential Equations To Design State of the Art Residual-Style Layers, A walkthrough of the theory behind Residual layers with comments on current research.
- - Learning About Deep Reinforcement Learning (Slides), My slides from a talk for Data Philly on Deep Reinforcement Learning.
- - Understanding Attention in Neural Networks Mathematically, Explains the Attention Mechanism's equations and demonstrates them geometrically and probabilistically.
- - Adversarial Dreaming with TensorFlow and Keras, Details a loss function to perform adversarial dreaming in Python.
- - Hogwild!? Implementing Async SGD in Python, Explains the Hogwild! algorithm and walks through an implementation using the multiprocessing library.
- - Covariate Shift, i.e. Why Prediction Quality Can Degrade In Production and How To Fix It, Describes the Kullback-Leibler Importance Estimation Procedure and introduces a python package to use it.
- - An Annotated Proof of Generative Adversarial Networks with Implementation Notes, Exploring gotchas associated with coding your own DQN implementation based on Google DeepMind's Nature paper.
- - Visualizing the Learning of a Neural Network Geometrically, Walking through how to visualize the training process of a neural network.
- - Dealing with Trends. Combine a Random Walk with a Tree-Based Model to Predict Time Series Data, Using statistics and machine learning for time series data.
- - Leveraging Factorization Machines for Wide Sparse Data and Supervised Visualization, A nonlinear model and visualization approach great for sparse data.
- - Why Blurring an Image is Similar to Warming Your Coffee, Showing the connection between the heat equation and the Gaussian blur.
- - On Solving Partial Differential Equations with Brownian Motion in Python, When random walks solve deterministic equations
- - Eigen-vesting IV. Predicting Stock and Portfolio Returns With Bayesian Methods, Part four in a Series on How Math Fits in Modern Portfolio Theory
- - Train a Neural Network to Play Black Jack with Q Learning, It turns out the traditional "stay at 15" is almost the best strategy.
- - Making a Markov Chain Twitter Bot in Python, For those of you who want to spam their followers with gibberish.
- - Eigen-vesting III. Random Matrix Filtering in Finance, Part three in a Series on How Math Fits in Modern Portfolio Theory
- - Connect The Dots. Least Squares, Linear Regression, and Bayesian Regression, Sometimes it helps to draw a line or two.
- - Don't Solve-- Simulate! Markov Chain Monte Carlo Methods with PyMC3., A lightning tour of PyMC3 and Bayesian inference to solve (somtimes frustrating or impossible) pen and paper problems.
- - Eigen-vesting II. Optimize Your Portfolio With Optimization, Part Two in a Series on How Math Fits in Modern Portfolio Theory
- - Eigen-vesting I. Linear Algebra Can Help You Choose Your Stock Portfolio, A Series on How Math Fits in Modern Portfolio Theory
Statistics
- - Calculating Statistical Power When Your Analysis Requires the Delta Method, Demonstration of a statistical power calculation when the variable in question requires the use of the delta method-- i.e., it is a ratio metric.
- - CUPED with Multiple Covariates and A Simpler the Delta Method Calculation, Theoretical derivation of the formula for CUPED with multiple covariates and a calculation trick to make the delta method tractable for this situation.
- - Connections Between the Delta Method, OLS and CUPED, Illustrated, Empirical evidence of the equivalence of OLS and CUPED, plus special considerations for page-level metrics.
- - Dealing with Trends. Combine a Random Walk with a Tree-Based Model to Predict Time Series Data, Using statistics and machine learning for time series data.
- - Bayesian Hierarchical Modeling Applied to Fantasy Football Projections for Increased Insight and Confidence, How to make fantasy football projections better with Bayesian techniques.
- - Making Fantasy Football Projections Via A Monte Carlo Simulation, How to make fantasy football projections with historic data and Monte Carlo techniques.
- - Eigen-vesting IV. Predicting Stock and Portfolio Returns With Bayesian Methods, Part four in a Series on How Math Fits in Modern Portfolio Theory
- - Making a Markov Chain Twitter Bot in Python, For those of you who want to spam their followers with gibberish.
- - Don't Solve-- Simulate! Markov Chain Monte Carlo Methods with PyMC3., A lightning tour of PyMC3 and Bayesian inference to solve (somtimes frustrating or impossible) pen and paper problems.
Reinforcement Learning
- - A Tour of Gotchas When Implementing Deep Q Networks with Keras and OpenAi Gym, Exploring gotchas associated with coding your own DQN implementation based on Google DeepMind's Nature paper.
- - Train a Neural Network to Play Black Jack with Q Learning, It turns out the traditional "stay at 15" is almost the best strategy.
Python
- - Why Blurring an Image is Similar to Warming Your Coffee, Showing the connection between the heat equation and the Gaussian blur.
Data
- - (My Opinion of) Best Practices for a Data Scientist in Industry, Giving unsolicited advice to upcoming Data Scientists.
- - Parsing HTML Tables in Python with BeautifulSoup and pandas, How to use BeautifulSoup and pandas to grab data from the web.
- - Detect Communities in Your Steam Friends List with the Steam Web API and Graph Theory, Using discrete math to analyze your Steam friends.
A/B Testing
- - Calculating Statistical Power When Your Analysis Requires the Delta Method, Demonstration of a statistical power calculation when the variable in question requires the use of the delta method-- i.e., it is a ratio metric.
- - CUPED with Multiple Covariates and A Simpler the Delta Method Calculation, Theoretical derivation of the formula for CUPED with multiple covariates and a calculation trick to make the delta method tractable for this situation.
- - Connections Between the Delta Method, OLS and CUPED, Illustrated, Empirical evidence of the equivalence of OLS and CUPED, plus special considerations for page-level metrics.
Recommendations
- - Learning RecSys through Papers Vol III- Mixed Negative Sampling + Odds and Ends, Another modern-ish implementation of the candidate generation step of a recommender system in PyTorch with a an implementation of Mixed Negative Sampling and a comparion to previous methods in this series of posts.
- - Learning RecSys through Papers Vol II- The How, What, and Why of In-Batch Negatives, Another modern-ish implementation of the candidate generation step of a recommender system in PyTorch with a sketch of the proof of the LogQ correction for in-batch negatives.
- - Learning RecSys through Papers- Implementing a Candidate Generation Model, A modern-ish implementation of the candidate generation step of the "Deep Neural Networks For YouTube Recommendations" by Covington et al. with a discussion of next steps from other papers.
RecSys
- - Learning RecSys through Papers Vol III- Mixed Negative Sampling + Odds and Ends, Another modern-ish implementation of the candidate generation step of a recommender system in PyTorch with a an implementation of Mixed Negative Sampling and a comparion to previous methods in this series of posts.
- - Learning RecSys through Papers Vol II- The How, What, and Why of In-Batch Negatives, Another modern-ish implementation of the candidate generation step of a recommender system in PyTorch with a sketch of the proof of the LogQ correction for in-batch negatives.
- - Learning RecSys through Papers- Implementing a Candidate Generation Model, A modern-ish implementation of the candidate generation step of the "Deep Neural Networks For YouTube Recommendations" by Covington et al. with a discussion of next steps from other papers.
Online
- - Comparing Online Linear Regression and Kalman Filter Approaches to Online Models Plus Derivations, A derivation and implementation of Online Linear Regression and Kalman filters to estimate the weights of a linear model online.
Recommender System
- - Combining Thompson Sampling with Neural Networks for Classification Tasks, An implementation of a training and serving a neural network for classification tasks with Thompson sampling built-in.
- - Online Calculation of the Mean and Variance of Logistic Regression Weights for Thompson Sampling, Deriving the online update for logistic regression using a Kalman filter to yield a mean and variance for Thompson Sampling.
- - Predicting Watch Time like YouTube via Weighted Logistic Regression, An explanation of how YouTube uses weighted logistic regression to predict video watch times.
Ranking
- - Predicting Watch Time like YouTube via Weighted Logistic Regression, An explanation of how YouTube uses weighted logistic regression to predict video watch times.