Arch Madness

less than 1 minute read

Published: May 16, 2026

Sorry for the lack of blog posts the past few months, things have been a little crazy.

POMDP

1 minute read

Published: February 21, 2026

Minesweeper as a Markov Process

As promised two weeks ago, I’ve started my attempt to formulate Minesweeper in the language of reinforcement learning (RL). The mathematical backbone of RL is a Markov decision process (MDP), consisting of a state space $\mathcal{S}$, an action space $\mathcal{A}_s$ for each state $s \in \mathcal{S}$, transition probabilities $P_a(s,s’)$ for states $s,s’\in\mathcal{S}$ and action $a \in \mathcal{A}_s$, and the reward functions $R_a(s,s’)$. The goal of the MDP is to find the policy $\pi$ that determines the best action $a$ from any given state, i.e., \begin{equation} \pi^* = \arg\max_\pi \mathbb{E}_{s \sim \pi, a \sim \pi(s)} \left[ \sum_{t=0}^\infty \gamma^t R_a(s_t, a_t) \right] \end{equation} for some discount factor $\gamma \in [0,1]$.

Rich Sutton Plays Minesweeper

2 minute read

Published: February 07, 2026

Month Two

Turns out, a new year’s resolution to blog every week is too ambitious. Maybe I can update that to once a month? Here goes a simple February blog post, coming back into the CompCath sandbox to see what I can put together.

New Year, New Me

1 minute read

Published: January 10, 2026

Trajectory

With the arrival of the arbitrarily chosen day set aside by our society as the first of the year comes the ritualistic resolutions meant to change our behavior. One such resolution of mine was to revive Computational Catharsis as a sandbox for my thoughts. Here goes nothing.