Exploration is the drum beat of any intelligent, self-improving system.

Key: Red: visited locations. Yellow: Last path taken. Cyan: Lowest-density frontier.

Most modern AI systems are built on machine learning, and machine learning achieves nothing without good, comprehensive data about its task. That’s why I’ve written a new paper, Meta-learning to Explore via Memory Density Feedback, demonstrating an improved approach to autonomous exploration. Watch above as the new RL-based agent, able to see only its own current coordinates and output movements, nonetheless learns to rapidly probe various maze problems, pushing its frontier and returning to lesser visited areas along the way.

This app may have performance issues in Firefox. Try a chromium browser.

Full view of app and control panel here

If this is what brain activity looks like, then how do algorithms fit into the picture?

In my research, I occasionally come back to this simulation because it is so provocative and mesmerizing. Nothing else I have made looks as clearly organic and alive. Yet, whereas other simulations focus on performance of tasks, this simulation does nothing useful on its own. It simply demonstrates a principle: that neurons, even in extremely simplified models, can produce a vast diversity of dynamical regimes, separated by small changes to a few parameters.

Could we evolve entirely unforeseen kinds of AI from mathematical expressions? Engineering works within the narrow bounds of what reason can foresee. Evolution, though blind and inefficient, often discovers systems no one could have predicted from first principles.

In 2024, I briefly dived into genetic programming and produced code for mutating and recombining tree-shaped genomes of mathematical expressions. The idea was that, while we did our best to engineer algorithms, some dedicated computing cluster could asynchronously churn out algorithms for fundamental problems, like localized learning rules, new dynamical systems for memory and computation, and model architectures. It isn’t a realistic goal without an extremely large, parallel, computing environment, but I had just gotten my head around JAX and thought it worth exploring.

In digging through old models and scripts, I came upon one to illustrate the concept of stochastic resonance. In short, when noise is added to periodic signals with local stability, it can cause them to jump between equilibria at about the dominant frequency:

We are always looking for interesting statistical phenomena like this for models in computational psychiatry, where symptoms of mania or depression, for example, are often episodic and make large, spontaneous leaps after periods of relative stability.

There is an obscure and useful function that makes it easy to fit stochastic differential equations to data insofar as the model can be linearized without causing too much trouble. The function discretizes the continuous-time (i.e., differential equation) state matrices A, the drift or state transition matrix, B, the input or covariate coefficient matrix, and Q, diffusion or noise covariance matrix. That means that the function essentially takes the differential equation in matrix form and solves it for a given time step. The discretized matrices function like those of an autoregressive process. Some details of this approach and what this does can be found here but not exactly a complete implementation, namely with matrix B. So this is one of those code blocks I just have backed up in several project folders in various languages.

In putting together educational materials related to Stan and posterior sampling, I remembered two good ones.

The MCMC interactive gallery is the best set of MCMC visualizations I’ve found.

Stan uses NUTS, so it has to calculate numerous gradients for each new sample and does so by autodifferentiation. I recommend this video for understanding autodiff. It helps a lot to know what Stan is doing with the model code to avoid giving it more work than necessary.

Some process photos from my painting on a harpsichord lid.

I painted this harpsichord lid for a commission in 2016. I started with a photoshop mockup:

The first stages of the actual painting are interesting to look back on because they show how fundamental color and movement are to the composition:

Final result:

Art gallery now works. Woohoo!

Check out the paintings

As of this first post, I’ll be using this site to document my various projects, code, minor discoveries, or other thoughts that probably won’t be published anywhere else.