# Programming

### Vertex Shader Domain Warping with Automatic Differentiation

May 14, 2024

I worked out some math to be able to apply arbitrary warps to 3D meshes in a shader and still get correct lighting and reflections.

Domain warping is a technique commonly used in creative coding to distort graphics and add visual interest to a work. The approach has the potential to be used in 3D art as mesh vertices can be efficiently warped using a vertex shader in a WebGL pipeline. However, 3D models packaged for the web typically come with baked-in normal vectors, and these need to be updated when vertex positions change for lighting calculations to work. This is typically done via finite differences, which requires parameter tuning to achieve optimal visual fidelity. We present a method for 3D domain warping that works with automatic differentiation, allowing exact normals to be used without any tuning while still benefiting from hardware acceleration.

### Bean There, Done That: Computer-Assisted Design of Bean Sculptures

Proceedings of SIGBOVIK, March 23, 2024

An algorithm for optimizing a unique bean sculpture that best fits existing architecture.

Chicago's status as a world-class city is cemented by its iconic bean sculpture. Other cities, wanting to replicate the success, have muddied the bean waters by introducing their own bean variations: New York City has a bean sharing similar properties, and Ottawa has a sphere, dubbed the "Ottawa bean" by locals. Our economic analysis proves their worth, so naturally other cities will want their own. We present a mathematical model of the space of all bean sculptures, and an algorithm to help cities replace existing landmarks with beans.

### A Jalgorithm for Japplying Jeans to Jobjects

Proceedings of SIGBOVIK, April 1, 2023

A super serious use of differentiable rendering.

For years, society has been plagued by the lack of a conclusive answer to the question of how arbitrary things would wear jeans, if they wore jeans. This paper proposes a jalgorithm to determine the mathematically optimal configuration of pants on a silhouette, leveraging recent insights in the fields of computer vision and differentiable rendering.

### p5.Framebuffer

June 4, 2022

A library for efficiently drawing to a texture in p5 WebGL mode.

This gives you access to Framebuffers, which are WebGL-only and similar to a p5.Graphics, but faster, and includes depth information.

### GLSL Autodiff

August 13, 2021

Tired of doing math to get normals in your vertex shader? Same. Use this library to write your function once and generate derivatives automatically!

Sometimes, I want to displace mesh vertices in a vertex shader. After doing this, the normals of a surface should change. However, per-vertex normals don't automatically update! Manual updating of vertex normals requires you to take the derivative of your displacement function. This library automates that process so that it takes less time and is less prone to user error.

### StrokeStrip: Joint Parameterization and Fitting of Stroke Clusters

Written in collaboration with Chenxi Liu, Nicholas Vining, Mikhail Bessmeltsev and Alla Sheffer.

ACM Transactions on Graphics, August 1, 2021

Artist-drawn vector sketches use clusters of strokes to convey intended curves. StrokeStrip fits intended curves to such clusters by computing and utilizing a joint arc length parameterization of each cluster, producing clean lines consistent with viewer expectations.

We observe that viewers perceive stroke clusters as continuous, varying-width strips whose paths are described by the intended curves. An arc length parameterization of these strips defines a natural mapping from a strip to its path. We therefore recast the curve fitting problem as one of parameterizing the cluster strokes using a joint 1D parameterization that is the restriction of the natural arc length parameterization of this strip to the strokes in the cluster.

### 2020

Written in collaboration with Jerry Yin.

April 14, 2020

We propose a differentiable formulation for soft shadows, allowing optimization of rendered images that include soft shadows with respect to scene parameters.

Our implementation enables applications such as optimizing a model with respect to its shadow. An example task motivating this work is the algorithmic generation of shadow art, in which a 3D shape produces different artist- specified shadow shapes when lit from different angles. An example of shadow art can be found on the cover of the book Gödel, Escher, Bach. Our goal is to find a reasonable tradeoff in this optimization task between visual quality and performance.

### Locomotion of Boneless Creatures with Distributed Control

Written in collaboration with Tim Straubinger.

April 14, 2020

We are interested in exploring how plausible it is to control creatures whose bodies contain muscle and no bone.

Given various elastic and deformable body plans, we would like to see what gaits emerge based on body shape and body material properties. Through experimentation, we comment on the effectiveness of genetic optimization over policy gradient optimization, and the expressive capability of local controllers with limited views of the world compared to single, comprehensive controllers.

### Automatic Secondary Motion with Dynamic Kelvinlets

Written in collaboration with Jasmeet Singh.

December 1, 2019

Given only primary motion skeletal animation, squash and stretch, overshoot, and jiggling are added automatically in real time.

Dynamic Kelvinlets model the time-varying elastic deformations of objects in response to input forces. Pixar uses them to add an extra level of realism to animations by introducing a secondary motion. We implement a system to automatically apply Dynamic Kelvinlet deformation to achieve visually plausible secondary motion at real-time simulation rates.

### Controlling procedural modelling interactively with guiding curves

Written in collaboration with Abhishek Madan, Andrew McBurney, Paul Bardea and Tammy Liu.

Proceedings of Graphics Interface, March 28, 2019

A method of interactive procedural model searching, written for my fourth year research project.

We present a likelihood function for Sequential Monte Carlo sampling that lets artists draw guiding curves to control the output of generating grammars. This framework enables the high-level structure of models to be intuitively specified while allowing for sufficient variation in the low-level details. Our method can be computed at interactive rates to enable the short feedback loops required for exploratory design.

### The Engulfed Cathedral: CS488 final project

July 25, 2018

A raytraced 3d renderer and short animation created with it for Waterloo's computer graphics course.

I used this project to implement graphics techniques such as photon mapped lighting, ambient occlusion, volumetric materials, constructive solid geometry, and inverse kinematics, along with sampling methods for inverse procedural modelling.

### Frontier: Programming Generative Art

June 14, 2015

2D procedural landscape generation.

I make use of Perlin noise for terrain generation and describe how to think about breaking complex shapes into recursive fractal-like structures to generate trees and houses.