I’ve always been fascinated by shaders. The idea that pieces of code can create some of the most awe-inspiring visuals you see in video games, movies, and on the web has driven me to learn as much as I can about them.
During that journey, I came across a video by Inigo Quilez called Painting a Character with Maths. It’s a frankly mind-blowing example of a technique called Raymarching. Essentially, it’s a way to build or render complex 2D and 3D scenes in a single fragment shader without needing complex models or materials.
While that example is really impressive, it is also quite intimidating! So, to ease us into this concept, we’ll explore things similar to metaballs, those extremely cool-looking gloopy, liquid shapes that seem to absorb into each other in interesting ways.
Raymarching is a huge topic to cover, but there are some excellent, in-depth resources and tutorials available if you’re interested in going a lot deeper. For this tutorial, we’re going to base the raymarching techniques on this tutorial by Kishimisu: An Introduction to Raymarching, with many references to 3D SDF Resources by Inigo Quilez. If you’re after something more in-depth, I highly recommend the excellently written Painting with Math: A Gentle Study of Raymarching by Maxime Heckel.
In this tutorial, we will build a simple raymarched scene with interesting lighting using React Three Fiber (R3F) and Three.js Shader Language (TSL). You will need some knowledge of Three.js and React, but the techniques here can be applied in any shading language such as GLSL, and any WebGL framework (so, OGL or vanilla is absolutely possible).
The Setup
We are going to be using Three.js Shading Language, a new and evolving language that aims to lower the barrier of entry for creating shaders by providing an approachable environment for those who aren’t so familiar with things like GLSL or WGSL.
TSL requires the WebGPURenderer in Three.js at the moment. This means that if WebGPU is available, the TSL we write will compile down to WGSL (the shading language used in WebGPU) and will fall back to GLSL (WebGL) if needed. As we’re using R3F, we’ll set up a very basic canvas and scene with a single plane, as well as a uniform that contains information about the screen resolution that we’ll use in our raymarched scene. First, we need to set up the Canvas in R3F:
import { Canvas, CanvasProps } from '@react-three/fiber'
import { useEffect, useState } from 'react'
import { AdaptiveDpr } from '@react-three/drei'
import WebGPUCapabilities from 'three/examples/jsm/capabilities/WebGPU.js'
import WebGPURenderer from 'three/examples/jsm/renderers/webgpu/WebGPURenderer.js'
import { ACESFilmicToneMapping, SRGBColorSpace } from 'three'
const WebGPUCanvas = ({
webglFallback = true,
frameloop = 'always',
children,
debug,
...props
}) => {
const [canvasFrameloop, setCanvasFrameloop] = useState('never')
const [initialising, setInitialising] = useState(true)
useEffect(() => {
if (initialising) return
setCanvasFrameloop(frameloop)
}, [initialising, frameloop])
const webGPUAvailable = WebGPUCapabilities.isAvailable()
return (
<Canvas
{...props}
id='gl'
frameloop={canvasFrameloop}
gl={(canvas) => {
const renderer = new WebGPURenderer({
canvas: canvas,
antialias: true,
alpha: true,
forceWebGL: !webGPUAvailable,
})
renderer.toneMapping = ACESFilmicToneMapping
renderer.outputColorSpace = SRGBColorSpace
renderer.init().then(() => {
setInitialising(false)
})
return renderer
}}
>
<AdaptiveDpr />
{children}
</Canvas>
)
}Now that we’ve set this up, let’s create a basic component for our scene using a MeshBasicNodeMaterial where we will write our shader code. From here, all of our code will be written for this material.
import { useThree } from '@react-three/fiber'
import {
MeshBasicNodeMaterial,
uniform,
uv,
vec3,
viewportResolution
} from 'three/nodes'
const raymarchMaterial = new MeshBasicNodeMaterial()
raymarchMaterial.colorNode = vec3(uv(), 1)
const Raymarch = () => {
const { width, height } = useThree((state) => state.viewport)
return (
<mesh scale={[width, height, 1]}>
<planeGeometry args={[1, 1]} />
<primitive object={raymarchMaterial} attach='material' />
</mesh>
)
}Creating the Raymarching Loop
Raymarching, at its most basic, involves stepping along rays cast from an origin point (such as a camera) in small increments (known as marching) and testing for intersections with objects in the scene. This process continues until an object is hit, or if we reach a maximum distance from the origin point. As this is handled in a fragment shader, this process happens for every output image pixel in the scene. (Note that all new functions such as float or vec3 are imports from three/nodes).
const sdf = tslFn(([pos]: any) => {
// This is our main "scene" where objects will go, but for now return 0
return float(0)
})
const raymarch = tslFn(() => {
// Use frag coordinates to get an aspect-fixed UV
const _uv = uv().mul(viewportResolution.xy).mul(2).sub(viewportResolution.xy).div(viewportResolution.y)
// Initialize the ray and its direction
const rayOrigin = vec3(0, 0, -3)
const rayDirection = vec3(_uv, 1).normalize()
// Total distance travelled - note that toVar is important here so we can assign to this variable
const t = float(0).toVar()
// Calculate the initial position of the ray - this var is declared here so we can use it in lighting calculations later
const ray = rayOrigin.add(rayDirection.mul(t)).toVar()
loop({ start: 1, end: 80 }, () => {
const d = sdf(ray) // current distance to the scene
t.addAssign(d) // "march" the ray
ray.assign(rayOrigin.add(rayDirection.mul(t))) // position along the ray
// If we're close enough, it's a hit, so we can do an early return
If(d.lessThan(0.001), () => {
Break()
})
// If we've travelled too far, we can return now and consider that this ray didn't hit anything
If(t.greaterThan(100), () => {
Break()
})
})
// Some very basic shading here - objects that are closer to the rayOrigin will be dark, and objects further away will be lighter
return vec3(t.mul(0.2))
})()
raymarchMaterial.colorNode = raymarchWhat you might notice here is that we’re not actually testing for exact intersections, and we’re not using fixed distances for each of our steps. So, how do we know if our ray has “hit” an object in the scene? The answer is that the scene is made up of Signed Distance Fields (SDFs).
SDFs are based on the concept of calculating the shortest distance from any point in space to the surface of a shape. So, the value returned by an SDF is positive if the point is outside the shape, negative if inside, and zero exactly on the surface.
With this in mind, we really only need to determine if a ray is “close enough” to a surface for it to be a hit. Each successive step travels the distance to the closest surface, so once we cross some small threshold close to 0, we’ve effectively “hit” a surface, allowing us to do an early return.
(If we kept marching until the distance was 0, we’d effectively just keep running the loop until we ran out of iterations, which—while it would get the result we’re after—is a lot less efficient.)
Adding SDF Shapes
Our SDF function here is a convenience function to build the scene. It’s a place where we can add some SDF shapes, manipulating the position and attributes of each shape to get the result that we want. Let’s start with a sphere, rendering it in the center of the viewport:
const sdSphere = tslFn(([p, r]) => {
return p.length().sub(r)
})
const sdf = tslFn(([pos]) => {
// Update the sdf function to add our sphere here
const sphere = sdSphere(pos, 0.3)
return sphere
})
We can change how big or small it is by changing the radius, or by altering its position along the z axis (so closer, or further away from the origin point)
This is where we can also do some other cool stuff, like change its position based on time and a sin curve (note that all of these new functions such as sin, or timerLocal are all imports from three/nodes):
const timer = timerLocal(1)
const sdf = tslFn(([pos]) => {
// Translate the position along the x-axis so the shape moves left to right
const translatedPos = pos.add(vec3(sin(timer), 0, 0))
const sphere = sdSphere(translatedPos, 0.5)
return sphere
})
// Note: that we can also use oscSine() in place of sin(timer), but as it is in the range
// 0 to 1, we need to remap it to the range -1 to 1
const sdf = tslFn(([pos]) => {
const translatedPos = pos.add(vec3(oscSine().mul(2).sub(1), 0, 0))
const sphere = sdSphere(translatedPos, 0.5)
return sphere
})Now we can add a second sphere in the middle of the screen that doesn’t move, so we can show how it fits in the scene:
const sdf = tslFn(([pos]: any) => {
const translatedPos = pos.add(vec3(sin(timer), 0, 0))
const sphere = sdSphere(translatedPos, 0.5)
const secondSphere = sdSphere(pos, 0.3)
return min(secondSphere, sphere)
})See how we use the min function here to combine the shapes when they overlap. This takes two input SDFs and determines the closest one, effectively creating a single field. But the edges are sharp; where’s the gloopiness? That’s where some more math comes into play.
Smooth Minimum: The Secret Sauce
Smooth Minimum is minimum, but smooth! Inigo Quilez’s article is the best resource for more information about how this works, but let’s implement it using TSL and see the result:
const smin = tslFn(([a, b, k]: any) => {
const h = max(k.sub(abs(a.sub(b))), 0).div(k)
return min(a, b).sub(h.mul(h).mul(k).mul(0.25))
})
const sdf = tslFn(([pos]: any) => {
const translatedPos = pos.add(vec3(sin(timer), 0, 0))
const sphere = sdSphere(translatedPos, 0.5)
const secondSphere = sdSphere(pos, 0.3)
return smin(secondSphere, sphere, 0.3)
})Here it is! Our gloopiness! But the result here is pretty flat, so let’s do some lighting to get a really cool look
Adding Lighting
Up to this point, we’ve been working with very simple, flat shading based on the distance to a particular surface, so our scene “looks” 3D, but we can make it look really cool with some lighting
Adding lighting is a great way to create depth and dynamism, so let’s add a variety of different lighting effects in TSL. This section is a bit of an “added extra,” so I won’t go into every type of lighting. If you’d like to learn more about the lighting used here and shaders in general, here is an excellent paid course that I absolutely recommend: https://simondev.teachable.com/p/glsl-shaders-from-scratch.
In this demo, we’re going to add ambient lighting, hemisphere lighting, diffuse and specular lighting, and a fresnel effect. This sounds like a lot, but each of these lighting effects is only a couple of lines each! For many of these techniques, we will need to calculate normals, again thanks to Inigo Quilez.
const calcNormal = tslFn(([p]) => {
const eps = float(0.0001)
const h = vec2(eps, 0)
return normalize(
vec3(
sdf(p.add(h.xyy)).sub(sdf(p.sub(h.xyy))),
sdf(p.add(h.yxy)).sub(sdf(p.sub(h.yxy))),
sdf(p.add(h.yyx)).sub(sdf(p.sub(h.yyx))),
),
)
})
const raymarch = tslFn(() => {
// Use frag coordinates to get an aspect-fixed UV
const _uv = uv().mul(resolution.xy).mul(2).sub(resolution.xy).div(resolution.y)
// Initialize the ray and its direction
const rayOrigin = vec3(0, 0, -3)
const rayDirection = vec3(_uv, 1).normalize()
// Total distance travelled - note that toVar is important here so we can assign to this variable
const t = float(0).toVar()
// Calculate the initial position of the ray - this var is declared here so we can use it in lighting calculations later
const ray = rayOrigin.add(rayDirection.mul(t)).toVar()
loop({ start: 1, end: 80 }, () => {
const d = sdf(ray) // current distance to the scene
t.addAssign(d) // "march" the ray
ray.assign(rayOrigin.add(rayDirection.mul(t))) // position along the ray
// If we're close enough, it's a hit, so we can do an early return
If(d.lessThan(0.001), () => {
Break()
})
// If we've travelled too far, we can return now and consider that this ray didn't hit anything
If(t.greaterThan(100), () => {
Break()
})
})
return lighting(rayOrigin, ray)
})()A normal is a vector that is perpendicular to another vector, so in this case, you can think of normals as how light will interact with the surface of the object (think of how light bounces off a surface). We’ll use these in many of our lighting calculations:
const lighting = tslFn(([ro, r]) => {
const normal = calcNormal(r)
const viewDir = normalize(ro.sub(r))
// Step 1: Ambient light
const ambient = vec3(0.2)
// Step 2: Diffuse lighting - gives our shape a 3D look by simulating how light reflects in all directions
const lightDir = normalize(vec3(1, 1, 1))
const lightColor = vec3(1, 1, 0.9)
const dp = max(0, dot(lightDir, normal))
const diffuse = dp.mul(lightColor)
// Steo 3: Hemisphere light - a mix between a sky and ground colour based on normals
const skyColor = vec3(0, 0.3, 0.6)
const groundColor = vec3(0.6, 0.3, 0.1)
const hemiMix = normal.y.mul(0.5).add(0.5)
const hemi = mix(groundColor, skyColor, hemiMix)
// Step 4: Phong specular - Reflective light and highlights
const ph = normalize(reflect(lightDir.negate(), normal))
const phongValue = max(0, dot(viewDir, ph)).pow(32)
const specular = vec3(phongValue).toVar()
// Step 5: Fresnel effect - makes our specular highlight more pronounced at different viewing angles
const fresnel = float(1)
.sub(max(0, dot(viewDir, normal)))
.pow(2)
specular.mulAssign(fresnel)
// Lighting is a mix of ambient, hemi, diffuse, then specular added at the end
// We're multiplying these all by different values to control their intensity
// Step 1
const lighting = ambient.mul(0.1)
// Step 2
lighting.addAssign(diffuse.mul(0.5))
// Step 3
lighting.addAssign(hemi.mul(0.2))
const finalColor = vec3(0.1).mul(lighting).toVar()
// Step 4 & 5
finalColor.addAssign(specular)
return finalColor
})Where to go from here
So we did it! There was a lot to learn, but the result can be spectacular, and from here there is so much that you can do with it. Here are some things to try:
- Add a cube or a rectangle and rotate it.
- Add some noise to the shapes and get gnarly with it.
- Explore other combining functions (max).
- Use fract or mod for some interesting domain repetition.
I hope you enjoyed this light introduction to raymarching and TSL. If you have any questions, let me know on X.
Credits and References