speice.io/blog/2024-11-15-playing-with-fire/3-log-density/index.mdx

---
slug: 2024/11/playing-with-fire-log-density
title: "Playing with fire: Tone mapping and color"
date: 2024-11-15 14:00:00
authors: [bspeice]
tags: []
---

So far, our `plot()` function has been fairly simple; map an input coordinate to a specific pixel,
and color in that pixel. This works well for simple function systems (like Sierpinski's Gasket),
but more complex systems (like our reference parameters) produce grainy images.

In this post, we'll refine the image quality and add color to really make things shine.

<!-- truncate -->

## Image histograms

:::note
This post covers sections 4 and 5 of the Fractal Flame Algorithm paper
:::

To start, it's worth demonstrating how much work is actually "wasted"
when we treat pixels as a binary "on" (opaque) or "off" (transparent).

We'll render the reference image again, but this time, count each time
we encounter a pixel during the chaos game. This gives us a kind of "histogram"
of the image:

import chaosGameHistogramSource from "!!raw-loader!./chaosGameHistogram"

<CodeBlock language="typescript">{chaosGameHistogramSource}</CodeBlock>

When the chaos game finishes, find the pixel we encountered most frequently.
Finally, "paint" the image by setting each pixel's alpha value (transparency)
to the ratio of times encountered, divided by the maximum value:

import CodeBlock from "@theme/CodeBlock";

import paintLinearSource from "!!raw-loader!./paintLinear"

<CodeBlock language="typescript">{paintLinearSource}</CodeBlock>

import {SquareCanvas} from "../src/Canvas";
import FlameHistogram from "./FlameHistogram";
import {paintLinear} from "./paintLinear";

<SquareCanvas><FlameHistogram paint={paintLinear}/></SquareCanvas>

## Tone mapping

While using a histogram to paint the image improves the quality, it also leads to some parts vanishing entirely.
In the reference parameters, the outer circle is preserved, but the interior appears to be missing!

To fix this, we'll introduce the second major innovation of the fractal flame algorithm: [tone mapping](https://en.wikipedia.org/wiki/Tone_mapping).
This is a technique used in computer graphics to compensate for differences in how
computers represent color, and how people see color.

As a concrete example, high dynamic range (HDR) photography uses this technique to capture
nice images of scenes with wide brightness ranges. To take a picture of something dark,
you need a long exposure time. However, long exposures can lead to "hot spots" in images that are pure white.
By taking multiple pictures using different exposure times, we can combine them to create
a final image where everything is visible.

In fractal flames, this "tone map" is accomplished by scaling brightness according to the _logarithm_
of how many times we encounter a pixel. This way, "dark spots" (pixels the chaos game visits infrequently)
will still be visible, and "bright spots" (pixels the chaos game visits frequently) won't wash out.

<details>
    <summary>Log-scale vibrancy is also why fractal flames appear to be 3D...</summary>

    As explained in the Fractal Flame paper:

    > Where one branch of the fractal crosses another, one may appear to occlude the other
    > if their densities are different enough because the lesser density is inconsequential in sum.
    > For example, branches of densities 1000 and 100 might have brightnesses of 30 and 20.
    > Where they cross the density is 1100, whose brightness is 30.4, which is
    > hardly distinguishable from 30.
</details>

import paintLogarithmicSource from "!!raw-loader!./paintLogarithmic"

<CodeBlock language="typescript">{paintLogarithmicSource}</CodeBlock>

import {paintLogarithmic} from './paintLogarithmic'

<SquareCanvas><FlameHistogram paint={paintLogarithmic}/></SquareCanvas>

## Color

Finally, we'll introduce the last innovation of the fractal flame algorithm: color.
By including a color coordinate ($c$) in the chaos game, we can illustrate the transforms
responsible for each part of the image.

### Color coordinate

Color in a fractal flame uses a range of $[0, 1]$. This is important for two reasons:

- It helps blend colors together in the final image
- It allows us to swap in new color palettes easily

We'll give each transform a color value ($c_i$) in the $[0, 1]$ range.
Then, at each step in the chaos game, we'll set the current color
by blending it with the previous color and the current transform:

$$
\begin{align*}
&(x, y) = \text{random point in the bi-unit square} \\
&c = \text{random point from [0,1]} \\
&\text{iterate } \{ \\
&\hspace{1cm} i = \text{random integer from 0 to } n - 1 \\
&\hspace{1cm} (x,y) = F_i(x,y) \\
&\hspace{1cm} (x_f,y_f) = F_{final}(x,y) \\
&\hspace{1cm} c = (c + c_i) / 2 \\
&\hspace{1cm} \text{plot}(x_f,y_f,c_f) \text{ if iterations} > 20 \\
\}
\end{align*}
$$

### Color speed

:::warning
Color speed as a concept isn't introduced in the Fractal Flame Algorithm paper.

It is included here because [`flam3` implements it](https://github.com/scottdraves/flam3/blob/7fb50c82e90e051f00efcc3123d0e06de26594b2/variations.c#L2140),
and because it's fun to play with.
:::

Next, we'll add a parameter to each transform that controls how much it affects the current color.
This is known as the "color speed" ($s_i$):

$$
c = c \cdot (1 - s_i) + c_i \cdot s_i
$$

import mixColorSource from "!!raw-loader!./mixColor"

<CodeBlock language="typescript">{mixColorSource}</CodeBlock>

Color speed values work just like transform weights. A value of 1
means we take the transform color and ignore the previous color state.
Similarly, a value of 0 means we keep the current color state and ignore the
transform color.

### Palette

Now, we need to map the color coordinate to a pixel color. Fractal flames typically use
256 colors (each color has 3 values - red, green, blue) to define a palette.
Then, the color coordinate becomes an index into the palette.

There's one small complication: the color coordinate is continuous, but the palette
uses discrete colors. How do we handle situations where the color coordinate is
"in between" the colors of our palette?

One way is to use a step function. In the code below, we multiply the color coordinate
by the number of colors in the palette, then truncate that value. This gives us a discrete index:

import colorFromPaletteSource from "!!raw-loader!./colorFromPalette";

<CodeBlock language="typescript">{colorFromPaletteSource}</CodeBlock>

<details>
    <summary>As an alternative...</summary>

    ...you could also interpolate between colors in the palette.
    For example: [`flam3` code](https://github.com/scottdraves/flam3/blob/7fb50c82e90e051f00efcc3123d0e06de26594b2/rect.c#L483-L486)
</details>

In the diagram below, each color in our palette is plotted on a small vertical strip.
Putting the strips side by side shows the palette used by our reference image:

import * as params from "../src/params"
import {PaletteBar} from "./FlameColor"

<PaletteBar height="40" palette={params.palette}/>

### Plotting

We're now ready to plot our $(x_f,y_f,c_f)$ coordinates. After translating from color coordinate ($c_f$)
to RGB value, add that value to the image histogram:

import chaosGameColorSource from "!!raw-loader!./chaosGameColor"

<CodeBlock language="typescript">{chaosGameColorSource}</CodeBlock>

Finally, painting the image. With tone mapping, logarithms scale the image brightness to match
how it is perceived. When using color, we scale each color channel by the alpha channel:

import paintColorSource from "!!raw-loader!./paintColor"

<CodeBlock language="typescript">{paintColorSource}</CodeBlock>

And now, at long last, a full-color fractal flame:

import FlameColor from "./FlameColor";

<SquareCanvas><FlameColor/></SquareCanvas>

## Summary

Tone mapping is the second major innovation of the fractal flame algorithm.
By tracking how often the chaos game encounters each pixel, we can adjust
brightness/transparency to reduce the visual "graining" of previous images.

Next, introducing a third coordinate to the chaos game makes color images possible,
the third major innovation of the fractal flame algorithm. Using a continuous
color scale and color palette adds a splash of color to our transforms.

The Fractal Flame Algorithm paper goes on to describe more techniques
not covered here. Image quality can be improved with density estimation
and filtering. New parameters can be generated by "mutating" existing
fractal flames. Fractal flames can even be animated to produce videos!

That said, I think this is a good place to wrap up. We were able to go from
an introduction to the mathematics of fractal systems all the way to
generating full-color images. Fractal flames are a challenging topic,
but it's extremely rewarding to learn more about how they work.