Meaningful Complexity, part 3

How do we quantify the "right" amount of complexity?

Caveat Lector: This is a Junk Drawer post, which means it’s very niche and technical. This is part three of a series documenting (in detail!) my recent exploration of a complexity rabbit hole invoked in

Joseph Rahi

’s April post, A Formula for Meaningful Complexity.

That post is required reading as are the previous two posts in this series (part 1 and part 2). I’ve laid enough groundwork in the previous posts to jump right into things here (and in future posts in this series).

The Emergence Factor (EF) Process

The last post ended with the EF formula and described how it tracks the curvature of the “compression curve”. Now we’ll get into exactly what that is and how it’s generated — though it should be familiar from Rahi’s post.

The basic notion is that we want to take a text or image file and progressively add random “noise” to it until we have a file that’s 100% random noise. For definiteness, say we increase the noise by 10% each step. Then we end up with eleven files: 0%, 10%, 20%, … 100%. Presumably, the original (with 0% noise) compresses to the smallest possible size for its content while the final file (with 100% noise) cannot be compressed at all. The amount of compression for each file comprises our compression curve.

I’ll explain more about how we add noise below, but let’s start with some examples. And because they’re more fun than texts, we’ll start with images. I have six camera photos I’ve been using, plus some other images for more variety (like flags or Mandelbrot plots).

You’ve already seen these two (in the first post):

“Clouds” and “Mock Apples”

As well as these two:

“Oranges” and “Bentley”

Here are the last two:

“Tree” and “Trees”

All six come in four sizes:

1. 4032x3024 (original photo)

2. 2400x1800

3. 1200x900

4. 400x300

Because I wanted to see what effect image size had on the numbers. Not to keep you in suspense, it does have an effect. I’ve already mentioned how the final score, the ESC (Emergent Structural Complexity) is dependent on file size.¹ But SS and EF are not (or not as much), and I’ll stick mostly with those for now.

An easy and obvious approach for adding noise to images is to replace a given percentage of the original image pixels with randomly generated pixels. For instance, here’s the “Bentley” image with 10%, 20%, 30%, and 40% noise:

And here it is with 70%, 80%, 90%, and 100% noise:

The final image is completely random and does not compress at all. If we apply this process to all four image sizes, we end up with four similar compression curves. For the 400x300 size:

For the 1200x900 size:

For the 2400x1800 size:

And finally, for the 4032x3024 size:

Note that the blue and red curves are in terms of bytes and vary depending on the file (or in this case, image size). The blue curve is the size of the file — I use BMP files so it’s the same for each image in the series. The red curve is the compressed size (in bytes). In these plots, it approaches the original file size as noise increases. In the final image, with 100% noise, it’s the same because that file cannot be compressed, just stored as is.

The green curve is a normalized measure of compressibility (calculated the same as the SS discussed last time):

\({SS} = \frac {\textit{file-size} - \textit{compressed-size}} {\textit{file-size}}\)

So, it has a range of 0.0 to <1.0 (note scale on right side of graphs). The larger its value, the more compressible the file is. The left-most point on the green curve in each graph represents the maximum compressibility — the compressibility of the original image.² Note how the smallest image size is only ~26% compressible whereas the largest size is ~50% (with the other two at ~36% and ~43%, respectively).

Using these data to calculate the SS, EF, and ESC for each image, we get:

• 4032x3024: 0.501230; 1.692117; 15,473,620.76

• 2400x1800: 0.429622; 1.708146; 5,424,765.15

• 1200x900: 0.363548; 1.738861; 1,303,600.64

• 400x300: 0.263983; 1.818803; 127,237.99

The EF values are close but decrease somewhat with image size whereas the image compressibility increases. Similar EF values indicate similar compression curves.³

The ESC, as mentioned, is subject to image size and varies the most, going from over 15 million down to just under an eighth of a million.

“Tree” with 10%, 50%, and 90% noise (left-to-right)

The “Bentley” image is roughly in the middle of the six camera photos in terms of its SS and EF values. The “Tree” photo is at one end of the range. Here is its graph for the smallest size:

And the largest size:

The other two fall between just as with the “Bentley” photo.

The curve is noticeably different, and we get different numbers:

• 4032x3024: 0.539428; 1.564040; 14,213,566.09

• 2400x1800: 0.495350; 1.577351; 5,110,194.87

• 1200x900: 0.459641; 1.604392; 1,291,111.89

• 400x300: 0.392144; 1.693803; 145,370.69

The bigger file is slightly more compressible, but the small one is considerably more so. As above, the EF decreases with larger file size, and compression increases.

The ESC values are comparable to above, though — except for the 400x300 size — which are just below the “Bentley” values.

“Trees” with 20%, 50%, and 80% noise (left-to-right)

At the other end of the range, the “Trees” photo has lots of small-scale structure and doesn’t compress well at all. Here’s the graph for the smallest size:

And here’s the one for the biggest:

This photo is almost uncompressible even without noise. This is reflected by the SS and EF values:

• 4032x3024: 0.095900; 2.250219; 7,136,494.34

• 2400x1800: 0.062774; 2.449934; 1,868,048.35

• 1200x900: 0.061294; 2.556542; 476,601.17

• 400x300: 0.077816; 2.551323; 65,920.44

Considerably higher EF values than in any of the other five photos (and note the extremely low SS values — along with how the smallest size is slightly more compressible than the intermediate sizes).

In this case, the ESC values are about half what they were for the “Bentley” photo, which I don’t know what to make of. The “Tree” photo — less complex than the “Bentley” photo — had slightly lower value ESC values, but this “Trees” photo — which is quite complex — has ESC values about half of the “Bentley” photo. I still haven’t quite figured out what ESC is telling me, but I haven’t spent that much time analyzing it yet.

Given the six photos in four resolutions each, there are 24 sets of values:⁴

The six photos are ordered top to bottom by their EF values from lowest to highest. This generally correlates with the SS value decreasing from top to bottom, but there are quite a few exceptions. For instance, “Clouds” is more compressible than “Tree”. There also seems more variance over image size with SS.⁵

This seems like a good stopping point for now. I’ll pick this up (soon) with stats on the other seven images I’ve been using. After that I’ll get into the noise process with text files.

I’ll end with something I’ll explore in more detail in the next posts: the actual technique of adding noise. Above I mentioned that an easy and obvious choice is to replace a percentage of pixels with randomly generated pixels. The end goal — with 100% noise — is a fully random file (that cannot be compressed).

But there are other “modes” of adding noise. For instance, rather than replacing a pixel with an entirely random pixel, one can perturb a pixel — change its original value to something else. Which brings in another potential parameter: the distance we perturb the pixel. (This turns out to be a lot more interesting and fun when it comes to text.⁶)

Here’s an example:

The upper image has “mode 0” noise — 50% of its pixels have been replaced with randomly generated pixels (which can have any color). The lower image has “mode 7” noise — all of its pixels have been 50% perturbed away from their original values. As you can see, the lower image is a “softer” distortion whereas the random pixels in the upper image give it a more “jagged” look.

Unfortunately, while the upper series of noise-distorted images (from 0% to 100%) gives us the same sort of curve as we got with the photos:

The lower image gives us something quite different:

Even the first file, with 10% perturbation (of all pixels, though!) is barely compressible. Oddly, that image doesn’t look particularly distorted:

But it does have a slight “graininess” to it, and that eliminates smooth stretches of a single color that can be easily compressed. The compression series starts at 35% but drops to 4% on the 10% noise image, so it doesn’t have far to fall to reach 0% for the 100% noise image. That makes its compression curve close to a flat line.

The SS, EF, and ESC values for the two noise modes:

• Noise Mode 0: 0.349754; 1.786773; 1,110,297.94

• Noise Mode 7: 0.349754; 5.041971; 3,133,072.51

Both have the same max/min sizes: 2,732,310 bytes and 1,776,675 bytes, so both have the same SS value. But noise mode 7 has a very high EF because of that weird compression curve — between min and max, most of the area is above the curve. The ESC values differ and, because AC is the same for both, file size is not a factor in that difference. In fact, it’s all due to the different EF values.

I’ll start with this image and others in the next post.

Until next time…

1. Because one of the factors — the AC — is the minimum (compressed) file size.

2. And this is the SS value assigned to the ESC calculation.

3. I think EF might be a good indicator of image structure.

4. Apologies for posting information in an image but Substack has no ability to insert tables into a post.

5. But I’ll have to do some calculating to determine their actual variance.

6. Interesting and fun but, alas, not particularly useful for our purposes.