1. This all seems pretty simple, but it is worth sharing anyways as a primer on the subject (there may be a follow-up on the same subject)

2. I’ve done this as an exercise in deriving things from first principles (no internet, pick three textbooks and go away for a while)

3. I’ve primarily done this to solve a problem, and like many problems, the solution involves touching computational grass, so my interest in this is limited to what can be applied directly

I work with hard-real-time hardware systems in my day-to-day by writing firmware and such system have a lot of constraints, namely:

1. There is not a lot of RAM

2. Many paths are bottlenecks to full system performance

3. Many paths have lower speed limits before quality-of-service issues cause functional problems

I’m interested in pulling a lot of data out of this system, but the above points are pretty annoying constraints. However, if we assume the following:

1. Measurements are independent from each other

2. Flows are (reasonably) deterministic

then we can spread the impact in at least one of two ways:

1. Probabilistically measure all values at once along a certain flow

2. Measure the values exactly and split those counters, running multiple versions along the same flow

This blog post is all about (1). (2) has shown itself to be very useful, but given what I do that’s off limits.

Counters

A lot of the data I’d like to collect involves counting the frequency of events. This has a few interesting properties:

1. Although the values of the counters may be related, the act of measurement is independent

2. A counter is relatively small, the number of counters is relatively large

Additionally, I’m only interested in orders of magnitude difference in values.

Think of a counter like a function, like this:

Obviously, it has a “derivative” of 1.

Consider another type of counter, like this:

This derivative depends on the indicator variable sampling from some probability distribution. That isn’t specified beyond the fact it depends on the current value. The key difference here is the value is not the number of increments, but the value after some number of trials. The relationship between the number of increments and the stored value is determined by the underlying distribution. Counters of this type are called Morris counters and the algorithm is called the approximate counting algorithm.

Calculus is tempting, but let’s not use that for now. I’ve found the following two ways of reasoning about this effective: nested polynomials and Markov Chains.

Nested Polynomials

Let’s define the following (for the case of log base 2):

Let’s also define a split, which defines the total number of hits (H). Splits 3, 2, 1 and 0 are defined as follows (items in braces are to be interpreted as a sequence of events in the order listed):

Each hit is an increment by one, so an interpretation as products and a summation should provide us with an expected value. From a probability perspective, the order of events does not matter, so all “hits” can be factored out to the front:

The polynomial can be seen as a valid probability because the events are all disjoint from each other (as each addition represents a unique path in the expansion and all paths are disjoint by definition). This can generalize to nested summations for arbitrary values across all possible splits. I’ve done this for my own purposes, but the pattern should be self-evident.

Markov Chain

Another approach is with Markov chains. The transition matrix is defined as such:

The initial state is the vector with only the first element set to 1, all other values 0. We need to know up-front how many trials to run to determine some constants (the number of constants is denoted by “n”):

1. The matrix is of size (n + 1)x(n + 1) since the initial state starts with a constant value

2. The transition matrix is multiplied n times

The resulting vector needs to be item-wise multiplied by its position to produce an expected value.

Final Notes

Both of these approaches are different lenses into fundamentally the same concept. The matrix approach encodes extra information, but in a structure that we all know and love (matrices). All algorithms listed have numerical stability issues that may be addressable with more thought. I’ve just deployed arbitrary precision rational arithmetic libraries at the problem. I may have a follow-up blog-post reasoning about the variance, but I need better performance on the numerics side (mostly accuracy with floating point numbers) before I venture onto that (I’ve done work on that front, but the numerics aren’t consistent with the theory, so I’m skeptical to publish that as-is).

The paper defines four distinct problems:

1. Generate all valid “slice states”

2. Generate all valid relations between slice states

3. Generate all compositions of slice states (similar to a recurrence relation)

4. Generate a distinct tiling given a valid composition

The Basics

Slice States

A slice state is a layer in the recurrence relation. A slice state is defined by the following:

1. The length of each piece ID and the area beneath it in future slice states

2. The piece ID at each position in the slice sate

Here are some requirements that a slice state must satisfy:

1. Each position has a mapped piece ID

2. Each piece ID has a non-zero length

3. The sum of all volume under a piece ID (inclusive) must be divisible by the width of the board

4. Horizontal flips of the same piece are considered identical

5. Every position of a piece ID must be connect-able to every other position on the same slice through another slice

There are others, but these have the clearest goemetric interpretations (an example of a non-geometric requirement is a unique isomorphism of IDs).

Slice Relation

A slice relation is a mapping from one slice state to another slice state. A slice relation is defined by the following:

1. A source slice state and a destination slice state

2. A mapping from the source slice state piece IDs to the destination slice state piece IDs

3. Whether to flip the first slice state (the choice of first is arbitrary, continue to learn more)

The search space is much larger, but there are fewer constraints. Here are some requirements that a slice relation must satisfy:

1. There are no internal borders (i.e. the interiors of distinct pieces aren’t connected)

2. Either the volume of every piece increases by the number of occurrences in the new slice state or the piece does not exist in the next slice state

3. The mapping of pieces from the source to destination is valid

4. Every mapping of a piece has at least one point where the two connect at an edge

5. Only unique flips will generate a flip, non-unique flips must only be represented once

Recurrence Relation

The beginning and end of the tiling region is determined by the n-wide piece (it behaves in the same way as the outside). I can adjust the execution of the recurrence relation so it only generates the height with no excess on the left/right, but there is a 1-1 mapping between the two tilings so it does not matter at this time. Every function and it’s flip is checked, so we don’t consider flips of entire functions but do consider flips of only the source slice state (the choice of first is arbitrary, as the flip of the entire mapping is equivalent to only flipping the bottom).

Beyond the Basics

Generating a Tiling

Technically, the paper only “outlines” a method for generating an explicit tiling, so any generated result is beyond the original paper (granted, I haven’t ran to the 9x9 case, but computing some visual representation for any single tiling isn’t hard, the issues are storing the results and specifying which subset to run it on).

Reasonable Visualization

The basics are correct but the representation is pretty ugly. It looks like this:

I understand how to read this, but it’s very ugly and difficult to find bugs. I need a representation that looks like Tetris. Adjacency information is encoded on a slice-by-slice basis, and I can back-solve for mappings by composing piece maps together. Using these tools, I can:

1. Construct global graph piece IDs by reversing the piece maps of every slice relation above it

2. Construct a graph of global piece IDs by connecting all global piece IDs together by adjacent local slice state piece IDs

3. Color the graph (currently via brute-force, but the Four-Color Theorem gives us an upper complexity bound, which works well for us)

I did this and now have the following:

This looks much better, doesn’t it?

Future Steps

I’ll make it much faster, allow for rendering a specific tiling, and optimize for memory use. I’ll detail specific optimizations as I write them as individual blog posts in a series.