I had the pleasure of seeing Mandelbrot in person when he visited Microsoft, and he described his lifelong fascination with things chaotic. One of the great accomplishments of complexity theory is that it proscribes some mathematical limits on how deep or how far we can understand certain otherwise deterministic phenomena. Robert Frost wrote, in his poem “Neither Out Far Nor In Deep”:

The people along the sand
All turn and look one way.
They turn their back on the land.
They look at the sea all day.

As long as it takes to pass
A ship keeps raising its hull;
The wetter ground like glass
Reflects a standing gull

The land may vary more;
But wherever the truth may be–
The water comes ashore,
And the people look at the sea.

They cannot look out far.
They cannot look in deep.
But when was that ever a bar
To any watch they keep?

Not for the proud man apart
From the raging moon I write
On these spindrift pages
Nor for the towering dead
With their nightingales and psalms
But for the lovers, their arms
Round the griefs of the ages,
Who pay no praise or wages
Nor heed my craft or art.

From “In My Craft or Sullen Art” by Dylan Thomas.

Or try:

Fear not the waking world, my mortal,
Fear not the flat, synthetic blood,
Nor the heart in the ribbing metal.
Fear not the tread, the seeded milling,
The trigger and scythe, the bridal blade,
Nor the flint in the lover’s mauling.

From “All All and All the Dry Worlds Lever“.

~

 (For Kellie Jones, born 16 May 1959)

Lately, I've become accustomed to the way
The ground opens up and envelopes me
Each time I go out to walk the dog.
Or the broad edged silly music the wind
Makes when I run for a bus...

Things have come to that.

And now, each night I count the stars.
And each night I get the same number.
And when they will not come to be counted,
I count the holes they leave.

Nobody sings anymore.

And then last night I tiptoed up
To my daughter's room and heard her
Talking to someone, and when I opened
The door, there was no one there...
Only she on her knees, peeking into

Her own clasped hands

~

THE DEATH OF NICK CHARLES

... And how much of this
do you understand?  I hide
my face, my voice twisted
in the heavy winter fog.  If I
came to you, left this wet island
& came to you; now, when I am young,
& have strength in my fingers.  To say,
I love you, & cannot even recognize
you. How much of me
could you understand?  (Only
that I love colour, motion, thin high air
at night?  The recognizable parts
of yourself?

We love only heroes. Glorious
death in battle.  Scaling walls,
burning bridges behind us, destroying
all ways back.  All retreat. As if
some things were fixed. As if the moon
would come to us each night (&
we could watch
from the battlements). As if
there were anything certain
or lovely
in our lives.

Sad
long
motion of air
pushing in my face. Lies,
weakness, hatred
of myself. Of you
for not understanding
this. Or not
despising me
for the right causes. I am
sick as, OH,
the night is. As
cold days are,
when we must watch them
grow old
& dark.

2

I am thinking
of a dance.  One I could
invent, if there
were music.  If you
would play for me, some
light music.  Couperin
with yellow hillsides.  Ravel
as I kiss your hair. Lotions
of Debussy.
I am moved by what? Angered at its whine;
the quiet delicacy of my sadness. The elements.
My face torn by wind, faces, desire, lovely chinese ladies
sweeping the sidewalks. (And this is not
what I mean. Not the thing I wanted for you. Not, finally.
Music, only terror at this lightly scribbled day.

Emotion. Words.
Waste. No clear delight.
No light under my fingers. The room, The
walls, silent & deadly. Not
Music.

If there were
a dance. For us
to make; your fingers
on my face, your face wet
with tears (or silence. For us
to form upon this heavy air. Tearing
the silence, hurting the darkness
with the colour of our movement! Nakedness?
Great leaps
into the air? Huge pirouette; the moon blurred
on ancient lakes. Thin horns
and laughter.

3

Can you hear this? Do you know
who speaks to you? Do you
know me? (Not even
your lover. Afraid of you, your sudden
disorder. Your ringless
hands. Your hair
disguised. Your voice
not even real. Or
beautiful.

           (What we had
I cannot even say. Something
like loathing
covers your words.

4

It grows dark
around you. And these words
are not music. They make no motions
for a dance. (Standing awkwardly
before the window, watching
the moon. The ragged smoke
lifting against
grey sheaths
of night.
You shimmer like words
I barely hear. Your face
twisted into words. "Love, Oh,
Love me." The window facing night, & always
when we cannot speak.

What shapes stream through the glass?
Only shadows
on the wall. Under
my fingers, trailing me
with a sound like
glass on slate. You cry out
in the night,
& only the moon
answers.

5

The house sits
between red buildings. And a bell
rocks against the night air. The moon
sits over the North river, underneath
a blue bridge. Boats & old men
move through the darkness. Needing
no eyes. Moving slowly
towards the long black line
of horizon. Footfalls, the
twisting dirty surf. Sea birds
scalding the blackness.

I sit inside alone, without
thoughts. I cannot lie
& say I think of you. I merely sit
& grow weary, not even watching
the sky lighten with morning.

              & now
I am sleeping
& you will not be able
to wake me.

No wonder interim CEO Cathie Lesjak has explicitly withdrawn herself from consideration in the search for a replacement CEO. The ideal CEO candidate must be male, white, talk like George Patton, have bulldogs and cigars, and know how to keep conniving women in their places.

Note that I haven’t found the theory I’m about to explain in any books or papers, so it might be wrong. But it seems simple and obvious, so I’ll wager that it’s well known and accepted.

Here’s the thing. Parent’s don’t teach the words for “mother” and “father” to their babies; babies teach the words to their parents.

Upon birth, before their first meal and well before they can control their breathing, babies have a reflex to open and close their mouths when they see an adult open and close a mouth. This reflex is ancient and involuntary, like breathing.

Later, babies learn to voluntarily control their breathing and their vocal chords, to voluntarily create sighing or humming noises. During this phase, when the babies are experimenting with noise, a parent need only trigger the mouth open/close reflex (even accidentally) for the baby’s humming to accidentally produce a “mama” or “baba/papa” noise. Try it yourself: make humming noises with your lips closed, and while continuing to hum, open and close your mouth the way that an infant reflexively does.

From here, the excited reaction of the parents (”He said ‘mama’! What a smart baby! Everyone come here!) is enough to reinforce the behavior, and the parents have been trained.

Here’s another way to think about it. Suppose that a culture were “advanced” enough to use the words “thrir” and “frish” instead of “mama” and “baba”. It would not be long before the parents gave up and stopped trying to get the babies to say “thrir” or “frish”. The babies would undoubtedly win in the end, and thrirs would joyfully adopt the name “mama”. Frishes would only slightly less joyfully accept the name “papa”, since “mama” is easier to say (as you should have easily verified in the humming exercise above), and everyone knows that, after baby, thrir always wins. ‘Cause when thrir ain’t happy, ain’t nobody happy!

The first person to solve the Parity Encoding Puzzle correctly after I published it was Vittorio’s wife, Iwona, with a very elegant and compact solution, which is slightly different from mine.

First, I will explain my thought process in solving the puzzle, then show my code, and then show you Iwona’s code (which makes my code look clumsy).  If you still want to try solving it for yourself, don’t read any further.  If you have solved it using another language or technique, feel free to post your solution here.

With a 2-bit buffer

The first step is to start with a smaller version of the problem.  If a buffer of size 2^n can store n bits of information, then a 2-bit buffer can store 1 bit.  All possible initial states of the random buffer are:

00
01
10
11

It’s pretty easy to see what to do.  If you want to send a ‘1’, just flip the first bit if it’s zero, otherwise flip the second bit.

At this point, you’re probably noticing that the second bit has no informational value; it’s sort of a “throwaway bit”.  If the rules were different, and you were permitted to either flip a bit or abstain, it would be similar (for the case of a 2-bit buffer).

With a 4-bit buffer

In a 4-bit buffer, you can store 2 bits of information (4 possible values).  A 4-bit buffer has only 16 possible initial states, so it’s easy to sketch on paper, and try the 4 possible coded values.

Encoding the first bit of the information is easy.  Just apply the pattern from the 2-bit solution, but use the parity of the first 2-bits of the 4-bit buffer as your value.  In other words, if the first bit of your information is a ‘1’, the parity of the left half of the 4-bit buffer should be even, otherwise it should be odd.

Switching the parity is easy; you just flip a bit in the target half, and the parity toggles.  But you’re then faced with the question of which bit to flip.  For example, if your starting buffer is 1011 and the value to code is 10, you need to flip a bit on the left half – you could do that by flipping either the first or second bit, resulting in either 1111 or 0011.  Additionally, you’re faced with the problem that you’ve used up your only allowed toggle, and you still haven’t encoded the second bit.

This latter fact should make the way forward obvious.  The bit that you choose to flip (resulting in a 0011 or 1111) should be determined by the second bit of information (10).  You can do this by saying that the parity of the opposite half (the final 2 bits in this case: 1011) must match the value of the second bit in the half being toggled (in this case, the 1011), if the bit being communicated is a 1.  You can arbitrarily decide that a bit value of 0 represents odd parity, and 1 represents even parity.  In this case, the second bit of information (10) is not a 1, so you want the value of the second bit to be different from the parity of the opposite side.  The second bit, representing odd parity (1011) is already different from the parity of the opposite half (1011) so you needn’t flip the second bit – flip the first bit instead.  The encoded buffer, therefore, is 0011.

You can easily prove this technique on paper or by writing a quick program.  It is worth working through the combinations yourself to make sure you understand what’s happening.

Note that you always think in terms of “opposite side”, since you’ll only be flipping a bit on the left side half of the time.  Also note that the decision to compare the second bit to the parity of the opposite side is completely arbitrary, just like the decision to use 1 to mean “even”.  You could change the rules, as long as you are consistent.

With an 8-bit buffer

At this point, you might be feeling confident, and think that you only need to recursively apply the 4-bit solution to manage a buffer of any size.  You would be right, but it’s easier said than done.  My first attempt worked 90+% of the time, but failed on some edge cases.

An 8-bit buffer holds 3 bits of information.  You could try something like this (colors show parity comparison):

1. 1010 1010 – first bit
2. 1010 1010 – second bit
3. 1010 1010 – third bit

Obviously, the position of the second and third bit will vary, depending on where the previous bits are selected.

Observation: If you implement this, you’ll notice 2 things (besides the fact that it fails some of the time).  First is that decoding has no idea whether you moved left or right in a given step, so half the time it will get the right answer only accidentally.  Second is that you’re leaving a lot of bits unused (for parity comparison purposes), which intuitively shouldn’t work.  It’s true that you will have a “throwaway bit”, but it should only be one bit.

This leads to the key intuition required to solve the puzzle generally.

The Solution

You need to “stripe” the parities; using every bit except the throwaway bit.  An simple example of decoding a value will make it clear.  Suppose that the received buffer is 10110000.  How do you decode it?

(For this case, we arbitrarily decide that parity equals the number of 1’s in the red area, and that 1 is “odd” [since it has an odd number of 1’s])

1. 1011 0000 – parity of left half = 1 (odd)
2. 1011 0000 – parity = 1 (odd)
3. 1011 0000 – parity = 0 (even)

Therefore, the encoded value is 110.

Step 3 is the key step: you notice that we are using 2 extra bits in our parity calculation more than we used in the failed attempt.  The only bit which is never used is the last bit, which you would flip only if the random buffer exactly matched the message you wanted to send.

You can also detect a pattern here:

1. Step 1: combine alternating blocks of 4 bits and calculate parity.
2. Step 2: combine alternating blocks of 2 bits and calculate parity.
3. Step 3: combine alternating blocks of 1 bits and calculate parity.

It’s easy to generalize this to a buffer of any size.  For example, if you are using a 128 bit buffer, you would use parity “stripes” of size 64, 32, 16, 8, 4, 2, and 1 (for 7 bits of information).

If you’d like to try writing your own code before looking at mine, you can pause here.

My Code

I used a very direct implementation of the process described above.  The “Parity” function takes a bit buffer and a width, and combines alternating blocks of size “width” before spitting out a parity.  The Encode and Decode functions use this Parity function.  You can trace through a  few examples on paper to see how easy it is.

public int Parity(int width, BitBuffer buffer)
{
    int p = 0;

    bool even = true;

    while (p < buffer.Length)
    {
        for (int i = 0; i < width; i++)
        {
            if (buffer[p + i] == 1) even = !even;
        }
        p += width * 2;
    }

    return (even ? 0 : 1);
}

public BitBuffer Decode(BitBuffer buff)
{
    string value = "";

    int width = buff.Length;

    while (width > 1)
    {
        width = (int)Math.Truncate((double)width / 2);
        value += Parity(width, buff).ToString();
    }

    return new BitBuffer(value);
}

public void Encode(BitBuffer encode, BitBuffer initial )
{
    int[] bits = encode.GetBuffer();

    int pos = 0;
    int width = initial.Length;

    for (int idx = 0; idx < bits.Length; idx++)
    {
        pos *= 2;
        width = (int)Math.Truncate((double)width / 2);

        int half = (int)Math.Truncate((double)initial.Length / width / 2);

        if (Parity(width) == bits[idx]) pos += 1;  // the left half is good; don't mess with it
    }

    if (initial[pos] == 0) initial[pos] = 1;
    else initial[pos] = 0;
}

Iwona’s Code

Iwona’s Code is very compact, using xors and bit shifts, and not needing a separate parity stripe test (solved for 64-bit, but will work on any buffer size).  Very nice!

private static UInt64 Encode(UInt64 buffer, byte number)
{
    byte i = 0;
    UInt64 tempBuffer = buffer;
    while (tempBuffer > 0)
    {
        if (tempBuffer % 2 == 1)
        {
            number ^= i;
        }
        i++;
        tempBuffer = (tempBuffer >> 1);
    }
    return buffer ^ ((UInt64)1 << number);
}

private static byte Decode(UInt64 buffer)
{
    byte i = 0;
    byte number = 0;
    while (buffer > 0)
    {
        if (buffer % 2 == 1)
        {
            number ^= i;
        }
        i++;
        buffer = (buffer >> 1);
    }
    return number;
}

A few weekends ago, a friend introduced me to this puzzle while we were walking around the park.  It took me a few hours to find the solution and write a program to test it.  It’s a fun puzzle which I have never seen before, so I’m posting it to my blog.  I’ll reveal the solution in a separate post, so you can try solving it without any spoilers.

Introduction

Imagine that you have a 64-bit buffer, where you can alter one bit only before passing the buffer on to a friend.

If the initial state of the buffer is known beforehand (all zeros, for example), you will be able to send 64 different unique values to your friend.  This means that you can send 6 bits of information (2^6 = 64) by flipping only a single bit of a 64-bit buffer.  This is enough to encode all upper and lower case letters of the alphabet, for example.

Random Initial State

Now, imagine that the 64-bit buffer you receive is initially filled with random information.  This time, your friend does not know the initial state of the buffer.  Ad before, you are allowed to flip only one bit in the buffer.

Now, how many bits of information would you be able to communicate to your friend?

Obviously, you would be able to communicate at least one bit of information (for example, a “yes/no”).  You could do this by agreeing with your friend beforehand that the “yes/no” would be encoded in the first bit, and then you would flip the first bit if necessary before sending the buffer on, otherwise, you would flip a different bit.

Challenge

It turns out that it’s possible to encode a full 6-bits of information by flipping only one bit on a random buffer, with the recipient not knowing the initial state of the buffer.

Note that the same holds true for any buffer size that is a power of 2.  For example, you can encode 8 bits of information in a 256 bit buffer, using this technique.

Your challenge is to figure out what technique needs to be used, and write a program to encode a message in a random buffer of arbitrary size, and a program that can decode the message without knowing the initial state.

The method signatures might look something like:

bit[64] encodeBuffer(bit[64] randBuffer, bit[6] value);
bit[6] decodeBuffer(bit[64] encodedBuffer);

// the below will always be true

decodeBuffer(encodeBuffer(r, val)) == val

Hint

At first, I started down the route of using some sort of run length encoding, but that quickly hit a brick wall.

Instead, think of this as information being encoded in the parity.  For example, you could encode a “yes/no” by saying, if the first 32 bits have an even number of zeros, the answer will be “yes”, otherwise “no”.  To encode the “yes/no” message, you just flip a bit on one half or the other to get the desired parity.

With this hint, you can set about generalizing the solution.

When entering comments, you just use your handle (or the handle of your favorite celebrity) from Facebook, Twitter, or whatever else, and the site automatically finds your avatar.  Of course it works with gravatar, too.

You can get this functionality on your own blog by using the free “Incarnate” WordPress plugin built by our team.  It also works with non-WordPress blogs.

At that time, I immediately defended the name Hulu, calling it a “great name”, based on it’s close auditory association with the candy known as Bing Tang Hu Lu.  (Yes, I beat Alex Bosworth to the punch on that one).  Now with the name “Bing”, we have appropriated the other part of that tasty treat’s name.  I can hardly contain my excitement!  Look at pictures of Bing Tang Hu Lu on Bing!

We should call it “Bing Bing” in China.  And we should hire the actress Bing Bing to be the spokesperson.

Some have theorized that the name comes from Groundhog Day’s “Ned Ryerson – Bing!” skit.  Others think it might be a riff on Mitch Kapor’s old project, Chandler.  Or Monty Python.  Yet others have pointed out the possible reference to the bar in The Sopranos or the sidekick in the Lethal Weapon series.  Jon Stewart has already “Disrespected the Bing!”

While the “bada boom, bada bing” reference seems to be hinted in our marketing materials, I believe that the Stallmanesque explanation probably holds some truth: The name is a recursive acronym, “Bing Is Not Google”.  How incredibly geeky.

His philosophy and approach reminded me a lot of Barfield, but I couldn’t find an explicit link. 

Here is an excerpt from a review of his book Hieroglyphics, describing his theory of “ecstasy”:

"Hieroglyphics" is Arthur Machen’s theory of literature, brilliantly exposited by that "cyclical mode of discoursing" that was affected by Coleridge. In it he promulgates the admirable doctrine that fine literature must be, in effect, an allegory and not the careful history of particular persons. He seeks a mark of division which is to separate fine literature from mere literature, and finds the solution in the one word ecstasy (or, if you prefer, beauty, wonder, awe, mystery, sense of the unknown, desire for the unknown), with this conclusion : "If ecstasy be present, then I say there is fine literature, if it be absent, then, in spite of all the cleverness, all the talents, all the workmanship and observation and dexterity you may show me, then, I think, we have a product (possibly a very interesting one) which is not fine literature."

There is something slightly incongruous about trying to rationally explain why great ideas cannot be communicated purely rationally.  This of course hasn’t stopped Aristotle with Poetics, Coleridge on Wordsworth, Steiner on Goethe, Bandler on Erickson, or countless others.  And Machen seems to practice what he preaches.  For example, this passage from Machen’s “Hill of Dreams”:

"Language, he understood, was chiefly important for the beauty of its sounds, by its possession of words resonant, glorious to the ear, by its capacity, when exquisitely arranged, of suggesting wonderful and indefinable impressions, perhaps more ravishing and further removed from the domain of strict thought than the impressions excited by music itself.  Here lay hidden the secret of suggestion, the art of causing sensation by the use of words."

You can easily see the influence on Lovecraft, who went on to influence Neal Stephenson’s “Nam-Shub of Enki”.  Machen’s friend A.E. Waite and Waite’s Golden Dawn colleagues were familiar with the stories of the Golem of Rabbi Judah Loew, which was perhaps the first variant of the “nam-shub” story in print 50-100 years before Lovecraft, and could be seen as a superior predecessor to Shelly’s “Frankenstein”.

Google Books has the full text of Hieroglyphics online.  The opening is delightful; I’m looking forward to reading the rest.