I wrote the following story to help beginners understand neural networks. This is part one, in which you will learn about the most widely used supervised training technique, backpropogation. In part two I will metaphorize a cutting edge NN training system which has shown even better results than BP. It may take some time to finish part two.

The Parable of Nyeralneht: Part One

Bakh’präp

The illustrious Bakh’präp is famous, even in legend today, as the wealthiest merchant in ancient Nyeralneht. He was known for having invented a system which could predict the number of shekels spice would be worth in the following week. When he knew the price would soon double, he stocked up and then sold once the price had risen.

His journal was only recently discovered, offered at an auction by an old hag who’d found it in a jar. With the wonderful but short time I have been able to get my hands on the journal (those archaeologists are annoyingly protective) I have finally been able to piece together his trading technique.

As was previously well documented, he had a network of informants who reported to him the sentiments of other spice merchants, the changing tastes of the populace, the predictions of disruptive weather, the average daily market price of spice, innovation in the agricultural and nautical technologies related to the production and transportation of spice, and even the volume of spice that was being traded at the Grand Bazaar of Nyeralneht on a daily basis. This was quite a network to manage, but many merchants besides Bakh’präp had networks of a similar sizes and complexities. Bakh’präp’s ability to filter and manage these various inputs was his real secret.

-§-

From his disintegrating journal I have learned that he employed two layers of advisors. There were lowly scouts and spies, some were little children, some even pickpockets, some corrupt government officials, and all the way to beggars on the street outside of his competitors’ distribution centers - his eyes and ears everywhere. The next layer, hidden to observers until my research, was a set of advisors that filtered and aggregated the noisy and seemingly intractable information coming from all the lowly informants. The advisors were trained by the insightful Bakh’präp to base their interpretations of informants’ info on how much they trusted the lowly minion and on how useful they thought that particular type of data was to making their estimate.

The illustrious merchant could never be certain that an advisor was giving him an accurate estimate, whether because their sources were untrustworthy or because the advisors were untrustworthy. To Bakh’präp, it was the same. This placed the burden of ranking the lowly underlings on the advisors. When he averaged each of his advisors’ estimates, he weighted them by his trust for the advisor- in the exact same way they had been trained to do.

-§-

He must have held absolute trust in his system and an utter lack of trust in human subjectivity. In Bakh’präp’s journal I found a table of numbers that appears to have grown over many months. At the top of each column of numbers was the name of an advisor. Each week, the merchant had assigned each advisor a number, either a little more or a little less than the previous week. These numbers quantified how much he trusted their estimates. Whenever the aggregated estimate for the week was inaccurate (causing the merchant to lose money on his trades!), he mostly decreased his trust in the ones he had trusted the most, and grew only slightly more distrustful of those he had not placed much faith in. The less he trusted someone, the less they were paid- so the advisors did the exact same thing as the merchant and placed less faith in the reports of those they’d paid attention to the most the first time, and listened more closely to those they’d dismissed before (paying accordingly).

While reading about his system I conjectured that it would spiral down to mediocrity if, by chance alone, an inaccurate estimate was generated. For then would not the deservingly trusted advisor be demoted and the conniving, worm-tongued vassal be promoted in the false upset? And then estimates would end up even further off, driving down the loyal counselor even further! But my reasoning was flawed. As long as over half of the estimates were accurate, then the trust in the top advisors would continue to rise to a certain equilibrium, where the available information reached its predictive limit. But if less than half were correct, then the ones Bakh’präp was paying the most attention to were leading him astray. So it did not matter that they had been the most trusted- their estimates were useless whether by inaccuracy of information gathered or by deceitful communication and they would continue to fall in favor.

The extensive entries in the journal narrate how difficult it was for the merchant to sometimes ignore the passionate recommendation of long-known advisors. It also shows how stressful it was for him to ignore the rumors which circulated the Bazaar and to only pay attention to his network of advisors. He had to merely trust that some lowly informant had reported the rumors and that it had been taken into account in the estimation. In a way, he was in the dark about the gears that drove his black-box system, but he had faith in the function by which it had been organized so he had faith in the whole.

Bakh’präp’s system is remarkable because it required no real thought once it was set up- so the merchant had plenty of time and money to indulge in a rajah’s lifestyle.