Enabling small software teams

A fundamental way in which the software industry differs from other industries is that, by leveraging open source, occasionally someone builds a software stack that allows a single person, or a small team, with limited knowledge, to build something commercially useful.

Here are two examples.

  • Ruby on Rails in the mid-2000s looked like magic.  Suddenly a single person, knowing a bit of Ruby but no SQL or HTML, could build a functioning database-backed website in a few hours or even minutes.  It did this through code generation and some simplifying assumptions about what the creator wanted to build.  It had limitations on performance and flexibility, but those very features enabled skeleton teams with limited programming chops to build large businesses by launching on Rails.  Canonical example: Twitter
  • In the 2010s, the best analog might be Node.js and similar JavaScript engines.  Suddenly you could build a whole application, front to back, including sophisticated client-side behavior, knowing only a single language.

The benefit of small teams is to minimize coordination issues, allowing extreme agility, and thereby faster innovation.  But the typical barrier to small teams is the fixed cost to learn multiple software stacks;  it’s rare to find one or two people that understand disparate enabling tech well enough to build a complete commercial solution.

The solution to that is to make a few simplifying assumptions about what people want to build, and then consolidate everything needed to build those things into a single language or framework.

Over time, the magic of these consolidated platforms ebbs.  To build a commercial Rails app today, you need to know a whole pile of non-Ruby stuff.  Presumably it’s the same with Node.js, though I haven’t worked with it.

What will the next consolidated platform be?


Implementation challenges for a carbon tax

Let’s stipulate that carbon exhaust generates a negative externality in the form of climate change.  Further stipulate that this externality is large enough to justify limiting carbon exhaust through a carbon tax and/or cap & trade.

Given those presumptions, how would you implement?  Here are some of the challenges.

  1. How to estimate the value of carbon emissions.  Cap and trade is supposed to deal with this through market pricing.  But that presumes markets have enough information to set a rational price.  Markets set prices through individual agents making specific cost/benefit calculations. The benefit of carbon reduction appears very large — but how large, specifically?  The specific benefit of carbon reduction, measured in dollars per kg, is not only unknown, but unknowable.  This remains true even when we all agree the number is large.  Markets cannot solve this problem.
  2. Prisoner’s dilemma.  All countries are better off if they all follow the rules.  But each individual country has an incentive to cheat, because carbon-dirty energy (coal) is, for now, cheaper than clean energy.
  3. Collusion.  Carbon buyers and sellers each have an incentive to form cartels.
  4. No global enforcement mechanism.  What do you do if a country simply ignores its carbon obligation, or does something like #2 or #3 above?  Since 1945, the United States has been the backstop for global multilateral agreements, backed up by force if necessary, whether in the form of trade, financial or military intervention.  As US GDP as a % of world GDP declines, and as the US population becomes less willing to intervene, this situation cannot persist indefinitely.  There is nothing to replace it.
  5. Incentive to cheat varies by country, due to differences by country in energy intensity, energy cost, energy mix and so on.  Markets are supposed to take care of this problem through cap & trade, but this leads back to the problem in #1 above.
  6. Costs are borne unevenly within countries.  In the most extreme case, a despot in a poor country might sell all his carbon credits and keep all the proceeds, reclining in his solar-powered palace as his subjects shiver in unheated hovels outside.  Actual countries won’t approach this extreme, but it’s easy to see that the greater the concentration of power, the more the benefits of cap & trade will go to leadership, while the costs are borne by the masses.  Therefore, cap & trade probably increases income inequality in non-democratic countries.

Note what this essay does NOT say.  It does NOT say that carbon limitations are a bad idea.  It does NOT deny climate change due to carbon emissions.  It solely concerns implementation challenges, which appear huge.


Skilled work: the missing link

Politicians and even economists refer to jobs as if they were a fungible commodity. “America needs jobs.” “We’re bringing jobs back to America.”

But jobs are not fungible. Instead, an ever-increasing share of employment requires specialized skills. And to an ever-increasing degree, only these specialized skills can sustain a middle class.

This misapprehension about the nature of employment helps fuel the mistaken idea that you can create prosperity through the creation of undifferentiated low-skill “jobs” — which, in turn, contributes to the misapprehension that if you just cut interest rates, prosperity will follow. Which leads to the policy error alluded to in a prior post.

Skill development is the missing link in American employment policy. It’s missing from colleges and universities. It’s missing from public schools. It’s missing even from many technical and trade schools. It’s not prioritized by economists or politicians.

This is mystifying, because obviously, in the long run, prosperity derives from productivity.

In the old days, the industrial era, productivity was fueled mainly by deploying natural resources: if you add more electricity and motors to a production line, unskilled assembly workers go faster, with no change in skills.

It’s different now. Everything has already been more or less optimized for automation through energy and motion.  To make that production line go still faster, you can’t just add more energy.  Instead, you may need to understand statistical analysis, or programming. Further productivity gains require skills.

US labor policy will continue to flounder until this becomes a priority.

Spanish colonial defaults

You’re probably aware Argentina has defaulted on its sovereign debt nine times since its independence in 1816.

You probably also know defaults are historically widespread across Latin America.  At least five former Spanish colonies have defaulted on their sovereign debts nine or more times each:  Argentina of course, and also Venezuela, Ecuador, Costa Rica and Uruguay.

What you may not know is the colonial backstory.  Argentina’s former colonial overlord, Spain, has defaulted on its sovereign debt at least 22 times that I could find:  1557, 1575, 1596, 1607, 1627, 1647, 1652, 1662, and 1666, plus another six times in the 1700s, and another seven times in the 1800s. (source, source)

Are Latin American defaults a cultural artifact of Spanish occupation?  Well, let’s compare to England.  Turns out that six of the ten countries that have never defaulted are England and former English colonies:  Canada, Malaysia, Mauritius, New Zealand, Singapore.  (The United States is an edge case, having paid interest late a couple of times, and having reneged on gold exchangeability of US bonds in the 1930s.)

When it comes to paying debts, there appears to have been a bug in the Spanish cultural program that was passed to its colonies.  What might that be?

It may be related to Spain’s essentially extractive view of colonization:  find gold and silver, ship it back to Madrid, and spend it on extending empire and Catholicism.  When that spending was not enough, Spain borrowed, and used precious metal extraction to pay the interest.  When that still was not enough, Spain defaulted and started over again.  And again.

Meanwhile, France and England developed industries to sell many of the things Spain was buying.  Spain’s extractive approach brought more long-run benefit to Spain’s rivals than to Spain itself.

Thus Spain was a victim of the Dutch Disease long before it afflicted the Dutch.  Instead of oil & gas, it was gold & silver that hollowed out the Spanish economy.  And Spain’s “bad choices” may to have led to the languishing not only of Spain, but also of certain of its former colonies, by transmitting to them a culture of spending rather than investment.

How the IT revolution leads to central bank policy errors

The Industrial Revolution reorganized the economy to replace labor with capital and/or “land” (resources). Progress was easy to measure. In contrast, the Information Revolution reduces the need for capital and resources, through much more efficient allocation. This is hard to measure, leading to policy errors.

Industrial progress was easily measurable by GDP and productivity. Before 1970, a policymaker could be confident that producing more stuff (more capex, more resource use) with fewer people (less labor) would improve quality of life. Therefore, rising GDP and rising productivity were good measures of progress.

That has changed. Free computing and communications instead let you reorganize to increase quality of life while reducing capital, land and labor, all at once, by using those inputs much more efficiently. We can do more with less. This leads to a measurement problem: lower GDP leads to higher quality of life.

What happens when I save money buying used on eBay? My quality of life goes up, because I spent less than buying new. Yet GDP goes down, because fewer new goods need to be produced. Information has replaced industrial production. A macroeconomist, using traditional measures, would wrongly conclude the economy is getting worse, when in fact it’s getting better.

Many of the innovations of the past twenty years look similar:

  • Fix things easily with online instructions like iFixit, so new items need not be built
  • Rent cars, houses, workspaces, so fewer of these need be built.
  • Phone-controlled self-driving cars empower car sharing, so you needn’t own a car

Of course, we can only participate in this increased efficiency if we have skills: reading, computer use, critical thinking. Demand for skills goes up. So we get skilled labor shortages, and unskilled unemployment, at the same time.

What will the traditional Fed chief do when presented with these conditions: rising unskilled unemployment, stagnant GDP, stagnant productivity? Why, they’ll cut interest rates, every time. They are trying to promote capex, to make GDP go up again.

This is a policy error. It falsely presumes that if GDP is stagnant, then quality of life must be stagnant. It also falsely assumes that stagnant GDP means capital is too scarce, when in fact, post IT revolution, the opposite is true. Artificially cheap money actually delays the natural free-market process of using information to replace capital, by making capex artificially more competitive with information.

The real scarce resource in the information-driven economy is not capital, but skills. The policy response to unskilled unemployment is to turn the unskilled into the skilled. I.e. education. Really nothing else will work.

Understanding economic security

Why are people with middle-class incomes more anxious about their economic situation than a generation ago?  The usual answers are either “income stagnation” or “you’re imagining this, there’s actually not a problem.”

I suggest it is something else:  an ever-rising percentage of middle-class expenses fall outside their control, leaving ever less economic autonomy and resilience.

You’ve probably seen this:

The graph is a political Rorschach test.  Depending on your pre-existing political beliefs, you will tend to think the high-inflation items on this graph (hereafter “HI”) outpaced the low-inflation ones (“LO”) because of one of these three things:

  • HI are government-subsidized, while LO are not.
  • HI are unfettered monopolies, while LO are not.
  • HI are produced domestically, while LO are offshored.

Girls, you’re all pretty.  Rather than argue over causes, think about the likely effect.

With few exceptions, LO can all be delayed for months or years in a pinch, while HI are necessities.  You can skip this year’s new TV, or buy one used off eBay.  You can’t delay or negotiate food, rent or a hospital visit.

Compounded over decades, it is easy to see that an ever-rising share of middle-class income consists of costs falling outside your control.

Which feels more secure:  a $50k after-tax income with $25k in non-negotiable fixed costs?  Or a $100k income with $99k in fixed costs?  You’d sleep better at night taking option A — but you can’t, because those fixed costs are not negotiable.  Many or most people with incomes below the 90th percentile have been forced, by the changing expense mix shown in the graph above, into option B:  superficially prosperous, but head barely above water, exposed to the slightest economic dislocation.

Wonder why people under 30 don’t own cars?  Why socialism is suddenly popular?  Why no one has more than a few weeks’ savings?  Why so many are so pessimistic about their future?  This.  This is why.


Income inequality = skill gap

For everyone except the top 0.01% of earners, “income inequality” is actually skill inequality.  To see why, consider two thought experiments.

  1. Suppose the US labor market starts in equilibrium with 10m skilled jobs done by 10m skilled workers, and 30m unskilled jobs done by 30m unskilled workers. Suddenly, we figure out how to train 10m unskilled to become skilled. We now have “too many” skilled, and “too few” unskilled, for the available jobs.  From Econ 101, what must happen to skilled and unskilled salaries?  Obviously, they get closer to each other.  Income inequality goes down.
  2. Suppose the US is in equilibrium with 30m unskilled jobs done by 30m unskilled workers. Then we bring in 10m unskilled immigrants. The new arrivals can perform only unskilled work, but they demand work of various skill levels (doctors, lawyers etc). Thus, even if the new arrivals’ overall impact on the labor market is exactly neutral, they will create a shortage of skilled workers, and a surfeit of unskilled.  Thus the price of unskilled work must fall, and for skilled work must rise.  Income inequality goes up, unless you can do #1 above — train lots of unskilled to become skilled.

This argues that immigration policy and education policy are inextricably linked.  Unless you are able to train unskilled workers, you must not admit more unskilled workers, or you will end up with rising inequality and political instability.


The end of industrial warfare?

Cyber warfare and decapitation strikes may signal the sunset of industrial warfare.

If a cyberattack can shut down a country’s infrastructure — disabling its power plants, refineries and ports, for example — then war starts to resemble a nation-scale ransomware attack.

If small, self-guided drones can kill key military or political leaders, then war starts to resemble a nation-scale decapitation duel.

You think war means blowing stuff up and shooting people, because that’s been true for centuries.  But that’s not the goal;  it’s just the means.  The goal is to force another country to do your bidding.  War, in the most general sense, is “the continuation of diplomacy by other means,” as Clausewitz put it.  

The industrial model was expensive. You had to devote much of your manufacturing base to making explosive widgets;  hiring semi-skilled labor, aka soldiers, to operate those explosive widgets;  and then using them to blow up your opponent’s soldiers and factories until they cried uncle.

If you can instead force another country to do your bidding without spending trillions, without firing a shot, then of course you will. If these new modes of warfare work, they will replace the old model.

This will upset some assumptions.  Under the old industrial model, it was a big advantage in warfare to have more people, more factories, and less constitutional democracy (those pesky voters always want to stop fighting).

But to win a decapitation duel, for example, you no longer need lots of soldiers or widget factories.  Instead, you need a constitutional republic, a political system that is sufficiently codified to continue functioning even when its leaders are repeatedly killed off by surprise drone attacks.  By contrast, autocracies in general, and cults of personality in particular, are highly vulnerable to this sort of warfare.

Sinus infection cure for engineers

TL;DR  Sinus infections resist antibiotics and your immune system by physically walling themselves off with biofilms.  Destroy this biofilm with any gentle surfactant, such as baby shampoo mixed into ordinary saline sinus rinse, and a normal immune system will quickly take out the undefended infection.  You can accelerate this further by adding topical antibiotic to the saline wash *after* first breaking up the biofilm.  Limit future reinfection by deliberately infecting yourself with lactobacillus, in the form of lacto-fermented foods.


Many doctors outside tropical Asia appear to be unaware of certain features of sinus infections, even though the information is widely available from Singapore and elsewhere.

Certain home remedies work well, but are usually so poorly explained or misunderstood, even by their purveyors, that they seem flaky or suspect.

Engineers and scientists don’t want to simply trust.  They want to understand mechanisms.

I suffered, and eventually cured, a ridiculously painful series of sinus infections several years ago — months of incapacitating pain day and night.  This post boils down years of reading and self-experimentation to a single explanation of how sinus infections work, and how to kill them. This is not a how-to-do-it, but just a how-I-did-it.  Your mileage may vary.

Understand these six mechanisms

Biofilms block antibiotics.

  • Sinus infections construct biofilms around themselves as a physical barrier that limits contact with your immune system and antibiotics.
  • Antibiotics and immune cells don’t pass through the biofilm, so the infection survives.
  • Hence oral antibiotics tend to kill every bacteria in your body except the sinus infection.

Surfactants easily defeat biofilms.

  • Biofilms are robust against antibiotics, but vulnerable to surfactants.
  • Johnson’s Baby Shampoo, for example, easily breaks up biofilms, and is specifically designed to be relatively painless on mucus membranes (“no more tears”).
  • Without its biofilm, the infection is defenseless against antibiotics or your immune system.
  • The delivery mechanism for the surfactant is saline sinus rinse, aka “neti pot.”

Sinuses are like your belly button.

  • Sinuses are an exterior but concave surface of your body, like your belly button.
  • Belly buttons get dirty unless you wash them, because of concavity.  Sinuses even more so, because they are more concave.
  • Viewed this way, there is nothing odd about washing your sinuses, and in fact it starts to seem weird not to wash there occasionally.
  • Hence sinus rinse, aka neti pot, is not weird.

Your body is a bacteria farm.

  • Many different strains, some harmful, some benign.
  • Benign bacterial strains limit the growth of harmful strains by competing for resources.

Oral antibiotics carpet-bomb the farm.

  • Oral and intravenous antibiotics kill all strains in your whole body indiscriminately.
  • This eliminates competition for resources, allowing surviving harmful bacteria to come roaring back, this time with little resource competition.
  • Topical antibiotics (unlike oral or intravenous) are precision weapons that don’t blow up the whole farm.

You can replant your bacterial farm to limit future infection.

  • The strains of lactobacillus bacteria found in everything from yogurt to kimchee are harmless to your body, yet compete for resources with other, more harmful bacteria.
  • Eating fermented foods has been shown to spread lactobacillus throughout your body, including your sinuses.
  • At least one strain of lactobacillus (lactobacillus sakei, the one in kimchee), when present in the sinuses, has been shown to limit sinusitis, again by competing for resources.

Sinus recovery plan

  • Buy equipment.
    • NeilMed sinus rinse squeeze bottle and salt packets.  My experience has been that off-brand and store-brand squeeze bottles don’t work right.  NeilMed works.
    • Johnson’s Baby Shampoo (or the NeilMed equivalent, which I haven’t tried).
    • (Optional but awesome) Find a doctor that will prescribe (and a compounding pharmacy that will mix) a small amount of topical antibiotic into sterile saline solution, to use as a secondary nasal rinse, as described below.  I invented this myself and am proud of it (though undoubtedly others have thought of it too).
  • Repeat the following every day, until cured.
    • Fill NeilMed bottle with water, add one drop shampoo and one packet salt.  Shake.
    • Rinse sinuses gently per package instructions.  Don’t squeeze the bottle too hard.
    • Wait 30 minutes for the surfactant to break up biofilm.
    • Repeat the above rinse cycle;  this second round will be much more effective at washing away the infection, because the biofilm is broken up.
    • (Optional) For this second cycle, instead of the normal mix, rinse with topical antibiotic described above.  This will put the antibiotic in direct contact with the infection you just exposed in the first cycle above.  In my experience, this is a neutron bomb, instantly destroying the exposed sinus infection while leaving buildings standing.
    • Recovery takes 3-5 days without the topical antibiotic, or just 1-2 days with it.
  • Replant the farm.
    • Buy kimchee, sauerkraut and yogurt.
    • Only works if it is a live culture:  when you first open kimchee or sauerkraut, gas should escape, and the contents should bubble up a little.
    • Eat these daily.
    • Feed the farm by also eating whatever the bacteria likes to eat.  I.e. for kimchee, eat steamed bok choy.  For sauerkraut, eat steamed cabbage.  For yogurt, drink milk (organic to make sure it doesn’t have trace antibiotic).
  • Redo the above procedures every time the infection recurs.

Things to worry about (and not to worry about)

  • The scare stories about Naegleria fowleri in nasal rinses are wildly overblown.  Yes, there is a nasty waterborne amoeba that kills you if it finds a hole from your sinus into your brain.  However:
    • Naegleria only occurs in warm untreated standing water, above 70 degrees.
    • Only a few hundred reported cases reported worldwide in 50 years.
    • Your odds of getting this from chlorine-treated urban tap water are very low.
    • If you use distilled water, the odds are lower still.
    • You are probably much more likely to die in a car crash driving to the supermarket for distilled water than you are to die from Naegleriasis.
  • Squeeze the NeilMed bottle gently.  If your sinus is blocked, water may flow through slowly, so if you squeeze too hard, water will seek other paths out, including your Eustacian tubes.  This can cause an ear infection.  If you are gentle, no ear infection.
  • After sinus rinse, wait 30 minutes before lying down, to let any water drain.  If you lie down too soon, water can drain into your Eustacian tubes or middle ear, creating conditions for an ear infection.
  • Recurrent sinus infection, as the name suggests, is never fully cured.  When it comes back, you just treat it again, and are fine again for a few months or years.

I do not assert works for anyone else, but it totally, completely works for me.  It seems to have the added benefit of making colds less severe, probably by washing away the nasal component of the infection.

Using my topical antibiotic idea in particular, lasting relief came in 1 to 2 days, after months of chronic sinus pain.

False choices: gun control

As early as preschool, I remember boys arguing about cars. Allegiances fell into simple A/B categories: Ford vs Chevy, Ferrari vs Porsche (our preschool was in Newport Beach, after all).

These arguments were sometimes heated. Yet even then, at age five, it was clear they argued over nothing. Obviously so, in the sense that our feet did not yet reach the pedals. Less obviously so, in that Ford and Chevy were not meaningfully different from each other, and presented a false choice between only two options, at a time when Toyota and Honda were quietly eating their lunch.

American politics today are like my preschool, and not only in their maturity level. Democrats and Republicans are superficially more polarized than at any time since WWII. But polarized over what? Party platforms look like an army of straw men, false choices trumped up to stoke false outrage, shallowly argued, as legitimate issues are ignored.

Take public safety. For most “coastal elites” like myself, public safety means gun control. Let’s stipulate that owning a gun is statistically not worth the risk. That’s why I don’t own one and don’t want one. But that’s not the question. The question is, are guns so dangerous that I should make it a political priority to force my personal choice on everyone else? The answer is no.

Large numbers of voters want their guns. America represents us both; their voices matter as much as mine. The Supreme Court has already ruled for gun rights. The great majority of gun deaths are suicides, household accidents, or inner-city mayhem. With the first two, consequences fall on the owners themselves; with the third, the problem is lawlessness in urban areas that already have tight gun laws, so more regulation would have no effect. High-profile events like mass shootings are statistically not meaningful.

In short, the gun owner in Ohio may be objectively illogical, but I don’t see a compelling reason to abridge his freedom, when both the courts and big chunk of the population are on his side. Fighting over this is an unproductive waste of energy.

While we hyperventilate about this unresolvable and useless gun debate, we leave synthetic opioids untouched. Should they be a public safety priority? Well, they now kill about as many people as guns or car accidents, and are still rising exponentially. Opioids are not defended by the Constitution. They do not provide benefits anywhere near the costs. They have no eager constituency in either party. There is no debate. Everybody agrees they are bad. It should be easy to collaborate to shut them down. Yet nothing happens.

There are many false choices like this — more in future posts.