Tuesday, August 4, 2015

Episode 4 is up, so it's time for some cryptic remarks about Silence


There is a sentence in "Silence Like Diamonds"  -- oh,that? How nice of you to ask. The fourth episode is up now, over at Light Reading, and if this is your first time here you can find my explanation here and Light Reading's explanation there. If you really just wanted to read a good story, go read! This will wait!
(Smiling and waving as a few folks leave the room)
RUN FOR YOUR LIFE, IT'S  ABOUT TO BE MATH IN HERE!
(crouching behind podium until pandemonium subsides, addressing the few remaining cool people in the front row) 
All right then. 
As I was saying, there's a sentence in "Silence Like Diamonds" that draws a lot of questions from readers, so I thought today I'd talk about that.  It's in Episode 3, which ran last Friday:
Ever since the Yan-Dimri fast factorization algorithm had flipped the advantage from the encryptors to the cryptanalysts, only isolated systems could be really secure (at the cost of being really useless).
I'm happy to note that in the 72 hours after that, "Yan-Dimri fast factorization" drew a few search hits on Google, so to begin with: I made that up. We call this stuff science fiction, you know?
But I'm guessing that the reason why it drew Google attention was because there are plenty of people who recognize the implications in that sentence; in fact I'm a bit surprised that more people don't.
So to begin at the beginning: since about the 16th century in the European cultural sphere, and no later than 1900 everywhere, there's been a relentless arms race between encryptors (codemakers) and cryptanalysts (codebreakers). Mathematical tools have gotten better and better on both sides, and theoretical understanding of encryption/decryption has advanced rapidly, driven first by the competing great powers in the last half of the 1800s, and then by war and cold war, and now by a huge international information market.
The balance began to swing -- apparently permanently -- toward encryption with the introduction of keyword, master key, etc. systems. I'm skipping over a vast amount of detail here, so real crypto people please forgive all the variations and interesting sidelines that I might introduce.  Here's a too-brief and too-undetailed version of how we got where we are now, and how the world of "Silence Like Diamonds" is different.
Think of the code that small children used to use, where A=1, B=2, etc. Suppose we encode a block of text:
HOW ALL OCCASIONS DO INFORM AGAINST ME AND SPUR MY DULL REVENGE
For our purposes in this example, I'll use 0 for space. Encoded in that simple fashion, the message is now:
8-15-23-0-1-12-12-0-15-3-3-1-19-9-15-14-19-0-4-15-0-9-14-6-15-18-13-0-1-7-1-9-14-19-20-0-13-5-0-1-14-4-0-19-16-21-18-0-13-25-0-4-21-12-12-0-18-5-22-5-14-7-5

To read it, then, the recipient would simply replace the numbers with letters (and zeros with spaces).
Of course  that cipher would be pathetically easy to break, especially with modern gear like any old  spreadsheet. In fact, if you knew it was that simple a code, you'd probably just set up a list of permutations in your spreadsheet (a row where you tried A=1 B=2, a row where you tried B=0, C=1, etc) and read down a few lines till you saw an English sentence. 
Yet systems like that were used down through the American Civil War; you can find Edgar Allen Poe explaining how to break that sort of code in "The Gold Bug".
Many other ideas were tried, but the road that proved most profitable was the one that leads through the Enigma machine, originally created for high-security business cables in the 1920s and then adapted by the Germans during World War II.  Leaving aside all the fascinating mechanics -- and they really are worth your while if you've never read the story -- the math of it was simply that if instead of a simple substitution, you had some number of encoding rules, and a key that told you which rule to use for each successive character, then every character of the clear text would be represented by several different characters, eventually all the possible ones, in the code.  For example, suppose we add the successive digits of  π to the substitution above; then the new code is:
11-16-27-1-6-21-14-6-20-6-8-9-28-16-24-17-21-3-12-19-6-11-20-10-18-21-21-3-3-14-10-14-14-21-28-8-17-6-9-8-15-10-9-22-25-30-21-7-18-26-0-9-29-14-12-9-25-9-31-9-18-12-14
Of course it doesn't have to be  π and it doesn't have to be addition; I could have multiplied each by the successive digits of √5 and then subtracted successive digits of √22, so that the message recipient would then decode by first adding  successive digits of √22, then dividing by the successive digits of √5, and finally substituting that simple numeric code. Important thing to note right here -- you'll see why later -- in this primitive version, it takes just about as long to encode as it does to decode.
This is a considerable improvement over the simple substitution ciphers; Herbert Yardley's codebreakers, in the United States's Black Chamber around World War I, would have had to work at it for a few days, especially with  message this short. Notice that the E's in "revenge" are now represented by 9, 9, and 14.  The spaces are represented by 1,6,3,3,8, 9,9,7,0, and 9 successively, the double L in all is 21-14 but the double L in dull is 14-12, and so forth. With a longer sample (say 1500 characters instead of the 62 here) and a long afternoon (and ideally a spreadsheet), though, a good amateur cryptographer could crack this while barely breaking a sweat. (I'm skipping how, here; let me recommend Simon Singh's THE CODE BOOK as a good place to start  a fascination with cryptography -- I wish it had been around when I first became interested, confidential information ago).
But rudimentary as my little demonstration here is, it introduces the basic tricks of modern encryption: there's a key, there's an operation performed by combining the message with the key, and there's a resulting coded message, which is hard to read without the key, but easy to read by applying the key through a defined operation. As long as the sender and the intended receiver both have the key, and the would-be interceptor doesn't, the would-be interceptor has a fairly-tough-up-to-impossible math problem to crack.
Some things make the problem harder than others. There are patterns in every language -- ETAOINSHRDLU is a famous one, that is, in a long enough passage of random English, E will be the most frequent letter, followed by T, A, etc. If there is also a pattern in the key -- say, for example, it is the digits of  π, beginning over again with 3 for every message -- then in a big enough sample, the combined pattern will emerge and can be teased out, especially if something is a frequent part of a message (you might have seen The Imitation Game and remember what a difference it made when British cryptographers that realized Heil Hitler would occur frequently in the encrypted messages).
But what a complicated world is hidden in that phrase "the sender and the intended receiver both have the key, and the would-be interceptor doesn't."  
 A perfect, unbreakable key would simply be a list of random digits held by both sender and receiver, as long as the list was used once and then thrown away; with no pattern in the key, there'd be no breaking it.  But that would require that every spy, warship, infiltrator, or whatever go out with an extremely long random number list of his/her own, in some cases smuggling it through enemy lines, and never accidentally losing it or having it confiscated. Whenever they can, modern encrypters do use those unbreakable "one time pads" ("pad" because the original WW2 version was a printed, gridded pad of sheets, on each of which you wrote your message on  a top line, added the random numbers from the middle line, and sent the resulting code from the bottom line, destroying it as soon as you finished. For a really fascinating story about how that all worked in practice, see Leo Marks's  BETWEEN SILK AND CYANIDE).
Suppose, though, that you can't send a one-time pad with someone. Perhaps they will be going elsewhere indefinitely, or forever. Perhaps a one-time pad is too dangerous for them to carry (since it's pretty obviously incriminating evidence).  What can you do instead?
For World War II and most of the Cold War, the answer was highly unsatisfactory: you made hard-to-figure-out keys that didn't look like keys. Another example from The Imitation Game, (a great movie if you're not trying to substitute it for reading Andrew Hodge's brilliant biography Alan Turing: The  Enigma): the Soviet mole, who is working inside a far-above-top-secret facility into which he dare not take any suspicious object, was using passages from the King James Bible as his key.  Though it still produces some very difficult code to break manually, presumably there will be recurring patterns produced by overlaps of frequent words. For example, if GERMAN and BEGAT line up every so often (one is likely to be frequent in the message, the other in the key), and if what we're doing is adding the numbers for the two letters, subtracting 26 if necessary, and then converting to a letter, the letter combination IJYNU would show up extra frequently.
The genius of the Enigma machine was that the key wasn't written out as a key, but an initial setting for a machine that then automatically generated the key as needed; each time you pushed a key to encode a letter, three internal rotors turned by specified amounts, reorganizing the substitution of the next letter. To read the message you had to know which three (of a possible 5) rotors were in which order at which starting positions, plus how a plugboard was arranged, and since the pattern was changed every night at midnight, the Germans thought it couldn't be broken at all, given the huge number of possible combinations. But again, it was breakable because there was a pattern to the Enigma's generation of the key, and a pattern to the German messages underneath, and an enormous sample (when there's that much of a war going on, and everyone is talking to everyone else, the message volume is huge).
So the Cold War began in a rough tie between encrypters and decrypters (since the British had shared some of their codebreaking methods with us voluntarily and others with the Soviets via espionage).  One-time pads were unbreakable (and still are and always will be as long as the numbers are truly random) but you had to hand off the pad between sender and receiver somehow, and the pad itself could be stolen, intercepted, lost, etc. Generated keys had patterns in them and were therefore breakable, even with enormously complex and sophisticated key-generation machines (and later, software). If a government, agency, business, or other organization changed keys often enough, and the key wasn't reused too much, and the process of generating it was not too obvious or wasn't stolen by the opposition, then most messages could be kept secret for long enough, most of the time, with the codebreakers occasionally getting ahead of the game because they got bigger, faster computers to run the breaking software on, or because the encrypters made a mistake somewhere.
And then in the 1970s some nice mathematicians came along and put the encrypters so far ahead that they've been there ever since.  They figured out a variety of processes using (what they hope are) one-way functions.
A one-way function is simply a process with numbers that is easy to do in one direction but hard to do in reverse. "Easy" and "hard" might have a genuine universal meaning related to the P=NP question, but for applied engineering purposes, it's always just "relative to the capabilities of the people with the secret and the people trying to get the secret."
In particular, the encryption mathematicians discovered a process in which it was quite easy to encode using the digits of a very large number (say two thousand bits, which is about a 600-digit number in decimal) that was a semi-prime, i.e. the product of two primes, but the process only worked backwards with enormous effort, so the message was hard to decode -- unless you had the "trap door," the value of one of the two primes, in which case it was easy.
So the Chief of the Global GoodORBad Guys Org (GG|BGO) could simply broadcast the semiprime and the encryption rule, and anyone who copied it down could send him a message, with no one else being able to read it.  Using one of the two prime factors that he had multiplied to create the semi-prime, the Chief could decode that same message quickly and easily. If he kept those factors secret, to break the code and read his agents' messages, someone would have to factor the public key.
Furthermore, his agents could simply mail him a public key (presumably generated on their laptops or phones) from any anonymous mailbox or email, keeping their trapdoor primes somewhere concealed (a few hundred digits is not hard to conceal, after all), and when he wanted to reply he could send it by any public channel. (Ever wonder what those numbers stations on the shortwave might be?)
And there matters have rested, because factoring is a hard operation. You can see this a little by just looking at the time it takes to factor a semi-prime by hand: 35 is a semi-prime, and its primes are 5 and 7, as anyone who remembers the multiplication table figured out in less time than it took you to read this. How about 2491? How long did it take you to find 47*53? (highlight to read)  Now, if you're really a demon at factoring, try 307,961.  (If you really must know, that's 547 times 563).
Here's the interesting thing, from a computing standpoint: not only is factoring a hard operation but it gets much harder (in terms of machine time taken) the bigger the number, and in modern crypto, it's just not practical at all.
So that's why RSA and its many cousins, which rely on semiprimes, are nearly as unbreakable as a 1-way pad. And a good thing too, because those methods of encryption are keeping thieves out of your bank account, your boss out of that naughty website you frequent, trolls out of taking over your i.d. and sending threatening notes to the president, and all the rest.  True, if the white hats don't change the key often enough, and the black hats can keep guessing for years or centuries, eventually they might manage to factor that semiprime ... In fact a few very limited tricks are known for doing just that on a few very special classes of large semiprimes (look up Fermat's method for one). The public record so far is for factoring a 232-digit number, which took hundreds of hours of computer time across about two years, and was essentially just a very-well-thought out brute force search.
So ... since it's easy to create those really big semiprimes,  and since factoring takes so long that the key will change long before anyone factors the public key and gets in, it's an encryption paradise. As long as the government doesn't make everyone give them one of the factors and leave all of them in a pile on someone's desk over the weekend we should be fine, right?
Except.
No one has actually satisfactorily shown that there can't be a fast way to factor a very large semiprime.
Except, if it's really impossible, someone ought to have proved it's impossible by now, given how big the economic incentive is.  Or if it is possible, you'd think someone would have done it.
And math problems can have a really long hang time for solution.
Remember, Fermat's Last Theorem took 350 years. The four-color problem stood open for about 130. This Wolfram article on unsolved problems contains many that have been open for a century or more.  
Right now, somewhere out there, mathematicians who would like to have a name in the history books are working on factoring large semi-primes, and other equally ambitious mathematicians are working on proving it's impossible. (Technically, too, someone might show it to be possible without showing how, but that idea didn't lead me to a story in whicht things blow up and people daringly do deeds of derring-do).
If the factoring side wins the race -- which they could do at any time, and it's always possible that a spy service somewhere might already have done so -- instantly,
"... there is nothing covered, that shall not be revealed; neither hid, that shall not be known. Therefore whatsoever ye have spoken in darkness shall be heard in the light; and that which ye have spoken in the ear in closets shall be proclaimed upon the housetops."
Everything. All at once. Private bank accounts and passwords, credit card accounts, corporate records, everything; the only limit will be how fast human beings can absorb and use the information.
There are other encryption methods possible, but most of them are also driven by large semiprimes.
So ... in my imagined future world, two mathematicians named Yan and Dimri came up with a fast-factorization algorithm somewhere between 2020 and 2025. It's a world where all encryption, everywhere (except for one-time pads) is broken quickly; encryption only slows corporate and political espionage down a little bit.  And thus it's a world where people like Yip, Yazzy, Dusan, and Markus are never short of work or things to do; in a world where there are no good locks, there's always a job for a guard.
Do I think that world will happen?
I think no one yet knows (or at least no one yet admits they know) whether it could. So, for the moment, it's perfect for science fiction.
And if I were running a large operation's security, I might deputize a math-literate person to watch the literature on semi-prime factorization.
Anyway, enjoy the story, see you Friday with some other topic, and this time I got to talk about two favorite subjects, math and fiction, so consider yourself lucky that I only wrote this much.

Friday, July 31, 2015

Episode 3 of Silence Like Diamonds is up early, and I was up late and had some futuristical thoughts about communication relay drones.


So once again I'm flogging my serialized novelet over in Light Reading, an I-hope-fun bit of light summer adventure fiction, set in the near future. And since there's another episode up -- Episode 3, "Principle One" --  go ahead, scoot on over and read that! -- I thought I'd talk about something that's already been in the story in Episodes 1 and 2, since Mitch has made dire threats about what he will do if I blog spoilers for my own story. (I won't, Mitch. Really, I won't. Could you release Dad, now, please?)
Since drones are a hot topic in the communications field today, and since the original story request mentioned them with considerable passion, here a few drone-thoughts, not necessarily in any order:
•Crewed aircraft nowadays are limited, more and more, by the crew. A human body can only take so much acceleration, insists on having continuous access to heavy and hard-to-handle materials like oxygen and water, and has a dreadfully slow narrowband interface to its environment, coupled to an internal electrochemical processing system that is even more dreadfully slow. As designers are becoming free of the pilot, amazing possibilities are coming up; there was no point in trying to engineer a 20-g turn that would kill everyone aboard, but now that there's no one aboard, that limit is gone. You couldn't do much with an aircraft that fit into a suitcase if it had to have a cockpit big enough for at least a jockey; now there are already drones out there smaller than most birds.
This trend is only going to accelerate as a new generation of designers comes into the workforce never having had to think about pilots or passengers. I played around with that idea with the Griffon, the super-drone that circles communications hotspots at 35,000 meters* and reshapes itself for convenience, usually shaped like an airplane, but ascending like a blimp, and descending like a dart as needed.
•Which brings me to another potential that isn't yet fully realized by either us sci-fi folk or more serious tech people: the revolution in materials science is just getting underway. For one thing, computer time and storage and speed are only beginning to make real computational molecular design possible to contemplate. We don't even really know what to wish for yet.  I'll give you one I thought about describing for the Griffon and then decided was too long to go into: if you had thin, flexible tubes that could handle the internal pressure required, you could use them to hold a thin, light envelope of some other extremely strong material open ... and thus your balloon could be just an inflated shell with a near-vacuum inside.  Not only does that make for a less explosion-prone, better-lifting balloon filler, but with enough energy and the right gear, you can always make more vacuum -- it might be a long time before you needed to refill the tanks.
•What's so great about a stratospheric drone?  Well, at the height the Griffon is flying, the horizon is about 670 km away -- the one over Arcata could talk directly to San Luis Obispo, Portland, and Reno, almost to Boise. And if it's communicating with another Griffon at the same altitude, that doubles the distance -- from Tijuana to Vancouver BC, all the way out to Calgary and Grand Junction. With a drone over every population concentration, and a few over the oceans, that puts the travel distance for a signal from any point on Earth to anywhere else at about 20,150 km, maximum, which is about 67 microseconds at light speed. Compare that with 204,000 km and 680 microseconds for geosynchronous, and you're looking at an unbeatable advantage.
•The other drone I made up was the Roverino, which Markus describes as "common as crows around a tech town." The idea I had here was that you've got a communications drone the size of a middling model airplane but it's smart; it records billable milliseconds for every internet packet it passes on, relaying that information to its owner, and it wanders around seeking out high-value transmitters. Besides doing obvious things like following always-on-the-internet people around, circling office buildings to add wireless capacity, and swarming to emergency sites or news stories to provide extra bandwidth, Roverinos would be getting much of their traffic from other Roverinos; they really would flock and swarm like birds. Probably, like birds, they'd also learn and adopt different strategies; you'd get some "loners"  and "pioneers" who  would be looking for isolated hotspots they could have all to themselves, lazy "freeloaders" who would simply follow the biggest flock, perhaps even "alphas" that many other Roverinos would follow.
•And of course that's just two possibilities. I think the drone-relay world is going to look more like an ecology than an economy.
•But before we start feeling all Brautigan about being watched over by machines of loving grace: consider too that in a true Internet of Things, anything can be weaponized.  The 9-11 terrorists turned four airliners into cruise missiles (three successfully), but it cost them 20 of their own to do it.  The future is going to look more like the Stuxnet attacks on Iran: one day the centrifuges went berserk and tore themselves apart, effectively shutting down the nuclear program. 
But it's also going to be a future of big data, and that's why I think the principle of stochastic terror will play a bigger role than people are realizing. If you haven't run across the concept before,  "stochastic terror" is the technique of broadcasting or publishing in a way intended to set off sympathetic-to-your-side "lone wolves" or "lunatics" who then carry out violent attacks on your enemies.  Conservatives tend to see it in cases like Mohammed Youssef Abdulazeez and Dzhokar Tsarnaev; liberals see it just as clearly in Dylann Roof and Jared Loughner.
But a converted and riled-up lunatic is a poor weapon compared to a virused drone. First of all, other people notice when another human being begins to consider ultra-violent crimes; their behavior changes and there's a good chance someone will notice and catch them. But a virus can lie dormant until the moment comes.
Moreover, a virus doesn't start to have second thoughts, or get a good lover, job, or medication and start to think it has something to lose.
And finally, a virus is eternally vigilant. So imagine, if you will, that a malign and patient virusmaker sets something loose among the drone population that lies in wait until a national political convention; and then one day, with hundreds or thousands of officeholders, party officials, and activists of one party all in the air as they arrive or leave, all the drones in a city swarm toward the arriving or departing flights, heading straight into cockpit windshields or jet engine intakes (but only of the planes actually carrying "targets", since the system could also know who was on board each one.
Pleasant dreams, everyone!
Meanwhile, nothing that dark is happening in "Silence Like Diamonds." Better go cheer yourselves up over there.
§

*35,000 meters is about 114,000 feet, or 21 miles, for the incurably US-system reader. That's up in the range that the media inaccurately describe as "the edge of space" when crazed engineers parachute from it  or fifth-grade girls send instrument packages up to it on  balloons

Tuesday, July 28, 2015

How nine heuristic rules, a handful of points, links, and curves, and some historical parallels all come together in a serial


Some of you have started reading the serial novelet by me over at Light Reading, "Silence Like Diamonds." Episode 1 came out last Friday; as I write this, Episode 2 is a few hours from being officially live, and that link should already be working for anyone who likes that "advanced peek" feeling.

While the serial lasts, I'm going to try to say something around the time each episode comes out.

The whole trick with this "credible near future" stuff is arithmetic and minimum times. Yip and her sister Yazzy are somewhere in their mid thirties at the time of the story, since each of them have established careers in a difficult field, but they're not grizzled old veterans, and what they actually do for a living (more about that in a future blog) is an occupation that doesn't fully exist in 2015.

The story is set around 2030, plus or minus maybe two years. So right now, Yip and Yazzy are going to some university and will graduate within a couple of years. They're at the young end of Millennials or the old end of Generation To Be Named.

So that's who they are: your kids (if you're my age), the summer intern where you work now, perhaps your students, maybe you on your first real job (assuming you just grabbed your first foothold on the tech ladder).

Now what about the world they're in?

The idea of near-term hard sf is to try to move things out of the lab into the real world, not to pull wish-fulfillments and sheer magic utterly out of fantasy. So this version of 2030 has no city on Mars, no hoverboards, no indistinguishable-from-people androids, no immortality injection and no franchise called "Just Fingers" that specializes in regenerating appendages for accident victims.

The time it takes from research results being announced to full commercial deployment varies a lot more than the time from early 20-something to experienced mid-30s, but there are some predictable aspects to that as well.

Here are nine of my favorite tricks for guessing how far  into the future the deployments of new technologies are:

• Everyone lies about how close lab-only things are to going into application, prototype, and production. It's essential for getting funding, after all. So things that are just being shown off in lab tours now, and are supposed to be shrink-wrapped and delivered next year, will probably happen in three.
 •home products come in at very high cost and fall rapidly in price, deploying swiftly as they do (some of you will remember when a smart phone was called a Blackberry and was a symbol of trendy wealth). It was a long, long trip from the Altair 8800 to the Apple IIe, TRS-80, and Commodore 64, about ten years to get to 5% market penetration -- but ten years after that, the home computer was ubiquitous and a majority of students going off to college were taking a computer with them.
•software and mathematical breakthroughs deploy all but instantly, because it's really just a matter of a proficient coder understanding what the math/logic says and writing it in valid code, and the transition from "one smart coder" to "a hundred thousand script kiddies" is nearly as quick as cut and paste.(This is particularly obvious in data science, where, for example, the time between a mathematician developing a new technique and a package to perform it becoming available in R, SAS, MySQL, etc. is measured in weeks rather than years).
•new business models flare up, go dormant, begin to grow quietly, and then suddenly have already taken over. "Oh, them, they're certainly trendy ... hey, they're still around but it's not exciting like it was ... gee, every year they have more market share, I wonder if ... I, for one, welcome the new masters of our humble economy..." For many years entrepreneur-coaches have used the example that anyone can make a better hamburger than McDonalds but making a better business model is something else again; I'd go farther and say that often even long after there has been a spectacular success, it's very difficult for most companies even to plagiarize a great business model. So no matter how advanced the tech, it will still have to be distributed via a large number of companies that don't know what they're doing, don't do it very well, and really aren't sure what they should be doing.
•"the street finds it own uses for technology," as William Gibson taught us all. Just the other day I saw an ad for a singing teacher who offers rates for "in person or Skype." I very much doubt that the developers of Skype intended that, any more than the developers at Google, Amazon, YouTube or Craigslist had any idea what else they'd be doing in the world. Whether it's as humble as WD-40 and duct tape, or as high end as DNA sequencing and nuclear magnetic resonance, somewhere out there someone will make it do something its creators never thought of.
•exponential growth without limit is exciting for sci-fi writers, but it is vanishingly rare; far more commonly, what looks like an "explosion" is merely the middle part of a logistic curve,
Wil McCarthy wrote a pretty funny alternate history a few years ago by treating the 19th century explosive growth and  connectivity of railroads as if the curve had been exponential rather than logistic. John Hersey's famous My Petition for More Space is another exponential that should have been logistic, about population growth (where do people in that world find time and space to do all that reproducing, and what exactly are they eating?). Infamously, the city of New York forecast in 1894 that within a few decades, all its resources would be needed just to haul horse manure out of the city (as we all know, this didn't happen to New York. It happened to Washington and Hollywood). The exponential curve may be fun and dramatic and sexy, but the logistic one is the one to bet on. What's growing rapidly today may saturate tomorrow and become as common as phones.
•almost as fun and entertaining as exponential growth is the often-forgotten possibility of reversal.  Nuclear arsenals grew at apparently exponential rates for more than a decade, but today there are only about as many as there were in 1960.

Certain high-fashion products show similar patterns: plumes for ladies' hats, plus-fours, and white plastic go-go boots.  And in the larger scale of history, there are the concluding lines  from Andrei Amalrik's Will the Soviet Union Survive Until 1984? (written in 1969, another attempt at a 15 year forecast):
Meanwhile, we are told, Western prognosticators are indeed worried by the growth of the cities and the difficulties brought on by the rapid pace of scientific and technological progress. Evidently, if "futurology" had existed in Imperial Rome, where, as we are told, people were already erecting six-story buildings and children's merry-go-rounds were driven by steam, the fifth-century "futurologists" would have predicted for the following century r the construction of twenty-story buildings and the industrial utilization of steam power. 

As we now know, however, in the sixth century goats were grazing in the Forum just as they are doing now, beneath my window in this village. 
What appears to be growing exponentially today may tomorrow reverse and become as dead as cassette tapes.
•"Information wants to be free" was not about politics, as is clearly shown by the very next sentence Stewart Brand spoke: "Information wants to be expensive." The root of this contradiction is that the value of information depends on who else has it and what they do with it, changes whenever it moves, and is catalytic for an enormous number of other processes. Information behaves rather like money with an unreadable expiration date or a randomly tested transaction limit; every time you gain value from it you risk making it worthless, and inevitably, sooner or later, something does.
•forecasts about coming ages and changes in human culture are generally rooted in observed truth, but nothing ever comes out as predicted. Robert Heinlein made up a nice little parable about the lab where they created four escape paths of exactly the same difficulty from a cage and put an ape in to see which one he'd use; the ape escaped the fifth way. Collectively, humans are that fifth-way ape; we often have some idea of where we're going but we rarely go there by the way that seems obvious, and when we get there, it doesn't look like much like what we thought it would.

I used all those tricks at one point or another in devising a world for "Silence Like Diamonds," but for the moment let me just talk about one way I used the last one.

You may have run across the currently-trendy notion of the Internet of Things, the idea of having almost everything in your physical environment wired to talk to almost everything else in the house. Eventually, presumably, your car (which is driving you home while you finish some paperwork) phones your house. It alerts the air conditioning that you are hung up in traffic and to delay coming on till you clear that bottleneck on the interstate. The refrigerator and blender start making up something soothing (and the fridge issues a call so that the drone from the local ice cream store schedules a restocking delivery). The music player queues up the "calm and happy" mix, the shower starts warming water in the reserve tank, and so forth. 

But even away from home, your phone records (by checking RFIDs or their successors) all the things you look at in the store, and reports it to the store chain's intelligence system (with the phone company collecting a small fee).  Face recognition software in half a dozen businesses around your workplace pick you up from security cameras and know what way you take from the parking lot to work. Knowing who walks by and what they like, stores on your route put more clothing you like in their windows. 

All the gadgets in your life gang up to make you happier.

It always strikes me that in most such scenarios, we assume that the human being is a petty tyrant having a bad day. The machines either sound as if they are living in fear of our rage, or acting as very large comforting invisible nannies who will have our milk and cookies and a big hug ready for us no matter how bad the world has been, or like a neurotic parent desperately trying to please a tantrum-prone jaded brat.  It all seems very infantilizing.

Compounding that infantilization, to give you everything you really want, before or at latest when you want it, an Internet of Things has to know everything you want or might want, including the wants you can't admit to and the wants you haven't felt yet and the wants that are only just emerging from the creative software of the marketing people. The liberal-feared surveillance state and conservative-feared nanny state are both much less intrusive than the probably-purely-commercial instant-gratification state. But of course, it's all much less scary because it's not about preventing you from doing what you want (like the cop on every corner), or making you do what someone else wants (like the nanny in every bedroom). No, not at all.  It's about always giving you what you want as soon as you want it, and knowing enough to do that.

Like Santa. Remember, he sees you when you're sleeping, he knows when you're awake ...  (As Allan Sherman remarked, who did he think he was, J. Edgar Hoover?)

So you've got a world that runs on the ability for everyone to quickly know anything that is public -- and the value of keeping information private is astronomical. The relationship between cybersecurity and security breaches becomes something like the relationship between health care and death: you can buy huge amounts of the former, and make the latter very unlikely, but the reaper (or the hacker) always wins in the long run. In such a world, the number and variety of communications security services and systems is going to go through a "Cambrian explosion" -- the evolutionary phenomenon that when there is a drastic increase in the variety of niches, all sorts of strange things grow, making more niches in which more things grow --

Until the reversal, or until the first derivative of that logistic curve starts to bend downward.  There, that ought to be a cliffhanger to hold you for a while ...

Friday I'll probably talk a bit about the drones that have been a big part of the plot so far, since I've gotten a few emails about them in the last couple days. After that ... well, information wants to be expensive.  And time is money.  So after a bit more time, more information.

Thursday, July 23, 2015

Tomorrow morning: Fresh serial!


First of all, this is all about promoting my serialized novelet, "Silence Like Diamonds," which begins appearing on Friday, July 24, in Light Reading, which is a moderately serious online journal covering the tech-management interface in the advanced communications industry (when it was founded, optical was the hottest area; they've broadened a bit since). So if you'd rather just read some fiction, get on over there;  the web link is live now, so you can get a sneak peek, the equivalent of crawling under the tent into the sideshow while the geek is still saying his bye-byes to the chickens.
And with that promotional bit taken care of, I shall now digress all over the landscape of the ruins of my once-active mind, as is my wont.
A long time ago I wrote a long-forgotten blog post about the difference between novellas and novelets.   I think it still holds up.
The gist of it is this: the magazine length standard originated back when reading was the main off-work entertainment for a very large (compared to the present) fraction of the population. Fewer people were able to read in the 1910-50 era (though perhaps not as few as some people imagine), but the readers of the time were at least proficient decoders, they didn't have a lot of alternatives, and they read a lot.
So magazines, knowing that they had to produce reliable entertainment to fit into people's lives properly, very sensibly started labeling stories with their length.  "Short story"= read at one sitting, i.e. an hour or so.  "Short-short," read on your fifteen minute break. "Novelet" or "novelette"* meant "after dinner before earlyish bed or a good radio program," or perhaps "read over a few trolley rides." Novella meant "long Sunday afternoon" or "about a week of commutes."
But the words themselves came from the literary sphere.  I guess I'll just quote myself from the earlier post; I don't think I'm going to say it better this time:
Novelet: Novelettes, in the 19th century popular press where the word was popularized, were originally "good parts versions" of adventure stories – all the action scenes (action broadly defined – not just explosions and fights, but also kisses, quarrels, revelations, oaths, all that other stuff that is memorable in a book) with just enough narrative summary between so that the reader could follow the story – lots of do and minimal be.  You could call them self-abridgements of never-finished novels, and because they were a way to present blood and thunder in a small package, oriented as much toward pure entertainment as any form ever has been, a stain of disreputability used to cling to the term.
Novella: Novellas, on the other hand, were conceived as a kind of fusion between short stories and novels; their origin is much farther up the brow.  A flock of artsy-serious types in the 1880-1920 era thought short-story single powerful effects were great but wanted to do them with novel-like complexity.  It turned out you could do that, but it was pretty hard to sustain at the kind of length that you find in Dickens, Thackeray, or Trollope (even Dickens couldn't – A Christmas Carol  is a novella).
Novellas became a somewhat awkward form commercially (which only enhanced their prestige) because they made for a too-slim volume for book buyers (who wanted to make sure they were getting enough literature per expenditure) and too long a piece for most magazines (whose readers wanted variety, something harder to give them if you let one novella take up room that could be occupied by five to seven short stories.)  It's a heavy-on-the-be form in which a dense structure of meaning is laid onto a few interesting incidentes (sometimes only one).  Think of how much The Secret Sharer, Beyond Bedlam, or The Last of the Winnebagos revolve around what it's like to be standing there in the moment when a conventionally honest man makes a self-admitted killer his best friend and confidant, when several people who are by our definitions mad come to realize how much they prefer what we call madness to what we call sanity, or just to be the owner/keeper of one of the world's last dogs and to have to cope with its death. 
As longtime readers have probably noticed, I tend to think that the real distinction should not be length at all.
Novelets are about a high speed ride thrill ride in which we skip most of that "makes it real" state of being stuff and just get to the stuff blowing up. Novellas are about a state of being a particular person at a particular time. You can mix them, for sure, but you tend to gain words as you do, and end up at the novel, which is not what I'm talking about today.**
Well, a few weeks ago, Mitch Wagner, who is a general purpose cool guy and one of my favorite editors to work with, called me up with an idea; we hadn't worked together in a while, and he'd come up with an idea for livening up things at Light Reading by adding some near-future hard sf, maybe playing around with some of the ideas that are in the labs now and will be busting out to disrupt all our lives***  The unusual venue and form dictated a few things:
•it would be published in a tech magazine with strict space limits,
•they wanted a serial to keep the regular readers looking in during the slow summer months
•they wanted science fiction to lure more of the techish audience to see what they're doing at Light Reading****
I was apparently a good guy to talk to because I've written a certain amount of science fiction, some of it hard and some of it adventurish, and moreover I'd recently done some tech journalism covering communications issues (example).
So we cooked up the basic rules: Exactly ten episodes, as close to 1000 words each as possible (to be broken someplace very close to the middle), some kind of cliffhanger at the end of each episode, and a fair bit of tech speculation over the whole thing.  
And to my surprise, I found that was fun to work within.  It's like haiku, sonnet, rondel, sonata, twelve-bar blues, or the well-made play; the form is strict but it somehow pushes you into  creating rather than strangles your drive.
Quintessentially, those rules pushed me into a real old-fashioned, literary-not-word-count sense novelet. At least  I think so. There's actually enough story rammed into there for a short novel, and I spend as much time in the action scenes as I can make myself do, but the whole thing can be read by a quick reader in around an hour.
At the end of that post a few years ago, I found myself wistfully saying that I wasn't reading enough of the old-fashioned kind of novelets, i.e. the ones where the definition wasn't about how many words, it was about the excitement and the lighting-fast display of exciting scenes.***** And there's an old saying in the writing/publishing business that a writer does pretty well if they write the sort of book they themselves want and can't find. (I suspect this is true if you're always looking for great big romps full of sex and violence. On the other hand if you're looking for the story of a romance between two shy, grumpy, older bus drivers who are running against each other for a position on the board of their church ... well, write it, then. Prove me wrong).
Meanwhile, though, while you're planning Love on the Rush-Hour Crosstown, you might want to go over and check out "Silence Like Diamonds." More about that soon -- along with stray thoughts, math, the usual sort of Approachably Reclusive stuff.
§
*wonder if anyone spelled them differently according to the gender of the reader, or the protagonist, or the writer?
**kind of like you can mix the energy of garage-band rock and the vocal and production technique of the "American songbook" performers, and where you end up is called pop and it's a whole other subject.
***possibly for the better. I myself am quite fond of horseless carriages, antibiotics, and movies in color.
****quite a lot of good things, by the way; after you finish each episode, take a look at the sidebar -- there's a lot more cool stuff in the future than I had room to put into one story!)
***** in my first draft I typed "the length didn't matter, it was the rapid, intense, continuous motion." Probably should have left it in, but I thought you might be distracted. But isn't distraction the point of  entertainment? Is it time for lunch?

Sunday, June 7, 2015

A post with nothing to do with math but something to do with smart kids with problems


I have a story, "The Soul Remembers Uncouth Noises," in Steve Stirling's The Change: Tales of Downfall and Rebirth anthology.  For those of you who don't know the genre publishing racket, successful franchises (and Steve's Emberverse is a very successful one) eventually lead to people other than the original author writing in that world.  There are lots of reasons why other writers will do that, but the only one that matters for you-the-reader is that writing in someone else's world is fun.


See? Knight in armor, Plains Nation warrior, wrecked helicopter. How much more fun could you want?





Of course, all the fan fic folk out there could tell you that writing in someone else's world is fun. Some of the fanfictioneers write in other, established worlds as a bridge to creating their own, but most of them are well aware that they'd probably be doing themselves more good by creating their own right from the beginning.  The truth is, at the bottom, writing in a world someone else has created is fun.  That's the one reliably good reason to do it. 



For a longtime professional fiction writer, it's almost exactly as challenging as the writer wants it to be. The challenge I set myself was to take Steve up on one of his observations, that he'd figured that if there were a truly strange apocalypse, the survivors would also be the truly strange, and the postapocalyptic culture or cultures would be formed out of the weird fringes of our preapocalyptic world. 

For the past decade or so I've been dipping a toe in the increasingly-popular Young Adult waters, so I liked the idea of teenage characters surviving in a world where adults didn't.



Furthermore, one way and another, I've become interested in twice-exceptional kids -- the category that could probably be more honestly called "weird geniuses," children and adolescents who are unquestionably gifted in one area with major difficulties in another: math or music prodigies with severe dyslexia, fourth graders who have twelfth-grade reading skills but tantrum like two year olds, and so forth.  To me, anyway, one of the most interesting things about the twice exceptionals is not their difficulties, which tend to be obvious, but their ways of coping with them, which are wildly diverse and creative. 



Also, school-age twice exceptionals tend to form close friendships with each other. Part of this may be that there are increasing numbers of programs for them, so they meet there. A bigger part, I think, is that two kids who feel like aliens, though their gifts and problems are very different, are more likely to establish emotional rapport with each other than they are with more typical people with whom they share a gift or problem.  Somehow or other being regarded as weird, and having trouble explaining yourself to the world, is a more foundational experience than merely being extremely good at some things and extremely poor at others.



So I set my story, "The Soul Remembers Uncouth Noises," in the part of Denver where I live, on the day of the Change.  (For those of you who haven't read Dies the Fire or any of the other Change/Emberverse books: the world of the books diverges from ours because at 6:15 pm Pacific Time, on March 17, 1998, all over the world, electricity, explosives, internal combustion engines, various other such high-energy-density systems stopped working abruptly).  I put together three twice-exceptional ninth graders, gave them just enough luck to get started, and thought about what might happen to them and who they might turn out to be.



Now, that's a very contemporary YA kind of story, the Understanding Difference story. And the frame story is actually pretty much a stock "cavalry western" (that is, back when westerns were a big part of pop literature, there were several subgenres named after who the main character would be: lawman westerns, cowboy westerns, gunslinger westerns, etc.).  For various reasons I don't think Steve will be doing any contemporary Understanding Difference YA soon, but sure enough, there was plenty of room in the Emberverse for one, along with a cavalry western.  And as some of you may know from my notes about the Daybreak books and the Jak Jinnaka books, I deeply love the idea of a fictional world big enough to tell any kind of story you like.



And it was fun.  Lots of it.  That's what matters.



You should go buy that book, and Dies the Fire if you haven't already yet, and lots more.  Read it so that one of these days, when it's a miniseries, you can smugly tell all your friends how much better the book was.