Fictional Text Analytics

There’s a great moment in John Scalzi’s Redshirts where statistical analysis is mentioned, and it comes down to comparing texts:

“So what you’re saying is all this is impossible,” Dahl said.

Jenkins shook his head. “Nothing’s impossible,” he said. “But some things are pretty damned unlikely. This is one of them.”

“How unlikely?” Dahl asked.

“In all my research there’s only one spaceship I’ve found that has even remotely the same sort of statistical patterns for away missions,” Jenkins said. He rummaged through the graphic elements again, and then threw one onto the screen. They all stared at it.

Duvall frowned. “I don’t recognize this ship,” she said. “And I thought I knew every type of ship we had. Is this a Dub U ship?”

“Not exactly,” Jenkins said. “It’s from the United Federation of Planets.”
Duvall blinked and focused her attention back at Jenkins. “Who are they?” she asked.

“They don’t exist,” Jenkins said, and pointed back at the ship. “And neither does this. This is the starship Enterprise. It’s fictional. It was on a science fictional drama series. And so are we.”

Science Fiction Stories by Keith Laumer in the Public Domain

If like me you found yourself very excited by the news on Boing Boing that possibly as much as 80% of the books published between 1924 and 1963 might now be in the public domain thanks to their copyrights not being renewed, then like me you also clicked on the links to the New York Public Library’s explanation as well as Leonard Richardson’s discussion. What was most exciting, to me, to discover was that a fair amount of science fiction from that period, which includes the so-called golden era, might be in the public domain.

That’s the good news. The bad news, or the news that requires patience, is that you still have to track down that work, much of which hasn’t been scanned. Some of it has been scanned, and is possibly available through the Hathi Trust, but it hasn’t been OCRed and curated into clean digital versions. But some of it has, and in the case of some work by Keith Laumer, a favorite of mine, it’s available on Gutenberg.

The list of texts is all sitting on one page, and it’s only 12 works, so writing a BeautifulSoup script seemed like overkill, especially when my preferred plain text note application, [Bear][], does a terrific job of turning HTML into easily edited markdown. From there, I edited the URLs following the pattern I gleaned from one of the texts using Textmate’s block edit functionality. I got the following list:

I saved it to a file, cded into my texts repo and ran wget:

wget -w 2 -i ~/Desktop/laumer.txt

A half minute later it was done:

FINISHED --2019-08-14 19:01:18--
Total wall clock time: 24s
Downloaded: 12 files, 1.2M in 1.4s (874 KB/s)

You can do the same, or you can grab the collection of plain text files out of my GitHub texts repo: Laumer stories.


A Culture GSV and Its Escorts by Ex-Pacifist on DeviantArt.

An imagining of what a Culture GSV looks like by Ex-Pacifist on DeviantArt.

About once or twice a year, something happens that reminds me of something from an Iain M. Banks novel, and I go that spot on my bookshelves, actual or virtual, and pull one down to re-read. This past month or so, Excession has been my concern, and, as its name suggests, I found myself wondering how I could have missed so much of the novel the first time?

Banks’ Culture novels present a post-scarcity civilization of mixed humans and various ranges of artificial intelligences. At the top of the proverbial heap, and largely running the show, are the Minds, vast intelligences usually tasked with operating a variety of sizes of ships, orbitals, hubs, and other facilities that range in size, in terms of complexity, from small cities (in space) to entire planets. At the top of the hierarchy are the Minds that are also the largest ships in the Culture, the General Service Vehicles (GSVs). The capabilities of the GSVs, as is revealed throughout the series but is also the focus of Excession, are virtually limitless.

Despite their being the true power behind the throne, much of the novels in the Cultureverse are told about and through humans. Speaking of that perspective, it should also be noted that, for the most part, the humans involved are rather close to humanity as we understand and experience it. So, while Culture humans have the ability to change form, change sexes, and lead lives several hundred years, and possibly up to a thousand years, in length. Most of our guides through the Culture are happy to stick to the basics, with the occasional need to gland something to make them more calm or more quick to respond, or whatever is called for by the current situation.

Some human stories are a part of the braided narrative of Excession, but the novel is also fascinatingly most concerned with the actions of various Minds, whose awareness of the shifting nature of it all, especially politics, seems to be what plagues them. The eponymous entity ostensibly at the center of the story, the Excession, seems more a MacGuffin than anything else, save a postscript at novel’s end, but its appearance somewhere within the Culture’s zone of influence is what puts a large number of diverse plot points in motion. (Spoilers ahead.)

In Banks’ Cultureverse, ships move about by tapping into the energy grids that lie “above” and “below” reality as we know it: thus above real space lies “ultra space” and below it lies “infra space.” (The obvious analogues here are the two kinds of radiation that lie just outside the human-visible spectrum we call light, ultraviolet, at a higher frequency, and infrared, at a lower frequency.) Either grid can be tapped, but not both at the same time. The object that gets dubbed the Excession appears to be able to tap both at the same time. This appears to be the limits of its expressibility. Initial attempts to communicate with it result in the loss of the ship. Thereafter, the various ships that come to witness the Excession for themselves keep their distance.

Despite the desire of the Culture minds that the excision might bring some of the known “Elder” civilizations to the scene, the only other civilization that seems interested are the Affront, a brutish, egocentric species that lives to establish their superiority through a variety of means, the more painful the better. The Affront represent, for some in the Culture, a mistake in need of mending, and so the appearance of the Excession, with its possible promise of unknown technological riches, is allowed to lure the Affront into war with the Culture, a war the reader knows, and the Minds involved know, from the first novel in the series, Consider, Phlebas, that the Culture will inevitably win.

One of the concerns of Excession is the Culture’s impulse to intervene in the affairs of others. The goal is always, of course, to make them less brutish, militaristic, and prone to see others as either potential slaves or cannon fodder: in short, to make them more cultured, and the irony of the verb form is not lost on Banks’ and his Culture denizens. There are glimpses of reconsiderations of, and possible recriminations for, the active peace-making in which the Culture engages, in other Culture novels. Echoes of the Idiran war echo in Look to Windward, with one of the Minds being particularly wracked with guilt by events that took place centuries ago. There is also a hint of the Culture having gotten into trouble for its busybody nature in Surface Detail.

But only in Excession is the matter taken up so centrally and by the players who seemingly matter the most but who are largely a backdrop in other novels, the Minds themselves. Like any good speculative fiction opera, there are plots and counterplots and plots within plots and subterfuges. What fascinates is how Banks manages to make matters both mundane enough for the reader to follow as well as surreal, quite literally, enough that we recognize in some fashion that the experiences of the Minds is necessarily not at all like our own. (He drops his guard here regularly when he resorts to analogies like “The giant ship watched the Excession, still billowing out towards it. For all its prodigious power, the Sleeper now felt as helpless as the driver of an ancient covered wagon, caught on a road beneath a volcano, watching the incandescent cloud of a nueé ardente tearing down the mountainside towards it.” Really? A ship with a Mind that does “metamath” for fun and which has, we are told, just finished constructing in record time and with complete stealth some thirty thousand war ships and its first impulse is to imagine itself in a covered wagon?) The problem is, of course, how not like us the Minds are, and yet with their politicking and guilt, they are like us.

And guilt plays a large role in Excession, as it does elsewhere in the Culture books — it is perhaps one of the central themes for Banks, the one emotion a post-scarcity world is sure to feel, one supposes. At least two Minds destroy themselves out of guilt, which is not unlike the Mind that does so in Look to Windward. In the case of all three Minds, they have been guilty of participating in war. In all three circumstances, there were possible mitigating circumstances: the Affront do seem rather horrid and there more horrible impulses would be better curbed than continued to be let loose upon any sector of the galaxy. But, it seems to be the case, in the overall trend of the Culture novels, that it’s better to achieve such means through skullduggery or a bit of carrot-and-sticking. Given, how many of the novels are about the exploits of, willing or unwilling, agents of Contact and/or Special Circumstances, Excession is one tale in which the Culture’s inner workings on front and center and it would appear that they are as willing to dig into their own skulls as those of the civilizations which they seek to improve through their “involvement.”

Rogue Considerations

We took our daughter and a friend to see Rogue One a few nights ago, and it turned out to be one of the better offerings in the Star Wars franchise. Perhaps none of the films will live up to the promise, the hope (more on this in a moment), of A New Hope, but, in my current moment, I would put Rogue One up there with The Empire Strikes Back as offering the promise of a continuing story which has a nice mix of characters and world(s) and a larger story, or braided collection of stories, to be told.

That is not to say, however, that the film isn’t without its problems, many of which are inherent in the Star Wars universe itself. I’m going to put aside the light saber, which is both ridiculous and cool at the same time, as a necessary fiction, like faster-than-light travel, and FTL communication, is to much of science fiction. What I can’t put aside is the weird reliance that Star Wars has on things like crystals. Early on, we learn that Jedda is being mined for its crystals, because that’s the power behind the Death Star’s weapon. It felt like a throwback to the original Star Trek‘s "dilithium crystals" as the basis for power. Even worse, at some point a minor character says something like "these crystals are the power behind the brightest suns" or some such nonsense. In a universe that already has ion cannons and some sort of fusion drive, we need to have crystals, too?

In the same vein, the SW franchise seems to be sticking with the need for its big bad weapon to be an energy weapon — at least with the Death Star, both the weapon and its target need to be in some kind of physical proximity, unlike The Force Awakens where a radiation-based weapon, which we see as light, can cross trans-galactic distances at hyper-light speeds. Alas, in Rogue One our planet-killer is not quite up to speed and it can only kill cities, but, oh, this is impressive. So, we are led to believe that a civilization based on advanced technology has no knowledge of an atom bomb, which is an effective city killer and at a fairly small cost, all things considered; nor is it aware that simply speeding an asteroid of a decent size will accomplish the same thing? (This is something that Babylon 5 got exactly right, and the scene where one of its main characters stands and watches his civilization’s fleet shoot rocks at another civilization’s home planet is quite effective in capturing the mixed emotions of destroying a fellow civilization.) Worse, the empire functionaries, here in the form of a CGI-revenant Peter Cushing as Governor Tarkin, seem reasonably impressed with the results. In reality, if all you could do is destroy a city from space, given what the empire has spent, your empire overseer should be pretty pissed.

But let’s put under-imagined — and that’s what it is, just flat out under-imagined and uninspired — science and technology aside and discuss a few things that are central to the Star Wars universe, woven into the very fabric of its plots to the point where it becomes almost ideologically necessary, it seems. There are two, one major and one minor: the major device is orphans; the minor, revenants.

At this point my wife walked into my study and shook her head, noting that orphans are a very common narrative figure / trope and that I shouldn’t hold Star Wars responsible for killing off parents on such a scale that Obi Wan Kenobi might very well feel a disturbance in the force. If we begin with the fictional chronology, we have, of course Anakin Skywalker, who is already sort of orphaned — the mother seems a pretty minor character — and who gets officially orphaned by the third episode of the series.

That leads us to the orphaning of both Luke and Leia, the former of which is raised by an uncle who is never explained — we have to assume he’s either Anakin’s (older?) brother, who’s a complete bum for letting his mom rent his younger brother out to pay off debt or he’s a great uncle by being the mom’s brother or he’s just sort of a avuncular character that Obi Wan knows, likes, and trusts with a kid because, well, Tattooine! (The SW need for desert planets is something we can discuss another time.)

So Darth Vader is an orphan, and, as it turns out, Luke isn’t really an orphan since he had a dad the whole time, but then dad gets killed, as does another orphan’s dad, Galen Erso, father of Jyn, and thus, as the father of a Star Wars hero, doomed to die. The heroine of The Force Awakens, Rey, is also an orphan, making her way through another desert world, Jakku, all alone, only to discover she has a bit of family, and, oh yeah, she may be the daughter of Luke?

While you may have your doubts about Jedi parenting — after all, one could argue that Qui Gon and Kenobi do a terrible job of wrangling the terrible teenager Anakin and the galaxy pays the price (Oh! What a millennial he is!) — what you cannot doubt is that having a family means you’re aren’t going to be having a grand adventure any time soon. Granted that a character being orphaned is simply a way to dramatize that feeling of being alone that all of us encounter, and our loneliness may actually be considered a strength. Star Wars has turned being orphaned into something like a fetish. If I were a kid in a galaxy far, far away, I’d want to get rid of my parents to increase my chances of getting in on some adventure, hopefully the evil empire ending kind of adventure, but whatever.

The minor fetish, er, regular plot device in Star Wars is, of course, the revenant. Darth Vader is our prime example — and was no one taken between the sadness of old Kenobi remembering in A New Hope that Vader killed Luke’s father and the hacking at limbs of the young Kenobi? Maybe it was just me. The business of bringing people back from the dead was brought home to me when we got to witness Vader in one of the life support tubes — perhaps left over from the second or third Alien film, but it was highlighted even more watching the creepy CGI version of Peter Cushing — wouldn’t another Grand Moff had done, and he, or she, could simply have said, “When Grand Moff Tarkin gets here, he’s gonna be pissed.” This is something older films get right: oblique is better than the creepy computer zombie of a beloved character actor. That goes for zombie Princess Leia as well: just have a woman in the white costume glimpsed only from behind. The audience will get it. (Lucas, and now Disney, has never had much confidence in his audience.)

And how are they going to bring back Kylo Ren from the blowing up of the death planet thingy? He’ll come back. The same way that there’s some talk of the Rogue One character re-appearing in the next Star Wars films as the Knights of Ren. Because why complicate things with new characters when you can recycle familiar ones?

Iain M. Banks’ Science Fiction Novels

## The Culture series

1987 – Consider Phlebas
1988 – The Player of Games
1990 – Use of Weapons
1996 – Excession
1998 – Inversions
2000 – Look to Windward
2008 – Matter
2010 – Surface Detail
2012 – The Hydrogen Sonata

## Other science fiction novels

1993 – Against a Dark Background
1994 – Feersum Endjinn
2004 – The Algebraist

Re-Reading Iain M. Banks’ “The Player of Games”

I remember the prominent display that Iain M. Banks’ _The Player of Games_ enjoyed in the London bookstore from which I bought it. I remember the general layout of the store, and I have a sense of the street onto which it fronted, but I don’t remember the reason why I picked up the novel. I was looking for something to read, and I guess something about the cover appealed to me. I had not heard of Iain M. Banks previously, and while perhaps some part of me now wishes I had known about him sooner, it does not detract in any way from all the enjoyment I have received since from reading his Culture series — nor from the sadness I felt at learning he was leaving writing behind to enjoy his remaining time however he wishes. (And, as a family man myself, good for him. More on this family matter later, perhaps.)

_The Player of Games_ was first published in 1988, but I stumbled across it in that London bookstore in 1997. By that time, Banks had published four of the nine novels that would become _The Culture_ series and its one collection of short stories: _Consider Phlebas_ (1987), _The Player of Games_ (1988), _Use of Weapons_ (1990), _The State of the Art (1991) — the collection of short stories, and _Excession_ (1996). (I should note that I am using the Wikipedia entry to define novels in the series: it leaves out a novel like _The Algebraist_, which in my mind contained a reference to the Culture, but now I will need to re-read it.)

On a whim, in the middle of re-reading some of Alastair MacLean’s novels that constituted my introduction, for the most part, to adult literature, I found myself in front of our living room’s bookshelves and staring at the stretch of Banks novels. There are not that many such stretches of novels by single authors. Further along the same shelf are Gregory Benford’s Galactic Center novels, except, curiously, the first one, which I have never read. Other spans of books that I can name without looking at the shelves are the previously named MacLeans and Patrick O’Brian’s Aubrey-Maturin novels. (Curiously, they are all British authors.)

Re-reading _The Player of Games_, a novel in which the protagonist, Jernau Gurgeh, is, it turns out, tricked into playing the role of champion of the Culture in an exercise, a game, that lies at the heart of a empire. The Azadian Empire is a nasty affair, containing many, if not all, of the worst elements of our own human cultures: racism, sexism, willingness by the powerful to exploit everyone less powerful by any means not only necessary but also pleasurable. It is, as the ship _Limiting Factor_ notes at one point in the middle of the novel in a conversation with Gurgeh, “a guilty system [that] recognizes no innocents” (171).

The novel’s focus is Gurgeh’s own developing consciousness of, and involvement in the game of Azad, the same name as the empire itself and the game through which positions within the power structure are determined, and some of the most compelling moments in the novel are those in which Gurgeh’s consciousness is fully immersed in the game play and Banks’ narration allows us to see the development of game play as Gurgeh himself sees it. Eventually Gurgeh realizes that he has been playing as a Culture person would play. In fact, for those familiar with _Consider Phlebas_ and its description of how the Culture came to rally itself in the moment of the Idiran War. The history we glimpse in the early novel is that the war did not go well for the Culture at first, which, playing by its own rules of prizing life and peace above all else did not know how to respond to the single-minded, religiously-guided aggression of the Idirans. As the Culture came to grips with the fact that it was at war, it exercised a kind of fatalistic logic: allowing certain parts of its civilization to be taken or destroyed or destroying those parts it knew it would be too dangerous to turn over to the Idirans. The Idirans experience this turn as simply capitulation on the Culture’s part, but it is, in shades of Yamamoto’s observation about Japan’s attack on Pearl Harbor, more a case of a great giant having been awoken but not yet ready to act.

The ramifications of the Culture’s mobilization for war are a constant reminder throughout the series that probably deserves more serious treatment than I can give it here. In _The Player of Games_, Gurgeh is transported by the warship _Limiting Factor_ which has been mothballed for 800 years — though, given that we learn that the Culture has had its eye on the Azad Empire for hundreds of years, that may not be entirely true either. In fact, much of what happens in the Culture series is really about necessary deceptions. Later, in _Excession_, when an entire fleet of mothballed warships are allowed to be taken by yet another empire about which the Culture has some concerns, we learn that a good portion of the entire Excession event was contrived by a group of Minds in order to put the Affronters in their place.

As always, the exercise is performed by an individual who is somewhat at odds with the Culture himself. In _Consider Phlebas_, our shapeshifter is turned off by the Culture’s logic and willingness to think like the machines upon which it depends–though he later comes to realize he has more in common with the Culture than his Idiran employers. In _The Player of Games_, Gurgeh comes to relish, in many ways, the vividness of life under the constant threat of death, dismemberment, or defamation. In _Excession_, Byr Genar-Hofoen actually opts out of the Culture at the end, choosing to become, biologically, an Affronter. In _Use of Weapons_, we find that we have been in the company of a homicidal monster the entire time, a deeply sorry one, but nevertheless a monster.

I don’t think it’s always the case with Banks’ protagonists, but I think a more clear enumeration of them and their development would make for an interesting project. Any analysis of Banks would make for a fun project. Thanks you, Iain (M.) Banks for all the fun.

Science Fiction Saturday

Saturdays are, or should be, about those other dimensions, or parts, of our lives that we like to nurture as best we can. When I am not reading about narratology or statistics or working out how the two can be combined in interesting ways — that is, in ways interesting to others as well as to myself — I read science fiction. Right now, I am making my way through Robert Heinlein’s _Methuselah’s Children_, which is interesting both for some of its contents as well as for its plotting, which I want to say is more uneven than other Heinlein work, but, in all honesty, anyone who’s actually read _Starship Troopers_ knows it is about the most oddly paced novel you can imagine. (You can’t really call it plotted because, in some ways, the plot is just there to support the fabric of dense politico-historical tracts. *I know, I know* … no one reads it that way.)

More on _MC_ some other time. In the present, I have the following:

* Gizmodo’s [_Sploid_][] has a [Youtube link][] to BBC’s _Horizon_ segment on the use of glass paintings in the making of _Raiders of the Lost Ark_ and _The Return of the Jedi_. For those of us who grew up in the era of film and grok the idea of multiple exposures, you immediately have a sense of “yes, of course, that’s how they did it.” I showed it to my daughter, who has grown up in a world entirely of digital effects production, and I don’t know what she thought — though, to be her credit, she and her friends have been making Minecraft videos in which they animate Lego figures standing in front of backdrops made up of iPad screens, which they shoot from another iPad. So, in a sense, they have re-discovered/invented [*rear projection*][].
* [io9 has a link][] to a [Castalia House blog post][] which itself is a rummaging about in an October 1991 article in _Publisher’s Weekly_ titled “Science Fiction: Expanding, Experimenting.” The article was written by Robert J. Kilheffer, then SF editor and reviewer, and senior editor at _Omni_ magazine. (_Omni_! Oh, my heart leapt at the memory of it!) Kilheffer’s *modus operandi* was to interview editors at various publishing houses to have them enumerate the trends they were seeing and/or anticipating and then compiling those views in the article. 1991 turns out to have been an interesting moment: the trend toward fantasy had just begun and no one quite anticipated the shift from Walden and B. Dalton book stores in malls to the big box stores of Borders and Barnes and Noble. Certainly no one then was thinking about e-books and/or Amazon and/or self-publishing.
* The comments in the io9 thread are absolutely worth the read, because there are several mentions of ebook experiments going on then, one of which involved a wildly interactive version of Marvin Minsky’s _Society of Mind_.
* A [thread on Reddit][] illuminates the back story behind Leigh Brackett and Edmond Hamilton compiling _Best of_ volumes of each other’s work: they were married and died within a year of each other in 1978-79. And it turns out that the [first draft of _The Empire Strikes Back_][] was written by Leigh Brackett. ([Geeks of Doom have a summary.][]) In Brackett’s version, Darth Vader is not (yet) Luke’s father and the rebel base on the ice planet, not yet named Hoth, is worried about attacks from ice creatures who are looking to get rid of these invaders. It’s only Luke’s force sense that alerts the base to the incoming imperial attack. Later, our merry band of adventurers end up on the cloud planet of Hoth. And, of yeah, Hans Solo has a step-dad.

[Youtube link]:
[*rear projection*]:
[io9 has a link]:
[Castalia House blog post]:
[thread on Reddit]:
[first draft of _The Empire Strikes Back_]:
[Geeks of Doom have a summary.]:

Computing Counterfactuals

In an article on how cold war priorities in the development of computing affected the kinds of games that we now play on our post-cold war computers, [Peter Christiansen offers a series of “counterfactual” scenarios][pp]. E.g., what would computing now look like if, instead of being driven by the need to calculate ICBM missile trajectories, the Jacquard loom had continued to develop? Christiansen has a number of interesting sources:

> In his book _From Sun Tzu to Xbox_, Ed Halter (2006) makes a very convincing argument that many of the conventions in videogames that we take for granted can be traced back to constraints that were placed by many of the early developers of computing technology. As he notes, early computers like the ENIAC , with its stored memory and its binary language of ones and zeros, were created for purposes such as calculating ballistic tables.

Jacquard Loom