Tuesday, 15 September 2015

Making a Mobi file for Amazon

Mrs' latest book needed to be uploaded to Amazon this week. We've not done this ourselves for her previous books, but this time it made sense to. That meant figuring out the process.

Amazon publish a guide which seems quite comprehensive but there are odd gaps in it which took some thinking about, so this describes the process that worked for us. I gather it changes fairly often so your mileage may vary, and that adds to the challenge to keep the documentation up to date as well.

One important thing this sorts out is the problem a lot of people have that means the Kindle reader opens the book on the wrong page the first time. This will open it on the cover. Okay, I haven't tried it on every device, but the ones I tried worked fine.

So we start with a .doc file. It was created with LibreOffice and it has very simple style settings. Almost everything is using the default style except chapter headings. We want italics, of course, and perhaps section breaks, something like this:


Pull the github template and put all the files into a new directory.  

Save your book file as an odt, a necessary step before you save it to html, then export to html. If you're using Word or some other word processor your process will be different, but you want to end up with an html file and copy it into the template directory (the top level directory, the one that contains the file sed.sed).

Now run sed against your html file. Sed is a linux batch editor which we will use to clean up your html because what your word processor generates tends to be riddled with rubbish you don't need. Plus the html export (from LibreOffice) will put most of the text in one long line with no line breaks, which is a bit punishing for many text editors. To run sed my command line looked like this:
sed -f sed.sed myfile.html > t.html
That runs sed using a file of commands (sed.sed) and writing the result out to t.html. You will find the sed.sed file in the github directory you pulled earlier. The first thing it does it break up the one long line into shorter ones to make it easier to edit. The rest of sed.sed is a bit specific to me so you will want to edit it to handle the classes you find in your own t.thml file. The general idea is to remove as many class references in the t.html file as you can. Take a look at the /kindle/text.html_template to see how it ought to look (except for the header part which we'll fix next)

I also convert the obscure names into readable ones where I can so 'italic' instead of 'T3' and so on. When you're happy with the t.html file copy it into the new directory and rename it to text.html.

Now manually edit the text.html and remove everything above the body tag, yes everything. Now look at the text.html_template file and copy everything above the body of that file into the top of your text.html file. You'll see references to css files in that header information. We'll look at the styles.css file now.

The only css file you need to bother about is styles.css. The class references in your text.html file need to be defined here. If you don't know much about css try here but css is a big subject, you won't need to know much but you will need a little. Take a look at the examples in the styles.css file. They might be all you really need.

You can open your text.html file in a browser and get an idea how well your css changes are working.

Next up is the LoremIpsum.opf file. Rename it to something that looks like your title. The opf file is the file that refers to all the others. Open it with a text editor and edit in your title information etc. Do the same with the title.html and copyright.html files.

Now for the table of contents. There are three files much the same here. toc.ncx has a list of chapters with names and links. Check that your chapters have id attributes on them (in text.html) that match the references here. Then you have toc.html and nav.html. All three of these look similar but not quite the same. It is possible we only need two of these three, but I found it just as easy to produce all three.

Be careful to leave the landmarks section at the end of nav.html. This section ensures the kindle opens the book at the beginning the first time. It if it missing the book will open somewhere in the table of contents.

You'll need a cover image. Use a jpg file

Delete the _template file file from your directory.

Now download and install the Kindle Previewer from here. I used the
Windows one and ran it under Wine on my Linux laptop, it worked fine. You start it, tell it to open your opf file, and it will build and open a mobi file. It is easy to use and it will give you a list of warning and/or errors if you have problems to fix. The resulting mobi file goes into a subdirectory off the directory your opf is in.

The only bit that isn't obvious in the Kindle Previewer is how to see the compilation details (errors, warnings etc) so I've marked that above.

Take a look at in in the previewer and you might also like to copy it to a device that runs the kindle software. I used my Android phone. That allowed me to take a good look at it on a 'real' device.

Now you are ready to upload the mobi file to Amazon. You will need to load the cover separately as well, even though the cover is already in the file. This is, presumably, so that Amazon can use it on their site.

Wednesday, 9 September 2015

Where do we go from Here?

There's a bit of talk around these days that we need a backup for Earth, possibly because of Elon Musk's plan to colonise Mars. There's a good case to be made for taking some steps to ensure we survive the next great meteorite strike given that it will happen sooner or later. The one that hit us 60 million years ago wiped out pretty much everything and there seems to have been four or five similar events before that. Life bounces back somehow, but not all of it. Chances are there would be very little, if any, human civilisation left after an impact of that size.

Much further out the sun is going to nova. That's a long time away, but when it happens (again, there's no 'if') the earth won't be habitable. It will be a more comprehensive extinction than a meteorite strike. Between now and then there might be other things that can go wrong that we haven't a clue about. So a backup seems reasonable.

Mars is only a short(ish) term solution because when the sun blows up it will be toast as well. We would need to be further away.

But is it a solution at all?

The link above is meticulous on making its arguments about the need for a backup and even how to do it but, for me, there is a huge gap in the argument. Actually there are two gaps, but the second one is less obvious. So let's look at the first one.

Once you get into space you've escaped Earth's gravity well. Getting to orbit is
really, really hard. Getting past orbit, essentially into an orbit of the sun rather than Earth, is harder but not as hard as getting to orbit. Actually it is still very hard but mostly because you have to haul all the stuff you need into Earth orbit first. You're still in the sun's gravity well, and that's hard to escape, and when you do that you're still in the galaxy's gravity well. None of these wells should be underestimated. But the immediate concern is the Earth's.

Once you escape Earth's gravity well you want to think very carefully before you go back down another similar well. For example Mars' well. Getting down is about as hard as getting up again, so it had better be worth your while. So what exactly does going down Mars' gravity well get us?

There's stuff, like things we could mine. And there's the hiking. I like a good walk through the countryside under the sky. The sunshine, the beaches... oh, wait, there aren't any beaches. And if you go outside it has to be in a space suit because it is freezing cold and practically a vacuum. Like in space. There's a protective magnetic field to keep off the worst of the cosmic radiation, um... there is on Earth but not on Mars. Again, you might as well be in space. The one thing you have on a planet is... gravity. Mars' gravity is not as strong as on Earth's but it would probably help us not lose bone density as fast as we do in weightless conditions. Of course it takes so long to get there we'd probably have to solve that problem for space anyway.

What about the stuff? We can get stuff from asteroids without going down a gravity well. And we can probably simulate gravity in space using rotation the way science fiction writers have been telling us for decades. There's still the matter of radiation. It is less of a problem in Earth low orbit where the ISS lives because that's inside most of the protective magnetic field. But if we want to go out to the asteroids, or even to Mars, we'd need to do better. This is another area where planets can help because you can block radiation by tunnelling underground. So when you think of a Mars colony expect most of it to be in tunnels, not hiking and looking at the sky.

We could terraform Mars... eventually. We have nowhere near the tech to accomplish that today, and that means a long time in the tunnels before anyone can walk under the sky, if it ever happens.

Fortunately there has been a lot of thinking about this already. In 1975 I came across a paper in Physics Today by Gerard K O'Neill and then his subsequent book 'The High Frontier' which developed the ideas further. There's a lot of detail there but it boils down to the idea that we could build huge (really huge) structures in space and colonise them.

If we make them big enough (remember Babylon 5?) then we could walk under something like a sky and this is far more practical than terraforming a planet.

The building material would initially come from the moon rather than the Earth because it is nearby and has a smaller gravity well. O'Neill built a prototype of a magnetic launching mechanism that could throw buckets of material from the surface to Lunar orbit powered by solar panels. With today's technology we could automate that, saving the need to send humans up and down that gravity well.

O'Neill's plan involved generating heaps of cheap solar power and beaming it down from space, an idea that never quite convinced me but it was supposed to make the space stations pay for themselves. Subsequent energy price drops made the idea economically unattractive and it has never been pursued. Today's cheaper launch technology and higher automation capabilities make it more practical, though whether it would ever pay for itself is doubtful.

But it is definitely more feasible than colonising Mars. Multiple space habitats could be scattered over the solar system, not all of them would be hit by anything (that we know about) this side of the sun going nova. By then we'd be able to at least move them further out if we hadn't worked out how to move to another star.

Instead we have this fixation on planets which, when you think about it, is a bit infantile.

Which brings me to the second gap in the argument. We've sent rovers to Mars to explore it for us. They're not as good as sending a human, yet, but they are getting better and better. It seems likely that in a few years there will be AI systems on Earth a good deal smarter than we are. Once that happens they will be the dominant species on the planet. This doesn't have to be scary. Imagine the AI systems thinking that they are not in competition with us, they don't have to enslave or exterminate us. But they're quicker and smarter. They are the Next Step.

They also aren't bothered by things like lack of gravity and it is simpler (probably) to shield them from radiation. They don't have a need to go hiking under the sky etc. In short, if they want to colonise space they'd be fine. And when the meteorite strikes and we're all wiped out they'd survive.

"But...but that's not the same. No humans? That's not survival!"

 It depends how hung up you are on your genes. Let's imagine the meteorite hits us in 10,000 years. You won't be around, neither will your grandchildren. What we will have turned ourselves into by then (think electronic implants, genetic modification and neural backups, then multiply by a million) will be unrecognisable. The AIs might even seem more familiar.

So that's my plan. Forget about sending humans to Mars. Just send AIs into space. Actually just build the AIs on Earth and let them figure it out for themselves.

Thursday, 27 August 2015


Early this month I spent some time in Vanuatu, and some of it was on the island of Santo, including Palikulo Bay which you've probably no reason to have heard of. I've marked it on the map, assuming you can find Vanuatu at all (Google is your friend). What you need to know is that it is a tiny country in the Pacific, about the same latitude as northern Australia but about 2000km to the east, so there is a lot of ocean around it. Apart from the whole tropical island paradise thing they have going on (and they really do have that going on) these islands were important during WWII. There is debris left over from the military operations, not everywhere, but where you find it you find lots of it.

Santo was a forward base put together by the US forces to block the Japanese advance through the Solomon Islands. It was an enormous undertaking done at breakneck speed. They had to have a working air base established before the Japanese could occupy Guadalcanal 500km to the north west. They had mere weeks to get this done on an island with practically no infrastructure that was a very long way from anywhere.

At the time the local population must have been under 10,000. They had had contact with the outside world, there were trade routes to the nearby island groups that pre-date European discovery. Many of them had gone to work in Australia a couple of generations before (some of that was not voluntary), and they'd had missionaries living among them for decades. But you should imagine a low tech subsistence lifestyle. Most of them would not have seen an aircraft.

Suddenly huge ships landed carrying all kinds of strange equipment: trucks, bulldozers (to make roads for the trucks, there weren't any yet) and about 40,000 people. I can imagine that few of the locals would have thought there could be so many people anywhere. And next thing they were building these:

That's the first of at least four air fields they built. This one was usable in under two months. This was important because without an airfield they had to rely on boats and seaplanes for cargo and they needed lots of cargo fast. The first pilots to fly in did report the experience was more exciting than they hoped, but they did (mostly) manage to land safely.

They built a town on the coast just off the far end of this runway which had even had movie theatres as well as hospitals, workshops and so on. It seems impossible they could have moved so quickly on this, but they did.

While the scale of all this is fascinating I found myself wondering what the locals thought of it all. Fortunately they're happy to talk about it.

This is Lino standing on one of the old runways. He's a guide and if you go to Santos and want a look around he's excellent.

It was not an invasion. The forward parties made an effort to talk to the locals, they told them that it was not safe on the coast because the Japanese would kill them, and that they should move inland. I'm not sure if that was true but the forward parties probably believed it. The important thing is that the process was friendly. They also needed labour and were happy to hire locals and pay them in, well, US dollars. What did they do with the money? Many of them buried it in a box somewhere and there are stories of old guys, feeling their end coming on, telling their family about some box of money buried in a forest. Subsistence farmers who own their own land have little use for cash and they had no banking mechanisms. Perhaps not so much on Santos but further south, on Tanna, they were so impressed by all this technology they made it into a kind of religion and so the cargo cults were formed. I'm not at all surprised. When you go from grass skirts to aircraft in a matter of weeks you'll struggle to find ways to explain what is going on. Lino tells stories of ghost trucks making their way through the forest at night, though not now they've picked up much of the debris.

Relations seem to have stayed cordial between the military and the locals. It helped that a great many of the soldiers who arrived were black, just like the locals. Not actually the same race, but it meant people were less hung up about skin colour. At the end of the war everyone left. They bulldozed most of the buildings and then drove the bulldozers into the sea, which seems bizarre but there were reasons. During wartime everyone pulls together and those bulldozers (and trucks and everything else) were produced by US factories at cost. The manufacturers did not want them coming back to the US and being sold cheap, destroying their markets. They offered the gear for sale to Australia and New Zealand but not at an attractive enough price. With no supporting infrastructure they were no use to the locals, there was no source of fuel for a start, so they dumped them. However the stuff that was useful to the locals such as kitchen utensils and so on, they handed around.

It was mostly US forces, but not entirely. There were Australians and New Zealanders there, including my father who was at Palikulo Bay servicing electronics gear on the aircraft. He was always mad on electronics, built his own ham radio when he was 13, so this was an obvious path. He spent time on Guadalcanal and Santo, but he got there after the worst of the fighting was over. Nevertheless his diary records several air raids which probably were flying out of the Japanese base at Rabaul, close to PNG. Historical records of this time show there was little damage from these raids but they seem to have been fairly alarming.

My father grew up in a small rural town and in his early childhood they had no electricity. The farm he lived on relied on horses rather than tractors. So, while it was not such a huge leap in technology for him to see this vast undertaking on Santo it must have impressed him more than I realised when he talked about it.

I found myself wondering if we can learn anything from this incidence of a low tech culture meeting a high tech culture for when, say, we eventually make contact with space aliens. We can expect the difference in technology to be very much greater. It seems unlikely that any space aliens we ever meet will be less than a million years or so ahead of our development and the difference here was probably a lot less than a thousand years. However this was also a case of the high technology running at full blast, so that accentuated the effect. If my father had turned up there with a horse and a plough it might have been a bit interesting to the locals, but not astonishing. An armada of ships deploying bulldozers, throwing up huge buildings and mowing down the forest would impress anyone even now.