E-Book Enlightenment

A Booki Of Your Own

Reasons For Having Your Own Booki

e_books.png

For most it will not be necessary to have their own Booki software, and every reason not to.  Setting up your own Booki and backing up the data on a regular basis is a significant amount of work, and if you're going to share your work with the world there is really no reason not to use the shared Booki as well.

If you're not going to share, then having your own Booki makes sense.  If you don't have reliable access to the Internet having your own Booki might make sense as well.  A school with XO laptops that can connect to each other on a network but not to the Internet might find a local copy of Booki quite valuable.  In my own case I set up Booki on a computer at home and another at the office.

My day job involves teaching people how to use software I have written.  Teaching people can be a challenge, and teaching people who live and work on the other side of the Earth is a greater challenge.  We had been using articles on a website, plus a Wiki, to contain the training materials, but after writing two FLOSS Manuals I came to the conclusion that what my company needed was an honest to gosh Manual.  I got permission to investigate using Booki for that purpose.

I installed the software at home because:

  • I wanted the work install to go as quickly and smoothly as possible
  • I want, eventually, to write a book that I will not share with the world.  After self-publishing my first FLOSS Manual on Lulu I honestly felt that it would be easier to use Booki to lay out this book (working title: Jim's Oprah Book) than to do the same thing with Open Office or MS Word.

If your reasons are like mine, then let's set up our own Booki!

 

Getting The Software

You will need a recent version of Linux to run Booki on.  Windows or a Mac will not run Booki.  You won't need much of a computer to run it on.  The computer I used at home was a refurbished IBM NetVista which I had bought online for about a hundred dollars.  The computer at work was a discarded desktop model which was even older.  I would not recommend trying to install this software on an XO laptop, but any desktop computer made in the last few years should be fine.

I used Fedora 13 for these Booki installs, but I don't recommend it.  Fedora is used on the XO laptops, and since I write software for that platform I use it on my desktop computers as well.  The downside of Fedora is that, so far, every time I've upgraded to a new version of Fedora I've had to back up all my data and do a complete reinstall.  Other than this, the different brands of Linux are more or less the same.  If I was going to recommend a Linux for Booki my choices in order would be:

  • Whatever Linux you already have
  • Ubuntu.  If you've never used Linux, this is probably the easiest, and I have had good experiences with it.

The rest of these instructions will assume that you have Linux installed and have become comfortable running it.  Getting comfortable with Linux is the subject for another book, so if you've never used Linux it would be a good idea to find someone who has to help.

Often the install program for Linux will ask if you intend to use it as a web server or if you are doing programming or if you want office software.  Answering "Yes" to all three will save some time.

There are actually two parts to Booki, and you'll need both:

  • Booki itself
  • OBJAVI 2, the part that creates PDF's, EPUBs, etc from your Book.

installing most software on Linux is no more difficult than checking a check box on an Add/Remove Software dialog, but when software is still under development like Booki is you'll need to get the source code and work with that.  The source code for both is kept in a Git repository, so you'll want to have Git installed.  Once you do, you can create a "src" directory in your home directory and from in that directory run these commands:

git clone git://booki-dev.flossmanuals.net/git/booki.git
git clone git://booki-dev.flossmanuals.net/git/objavi2.git

This will create two directories under "src": booki and objavi2, which will contain the source code for these products.

It is also possible to get the code without using Git, which may be necessary if your company firewall doesn't let you access the repository.   To do that go to this URL:

http://booki-dev.flossmanuals.net/git?p=booki.git;a=tree  

In the upper left of the page is a link named "snapshot".  Click on this link to get the latest code in a tar.gz archive and unpack it into your "src" directory, then rename the directory this gives you to "booki".  Then use this URL and the same procedure to get the code for OBJAVI:

http://booki-dev.flossmanuals.net/git?p=objavi2.git;a=tree

Before you can continue, you'll need to check the README.txt file in Booki and the INSTALL file in OBJAVI 2 to find out what other software you'll need to install.  For Booki the list is:

  • django
  • django south
  • wsgi
  • apache2
  • php5
  • python-simplejson
  • sqlite
  • redis
  • aspell plus dictionaries for the languages you will use.

If you're lucky the latest redis will be included as a package in your distribution.  Fedora users so far are not so lucky.  They will need to download the latest source code from http://code.google.com/p/redis/ and compile it using the Makefile in the time honored manner:

make
sudo make install

If you installed it from a package in your distribution you should check the Services to see that it is enabled and running.  If you compiled from source you can start it up like this (running as root):

redis-server &

You'll want to run this command every time your computer boots up.  For Fedora 13 you can put this command in /etc/rc.d/rc.local.

For OBJAVI the list is:

  • lxml
  • pdfedit (4.1+)
  • xvfb
  • fontconfig
  • pdftk
  • psutils
  • poppler-utils or xpdf-utils
  • wkhtmltopdf
  • open office 3
  • some fonts

You can get everything from your distribution's packages except wkhtmltopdf.  That you'll need to download from http://code.google.com/p/wkhtmltopdf/.  You should get a precompiled binary and copy it to the /usr/local/bin directory as user root.

When you set up Linux you should get fonts as part of the basic install, but it would be a good idea to install more.  In Fedora the Add/Remove Programs dialog has a section for Fonts that has over a hundred free fonts that you can install.  If you use the Roman alphabet only you can limit yourself to the Latin fonts.

Disable SELinux

Fedora 12 and later has SELinux.  SELinux is to Linux is as Aunt Polly is to Huckleberry Finn.  Its job is to keep programs, including the Apache Web Server, from doing things they should not.  Even without SELinux Linux is pretty secure.  The web server runs as a user (in Fedora's case the user is named "apache") and it is only allowed to do what that user is allowed to do.  On a personal or corporate network this is generally enough.

What SELinux does is add an extra layer of protection.  Programs are expected to do certain things.  If a program tries to do something that SELinux is not expecting, then SELinux stops it.  To get around this you have to tell SELinux to expect this behavior from this specific program.  Then it will be allowed.  This extra layer of protection makes it more difficult for a malicious programmer to break the system.  Sooner or later he will have to do something SELinux is not expecting.  He will be stopped and his actions will be logged.

As commendable as this is, if you have a program that does more than a few unusual things SELinux will be a real challenge.  OBJAVI falls into that category.  It would be a great deal of work to get SELinux to tolerate all the things that OBJAVI is likely to do, and if you're running it on a private network there wouldn't be much benefit.  We disable SELinux by editing a file /etc/selinux/config as user root.  The file should look like this:

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#       enforcing - SELinux security policy is enforced.
#       permissive - SELinux prints warnings instead of enforcing.
#       disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
#       targeted - Targeted processes are protected,
#       mls - Multi Level Security protection.
SELINUXTYPE=targeted

Change the value of SELINUX to disabled, save the file and reboot your computer.

Configure Booki and OBJAVI 2

Booki recently got a much improved setup script.  This will install most of booki under the directory /var/www/mybooki, which is as good a place as any.  The rest of it will run out of whatever directory you put the source code in.  In my case this was /home/jds/src/b ooki-HEAD-9db92a6, because I downloaded a snapshot and unpacked it.  You might want to rename the directory to be simply booki, but you don't have to.

You'll need to create a directory named /var/www/mybooki to copy everything into.  Chances are you'll need to be the root user to do this.  However, you need to change the ownership of this directory so that you, as yourself, can copy things into it and so the apache web server can write to it.  I did this with this command:

 chown jds:apache /var/www/mybooki
 chmod 664 /var/www/mybooki

jds is my own account, and apache is the group the apache web server belongs to in Fedora.  (It may be different in other Linux distributions).  The chmod 664 means that both jds and members of the group apache may modify the contents of this directory.

It is important when creating content in this directory to make sure all of it is group owned by apache. 

As yourself, change to the scripts directory within the directory where you have your booki source code and run this:

./createbooki --database sqlite /var/www/mybooki/

This will copy a bunch of stuff to /var/www/mybooki.  Now change to that directory and you'll see that it contains a file named settings.py.  There are several places you'll need to modify in this file.  First is the doc root:

BOOKI_ROOT = '/var/www/mybooki'  # edit this

Next you need to set up some URL's:

# use this objavi server
OBJAVI_URL = "http://127.0.0.1/objavi.cgi"
ESPRI_URL = "http://127.0.0.1/espri.cgi"
TWIKI_GATEWAY_URL = "http://127.0.0.1/booki-twiki-gateway.cgi"

#the name of the booki server (comment out to use os.environ['HTTP_HOST'])
THIS_BOOKI_SERVER = '127.0.0.1:8000'

IP Address 127.0.0.1 is of course the localhost IP address.  It is likely that you will want to change this to the IP address of your computer so that you can use Booki on the network.  You can of course use a DNS name rather than an IP address.

Notice that we have Booki running on port 8000.  We need to set up virtual hosts for both Booki and OBJAVI.  If you can give each one its own IP address or DNS name there is no reason you can't run both on port 80 like a normal web application.  If everything has to use the same IP address then you can distinguish your virtual hosts from each other using a port number.  Note that OBJAVI has to run on port 80, but Booki can use any port.  There is nothing magic about the number 8000.  It just needs to be a port that nothing else is using.  In the office I use port 86.

Next we have to set up the database entries:

DATABASE_ENGINE = 'sqlite3'
DATABASE_NAME = '/var/www/mybooki/database.sqlite' # Or path to database
# file if using sqlite3.
DATABASE_USER = ''             # Not used with sqlite3.
DATABASE_PASSWORD = ''         # Not used with sqlite3.
DATABASE_HOST = 'localhost'             # Set to empty string for localhost. Not used with sqlite3.
DATABASE_PORT = ''             # Set to empty string for default. Not used with sqlite3.

Booki can be used with Sqlite3 or Postgres.  If you are willing to tinker with the code you could get it to run with just about any database.  Sqlite3 is the easiest to set up and is completely adequate for Booki on a small private network.  The only entry you need to be concerned with is DATABASE_NAME, which is set to put the database in a directory that can be read and written to by user apache.  There is no need to create the database file.  Booki will create it.

Now from the command line, as yourself, run this:

. ./booki.env

Notice that there are two periods on that line, separated by a space.  If you just use one the environment variables needed by the next step will NOT be set.

Now run this:

django-admin syncdb

This will create your database tables. In the middle of doing this it will ask you if you want to create a superuser account.  You do not.  If you answer "yes" it will prevent the rest of the database initialization from happening.  Reply "no" when asked.

To finish up the database initialization you need to run this:

django-admin migrate

Now you can create the superuser you skipped creating before:

 django-admin createsuperuser

After that you can run Booki itself and check it out.  First run this as yourself:

./manage.py runserver

Once this is running you should be able to point your web browser to http://127.0.0.1:8000 and see Booki in operation.  You should also go to http://127.0.0.1:8000/admin and add a license like this (after signing in as the superuser you created):

Django Admin

You can fool around with Booki some more if you like, but be aware that this is not an adequate way to run Booki.  You really need to run it under a virtual host in Apache.  manage.py only supports one user at a time, and is only good as a quick sanity check to make sure everything is set up correctly.  When you're done looking at Booki you can kill manage.py by pressing Ctrl-C in the terminal where you started it.

Next we need to configure OBJAVI. You'll probably want to run objavi out of a directory like /var/www/objavi.   This means you'll want to set up this directory similarly to the one you created for booki, with the same permissions, and you'll copy the objavi source code there.  Before you do, be aware that the objavi/tests directory may contain subdirectories with epub examples that take up over 100 megabytes!  You'll have no use for these, so be sure and delete the files before you copy the files to /var/www/objavi.

OBJAVI has its own configuration file, objavi2/objavi/config.py.  You'll need to change some settings in this file, create some new directories, and make the directories readable and writable by the apache group.  The first setting to change is this one:

WKHTMLTOPDF = '/usr/local/bin/wkhtmltopdf'

When you downloaded the binary for wkhtmltopdf it may have had a name like wkhtmltopdf-static or something else.  Change the value of WKHTMLTOPDF to whatever name it has in /usr/local/bin.  (In my case I had renamed it to wkhtmltopdf, which hurt nothing).

The biggest change is to add an entry to SERVER_DEFAULTS.  The first entry for '127.0.0.1:8000' is my own new entry.  Use the actual IP address (or the DNS name) of your machine if you're going to use it over the network. 

SERVER_DEFAULTS = {
    '127.0.0.1:8000': {
        'css-book': '/static/simmons.css',
        'css-web': '/static/en.flossmanuals.net-web.css',
        'css-newspaper': '/static/en.flossmanuals.net-newspaper.css',
        'css-openoffice': '/static/en.flossmanuals.net-openoffice.css',
        'lang': 'en',
        'dir': 'LTR',
        'toc-encoding': None,
        'display': True,
        'interface': 'Booki',
        'toc_header': 'Table of Contents',
        },
    'booki.flossmanuals.net': {
        'css-book': '/static/en.flossmanuals.net.css',
        'css-web': '/static/en.flossmanuals.net-web.css',
        'css-newspaper': '/static/en.flossmanuals.net-newspaper.css',
        'css-openoffice': '/static/en.flossmanuals.net-openoffice.css',
        'lang': 'en',
        'dir': 'LTR',
        'toc-encoding': None,
        'display': False,
        'interface': 'Booki',
        'toc_header': 'Table of Contents',
        },
}

The entries that I changed are in bold.  I use my own style sheet when creating PDFs for books, and you may wish to do that too.  I indicate that I want to display this server in OBJAVI's list of servers and that I don't want to display the other entries.  You'll see why in a minute.

You need to create some directories under objavi2/htdocs and make certain they can be read and written to by the apache group:

  • books
  • booki-books
  • progress
  • tmp

You'll also need to make certain that the static directory already in objavi2/htdocs is writable by the apache group.  Finally, you'll need to set up a log directory under objavi2 and make sure that apache can create files there too.

Setting Up Apache Virtual Hosts

The simplest way to get Booki and OBJAVI 2 running under the Apache web server is to set up virtual hosts.  What I did was to edit the /etc/httpd/conf/httpd.conf file as the root user using gedit.  It is also possible to make configuration files outside of httpd.conf that will be loaded by Apache automatically.  For Fedora 13 you could make separate files for each virtual host and put them in directory /etc/httpd/conf.d.  (When I got everything working on my Booki install I moved the virtual host entries to files named booki_vh.conf and objavi_vh.conf respectively).

The entries I put at the end of httpd.conf looked like this:

Listen 8000

<VirtualHost *:8000>
     # CHANGE THIS
     ServerName booki.myhost.com
     SetEnv HTTP_HOST "booki.myhost.com"

     SetEnv LC_TIME "en_GB.UTF-8"
     SetEnv LANG "en_GB.UTF-8"

     WSGIScriptAlias / /var/www/mybooki/booki.wsgi

     <Location "/">
       Allow from all
       Options FollowSymLinks
     </Location>

     Alias /static/ "/var/www/mybooki/static/"
     <Directory "/var/www/mybooki/static/">
       Order allow,deny
       Options Indexes
       Allow from all
       IndexOptions FancyIndexing
     </Directory>

     Alias /site_static/ "/home/jds/src/booki/lib/booki/site_static/"
     <Directory "/home/jds/src/booki/lib/booki/static/">
       Order allow,deny
       Options Indexes
       Allow from all
       IndexOptions FancyIndexing
     </Directory>

     Alias /media/
     "/usr/lib/python2.6/site-packages/django/contrib/admin/media/"
     <Directory
     "/usr/lib/python2.6/site-packages/django/contrib/admin/media">
       Order allow,deny
       Options Indexes
       Allow from all
       IndexOptions FancyIndexing
     </Directory>

     <Location
     "/site_media/xinha/plugins/SpellChecker/spell-check-logic.php">
         SetHandler application/x-httpd-php
     </Location>
     <Location
     "/site_media/xinha/plugins/SpellChecker/spell-check-savedicts.php">
         SetHandler application/x-httpd-php
     </Location>
     <Location "/site_media/xinha/plugins/SpellChecker/aspell-setup.php">
         SetHandler application/x-httpd-php
     </Location>

     ErrorLog /var/log/apache2/booki-error.log
     LogLevel warn
     CustomLog /var/log/apache2/booki-access.log combined

</VirtualHost>

<VirtualHost *:80>
        ServerAdmin myname@gmail.com
        # limit MEM to 800 million bytes
        RLimitMEM 800000000

        #Sometimes it takes a while. Wait.
        TimeOut 600

        DocumentRoot /var/www/objavi/htdocs
        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>
        <Directory /var/ww/objavi/>
                Options +All +ExecCGI
                AllowOverride None
                Order allow,deny
                Allow from all
                AddHandler cgi-script .cgi
                # Remove output filters in case mod_deflate is being used.
                RemoveOutputFilter .cgi
        </Directory>

        DirectoryIndex index.html objavi.cgi
        ErrorLog /var/log/apache2/objavi-error.log

        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn

        CustomLog /var/log/apache2/objavi-access.log combined
        #ScriptLog /tmp/objavi-cgi.log

</VirtualHost>

A Virtual Host is a way of making Apache act like more than one web server.  We have one Virtual Host for Booki, and one for OBJAVI 2.  When you set up a Virtual Host you need some way for Apache to know which host is needed for a given request.  You can do this by giving your server more than one IP address, more than one DNS name, or in my case more than one port.  The Listen directive at the top says that we will be listening at port 8000 in addition to the normal HTTP port 80.  When a request comes in on port 8000 it will go to Booki and when it comes in on 80 it will go to OBJAVI 2.  Again, OBJAVI 2 must run on port 80.

For Booki we're using the wsgi plugin for Apache, so make sure it's installed.

The createbooki script will make two files you can use for your virtual host, and you'll find them in /var/www/mybooki.  The first one is booki.wsgi, and you can see we refer to it in the virtual host entry above.  The second file is wsgi.apache.  This contains the entry for the virtual host itself.  You can use this as the virtual host entry, but you'll need to make some modifications.  My example virtual host entries should give you some idea of what you'll need to modify.  Pay attention to the lines in bold in the example.  They should be in your booki virtual hosts entry.  If the wsgi.apache file doesn't have them, add them.  They are responsible for the spell check function of the Booki web page editor.

Using OBJAVI 2

If you've done everything right you should be able to go to http://127.0.0.1:8000 and see Booki running, and go to http://127.0.0.1 and see OBJAVI 2 running.  OBJAVI 2 looks like this:

OBJAVI 2

OBJAVI is run from within Book from the Export tab when you're editing a book.  If all you want to do is create PDF's and EPUBs you may never need to look at this page.  There is one thing you can do from here that you can't do from Booki's Export tab, and that is to create output as Templated HTML.  To do that you choose Templated HTML as the Document Type.

Templated HTML is not a kind of e-book, but it is worthy of a brief mention.  One of the ways that Booki is different from Wikis like Media Wiki (used for Wikipedia) is that with a normal Wiki anyone can edit any document and the edit is available to the readers of the Wiki instantly.  A normal static website makes it easy to control who can update the content, but this control means that updating the content is more work.  What Booki introduces is the idea of generating a static website from a Wiki.  The Wiki is used by the book authors but is not seen by the book's audience.  When the authors have something ready to publish to the world they use OBJAVI to generate a static website and copy it to the public web server.

By default the HTML looks like the FLOSS Manuals website.  Because it is generated using templates, you can easily add your own stylesheets, corporate logos, and the like to make the generated site look the way you want it to.

You should definitely think about creating a templated HTML version of your book if the contents are likely to be updated frequently.  The website version of your book can then act as a supplement to the e-book version.  The stable content will be in the e-book and the latest minor tweaks and corrections will be on the website.