Department of Physics and Astronomy

The Forbes Group

Installing Nikola

$\newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\uvect}[1]{\hat{#1}} \newcommand{\abs}[1]{\lvert#1\rvert} \newcommand{\norm}[1]{\lVert#1\rVert} \newcommand{\I}{\mathrm{i}} \newcommand{\ket}[1]{\left|#1\right\rangle} \newcommand{\bra}[1]{\left\langle#1\right|} \newcommand{\braket}[1]{\langle#1\rangle} \newcommand{\op}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\d}{\mathrm{d}} \newcommand{\pdiff}[3][]{\frac{\partial^{#1} #2}{\partial {#3}^{#1}}} \newcommand{\diff}[3][]{\frac{\d^{#1} #2}{\d {#3}^{#1}}} \newcommand{\ddiff}[3][]{\frac{\delta^{#1} #2}{\delta {#3}^{#1}}} \DeclareMathOperator{\erf}{erf} \DeclareMathOperator{\Tr}{Tr} \DeclareMathOperator{\order}{O} \DeclareMathOperator{\diag}{diag} \DeclareMathOperator{\sgn}{sgn} \DeclareMathOperator{\sech}{sech} $

Installing Nikola

Update January 2020: I recommend installing Nikola from a conda environment.yml file. I keep one on anaconda cloud so you can just go:

conda env create mforbes/blog
such as the following:

```yml
# environment.yml
name: blog
channels:
  - defaults
  - conda-forge
  - mforbes
dependencies:
  - jinja2           # Jinja2 Template engine for my styles.
  - nbstripout       # Cleaning output from notebooks
  - typogrify        # Required for the typogrify filter

  # These are needed by Nikola.  We install them so Conda has control.
  - babel>=2.6.0
  - blinker>=1.3
  - docutils>=0.13
  - doit>=0.32.0
  - lxml>=3.3.5
  - mako>=1.0.0
  - markdown>=3.0.0
  - natsort>=3.5.2
  - piexif>=1.0.3
  - pillow>=2.4.0
  - pygments>=1.6
  - python-dateutil>=2.6.0
  - requests>=2.2.0
  - setuptools>=24.2.0
  - unidecode>=0.04.16
  - yapsy>=1.11.223

  # Additional dependencies not in conda - they will be installed by pip.
  #- PyRSS2Gen>=1.1

  # Needed of ipynb format
  - nbconvert
  - notebook>=4.0.0

  # Needed for nikola auto
  - watchdog>=0.8.3  # Conda version gives "Failed to import fsevents. Fall back to kqueue"
  - aiohttp

  # My packages.
  - mmf_setup

  - pip
  - pip:
    #- nikola>=8.1.1
    # Need dev version for this issue with templates
    # https://github.com/getnikola/nikola/issues/3457
    - git+https://github.com/getnikola/nikola.git@master
    - pipdeptree
    #- watchdog>=0.8.3
    - ghp-import2         # Needed for github_deploy

    #- pyrss2gen>=1.1
    #- python-hglib==2.6.1

There is no conda recipe yet for Nikola, so it needs to be pip installed.

To uses some html features like SASS, I needed to install npm which I did at the system level.

conda env create -f environment.blog.yml
conda activate blog
npm install -g less            # lessc compiler required for CSS in zen- themes
sudo gem install sass          # sass compiler required for SASS based CSS
nikola plugin -i less
nikola plugin -i sass

Old Notes and Problems

I had some trouble pip-installing nikola with pip so had to install a few of the dependencies first with conda.

In [1]:
# This worked with python 3 in an anaconda install.
!pip install nikola webassets
!conda install ws4py watchdog   # Required for nikola auto
!pip install typogrify          # Required for the typogrify filter
!npm install -g less            # lessc compiler required for CSS in zen- themes
!sudo gem install sass          # sass compiler required for SASS based CSS
!nikola plugin -i less
!nikola plugin -i sass
Collecting nikola
  Using cached Nikola-7.7.6-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): webassets in /data/apps/anaconda/envs/work/lib/python2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): natsort>=3.5.2 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): docutils>=0.12 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): doit<=0.29.0,>=0.28.0 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): unidecode>=0.04.16 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): python-dateutil>=2.4.0 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): requests>=2.2.0 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): Pygments>=1.6 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): lxml>=3.3.5 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): husl>=4.0.2 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): logbook>=0.7.0 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): Pillow>=2.4.0 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): PyRSS2Gen>=1.1 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): mako>=1.0.0 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): Yapsy>=1.11.223 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): setuptools>=5.4.1 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages/setuptools-20.1.1-py2.7.egg (from nikola)
Requirement already satisfied (use --upgrade to upgrade): six in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from doit<=0.29.0,>=0.28.0->nikola)
Requirement already satisfied (use --upgrade to upgrade): macfsevents in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from doit<=0.29.0,>=0.28.0->nikola)
Requirement already satisfied (use --upgrade to upgrade): configparser in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from doit<=0.29.0,>=0.28.0->nikola)
Requirement already satisfied (use --upgrade to upgrade): MarkupSafe>=0.9.2 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from mako>=1.0.0->nikola)
Installing collected packages: nikola
Successfully installed nikola-7.7.6
In [14]:
# I needed this before with python 2
!conda install pillow
!pip install nikola livereload webassets
!conda update lxml
Fetching package metadata: ..........
Solving package specifications: .
# All requested packages already installed.
# packages in environment at /data/apps/anaconda/1.3.1:
#
pillow                    2.7.0                    py27_0  
Requirement already satisfied (use --upgrade to upgrade): nikola in /data/src/python/git/nikola
Requirement already satisfied (use --upgrade to upgrade): livereload in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages
Collecting webassets
  Downloading webassets-0.10.1.tar.gz (167kB)
    100% |################################| 167kB 386kB/s 
Requirement already satisfied (use --upgrade to upgrade): doit>=0.23.0 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): Pygments>=1.6 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): Pillow>=2.4.0 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): python-dateutil==2.4.0 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): docutils>=0.12 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): mako>=1.0.0 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): unidecode>=0.04.16 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): lxml>=3.3.5 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): yapsy>=1.10.423 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): PyRSS2Gen>=1.1 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): logbook>=0.7.0 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Collecting setuptools>=5.4.1 (from nikola)
  Using cached setuptools-12.1-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): natsort>=3.5.2 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): tornado in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from livereload)
Requirement already satisfied (use --upgrade to upgrade): six in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from doit>=0.23.0->nikola)
Requirement already satisfied (use --upgrade to upgrade): macfsevents in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from doit>=0.23.0->nikola)
Requirement already satisfied (use --upgrade to upgrade): MarkupSafe>=0.9.2 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from mako>=1.0.0->nikola)
Installing collected packages: setuptools, webassets
  Found existing installation: distribute 0.6.45
    Uninstalling distribute-0.6.45:
      Successfully uninstalled distribute-0.6.45

  Running setup.py install for webassets
    Installing webassets script to /data/apps/anaconda/1.3.1/bin
Successfully installed setuptools-0.6rc11 webassets-0.10.1
Fetching package metadata: ..........
# All requested packages already installed.
# packages in environment at /data/apps/anaconda/1.3.1:
#
lxml                      3.4.2                    py27_0  

Compiler

I had trouble installing macfsevents using the HPC compilers, so I had to disable them. When using the Xcode compilers, everything worked.

Lack of Root Permissions

If you don't have root access, then running npm or gem might cause issues (i.e. on Sage Mathcloud.) For npm, the suggestion here to:

npm config set prefix ~/.npm
cat > ~/.bash_aliases <<EOF

# Add local npm package to path
# http://stackoverflow.com/a/23889603/1088938
# http://stackoverflow.com/a/40830016/1088938
export PATH="$PATH:$HOME/.npm/bin"
export GEM_HOME="$HOME/.gem"
export PATH="$PATH:$GEM_HOME/bin"
EOF
. ~/.bash_aliases
npm install less
gem install sass

works.

New Blog/Website

These new instructions are based on my custom themes. See below to base your blog on public themes.

  1. Install Nikola (above) and activate the blog environment.

    conda env create -f environment.blog.yml
    

    You can also use anaconda-client to do this from my environment.blog.yml file on anaconda cloud. Note: Make sure there is no local environment.yml file, or creating the environment from anaconda cloud may fail.

    conda install anaconda-client
    conda env create mforbes/blog
    

CoCalc

Here is a complete walk-through using CoCalc. The example will be for the iSciMath main website. We do a couple of things here:

  • Store all images in a separate repo since these can be big.

Here is the play by play:

Initial Setup

  1. Create a new project on CoCalc and enable internet access. (This will require that you have some access to at least one upgrade... they needed to do this to prevent people from abusing the resources.)
  2. Create and activate the blog environment.

    anaconda2020                    # CoCalc-specific way of activating conda.
    conda install anaconda-client   # Not needed on CoCalc, but may be needed elsewhere
    conda env create mforbes/blog   # Make sure there is no local environment.yml file!
    conda clean -y --all            # Free disk space from downloads.
    conda activate blog
    

    As a check, you should now be able to run Nikola:

    $ nikola --version
    Nikola v8.1.1
    
  3. Create a Jupyter kernel (optional). Here we create a Jupyter Kernel so that we can activate the blog kernel from README.ipynb providing some interactive instructions. This is not needed for the general functionality of Nikola.

    mkdir -p ~/.local/share/jupyter/kernels/
    cp -r /ext/anaconda2020.02/share/jupyter/kernels/python3 ~/.local/share/jupyter/kernels/blog-py
    vi ~/.local/share/jupyter/kernels/blog-py/kernel.json
    

    Here is how I modified the kernel.json file:

    #kernel.json
    {
      "argv": [
        "/home/user/.conda/envs/blog/bin/python",
        "-m",
        "ipykernel_launcher",
        "-f",
        "{connection_file}"
     ],
     "display_name": "Blog",
     "language": "python"
    }
    
  4. Create a repo for your website and add some files:

    mkdir -p ~/repositories/iscimath_website
    cd ~/repositories/iscimath_website
    hg init
    tee .hgignore <<EOF
    syntax: glob
    
    syntax: regexp
    ^.output/
    ^.cache/
    EOF
    tee Makefile <<EOF
    auto:
        echo "https://cocalc.com/ff1cb986-feb2-4122-b55d-1700a5fd8553/server/8000"
        nikola auto -a 0.0.0.0 -p 8000
    .PHONY: auto
    EOF
    
  5. Use Nikola to create the site.

    (blog) ~/repositories/iscimath_website$ nikola init .
    Creating Nikola Site
    ====================
    
    This is Nikola v8.1.1.  We will now ask you a few easy questions about your new site.
    If you do not want to answer and want to go with the defaults instead, simply restart with the `-q` parameter.
    --- Questions about the site ---
    Site title [My Nikola Site]: iSciMath: Integrated Science and Mathematics
    Site author [Nikola Tesla]: Michael McNeil Forbes
    Site authors e-mail [n.tesla@example.com]: cdlg.iscimath+admin@gmail.com
    Site description [This is a demo site for Nikola.]: iSciMath Website.
    Site URL [https://example.com/]: https://www.iscimath.org/
    Enable pretty URLs (/page/ instead of /page.html) that do not need web server configuration? [Y/n] 
    --- Questions about languages and locales ---
    We will now ask you to provide the list of languages you want to use.
    Please list all the desired languages, comma-separated, using ISO 639-1 codes.  The first language will be used as the default.
    Type '?' (a question mark, sans quotes) to list available languages.
    Language(s) to use [en]: 
    
    Please choose the correct time zone for your blog. Nikola uses the tz database.
    You can find your time zone here:
    https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
    
    Time zone [Etc/UTC]: America/Los_Angeles
        Current time in America/Los_Angeles: 13:43:22
    Use this time zone? [Y/n] 
    --- Questions about comments ---
    You can configure comments now.  Type '?' (a question mark, sans quotes) to list available comment systems.  If you do not want any comments, just leave the field blank.
    Comment system: 
    
    That is it, Nikola is now configured.  Make sure to edit conf.py to your liking.
    If you are looking for themes and addons, check out https://themes.getnikola.com/ and https://plugins.getnikola.com/.
    Have fun!
    [2020-10-10 20:47:27] INFO: init: Created empty site at ..
    (blog) ~/repositories/iscimath_website$ tree
    .
    ├── conf.py
    ├── files
    ├── galleries
    ├── images
    ├── listings
    ├── pages
    └── posts
    
    6 directories, 1 file
    

    The main thing this does is create the conf.py file, which we should now commit to version control.

    (blog) ~/repositories/iscimath_website$ hg add
    adding .hgignore
    adding conf.py
    (blog) ~/repositories/iscimath_website$ hg com -m "Initial commit of Nikola-generated conf.py"
    

Content

Now we add some content. For demo purposes, we add simple markdown content from geometricanalysis.org.

  1. First we create a repo for the assets, and download some images

    mkdir -p ~/repositories/iscimath_website_assets
    cd ~/repositories/iscimath_website_assets
    hg init
    mkdir images
    
    wget https://images.squarespace-cdn.com/content/v1/5cb91b1c70468025fd21e98b/1555698446161-7CZZ16Y5BMUKXK4BXY15/ke17ZwdGBToddI8pDm48kHtOJuqdoEFYKNzpMbCdjQ1Zw-zPPgdn4jUwVcJE1ZvWQUxwkmyExglNqGp0IvTJZamWLI2zvYWH8K3-s_4yszcp2ryTI0HqTOaaUohrI8PI84wy8WKf6jeaYut84yuMMxu-RozL28G4_z9WCTpl1xsKMshLAGzx4R3EDFOm1kBS/events.jpg -O images/banner1.jpg
    wget https://images.squarespace-cdn.com/content/v1/5cb91b1c70468025fd21e98b/1555728634489-WHSXTS0E32WU1OHYS8Y6/ke17ZwdGBToddI8pDm48kNaADO1I1V8iOHJQMUQI2H1Zw-zPPgdn4jUwVcJE1ZvWQUxwkmyExglNqGp0IvTJZamWLI2zvYWH8K3-s_4yszcp2ryTI0HqTOaaUohrI8PIBCpzMMEguc8VVUBIRTMwFWs2R71F1SeBH0ohOKrYXboKMshLAGzx4R3EDFOm1kBS/upcomingevents.jpg -O images/ideas.jpg
    hg add
    hg commit -m "IMG: Added images"
    
  2. Link this into our main repo

    cd ~/repositories/iscimath_website
    mkdir _ext
    ln -s ~/repositories/iscimath_website_assets _ext/
    rmdir images
    ln -s _ext/iscimath_website_assets/images .
    hg add images
    
  3. Now create an index file. We can do this in a variety of formats, but ReStructuredText gives us more control than Markdown.

    .. title: Center for Geometric Analysis and Data
    .. slug: index
    .. date: 2020-10-10 15:03:35 UTC-07:00
    .. tags: 
    .. category: 
    .. link: 
    .. description: 
    .. type: text
    
    .. image:: images/banner1.jpg
      :width: 100%
      :alt: Alternative text
    
    The Center for Geometric Analysis and Data is the research group led
    by Kevin R. Vixie. As the name suggests, the research is focused on
    topics at the intersection of analysis and geometry as well as those
    suggested by challenges from data science or data generated by
    scientific questions. To find out more about Kevin and his circle of
    collaborators, visit the Group Members page, peruse the notes and
    papers, read the blog, or follow the links that abound throughout.
    EOF
    

At this point we now need to edit the conf.py file to convert this to a website rather than a blog. We follow the process described here. The following was changed:

# Modified conf.py
...
# Website: remove destination directory to generate pages in the root directory
 PAGES = (
    ("pages/*.rst", "", "page.tmpl"),
    ("pages/*.md", "", "page.tmpl"),
    ("pages/*.txt", "", "page.tmpl"),
    ("pages/*.html", "", "page.tmpl"),
)
...

# Website: we will provide our own index, so blog stuff goes here instead
INDEX_PATH = "blog"

Commit these changes:

hg add
hg com -m "Initial version of website."

Preview

Now you can build and view the page:

nikola auto -a 0.0.0.0 -p 8000

This will be hosted on CoCalc as described here as long as the nikola server is running. Only users authenticated to the CoCalc project can see it.

I have also put these commands into the Makefile, so one can do

make auto

Theming

To use my new themes, based on the zen-jinja theme, I needed to install that SASS plugin:

nikola plugin -i sass

and add the following to the conf.py file:

# conf.py
...
NAVIGATION_LINKS = {
    DEFAULT_LANG: (
        ('/index.html', 'Home', 'fa fa-home'),
        ('/archive.html', 'Archives', 'fa fa-folder-open'),
        ('/categories/index.html', 'Tags', 'fa fa-tags'),
        ('/rss.xml', 'RSS', 'fa fa-rss'),
        ('https://getnikola.com', 'About me', 'fa fa-user'),
        ('https://twitter.com/getnikola', 'My Twitter', 'fab fa-twitter'),
        ('https://github.com/getnikola', 'My Github', 'fab fa-github'),
    )
}
...
# Compiler to process Sass files.
SASS_COMPILER = 'sass'

# A list of options to pass to the Sass compiler.
# Final command is: SASS_COMPILER SASS_OPTIONS file.s(a|c)ss
SASS_OPTIONS = []

Setup Blog (Old Version)

  • nikola init test_blog
    • Answer the questions
  • Follow the instructions on http://www.damian.oquanta.info/posts/ipython-plugin-for-nikola-updated.html for setting up your blog to work with IPython notebooks:

    • nikola install_theme zen-ipython
    • Add the following to your conf.py file:

      NAVIGATION_LINKS = {  # Specific for the zen-ipython theme
          DEFAULT_LANG: (
              ('/index.html', 'Home', 'icon-home'),
              ('/archive.html', 'Archives', 'icon-folder-open-alt'),
              ('/categories/index.html', 'Tags', 'icon-tags'),
              ('/rss.xml', 'RSS', 'icon-rss'),
              ('http://getnikola.com', 'About me', 'icon-user'),
              ('https://twitter.com/getnikola', 'My Twitter', 'icon-twitter'),
              ('https://github.com/getnikola', 'My Github', 'icon-github'),
          )
      }
      ...
      POSTS = (  # Keep only one so that ``nikola new_post`` defaults to ipynb
          #("posts/*.rst", "posts", "post.tmpl"),
          #("posts/*.txt", "posts", "post.tmpl"),
          ("posts/*.ipynb", "posts", "post.tmpl"),
      )
      PAGES = (
          ("stories/*.rst", "stories", "story.tmpl"),
          ("stories/*.txt", "stories", "story.tmpl"),
          ("stories/*.ipynb", "stories", "story.tmpl"),
      )
      ...
      THEME = "zen-ipython"
      

Generating Metadata

I have a bunch of notebooks I want to "blog" but need to generate metadata. Here is some code to do this programatically from the file information.

In [13]:
import os.path
import time
import nikola.utils

POST_DIR = 'mmfblog/posts/'
all_files = os.listdir(POST_DIR)
new_posts = [_f for _f in all_files 
             if _f.endswith('.ipynb') 
             and _f[:-6] + ".meta" not in all_files]

_template = '''\
.. title: {title}
.. slug: {slug}
.. date: {date}
.. tags:
.. link:
.. description:
.. type: text
'''

TIME_FMT = '%Y-%m-%d %H:%M:%S %Z'

doit = True

for f in new_posts:
    subs = {}
    filename = os.path.join(POST_DIR, f)
    title, ext = os.path.splitext(f)
    slug = nikola.utils.slugify(unicode(title))
    date = time.strftime(TIME_FMT, time.localtime(os.path.getmtime(filename)))
    subs.update(title=title, slug=slug, date=date)

    if slug != title:
        new_filename = os.path.join(POST_DIR, slug+ext)
        print("mv {0} {1}".format(filename, new_filename))
        if doit:
            os.rename(filename, new_filename)
    meta_filename = os.path.join(POST_DIR, ".".join([slug, 'meta']))
    meta = _template.format(**subs)
    print meta
    if doit:
        with open(meta_filename, 'w') as _mf:
            _mf.write(meta)
.. title: 3d-visualization
.. slug: 3d-visualization
.. date: 2014-10-24 23:51:28 PDT
.. tags:
.. link:
.. description:
.. type: text

.. title: coroutines
.. slug: coroutines
.. date: 2014-06-10 10:49:57 PDT
.. tags:
.. link:
.. description:
.. type: text

.. title: cylindrical-dvr
.. slug: cylindrical-dvr
.. date: 2013-10-13 21:49:16 PDT
.. tags:
.. link:
.. description:
.. type: text

.. title: dotproduct
.. slug: dotproduct
.. date: 2014-06-10 20:04:04 PDT
.. tags:
.. link:
.. description:
.. type: text

mv mmfblog/posts/Freeze (For cluster computing).ipynb mmfblog/posts/freeze-for-cluster-computing.ipynb
.. title: Freeze (For cluster computing)
.. slug: freeze-for-cluster-computing
.. date: 2014-10-24 21:12:04 PDT
.. tags:
.. link:
.. description:
.. type: text

.. title: git-annex
.. slug: git-annex
.. date: 2014-06-03 02:25:07 PDT
.. tags:
.. link:
.. description:
.. type: text

.. title: interfaces
.. slug: interfaces
.. date: 2014-09-23 01:15:53 PDT
.. tags:
.. link:
.. description:
.. type: text

.. title: ipython-cluster
.. slug: ipython-cluster
.. date: 2014-10-16 21:52:55 PDT
.. tags:
.. link:
.. description:
.. type: text

mv mmfblog/posts/IPythonParallel.ipynb mmfblog/posts/ipythonparallel.ipynb
.. title: IPythonParallel
.. slug: ipythonparallel
.. date: 2015-02-15 18:43:33 PST
.. tags:
.. link:
.. description:
.. type: text

.. title: mayavi
.. slug: mayavi
.. date: 2014-03-24 02:44:58 PDT
.. tags:
.. link:
.. description:
.. type: text

mv mmfblog/posts/Mercurial Repository Conversion.ipynb mmfblog/posts/mercurial-repository-conversion.ipynb
.. title: Mercurial Repository Conversion
.. slug: mercurial-repository-conversion
.. date: 2014-04-12 02:04:56 PDT
.. tags:
.. link:
.. description:
.. type: text

mv mmfblog/posts/MHD.ipynb mmfblog/posts/mhd.ipynb
.. title: MHD
.. slug: mhd
.. date: 2014-05-05 15:42:04 PDT
.. tags:
.. link:
.. description:
.. type: text

.. title: optimization
.. slug: optimization
.. date: 2014-07-11 13:28:41 PDT
.. tags:
.. link:
.. description:
.. type: text

.. title: optimization_with_numba
.. slug: optimization_with_numba
.. date: 2013-11-26 12:45:25 PST
.. tags:
.. link:
.. description:
.. type: text

.. title: profiling_python
.. slug: profiling_python
.. date: 2014-02-09 23:21:14 PST
.. tags:
.. link:
.. description:
.. type: text

mv mmfblog/posts/PythonContainers.ipynb mmfblog/posts/pythoncontainers.ipynb
.. title: PythonContainers
.. slug: pythoncontainers
.. date: 2015-02-15 04:15:06 PST
.. tags:
.. link:
.. description:
.. type: text

mv mmfblog/posts/reStructuredText.ipynb mmfblog/posts/restructuredtext.ipynb
.. title: reStructuredText
.. slug: restructuredtext
.. date: 2014-08-24 16:43:25 PDT
.. tags:
.. link:
.. description:
.. type: text

.. title: saddlepoint
.. slug: saddlepoint
.. date: 2014-04-24 17:11:55 PDT
.. tags:
.. link:
.. description:
.. type: text

.. title: software_carpentry
.. slug: software_carpentry
.. date: 2014-06-01 18:45:44 PDT
.. tags:
.. link:
.. description:
.. type: text

mv mmfblog/posts/Weizsacker Term.ipynb mmfblog/posts/weizsacker-term.ipynb
.. title: Weizsacker Term
.. slug: weizsacker-term
.. date: 2014-09-29 00:27:25 PDT
.. tags:
.. link:
.. description:
.. type: text

mv mmfblog/posts/Widgets.ipynb mmfblog/posts/widgets.ipynb
.. title: Widgets
.. slug: widgets
.. date: 2014-07-18 20:03:17 PDT
.. tags:
.. link:
.. description:
.. type: text

In [10]:
f = "mmfblog/posts/optimization.ipynb"
time.strftime(TIME_FMT, time.gmtime(os.path.getmtime(f)))
Out[10]:
'2014-07-11 20:28:41 PST'

Updating Nikola

If you need the latest development version of Nikola, you can install form sources:

conda install notebook nbformat nbconvert pillow cloudpickle docutils lxml mako natsort python-dateutil requests unidecode

pip install git+https://github.com/getnikola/nikola.git webassets

I needed this before v7.3.1 to fix issue #1582.

Later I needed this before version v7.8.0 to deal with Ipython version 4.0 deprecations.

Later I needed this before this commit when sys.path was not including the root directory (where I keep a module that needs to be imported.)

In [25]:
import IPython.nbformat
IPython.version_info
IPython.nbformat.read
from IPython.nbconvert.exporters import HTMLExporter
In [29]:
exportHtml = HTMLExporter()
with open('test_blog/posts/test.ipynb') as f:
    nb_json = IPython.nbformat.reads(f.read(), IPython.nbformat.current_nbformat)

(body, resources) = exportHtml.from_notebook_node(nb_json)

MathJax

If you want to use LaTeX in your notebook, be sure to add the following to your conf.py file.

# If you are using the compile-ipynb plugin, just add this one:
MATHJAX_CONFIG = """
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
    tex2jax: {
        inlineMath: [ ['$','$'], ["\\\(","\\\)"] ],
        displayMath: [ ['$$','$$'], ["\\\[","\\\]"] ],
        processEscapes: true
    },
    displayAlign: 'left', // Change this to 'center' to center equations.
    "HTML-CSS": {
        styles: {'.MathJax_Display': {"margin": 0}}
    }
});
</script>
"""

Comments

I am trying the Disqus comment system since this was recommended in the Nikola manual. I had to create a Disqus account, and then register a new site to get the shortname. Some immediately apparent issues:

  • Disqus puts a lot of things in a global namespace. For example, the username (my usual handle was already taken) and when registering a new site, you need to guess a unique identifier <something>.disqus.com.

Website

Nikola can also be used for a static website. Here is the list of packages and versions I used when doing this:

In [1]:
!conda list -n blog
# packages in environment at /data/apps/conda/envs/blog:
#
# Name                    Version                   Build  Channel
aiohttp                   3.6.2            py36h1de35cc_0  
appnope                   0.1.0            py36hf537a9a_0  
argh                      0.26.2                   py36_0  
argon2-cffi               20.1.0           py36haf1e3a3_1  
async-timeout             3.0.1                    py36_0  
async_generator           1.10             py36h28b3542_0  
attrs                     20.2.0                     py_0  
babel                     2.8.0                      py_0  
backcall                  0.2.0                      py_0  
bleach                    3.2.1                      py_0  
blinker                   1.4                      py36_0  
brotlipy                  0.7.0           py36haf1e3a3_1000  
ca-certificates           2020.7.22                     0  
certifi                   2020.6.20                py36_0  
cffi                      1.14.3           py36hed5b41f_0  
chardet                   3.0.4                 py36_1003  
cloudpickle               1.6.0                      py_0  
cryptography              3.1.1            py36hddc9c9b_0  
decorator                 4.4.2                      py_0  
defusedxml                0.6.0                      py_0  
docutils                  0.16                     py36_1  
doit                      0.32.0                   py36_0  
entrypoints               0.3                      py36_0  
freetype                  2.10.2               ha233b18_0  
ghp-import2               1.0.1                    pypi_0    pypi
icu                       58.2                 h0a44026_3  
idna                      2.10                       py_0  
idna_ssl                  1.1.0                    py36_0  
importlib-metadata        1.7.0                    py36_0  
importlib_metadata        1.7.0                         0  
ipykernel                 5.3.4            py36h5ca1d4c_0  
ipython                   7.16.1           py36h5ca1d4c_0  
ipython_genutils          0.2.0                    py36_0  
jedi                      0.17.2                   py36_0  
jinja2                    2.11.2                     py_0  
jpeg                      9b                   he5867d9_2  
jsonschema                3.2.0                    py36_1  
jupyter_client            6.1.7                      py_0  
jupyter_core              4.6.3                    py36_0  
jupyterlab_pygments       0.1.2                      py_0  
lcms2                     2.11                 h92f6f08_0  
libcxx                    10.0.0                        1  
libedit                   3.1.20191231         h1de35cc_1  
libffi                    3.3                  hb1e8313_2  
libiconv                  1.16                 h1de35cc_0  
libpng                    1.6.37               ha441bb4_0  
libsodium                 1.0.18               h1de35cc_0  
libtiff                   4.1.0                hcb84e12_1  
libxml2                   2.9.10               h3b9e6c8_1  
libxslt                   1.1.34               h83b36ba_0  
lxml                      4.5.2            py36h63b7cb6_0  
lz4-c                     1.9.2                hb1e8313_1  
macfsevents               0.8.1            py36h1de35cc_0  
mako                      1.1.3                      py_0  
markdown                  3.3                      py36_0  
markupsafe                1.1.1            py36h1de35cc_0  
mistune                   0.8.4            py36h1de35cc_0  
mmf_setup                 0.3.0                      py_0    mforbes
multidict                 4.7.6            py36haf1e3a3_1  
natsort                   7.0.1                      py_0  
nbclient                  0.5.0                      py_0  
nbconvert                 6.0.7                    py36_0  
nbformat                  5.0.7                      py_0  
nbstripout                0.3.9              pyh9f0ad1d_0    conda-forge
ncurses                   6.2                  h0a44026_1  
nest-asyncio              1.4.1                      py_0  
nikola                    8.1.1                    pypi_0    pypi
notebook                  6.1.4                    py36_0  
olefile                   0.46                     py36_0  
openssl                   1.1.1h               haf1e3a3_0  
packaging                 20.4                       py_0  
pandoc                    2.10.1                        0  
pandocfilters             1.4.2                    py36_1  
parso                     0.7.0                      py_0  
pathtools                 0.1.2                      py_1  
pexpect                   4.8.0                    py36_0  
pickleshare               0.7.5                    py36_0  
piexif                    1.1.3                      py_2    conda-forge
pillow                    7.2.0            py36ha54b6ba_0  
pip                       20.2.3                   py36_0  
pipdeptree                1.0.0                    pypi_0    pypi
prometheus_client         0.8.0                      py_0  
prompt-toolkit            3.0.7                      py_0  
ptyprocess                0.6.0                    py36_0  
pycparser                 2.20                       py_2  
pygments                  2.7.1                      py_0  
pyopenssl                 19.1.0                     py_1  
pyparsing                 2.4.7                      py_0  
pyrsistent                0.17.3           py36haf1e3a3_0  
pyrss2gen                 1.1                      pypi_0    pypi
pysocks                   1.7.1                    py36_0  
python                    3.6.12               h26836e1_2  
python-dateutil           2.8.1                      py_0  
python-hglib              2.6.1                      py_0    mforbes
pytz                      2020.1                     py_0  
pyyaml                    5.3.1            py36haf1e3a3_1  
pyzmq                     19.0.2           py36hb1e8313_1  
readline                  8.0                  h1de35cc_0  
requests                  2.24.0                     py_0  
send2trash                1.5.0                    py36_0  
setuptools                50.3.0           py36h0dc7051_1  
six                       1.15.0                     py_0  
smartypants               2.0.0              pyh9f0ad1d_2    conda-forge
sqlite                    3.33.0               hffcf06c_0  
terminado                 0.8.3                    py36_0  
testpath                  0.4.4                      py_0  
tk                        8.6.10               hb0a8c7a_0  
tornado                   6.0.4            py36h1de35cc_1  
traitlets                 4.3.3                    py36_0  
typing_extensions         3.7.4.3                    py_0  
typogrify                 2.0.7                    py36_0    conda-forge
unidecode                 1.1.1                      py_0  
urllib3                   1.25.10                    py_0  
watchdog                  0.10.3           py36haf1e3a3_0  
wcwidth                   0.2.5                      py_0  
webencodings              0.5.1                    py36_1  
wheel                     0.35.1                     py_0  
xz                        5.2.5                h1de35cc_0  
yaml                      0.2.5                haf1e3a3_0  
yapsy                     1.12.2                     py_0    conda-forge
yarl                      1.6.0            py36haf1e3a3_0  
zeromq                    4.3.2                hb1e8313_3  
zipp                      3.3.0                      py_0  
zlib                      1.2.11               h1de35cc_3  
zstd                      1.4.5                h41d2c2f_0  
$ nikola init forbes-group
Creating Nikola Site
====================

This is Nikola v7.8.3.  We will now ask you a few easy questions about your new site.
If you do not want to answer and want to go with the defaults instead, simply restart with the `-q` parameter.
--- Questions about the site ---
Site title [My Nikola Site]: Forbes Group Website
Site author [Nikola Tesla]: Michael McNeil Forbes
Site author's e-mail [n.tesla@example.com]: m.forbes@wsu.edu
Site description [This is a demo site for Nikola.]: Forbes Group Research Site
Site URL [https://example.com/]: http://swan.physics.wsu.edu/forbes/
Enable pretty URLs (/page/ instead of /page.html) that don't need web server configuration? [Y/n] 
--- Questions about languages and locales ---
We will now ask you to provide the list of languages you want to use.
Please list all the desired languages, comma-separated, using ISO 639-1 codes.  The first language will be used as the default.
Type '?' (a question mark, sans quotes) to list available languages.
Language(s) to use [en]: 

Please choose the correct time zone for your blog. Nikola uses the tz database.
You can find your time zone here:
https://en.wikipedia.org/wiki/List_of_tz_database_time_zones

Time zone [America/Vancouver]: 
    Current time in America/Vancouver: 00:53:13
Use this time zone? [Y/n] 
--- Questions about comments ---
You can configure comments now.  Type '?' (a question mark, sans quotes) to list available comment systems.  If you do not want any comments, just leave the field blank.
Comment system: disqus
You need to provide the site identifier for your comment system.  Consult the Nikola manual for details on what the value should be.  (you can leave it empty and come back later)
Comment system site identifier: forbes-group

That's it, Nikola is now configured.  Make sure to edit conf.py to your liking.
If you are looking for themes and addons, check out https://themes.getnikola.com/ and https://plugins.getnikola.com/.
Have fun!
[2017-01-21T08:53:36Z] INFO: init: Created empty site at forbes-group.

Now version control the generated files:

cd forbes-group
hg init
hg add
hg com -m "Initial Commit"

We now switch the theme to zen-ipython which requires installing some themes:

$ nikola install_theme zen-ipython
[2017-01-21T09:01:18Z] WARNING: Nikola: Cannot load theme "zen-ipython", using 'bootstrap3' instead.
[2017-01-21T09:01:18Z] INFO: theme: Downloading 'https://themes.getnikola.com/v7/zen-ipython.zip'
[2017-01-21T09:01:19Z] INFO: theme: Extracting 'zen-ipython' into themes/
[2017-01-21T09:01:19Z] NOTICE: theme: This theme has a sample config file.  Integrate it with yours in order to make this theme work!
Contents of the conf.py.sample file:

    NAVIGATION_LINKS = {
        DEFAULT_LANG: (
            ('/index.html', 'Home', 'icon-home'),
            ('/archive.html', 'Archives', 'icon-folder-open-alt'),
            ('/categories/index.html', 'Tags', 'icon-tags'),
            ('/rss.xml', 'RSS', 'icon-rss'),
            ('https://getnikola.com', 'About me', 'icon-user'),
            ('https://twitter.com/getnikola', 'My Twitter', 'icon-twitter'),
            ('https://github.com/getnikola', 'My Github', 'icon-github'),
        )
    }

[2017-01-21T09:01:19Z] INFO: theme: Downloading 'https://themes.getnikola.com/v7/zen-jinja.zip'
[2017-01-21T09:01:19Z] INFO: theme: Extracting 'zen-jinja' into themes/
[2017-01-21T09:01:19Z] NOTICE: theme: This theme has a sample config file.  Integrate it with yours in order to make this theme work!
Contents of the conf.py.sample file:

    NAVIGATION_LINKS = {
        DEFAULT_LANG: (
            ('/index.html', 'Home', 'icon-home'),
            ('/archive.html', 'Archives', 'icon-folder-open-alt'),
            ('/categories/index.html', 'Tags', 'icon-tags'),
            ('/rss.xml', 'RSS', 'icon-rss'),
            ('https://getnikola.com', 'About me', 'icon-user'),
            ('https://twitter.com/getnikola', 'My Twitter', 'icon-twitter'),
            ('https://github.com/getnikola', 'My Github', 'icon-github'),
        )
    }

[2017-01-21T09:01:19Z] NOTICE: theme: Remember to set THEME="zen-ipython" in conf.py to use this theme.

Organization

In addition to static content, I would like to have posts on my webpage describe research, projects, etc. These can be organized in "sections" by placing them in appropriate directories in posts and then specifying the output directories in the POSTS variable in conf.py. One can then filter them, for example, with {% for post in posts if post.section_name().lower() == "highlights" %} in a template.

I use this with a special template templates/research_highlights_list.tmpl to format the research highlights on my index page.

Customization

Themes

Here we collect some notes about how themes work and can be customized. Themes are stored in the themes directory. For example, themes/zen-ipython. I have a separate repo for custom themes which I include in _ext/forbes_group_website_theme and then link into the themes.

A Nikola site can be customized in several ways:

  1. conf.py: Many customizations appear in this file, including some that might not be obvious:
    • INDEX_READ_MORE_LINK: This variable is used to customize the "Read more" link. We used this to modify these to look like a button matching the "Editorial" style.
    • NAVIGATION_LINKS: These appear in the side-bar, so you can specify the icons here (from Font Awesome if you use the zen-* themes).
    • POSTS/PAGES: Here you can specify which template files are used for posts and pages.
  2. Modify the templates. To do this, I found it easiest to copy all of the inherited template files into a new theme's template directory so I can see what is happening. Here are some important templates:

    • index.tmpl: Used for the index page collecting your blog posts.
    • post.tmpl: Used for the individual blog posts. This can be selected in POSTS variable of conf.py. These can also be controlled in the actual posts with the template variable.
    • story.tmpl: Used for static pages. This can be selected in PAGES variable of conf.py. These can also be controlled in the actual posts with the template variable.

zen-ipython

This directory contains a file themes/zen-ipython/parent that specify the parent theme - in this case themes/zen-jinja. Thus, to find assets, etc. one might need to look in several places. Here is the full hierarchy for the themes/zen-ipython theme:

The last two are pre-installed themes whose location on disk can be found using the specified command. I copied them into my site folder for easy reference.

Icons

The zen-ipython theme uses icons from Font Awesome. To see the available icons and names, look at the icon list for the appropriate version (version 3.2.1 here). You can then specify the appropriate fonts in conf.py.

Less/Sass

The zen themes use [less], which they tell you in the Readme.md file can be enabled by setting USE_BUNDLES = False in conf.py after installing the lessc compiler and nikola plugin -i less, however, you also need to rename/remove themes/zen-jinja/assets/css/main.cssmain.css file or else you will get the following error (since it is also built with lessc...)

ERROR: Two different tasks can't have a common target.'output/assets/css/main.css' is a target for copy_assets:output/assets/css/main.css and build_less:output/assets/css/main.css.

Targets for less are specified in the file themes/zen-jinja/less/targets.

I am using a custom theme which provides sass stylesheets. This requires installing the sass compiler and then one can use the sass plugin to generate appropriate CSS files. This theme was inheriting from the zen-jinja theme and so the main.css file was clashing with the previous file specified in themes/zen-jinja/less/targets. To resolve this, I added a custom less directory in the new theme with an empty themes/html5up-editorial/less/targets file.

Jinja2

Here are some tips for working with Jinja2 (see the references section below for details.)

  • You can see the value of variables using code like this: {{ post.tags|pprint }} or if you need to {{ post.__dict__.keys()|pprint }}.

Images

One issue I had is how to include images in posts. The usual approach for notebooks is to include the images in a file relative to the notebook. For example, if the files are in a folder images then one can refer to this as

![Image](images/eli-francis-100644.jpg)

There are two issues with this approach:

  1. Nikola will not copy these images to the output. This can be overcome by storing the files in the images folder at the root of your project, and then specifying where these should appear in the output by specifying IMAGE_FOLDERS = {'images': 'images'} (the default value) in your conf.py file. You can then symlink this file to your posts directory.
  2. The second issue is that this image will produce a link like this in the output:

     /highlights/prerequesites/images/eli-francis-100644.jpg
    
    

    I.e. this is a link relative to the post which would require a complicated set of non-standard IMAGE_FOLDERS = {'images': 'images'} mappings.

The standard Nikola solution is to place the images in a images/ folder in the top level of your project as above with IMAGE_FOLDERS = {'images': 'images'} in your conf.py file. Then refer with an absolute address:

![Image](/images/eli-francis-100644.jpg)

Unfortunately, this breaks in the notebook rendering which cannot see higher-level directories. For example, if you start the Jupyter notebook server in the same directory as the post, then one would have to use the following absolute path in the notebook in order to see the image:

![Image](/tree/images/eli-francis-100644.jpg)

This location could then be hacked with Nikola using IMAGE_FOLDERS = {'images': 'tree/images'}.

Solutions

I see three possible solutions:

  1. Use the modified "tree" hack above:

    1. Place images in

      images/eli-francis-100644.jpg
    2. Symlink this to an images folder next to your posts.

    3. Refer to your images as

      ![Image](/tree/images/eli-francis-100644.jpg)
    4. Use the following mapping in your conf.py file:

      IMAGE_FOLDERS = {'images': 'tree/images'}
  2. Mirror the structure of your posts in your images directory. For example, if your posts are in posts/highlights/my_post.ipynb, then:

    1. Place images in

      images/highlights/my_post/images/eli-francis-100644.jpg
    2. Symlink these images to your posts directory:

      link -s ../../images/highlights/my_post/images .
    3. Refer to images with a relative path:

      ![Image](images/eli-francis-100644.jpg)
    4. Map the images to the root level so they are included in the relative path:

      IMAGE_FOLDERS = {'images': ''}
  3. Use the Nikola recommend approach of linking to an absolute path and extend the jupyter notebook server to properly redirect these requests.

    1. Place images in

      images/eli-francis-100644.jpg
    2. Refer to your images using an absolute path

      ![Image](/images/eli-francis-100644.jpg)
    3. Run jupyter notebook from the top level (where images is) after installing and loading my notebook server extension mmf_nikola_nbserver_extension (this is done in the file jupyter_notebook_config.py:

       pip install -e mmfsite/mmf_nikola_nbserver_extension
       jupyter serverextension enable mmf_nikola_nbserver_extension

I am presently using the last approach.

Notebook Conversion

The notebooks are converted to HTML through the nbcovert process. This can be customized through templates or providing plugins.

  • The 'IPYNB_CONFIG' variable in conf.py is passed to the HTMLExporter. To see possible options, run jupyter nbconvert --help-all. For example, I have added a preprocessor mmf_nikola_nbserver_extension.ArXivLinks which replaces references to preprints with a URL. This is enabled as follows in conf.py:

    IPYNB_CONFIG = {

      'Exporter': {
          'preprocessors': ['mmf_nikola_nbserver_extension.ArXivLinks']
      }
    

    }

Plugins

Sometimes you just can't seem to do what you want with the existing tools. One way of extending Nikola is to use a plugin. First check the Nikola Plugins to see if you can find something, otherwise you can create your own in the plugins directory. To add a simple reStructuredText directive, for example, you can install and copy the book_figure plugin, which is a fairly simple plugin that creates a book_figure directive:

nikola plugin -i book_figure
mv plugins/book_figure plugins/mmf

then edit the plugin.

There are also some useful plugins: here is a list of the plugins I sometimes use:

nikola plugin -i less
nikola plugin -i sass
nikola plugin -i localsearch

References

People who have customized Nikola and describe the process:

Official Documentation:

Jinja Templating:

Some nice HTML and CSS templates:

  • HTML5 UP: Some nice, responsive, and clean HTML and CSS templates. We have tried to massage one of these into a website.
In [ ]: