Namespaces

Usage of namespaces:

Stasis: collecting directories and files. Ring middleware: Serving the correct content type. Prone Debug: Debugging Clojure code in the browser. Java Shell: Running Emacs to convert org files to HTML. Enlive: Metadata collection.

(ns docs-gen.render
  (:require [stasis.core :as stasis]
	    [ring.middleware.content-type :refer [wrap-content-type]]
	    [clojure.java.shell :refer [sh]]
	    [clojure.java.io :as io]
	    [prone.debug :refer [debug]]
	    [docs-gen.collect :as collect]
	    [docs-gen.transform :as transform])
  (:use [net.cgrand.enlive-html]))

Collector functions

(defn back-one-directory [s]
  (let [idx (.lastIndexOf s "/")]
    (subs s 0 (+ idx 1))))

(defn collect-vals [key config] 
(let [vals (map key (:sites config))]
(when (every? (complement nil?) vals) vals)))

Configuration

(def root-dir (back-one-directory (System/getProperty "user.dir")))

(def code
{:site "code"
:metadata [:loc]
:source (str root-dir "code/source")})


(def dissertation
{:site "dissertation"
:metadata [:words]
:source (str root-dir "dissertation/source")})

(def design
{:site "design"
:metadata []
:source (str root-dir "design/source")})

(def help
{:site "help"
:metadata []
:source (str root-dir "help/source")})

(def sites [code dissertation design help])

(def config {
:mode :serve ;:export
:render-mode :full
:serve {:root "http://localhost:3000"}
:export {:root "http://nclverse.github.io"}
 :output "/Users/Prabros/Dropbox/ncl/stage3/clojure/apps"
:sites sites})

URL Helpers

Replaces .org with given string

Org Converter

We convert each org page to HTML using Emacs.

Org to HTML

This uses a script stored inside the resources directory to convert org file to HTML using the config options specified per file. Need to decide if a global org file config makes sense.

(defn org->html [org-content]
"Prerequisite: Emacs with Org-Mode installed.
Converts given org file to html."
(:out (sh "emacs" "--script" "resources/scripts/org-to-html.el" org-content)))

Metadata Collection Helpers

Word Count

(defn word-count [entry]
   (let [nodes (select-nodes* (html-snippet entry) [text-node])
   text (apply str nodes)]
   (count (re-seq #"\w+" (.toLowerCase text)))))

(defn loc [entry]
   (let [nodes (select-nodes* (select (html-snippet entry) [:pre]) [text-node])
   text (apply str nodes)]
   (count (re-seq #"\n" text))))

#+ATTR_HTML :class smell

URL Components

Pretty contrived right now but supports all the current documentation sites. Better to make this support. Need to change this to support arbitrary amount of nesting. 5 seems like a good limit if arbitrary nesting is a bad idea. Need to reflect on this problem further.

(defn url-components [url]
  (zipmap [:category :title :subtitle] (filter not-empty (clojure.string/split url #"/"))))

Find Title

Gets the title of the converted HTML page.

(defn title [x]
  (first
   (select
   (html-snippet x) [:h1 text-node])))

Meta data function map

(defn metadata-fn [k]
(letfn [(not-found [x] "No such metadata collector exists")]
(or ({:loc loc
     :words word-count} k) not-found)))

Weave metadata

Collects metadata of each given function.

(defn weave-meta-data [metadata html-page]
  {:pre [(vector? metadata)] :post [#(map? %)]}
  (zipmap metadata (map #((metadata-fn %) html-page) metadata)))

(defn nav-element [metadata url page]
(let [components (url-components url)
meta-coll (weave-meta-data metadata page)]
(into components meta-coll)))

Collect metadata from HTML pages Creates a map of the form:

(defn structure-page [url page] {:page page :url (url-components url) :meta {:title (title page)}})

Renderers

Recombining the pages that have been transformed and rendering them. Comes in two flavours. One with meta data collection (involves expensive computation, slows down the generator considerably) and one without(quick).

(defn quick-render [root site _ pages]
(map #(transform/transform-page root site meta [] (structure-page site (org->html %))) pages))

Render with Meta

Weaves the metadata of all pages with the current page. Gives out a map of url with the pages weaved with meta data.

(defn nav-builder [root site metadata urls pages]
  (let [nav (map (partial nav-element metadata) urls pages)]
(sort-by #(vec (map % [:title :subtitle])) nav)))

(defn full-render [root site metadata urls pages]
(let [html-pages (map org->html pages)
      sites (collect-vals :site config)
      pages (map structure-page urls html-pages)
      nav (nav-builder root site metadata urls html-pages)]
  (map (partial transform/transform-page root site metadata sites nav) pages)))

(defn renderer [root site metadata mode]
({:quick #(quick-render root site %1 %2)
  :full #(full-render root site metadata %1 %2)} mode))

Build Site Map

Builds a website.

(defn build-site [root render-mode {:keys [site metadata source output]}]
  (let [render-fn (renderer root site metadata render-mode)
  assets (collect/collect-all site source render-fn)]
assets))

The progression is thus.

Collect -> Transform -> Render

The first two stages are done here.

Collecting Files

(defn collect-files [{:keys [sites mode render-mode] :as config}]
  (let [root (:root (mode config))]
    (stasis/merge-page-sources (apply merge-with into (map #(build-site root render-mode %) sites)))))

Finale

Servers or exports the pages. A multimethod to either render or export based on the configuration.

(defmulti render :mode)

(defmethod render :serve [config]
(wrap-content-type (stasis/serve-pages (collect-files config))))

(defmethod render :export [config]
  (stasis/export-pages (collect-files config) (:output config)))

(def app (render config))

(defn export-site-builder [site-map dir]
  (stasis/export-pages (stasis/merge-page-sources (build-site (:root (:export config)) :full site-map)) dir))


(defn exporter [site]
  (if-let [site-map 
	   ({"code" code
	     "dissertation" dissertation
	     "design" design
	     "help" help}
	    site)]
    (let [export-dir (back-one-directory (System/getProperty "user.dir"))]
    (export-site-builder site-map export-dir)) (println "No such site")))