These days there are a plethora of web frameworks to choose from when developing a new website. Name any language and it’s pretty much guaranteed that there is a web framework written for it. Haskell? yep. Erlang? yep. Brainfuck? most likely. I myself, am a fan of Lisps. I’ve spent a bit of time programming in Racket (a Scheme to be exact) Something about them just feels rather elegant to use and program in. I’m by no means an expert, but still I find them fun. In my day to day I am a C++ programmer. It seemed like fun to take a look at Clojure since it appears to be one of the more widely used Lisps. And there is a Clojure implementation for embedded devices called Ferret.
There were a couple different choices out there for web frameworks built around Clojure, but I ended up choosing Luminus. In this post I will discuss my process of learning and getting set up with a development environment, picking my project, and learning a bit of Clojure. In the end I made quite a bit of progress on the project and I hope to continue to work on it over the coming weeks. Also, I should not that in my exploration of Clojure, I have come to discover my love for Emacs (praise gnu). I have a little experience programming in the functional paradigm and I wanted to dive deeper into that world. C++ recently has added support for lambdas in to the language in one of the latest standards. The previous experience that I had was working with Racket, a scheme, mostly used in the academic sense. There t seem to be a whole lot in the way of libraries for Racket when it comes to web programming, and I t have time to write my own (nor do I really want to). Clojure has been praised for having excellent support for functional programming, preferring immutability by default unless specified.
It also has great support for concurrent programming with its software transactional memory system. One of the main reasons I ended up picking Clojure was the plethora of libraries available to it, since it runs on top of the JVM. Well, that and I needed an excuse to read through my copy of Clojure For The Brave And True. With picking Luminus, there were several reasons why I ended up at that choice. One, other frameworks I looked at seemed to be less mature or lacking documentation. Two, the projects seemed to have been abandoned and no longer updated. Luminus appeared to both have a good community and documentation. Additionally, I thought it was interesting that they present it as a microframework [https://en.wikipedia.org/wiki/Microframework]. This was a term that was new to me. After doing some research about microframeworks I figured it would be worth the experience of giving it a shot. The short story of how microframeworks differ is that they sacrifice features for simplicity. Microframeworks usually do not have direct support for things like\nauthentication or other advanced features.
For this post I wanted to do something a little different. I enjoying browsing the website Hacker News. The site has a lot of interesting content and has a very minimal interface. One of the downsides of the site is that you can only browse by currently popular stories, or new stories. There is no way to view old popular stories.
What I want to do is create a site which will track the current top story from Hacker News and when it changes store the old one in a database. You could have several different options available for users to look through old stories. You could show past top stories by week, month, or even year. This would be helpful for the occasional user of the site, so rather than look through the hundreds of stories that were posted to the site that week, they could narrow it down to the few dozen that were the most popular.
s go ahead and grab one of the first pieces of software ll need to get started; Leiningen. Do note that I am also assuming that you have Java JDK 1.8 installed on a Unix machine already (needed by Clojure at the time of this writing) Leiningen acts as your Clojure build tool. It handles downloading Clojure for you, creating new projects, and running your test suite. ll get very used to running the lein command. s install it. Navigate you browser to https://github.com/technomancy/leiningen. Download the
lein script. Now we need to allow the script to be executable; open a terminal, navigate to where you downloaded the
lein script and run this command:
$ chmod +x ./lein
We can now run Leiningen from that directory. If we want to be able to run it from any directory we need to make sure that it is in our $PATH. If we move the
lein script into the directory
/bin this should work on most systems. Run this command to move the script:
$ sudo mv ./lein /bin
Like most other programming languages you could edit your Clojure project files through a basic text editor like nano or gedit, but I would recommend setting up an IDE. Since I have been using s IDEA recently, I ended up using the Cursive plugin. It allows you to turn IDEA in a Clojure IDE. This worked out great for me and since Clojure has a REPL, it was very helpful to have quick access to that. The layout is also very familiar to you if you are familiar with IDEA. If Intellij is not to your liking there are still other options out there as well. The most popular seem to be Emacs.
While some view Emacs as part religion, part text editor, it has stuck around\nthirty years for a reason. I have been using Emacs for a brief period, just trying to get my feet wet. While it took me a while to figure out all of the keyboard shortcuts I needed to know, I will say I have been overall very impressed. Esspecially with the fact that no graphical interface is required, meaning I can develop my Clojure code over ssh just using a terminal. Lastly, I will mention that there is a Eclipse plugin called Counterclockwise for those who prefer that environment. I did not try out that one so I cannot speak to its quality. The project appears abandoned.
We will need to pick a database when we start our new project as Leiningen will configure it for us. I went ahead and chose the h2 database as it is well supported in Java. You could have just as easily picked MySQL, or SQLite, or MongoDB. Let navigate to where we want to have our new project located in the terminal. Now to create a new Luminus project in leiningen we can issue the following command:
$ lein new luminus hndottech +h2
We will need a way to scrape the top stories from hacker news website. There are plenty of mobile apps out there in the app stores for accessing the site, so I figured it t be too difficult to integrate into their site. Luckily while I was searching on GitHub, I did end up finding a Java library for interacting with hacker news. It appears that this library scrapes the data directly from the HTML (rather than interacting with some public API). This saves us a lot of work in trying to implement that ourselves. (There may still be a way to actually talk directly to HN’s API, I’m still looking into it). [Update 2019:] Yes, there is now an official Hacker News API you can call directly!
First, I tested out the library just in a Java project just to make that it worked. I had to make a few changes to the Java code to get it to work right. I had to set up the package name to get my classpath correct. This library was also complained about being used by Java 8, as this was written to be used by Java 6. Now that I have verified that the library will work I need to figure out how to interact with Java code from Clojure. Now that we have our environment and libraries set up, we can get into some Clojure code. Before we start with that though, s go back to that Leiningen command we ran earlier. We have created our skeleton application with lein. s take a look at the layout and get a feel for what we need to edit.
Leiningen gives a a good starting point here with our base project tree. We have things like a basic
production environment files. s go ahead and move our Java source files under the src directory in a new folder named java. Lein will automatically take care of compiling those for us. Next we will edit the
project.clj file. We need to add in a couple of dependencies as well as tell Lein where those Java source files will be located. Mine ended up looking like this:
Most of our other edits will take place in the resources directory. This is where we can edit our HTML templates, CSS, and database resources. I ended up changing the color scheme, favicon, navbar, and others to suit the appearance of the site. I wanted it to look like the original hacker news site. Also, in the src folder. We need to edit our routes file to have a particular route make a call to hacker news page get function. It ended up looking like this:
We need to initialize our database. First I edited the SQL migrations file to suit holding Page objects and ran the following command:
$ lein run migrate
Your output should be something similar to this:
We can pack up our site into a JAR for running on a production server by issuing the command:
$ lein uberjar
To host my application I ended using an OpenBSD instance from Vultr. That is my preferred hosting environment. You could choose to host this anywhere that you can install the JDK. Since I wanted to see this project through to completion I went ahead a registered a new domain name for it. I ended up settling on
hackernews.tech. Definitely worth it for a couple bucks a year. If you want to see the code for this project, take a peek at the GitHub page(dead). [Update 2019:] While the site is dead for now, I still like the idea and want to revive it. Just need to make it through some of the other items in my TODO list..
[originally posted 2017]