New writing workflow

I wrote Functional programming in R using Ulysses and have used Ulysses to write the first part of Object oriented programming in R as well. It is a very nice editor, and I am definitely using it in the future for other writing projects, but for the workflow with these books it doesn’t really work for me anymore.

The way I write the books is this: I write in R Markdown that I then need to run through ‘knitr’ to get the R code executed, this generates Markdown files that I then process using ‘pandoc’.

To get the full set of features in Ulysses, I have to use their internal file format, which means I have to export files ever time I need to compile the text. It is not a major hassle, but it is a little annoying.

It is especially annoying with Object-oriented programming in R where the second chapter has some performance measurement code that takes a while to run. I don’t want to run that every time I compile the book, so having a make file that makes sure I only compile and run the R code when necessary is the way to go, but either I have to be careful to export chapters in separate files (and remember to do that every time I change the text), or I have to just edit the raw Markdown files and not bother with the internal file format.

I have decided on the latter. I can’t use the more advanced features then, so I will edit with iA Writer instead, I think, just to give that a good try for comparison with Ulysses. I already use iA Writer for most Markdown editing, but I haven’t tried using it for a book yet. I will miss Ulysses’ sheets — they make it easy to move sections about — but I am sure I can manage without them.

With iA writer I can’t use the more advanced features like embedded files and such outside of iCloud, as far as I can see, but that wouldn’t really work well with the book compilation framework anyway. Embedding figures, for example, doesn’t really work for me because I need different figures when compiling to PDF and when making EPub. So I have to have the figures external to the text in any case.

You can see my current setup at github. I just have a Makefile for processing the files. First I send the R Markdown through a pre-processor — this is how I can have different code run depending on whether I’m making a PDF or an ebook — and after that through knitr and finally through pandoc.

Experimenting with GitHub for exercises

I have used GitHub to set and correct exercises in my Data Science class the last two weeks. More specifically, I used GitHub Classroom.

I wouldn’t recommend that.

What I find works well is using GitHub to describe the exercises — the README file that gets rendered nicely on the repository front page is good for that — and you can very easily provide template code and data.

I definitely should have given my students more information about how to clone repositories and how to push changes back, but lesson learned, so I will do better next time.

Commenting on changes in the students’ repositories is also very easy as is following up on comments and checking changes made to fix problems.

What does not work well is that the Classroom just give each student a repository that is completely disconnected from the template I made. The way it works is that for each student and each assignment you get a new repository where a template repository is used to initialise it. It does that, as far as I can see, by just cloning in the original repository and not by forking it.

In principle there is nothing wrong with that, but it gets a little annoying when correcting exercises. I can go in and see the change log on each project, but I don’t get informed when the students think they are done — as I would if it required a pull request — and what is worse is that I don’t get a diff from my original template to the current state of the students’ solutions. (I can get that with a little work, but if I just want to go in and comment on commits I get the changes in bites matching the commits, I don’t see the combined changes).

So the plan now for the coming weeks is to use forks and pull requests instead. I have high hopes for that.

I still need to figure out a good way of signalling that a pull request is accepted as a hand-in without merging it into my repository. I can’t merge all students’ pull requests, but ideally that would be the way to indicate that a request is accepted.

This is less than ideal…

Yeah, I knew that writing would never give me a decent hourly salary, but this is a little depressing…

Oh well, I will probably manage to get the $61 back, but I will have to reconsider my Grammarly subscription when it runs out and I probably have to think carefully about putting up a new book (I was planning to start one on meta-programming and one on functional data structures, not sure about the order, once I’m done with OOP in R).

Progress is a little slow on Object-oriented programming in R. I thought I would get inspired by #NaNoWriMo and pick up speed (I’m not planning on actually writing a novel, but I could write faster on OOP in R). That hasn’t really happened.

I have written a lot the last couple of weeks, but it has been on papers or teaching material and not my book, so when I finish work my brain is just not up to writing much more.

Still, while it may be going along slowly, it is moving along.

Royalties and publishing on Leanpub

Now that it costs me money to put books on Leanpub I have to consider if it is worth it.

I don’t write to make any money — I do it because I enjoy it and have some hope of maybe using my material in teaching in the future — but if it actually starts to cost me money to put my books online I have to think of another solution that Leanpub.

Don’t get me wrong. I like Leanpub. It is wonderful to have a place where I can put my half-written texts and get feedback before I am done with a book, but it is not something I am quite willing to loose money on.

I have pulled down the royalties I have gotten from the books and plotted them over time with a dashed line showing the break-even point. I have to pay $99 to put a book there now, which with VAT is $123.75.

If my books get to the break-even point I will continue putting them on Leanpub — and any money I make on top of the break-even point I will spend on trying to make print-on-demand physical copies — but if I can’t reach the break-even point I will just put them up somewhere else where people can get them for free and give up on making hard-copies.

The first book, Introduction to data science and statistical programming with R has reached the break-even point. Functional programming in R hasn’t yet. No one has paid for Object-oriented programming in R yet (nor should they, there is only one chapter written yet).

I will see how it goes with these three books and make up my mind later.