Last week

Last week time-wise was dominated by two things: a three day IUCN workshop on applying new technologies to the IUCN Red List, and attempting to pull together a paper on yirgacheffe for PROPL. I'll do a separate blog post on the former, and I now have a complete draft of the later, so will have something to submit by the deadline of Thursday.

Opam submission failure

I mentioned last week I submitted a package to opam for the first time, mostly as a learning experience. It was good that my main objective was to learn rather than get a package into opam, as that way I can claim success, otherwise the whole thing has been a frustrating exercise.

The style of package I needed to submit isn't well documented, so I scanned through opam to see if I could find an example that was like what I wanted and based my PR on that. Unfortunately I picked poorly, and I got two responses to my pull request telling me to do it a different way, but without links to examples or documentation. I had a third tell me to change the name, despite the fact I'm wrapping an existing popular C library, and that was where the name came from.

All of which is fine if I was a seasoned contributor, but I'd argue is a poor response for a first-time contributor to an open-source project to receive. If OCaml wants to draw in new contributors and widen its user pool, which is the tone I got from events like the FunOCaml conference I attended last year, then telling new contributors they're doing it wrong without support isn't going to encourage that, meaning you only want contributors who do this full time or are already embedded in the community. If it's a more casual contributor who isn't doing this as their primary function, this sort of interaction may well be the end of their attempt to participate.

On the plus side, I did receive a small PR for one of my other obscure OCaml packages (which isn't in opam) last week, so that was a positive OCaml community wise, and shows that not having things in opam is the end of discoverability.

Bon In A Box

I had a play with Bon In A Box, a containerised environment for running ecology data-science pipelines, and was quite impressed with it. Firstly it provides an environment in which for you to run your Python or R scripts, and to do so in a way that encourages reproducibility by making inputs and outputs explicitly at a higher level (no more hidden sideeffects in scripts, which I think is a major problem with using general purpose languages in this domain), and because it's containerised you have to use the package dependancies via their metadata setup, which is good also - a common reproducibility problem with running other people's scripts is missing package dependancies as they happened to have more installed on their system than what was in the requirements.txt or such (assuming that exists).

Beyond that they also have this very cool way for building up pipelines where you drag scripts into a visual editor, and because the inputs and outputs of each script are defined in a metadata file, they can link together scripts visually, which is super exciting to me who spends a lot of time trying to generate this visualisation as an after the event view:

Anil and I had a chat with some of the BIAB team, and it's still work in progress, so I hope there's some opportunities for us to collaborate there, as this solves a bunch of problems we were looking to tackle, and does so in more packaged and robust way than say our Shark project has. Not that time on Shark was wasted, but rather I'd be interested to see if we can contribute to an existing effort that does a lot of things right even if not quite how we'd do them, rather than duplicate a lot of effort to get our particular spin on things production ready.

As a concrete version of that: I originally built the LIFE and STAR pipelines to be executed by Shark, which was great for me, but because Shark is very much work in progress, I couldn't ask other people to do that, so I also had to ship a shell script to run the pipeline. The shell script isn't great, as it lacks the nuance of a proper build system. It looks like BIAB will be a nice in between, so I now want to try porting a part of LIFE or STAR to BIAB to get a feel for how it goes.

This week

I very much need to make technical progress this week, and submit my PRORL paper. On the technical front:

• Make some progress on AoH edge effects
• Start to work with the land cover foundation model from our group so I can try to apply it to the projects I've been working on

Thankfully I'm hidden up on the Wirral for the next two weeks, so hopefully I can just get my head down and get on with things.

I'll also be at the aforementioned Liverpool Makefest on Saturday 5th, trying to demonstrate to the people of Liverpool how to get started in building guitars. And so in the unlikely event you're in the area next weekend, do say hi.