Last week

ICFP and nostalgia

In the previous weeknotes I detailed my prep for my PROPL talk, which went okay as far as I can tell. You can see the presentation here if you're so inclined, but I suspect nothing there that regular weeknotes readers haven't heard before.

Despite being a remote attendee, I did watch a bunch of talks from ICFP throughout the week. I watched all of the PROPL sessions, most of the OCaml workshop, and one or two other bits that caught my attention. All the sessions were live-streamed on YouTube, so anyone can watch any of the sessions, which is good.

I also particularly enjoyed Satnam Singh's keynote on functional programming for hardware design. Partly this is because Satnam was my first year undergraduate lecturer in data structures and algorithms, and his lively delivery has not waned over the years. Partly because Satnam then employed me over the summer after my second year and got me linked up with up with doing FPGA work, which then shaped my PhD despite the fact by then he'd moved on to Xilinx, and FPGA knowledge got me my job at Intel Research also. Satnam's early choice to employ me that summer shaped a lot of my career.

(Going right to left that's Satnam, me, Adam, and Jim back in 1996 - a slightly silly photo but it's the only one I have from the time)

I still have a soft spot for FPGA work, and would like to do more with it, but there's never enough hours in the day. Programming data-science pipelines in Python is infinitely less cool to me personally, but the work we do is important given the state of things, so it's hard to justify stepping back from that to work on more personally exciting things even if the academic environment in which I operate would tolerate that.

The final reason I enjoyed Satnam's talk was that he was just getting into functional hardware description languages like Ruby (not the one you're probably thinking of, it predates that) and Lava when I first met him, and although I picked up his excitement for FPGAs, the functional programming course at Glasgow had totally left me cold, and so I never saw those things he was working on as something I was interested in. I've since changed my opinion, and been fortunate enough to meet Ruby creator Mary Sheeran at Chalmers last year and I can see how cool all that stuff is, just thirty years too late :)

The reason the FP course at Glasgow left me cold is also why I struggle to make the switch back to FPGA work even though it'd be easy for me to justify say hacking on HardCAML today. The FP course was presented in a very abstract way without application, and I've always been more interested in the application than the tool, at least until I understand how the tool gets me to the result better, quicker, more reliably etc. So at university attempts to teach me Haskell without ever getting into side-effects (as in, actually impacting the world), and Prolog, which when I asked my tutor why Prolog was useful he responded with "well, you can write a Prolog interpreter in just ten lines of Prolog!", all made me go back to my C courses with renewed vigour. Similarly, without knowing that I could use HardCAML and FPGAs to try somehow accelerate saving the planet I'm not sure I can motivate the switch. All of which is possibly a long winded way of saying I lack imagination.

Biodiversity pipelines and validation work

I maintain two similar-but-different biodiversity pipelines, the one for LIFE which is developed here in Cambridge, and an implementation of the IUCN's STAR metric. In practice what that means is I focus on one for a bit and then have to catch the other up. This time it was the STAR pipeline that needed a bit of catching up with all the changes I'd done to simplify the Yirgacheffe APIs. But making the changes necessitated also doing a full pipeline run, which I finally got around to. I also did a bunch of tidying of the docs to try make the pipeline a little more accessible.

Getting STAR active again is particularly useful, as I mentioned a few weeks ago that I'd implemented the occurrence side of the Dahal et al validation method, and STAR processes fewer species than LIFE, so makes for an easier first test case.

On the topic of validation work, I had a couple of good chats with Shane Weisz who has just started his PhD at the lab. He's interested in this general area, so we've been discussing how he might help with the validation work. Shane certainly knows stats better than I do, so I'm excited at the possibility of having him take a look at the problem.

The one other thing I started, but have not yet finished, is adding a script to STAR (which eventually will be ported to LIFE) to allow people to run both pipelines using the IUCN Redlist API using the Python package by Daniele Baisero. At the moment the pipelines require either that you have a Postgres database with all the Redlist data in it, not something most people have to hand, or you download a GPKG bundle using their data portal. I'm hopeful that using Daniele's work means I can make both pipelines more accessible.

Workload management

One of the reasons I'm also glad for Shane's interest is that I ended up overcommitted, and so the validation work which I'm very keen to move forward I've not had much bandwidth for. Having someone else help on it is a double win: more hands on the project, and it makes it easier for me to justify spending time on it. I've also been trying to remove myself from being on the critical path for other projects, not because they're not interesting, but because I'm on too many critical paths and it's starting to show as people get frustrated with lack of progress.

This is in part why I'm trying to make the pipelines easier to run: I need to make it easier for others to run them without me. I hope to run a little workshop in a month or so where I can show others how to do that, but I need to make it a little easier for them to do so first. It's also why I was grateful when Daniele took a look at some work on a plant AOH related project that had been sitting on my plate for too long.

My hope is that spending a little time this month tidying up these things will free me up to do the more interesting things in the coming months, even if it does mean I'm still blocking people for a few more weeks before that is complete.

This week

• Get Shane running the LIFE and/or STAR pipelines
• Get the Redlist API work into a good state so that I can merge it to both pipelines
• Get back to the hybrid habitat map
• Migrate my self-hosting to a new host - the current one is struggling and I'm finally running out of disk, so rather than just reprovision the disk at the current host, I'll move it to something with more CPU as well as disk.