Last week

Habitat mapping

After a few weeks of incremental improvement, I finally hit a stage where "progress" lead to regression in the results of my attempt to build up a habitat map. In part this comes from trying to infer certain things from Open Street Map data, and how different OSM communities approach things. For instance, in the UK any town will have a residential area enclosing polygon, but in practice that isn't the case in Brazil. If you happen to sample a point in a town that isn't an actual building then you are most likely just to get that as an unclassified area. In theory we could do some proximity inference, but for now I'm more interested in what is and isn't actually stated in the OSM data.

I did find a close proxy, which in Brazil is using administrative zones at level 10. The admin zones in OSM are interesting, as each country has different levels of administrative zone based on how that country manages sub-regions. For Brazil there is one that is down to the suburb/neighbourhood level, which isn't used in all towns and villages, but does seem to be somewhat used, so I thought I'd give that a go.

At the same time there's a lot of nature reserves in the parts of Brazil I'm looking at, and whilst they don't say what kind of nature they're preserving, I wondered if we could use those to rule out anthropogenic (human made) change. Turns out this isn't the case - most of Anglesey in Wales is actually under a preserve and includes towns. But I tried it anyway, but only where there wasn't some existing polygon explicitly stating town.

The net result of these is somewhat of a regression. I don't think they're particularly bad rules, more that in Brazil we're just data starved, and so any rule that happens to tweak the data one way is having exaggerated impact.

---

Sticking with OSM, I've made a habit of when I spot pasture now I see if it's correctly recorded in OSM, as my suspicion was that more than often it isn't properly recorded. This added an extra layer of fun to my trip to Yorkshire (see below) last weekend as I tried to remember where the fields I spotted with sheep and horses in happened to be :)

In general I see two things happening frequently in OSM in the UK:

• Pasture is misclassified as landuse=farmland. To my reading of the OSM wiki, farmland should be arable, whereas pasture should be meadow with a subtype of pasture. I've seen this both around Cambridge and on the Wirral; for an example, these are horse paddocks: https://www.openstreetmap.org/way/324028977
• There are walls/fences marked, but there is no actual polygon for the field at all, which I saw quite a lot around the area of Yorkshire we were in. I drove past all these fields with horses and sheep in them today: https://www.openstreetmap.org/#map=17/53.806429/-1.959402

I have corrected examples of 1, e.g., https://www.openstreetmap.org/way/188906871 where it was farmland but I know it's had sheep in it for several years at least - but that manual fixing by just one person doesn't scale.

This I think means two problems for my current approach on using OSM to refine occurrence data:

1. We under-represent pasture as there's just less of it
2. We probably over-represent arable as a result of pasture being labelled as farmland

In the short term I think I might need to treat OSM farmland as either, with a bias to arable if a species likes both. But in the long term I do wonder if we can use Tessera to help fix this. If we tag examples where we very much know what is arable and pasture in OSM (solid truths), then we should be able to use tessera to then work out at least the misclassified fields and create a list of suggested edits for review.

I spoke to some people better connected with the OSM community at the event I was attending, and they suggested if I had that I could try to engage the UK OSM community for one of their semi-regular themed events, so I feel that might be a good side project for us to run in the group, perhaps if we can find a student with some time.

---

Finally on this I had a good long conversation with Vinicius Tonetti, a collaborator who is very familiar with the Brazillian landscape, and he answered a lot of naive questions from me on what to expect on the ground in the regions of Minas Gerais and Espírito Santo. I have to work hard to keep a sense of scale for Brazil: it's tempting to think of these sub-regions of Brazil being like counties in England, or even like the individual states of Scotland, Wales, etc. but Minas Gerais alone is 2.5 times the size of the UK, and so I can't just assume a level of uniformity that my mind defaults to for that sort of sub-national unit.

Anyway, I was very grateful for for Vinicius's patience as he educated me about the makeup of the area, and it made me feel that at least in Espírito Santo, whilst we have some obvious problems (roads being marked as arable in my maps), and it's clearly not up to the standard we have in the UK, it's not terrible, and perhaps actually doing better than other habitat maps in broad strokes.

Claudius for the web

I went to Open Source Hardware Camp 2026, a small gathering of the UK open hardware community. Set in the wonderful Hebden Bridge, it's a slightly odd event where I think many of us go to meet people there more than to see the talks or workshops. Not that there weren't good talks: particular highlights being Jo Hinchliffe talking about what's new in FreeCAD and Stuart Childs talking about boding bare, unpackaged chips to PCBs in Shenzhen. But I think there was a sense, between the all male speaker list and with one of the community stalwalts passing earlier this year and generally it feeling like an aging community of where might it be going.

In the social gathering the day before the conference I felt we did find a couple of threads that might be pulled upon. Quite a few community members are involved in various computer preservation projects, from those volunteering at The National Museum of Computing to build a working EDSAC, and Kevin Murrell who gave a talk on emulated PDP's with physical interfaces. There was a common thread though that these not just be objects of worship, but things people can engage with. This brought back to mind that in the last OSHCamp in 2024 there was some members of the sizecoding retro demo-scene at the sister event the day before, who are people making new and exciting demo effects using tools like TIC-80 or old 8-bit computers - seeking the challenge of doing more with less, which is the essence of these historic machines. I feel there's a potential connection there if only we could find a way to bridge it. What would that crew get out of the blinken lights on these old computers?

To that end, in quite moments over the weekend I dusted off Claudius, the retro-style graphics library I made for OCaml and did something Anil Madhavapeddy had suggested a while back, and started porting it to run in a browser, using the Js_of_ocaml OCaml bytecode to Javascript compiler. Progress was quite swift, such that after just a day, and having never used JSOO before I was able to get an unmodified example from Claudius running in a browser, like so:

Claudius normally uses SDL for graphics, via the Tsld OCaml bindings, but I'd always assumed one day there would be different backends, so I've always made sure that the SDL code was relatively well contained. Claudius maintains an entire rendering pipeline to a virtual framebuffer, so once I'd tidied up the few SDL abstractions that had been missed, porting the run-loop from Claudius to target an HTML canvas element instead was relatively simple. That above example I just took the tunnel example tick function and copied and pasted it into the same file my new lashed up run-loop and it Just Worked™.

Why? Well, mostly it makes it much easier to share the output. In the past I've done this by running it, capturing a video of it, converting that video to a format suitable for the web, and putting it on my website. With this you should just be able to post the actual live demo to the web. And because of how Cladius runs, you could then do other things like present different variations and have web page controls change say the palette or let you parameterise the constants in the code to let people explore the math behind the demos more easily. It doesn't directly link the two worlds I outlined above, but it felt like a little contribution I could make to encourage more of that sort of behaviour.

There's probably a couple of days work before I can upstream all these changes to Claudius proper. I've refactored Claudius now to have the SDL backend be an optional import, and I have started to make the JSOO alternative backend. Keyboard input is in progress, and mouse input shouldn't be too hard once that is working. I think the main thing I want to think about is the abstraction for just working with it: the old SDL based method of code, compile, run I don't think quite makes sense for the browser - where are you going to run it? Should we generate a template holding page for the canvas element required so you can easily get started? Can we do that with a simple HTTP server too? But that looks quite different I think from saying you want to embed it within another page. I suspect a couple of cups of tea will sort it all out :)

Multithreading

Whilst on our way to Yorkshire we stopped off at Lancashire at the Queen Street Mill, a former cotton mill that still has a working steam engine connected to looms. I can highly recommend the guided tour they run, where we were shown the end-to-end process by which cotton was processed and turned into cloth, including a lot of working machines.

What interested me is how much parallelism is in play in a factory like this, which makes me feel there is a good parallel programming guide somewhere in this factory. There's some simple, obvious parallelism, such as making the back beam, which is the width wise set of parallel threads that will be woven together to be cloth. These rolls were made on machines that literally showed the multithreading at play:

They also had even wider machines of the same kind when they needed to make wider cloth, showing some simple scaling in parallelism:

There was also some less obvious but still relevant stories of parallelism too. The cotton for the shuttle that passes through the parallel threads from the back beam is on a much smaller reel, as the shuttle needs to be small to be able to pass through the loom back and forth. But the supplied reels didn't necessarily match the size of whatever you were using, so the would need to be rewound for a given mill's set of looms. In this mill they actually unwound these small input reels into larger conical bobbins with enough material from dozens of these smaller reels, which was a slow process, but they did this because they had another machine that could then automatically unwind this larger bobbin onto many final shuttle reels without the need for human intervention, a sort of job queue system perhaps (which was also running in parallel):

The point is that the raw inputs sometimes needed to go through many transforms, some of which appear slower than you'd want, but that is to enable some greater level of parallelism downstream. This is a common challenge in data-science, where the ideal path to throughput might not make sense at every single stage along the way.

And so far this has been all to get one machine running: in the main hall of the factory originally would have sat 900 or so looms (there are now around 300 IIRC), and a single operator would be managing four or more looms, trying to keep them fed (instruction scheduling on a CPU with multiple units :). All 900 machines were driven by a single steam-engine that has shafts and gears and belts overhead.

They also, for some of the fancier machines, had punch-card programming for the patterns:

Anyway, I highly recommend if you're around Burnley that you stop off and take the guided tour. They fire up the steam engine in full once a month, but if you miss that, like us, they have an electric motor drive a section of the factory so you can still get to see the machines in motion.

This week

• Wrapping up the minimal Claudius JSOO backend before that falls out my head again
• Writing up my paper outline for the habitat work to help ensure everyone knows what I'm planning
• Move to the new geotessera xarr code for my work as it is clear that what I'm currently doing with offline tiles isn't the future for how tessera is to be used.