I claimed last week that I'd have to put aside the fun I was having working through the Ray Tracer Challenge book in OCaml due to other commitments. And technically that was true, I didn't get to do any new features based on the book, I had a couple of longish train trips, and so I did go off piste a little and scratch a couple of itches with my ray tracer.

Performance

Firstly, I did a little work to improve the performance. The main improvement was to cut down the number of allocations that were happening on the heap, as I could see from profiling with Instruments that this was one of the dominant costs. The main culprit here was my implementation of the basic point/vector tuples and then matrices for multiplication work: I built these up as nice abstract data types without concern for performance, which is the sentiment of the book in the opening chapters too: understandability over performance is a key win of this book. But at some point this bites you and you need to reconsider things (something the book also directs you too towards the end). Not only was my implementation naive from a performance perspective, for instance using nested arrays to store matrices, requiring 5 allocations rather than 1 for each new matrix, but also when you make abstract data types in OCaml you can not violate that abstraction easily, and by following the book to have point/vector tuples separate from matrix meant there was a lot of conversion going on between the two abstractions.

To fix this I rewrote both types into a single type that is a bit specialised to 3D work rather than a more mathematically pure set of abstractions and then used a 1D array for storing the results rather than a 2D array as before. I also switched from using float array to the dedicated floatarray in OCaml. Even though as I understand it OCaml special cases float arrays I found that moving to the decicated floatarray type gave me a further performance increase.

These game me a nice performance increase, but not anything exponential. Still as I throw more and more complex shapes at my ray tracer any performance win is worth it. The chess example in last week's notes was already quite slow, and as I throw more unstructured sets of triangles at it that slow performance becomes more evident. Speaking of many unstructured triangles...

STLs

Wearing another hat I do a bunch of CAD work and 3D-printing, and I thought it'd be nice to use the ray-tracer with some of those models. The common format used in 3D-printing is STL, which is basically just a long list of triangles - with no other structure in it. It's not great for ray tracing, where ideally you'd have more complex shape data, but given how many STLs I have kicking around my various disks, I thought it'd be fun to try get it working.

It turns out STL has variations: one is ASCII based and human readable, and the other is binary. Both are literally just a list of triangles with very little other information in there. I had no appetite to write another text parser, and so I opted to implement a library to read the binary STL format. And shortly I was able to load models like this one of an amp I built quite a few years ago:

To get this model I just exported it directly from my CAD tool for 3D-printing:

My first attempt to write the STL parser failed to load any files from Fusion 360 because they were tripping an assert in my code. For each triangle in binary STL there is a 2-byte field called attribute length, which doesn't appear in the ASCII version of the format, which isn't used, and so I assumed must always be zero, and had asserts to that effect to help me check I was reading the file correctly.

However, it turns out that many CAD tools will stash other data into the unused space in the STL format, and such was the case with Fusion. It turns out Fusion will put a 15 bit RGB colour value in that field, so I can actually get my amp looking more like the CAD model with just a couple more lines of code:

There's also an 80 byte undefined header in the binary STL format that Fusion places some information about the version of Fusion used to generate the file, along with a global background colour.

I find this somewhat fascinating, as I've been exporting STLs form Fusion for years, and I've never seen any tool I load them into display the colours, to the point that I assumed it wasn't possible and so when we had Finley the summer student generate 3D-prints of geospatial data I'd specified that he use the less common 3MF file format because I knew it could store colour. I worry about what jokes might be out there on all the STL files on the Internet that are only ever displayed in mono but have colour information in there (this is what happens to your brain after you work in the security industry for a while).

On a more positive note, it's one of those amazing moments in coding whereby just a small amount of code (STL is really easy to parse) can have such a large impact on the feel of a product: the ray tracer suddenly feels so much more powerful now I can load up my own CAD models into it: it goes from toy to potentially useful tool. I didn't do much work to cause that change, but having don that just unlocks so much more. The challenge in coding is just finding the right few lines of code to write to cause that change in perception :)