toyrobot in golang

Wrote toyrobot in golang. Thoughts:

- Golang is kind of "dumb". It has a strong emphasis on simplicity. I found myself writing very simple code. However, it is still reasonably concise.
- Not having to worry about memory management felt weird in such a low-level language. Also, since there are still pointers, the only protection from null pointer dereferences is at runtime. I had several null pointer dereference bugs during my development. Fortunately they were all found by small and simple tests, so they weren't too hard to find and fix. In a large program that would suck.
- Getting the tools running in VSCode on linux sucked. I had golang 1.10 installed somehow, but wanted to use 1.12 package features. Trying to upgrade broke my vscode -- I installed it with the snap. Setting up GOPATH and stuff wasn't fun. VSCode still doesn't work properly, doesn't highlight syntax errors or test failures.
- Writing tests was pretty straightforward.
- The effectively instant build time is definitely go's best feature.
- I suspect I would have difficulty writing efficient go code. I haven't learned the complex rules for when a struct gets copied when passed by value or when something is allocated on the heap instead of the stack. Also, garbage collection means no hard-real-time applications; GC pauses mean you can't use it for financial trading, hardware interfaces, or anything else where being a few milliseconds late is a failure. I'm having trouble thinking of other situations where that might be a problem though.
- The error checking boilerplate is pretty annoying
- I didn't use concurrency or channels, but I can see that it would be pretty useful for any kind of server app that talks/listens to other programs
- I really liked the single-binary-that-contains-everything concept. No build folder!
- I found the dependency management pretty straightforward
- Golang.org / stackoverflow was pretty good for looking up how to do things like reading from a file or parsing strings.
- Maps and slices syntax is a bit awkward
- I LOVED the everything-is-plain-old-data-with-extension-methods. I think that design decision alone strongly guides programmers toward more maintainable (less tangled) architectures.
- Packaging system very solid. My code is available almost accidentally as a library because I posted it on github. The only thing you need to do to use it is to add `import "github.com/jnnnnn/toyrobotgolang"` at the top of the source files that need it -- no `package.json` or `conanfile.py` or anything else. Exporting things that start with a capital letter is neat too.

I would recommend golang for the purpose it was designed for, server-side application-level software. Which is probably most software these days. It's not going to work for native GUIs or really high performance stuff or hardware interfaces.

Devlogs 2

I was stressed on Wednesday this week, overwhelmed with keeping track of regressions caused by a change to a React component. However, I discovered a very useful way of dealing with it -- create a whole Trello board just for the problem. Here is a screenshot of the trello board once I'd fixed all the problems.



There are 11 things that gradually got added to the Problems list and got moved off it once I'd dealt with them.

I could also have written these things down on a notepad or post-its (or in my devlog) as I noticed them. Both those options would have worked pretty well too.

Without Trello

I would start the refactoring, do a manual test, and notice that some things were broken. I tried to remember every problem (there were 11 separate problems in the end) and quickly got overwhelmed. I hate missing things in work so this was quite upsetting, and I found myself looking for distractions instead of focusing on the work.

With Trello

When doing manual testing, whenever I noticed a problem, I just added a card to the new Problems list. And then promptly and happily ignored it and forgot about it. Once I'd finished the thing I was working on, I could come back to the Problems list to see what I needed to fix.

This also allowed me to prioritize work. Instead of having the "native scrollbars option" idea and then immediately rabbit-holing into the problem, I just added it to the list. When I came back to it, I decided that it wasn't worth doing -- although I didn't have to immediately discard the idea, keeping it around to discuss next time someone else on my team had time to talk.

Why Humour

Large haskell program architecture

I've been doing job interviews lately and reviewing applicant's code has got me interested in quickly understanding abstractions and architectures in a novel codebase. My first step is now to draw a dependency graph -- read the code, and every time a module/function/file mentions a name, add a node for that name and an edge to the graph. This quickly builds up a picture of how the parts of the codebase are tied together. Easily-understood architectures have few edges. Bad architectures have edges from every node to every other node.

I've started applying this approach to larger programs I am considering adopting as well. For example, here is the dependency diagram for Pandoc, generated with

find pandoc/src -name '*.hs' | xargs stack exec graphmod -- -q -p | tred > graph.dot
dot graph.dot -Tsvg -ograph.svg

"I don't know how to fix leaks in museums" is my new answer whenever I am unable to help someone when they ask for help.

Backpressure

Our software uses sequence-based recovery, just like TCP. Each write to the database records the sequence number, and when we start recovery we ask the other end of the connection for the next sequence number that wasn't written to the database.
However our system processes messages in several stages between several different applications. The first stage can process messages far faster than the part that writes to the database.
If there are lots of messages cued up on the other end, the database part can get overwhelmed. So if we go down for a few minutes at a busy time, we can never catch up.  We try to read in too many messages, the database writes a few then crashes, and then we start again.
The standard solution to this problem is back pressure. If the database part refuses to accept too many messages, then the reader part has to wait for the database to be ready before sending more. This means that we only request messages from the other end when we have space in the queues.
Also the database part get slower if we send it too many messages at once. Its memory fills up, the processor and eventually disk caches get less efficient, and after a while it runs out of memory completely.
So back pressure actually makes the system considerably faster.
And also means it doesn't crash under load.

peak complexity

One architecture

Here is the concept usage graph of a good architecture. Note that there is a clear hierarchy.

Another architecture

Here is the concept usage graph of a more spaghetti architecture. Note in particular the MultiStepAction -- I find this kind of abstraction (a group of other abstract actions) very common in spaghetti architectures.

I want a pony