Learn By Wrapping
January 30, 2022TLDR: I enjoy writing wrappers because they eliminate one of the major challenges I face when learning a new technology: figuring out something useful to build.
Learning a new library, tool, or technology has always been a pain in the ass for me. I don't have the attention span to read through the documentation. I learn by building something that solves a specific problem or scratches an itch. For example, at a prior job, I built a tool with the following requirements:
- Ships as a single executable that can run on older versions of Windows
- Can be cross-compiled from macOS
- Able to use Windows APIs via dynamic libraries
- Easy to set up a WebSocket server
Rust (sort of) fits the bill, but I chose Go because the learning curve was lower (sorry Rustaceans, I haven't worked up the patience to tussle with the borrow checker just yet). I was able to learn Go quickly because I was focused on solving the problem, not on learning the language.
Did I have a comprehensive and deep understanding of Go by the time I was done?
Hell no, but I enjoyed the experience and still learned a great deal.
I probably wouldn't have gotten very far with Go if I was just building a to-do app or a CLI tool that prints Hello World!
.
But what if I want to learn an API or command line tool? Maybe it's something I'm curious about or need to know at some point for my job. As I already mentioned, just reading through the documentation is like pulling teeth.
Here's my usual thought process when I encounter this situation:
I need to learn something new. Duh, use it to fix a problem.
Cool, I'll think of a problem. But you don't have a useful project idea.
What can I do then? Write a wrapper, ya dingus!
Over the years, I have written several wrappers around various tools and APIs. I built a Node.js wrapper around the Trello API. I've wrapped the Toggl and Clockify APIs for the web app I built to transfer time entries. And most recently, I switched gears and wrote a Go wrapper around the devcon CLI tool (it's super niche).
My most ambitious undertaking is a wrapper around QEMU (I'm a sucker for punishment). I have always found QEMU to be incredibly interesting, ever since I was introduced to it back in 2018. I no longer use it in a professional capacity, but I like to tinker with it for hobby projects.
If you've ever used QEMU, you know it has a batshit amount of command line options. Here's a little snippet of how you'd boot an image with some custom settings, a port forwarding rule, and a mounted ISO:
qemu-system-x86_64 -m 3G -smp 2 \
-netdev user,id=n,hostfwd=tcp:127.0.0.1:9000-:445 \
-device e1000,netdev=n \
-usb \
-device usb-tablet \
-k en-us \
-drive file=some-file.qcow2,media=disk,format=qcow2 \
-cdrom some-iso.iso
- Note
- I know there's more terse ways to achieve the same result above. Please just bear with me.
Believe it or not, that's a fairly simple invocation. Shit starts getting real when you need to define NUMA nodes:
qemu-system-x86_64 -machine hmat=on \
-m 2G \
-object memory-backend-ram,size=1G,id=m0 \
-object memory-backend-ram,size=1G,id=m1 \
-smp 2,sockets=2,maxcpus=2 \
-numa node,nodeid=0,memdev=m0 \
-numa node,nodeid=1,memdev=m1,initiator=0 \
-numa cpu,node-id=0,socket-id=0 \
-numa cpu,node-id=0,socket-id=1 \
-numa hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=access-latency,latency=5 \
-numa hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=access-bandwidth,bandwidth=200M \
-numa hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=access-latency,latency=10 \
-numa hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=access-bandwidth,bandwidth=100M \
-numa hmat-cache,node-id=0,size=10K,level=1,associativity=direct,policy=write-back,line=8 \
-numa hmat-cache,node-id=1,size=10K,level=1,associativity=direct,policy=write-back,line=8
- Note
- The above invocation was taken directly from the QEMU site, so you can stop bearing with me.
I got sick of passing all the options into exec.Command
in Go, so I started building Queso.
Queso is what I call a spicy command builder. To get the same command as the first invocation example I gave, your code would look like this:
q := qemu.New("qemu-system-x86_64")
q.SetOptions(
qemu.Memory("3G"),
qemu.SMP(qemu.WithCPUCount(2)),
// Network Settings
network.UserBackend("n",
network.WithForwardRule(
network.NewHostForwardRule(network.PortTypeTCP,
9000, 445).WithHostIP("127.0.0.1"))),
device.Device("e1000", device.NewProperty("netdev", "n")),
// USB Settings
qemu.EnableUSB(),
qemu.USBDevice(qemu.USBDeviceTablet),
// Drive Settings
blockdev.Drive(
blockdev.WithDiskImageFile("some-file.qcow2"),
blockdev.WithDiskImageFormat(diskimage.FileFormatQCOW2),
blockdev.WithDriveMedia(blockdev.DriveMediaDisk)),
blockdev.DiskDrive(blockdev.CDROM, "some-iso.iso"))
if err := q.Cmd().Run(); err != nil {
log.Println(err)
}
Is this better than a Bash script? That's up for debate. It is nice if you're running QEMU cross-platform, as you don't have to maintain two different script files.
- Disclaimer
- I promise this post isn't just a shameless plug for some code I wrote. In fact, as of this writing, I wouldn't recommend using Queso (at least not in production). It's still undergoing changes.
Remember how I mentioned "staying engaged with the documentation"?
As I was writing Queso, I went through each invocation option in the QEMU documentation and tried to come up with a corresponding wrapping mechanism.
I also included the documentation alongside each option, so hovering over qemu.Memory
in Goland gives you this:
Pretty nifty, eh?
I learned a great deal about QEMU because I wasn't just skimming the docs, I had to understand what each option meant. I spent long stretches of time reading and re-reading certain sections to make sure I got it right. I gained a much deeper understanding of QEMU's functionality, as well as some gotchas and features I probably wouldn't have come across in a Google search (mainly because I wouldn't really know what search term to use).
Wrapping is also great for learning a new language. Pick a command line tool or API you're already familiar with (or not!) and build a library in the language you're trying to learn. Some command line tools will write something meaningful to stdout that you can parse and convert to data structures (I did a lot of this in go-devcon). APIs provide an opportunity to wrap endpoints in functions with return values parsed from the corresponding payloads.
Reading through the documentation and writing functions that wrap commands or endpoints can be very zen. There are no ambiguous requirements, and you can wrap as much or as little as you like. You can focus entirely on what you're wrapping (or what you're wrapping it with) without the cognitive overhead of ideation.
Everyone learns differently, so this approach may not work for you. But next time you're tasked with learning a new tool, try whipping up a wrapper and see if it helps. You may end up retaining more knowledge than you expect.