Advent of Code 2015

For the past week or so, I’ve been having fun going through the older Advent of Code challenges. Every day in December, Advent of Code publishes a new puzzle to solve using whatever programming language you want to use. I’ve been trying to solve the puzzles for a few years, and it’s been interesting to see progress.

When I just started, I hardly got anywhere with the challenges. I hardly knew how to program anything at that point, so the simplest puzzles were already syntactical challenges—how do you even code? What should it look like? How do I write conditional statements? As I grew in experience, though, I find myself much more able to solve these puzzles. Looking back, I see a moment where I start experimenting with object-orientated programming, and solving problems more easily than before. The last week, I find myself easily pushing through the first seven or so challenges, where they would have taken me ages initially.

What I run into these days are the puzzles that are really more orientated towards people with formal education in programming. The puzzles will involve certain algorithms that are commonly taught in computer science that I usually don’t even know to look for. Think things like modelling geometrical shapes in three-dimensional coordinate spaces, or algorithms to optimize seating arrangements, or recursive algorithms to sort through instructions that refer back to themselves, and so on.

I work on these problems, and sometimes get to decent (but incredibly slow) solutions, and feel quite proud of myself. When I get really stuck, I try to find help on Reddit, and usually the answer will be “Ah yes, you’d have to apply X’s theorem” or “Y’s algorithm is the best approach”, and referrals to read up on concepts that I have no idea what they mean. Essentially, it shows the gap between an amateur hobbyist like myself and a trained programmer.

It’s great to learn about these types of things, and I’m also more than happy to take some of these points and think “Ah, that’s too much bother” and leave it be. Either way, I’m quite grateful that so many people make all this type of content for free, and help me learn and enjoy these things.

A New Perspective on Typing

Now that I have more time on my hands thanks to the summer holiday, I’ve been looking into a thing that I’ve been interested in for a while: alternative keyboard layouts. A few years back, I learned touch typing, and it’s massively improved my ability to type. Already, I was a fairly quick typist with a manner of ‘hunt-and-peck’ typing, using three fingers per hand to reach around 110 words per minute. However, it was also somewhat error-prone, and caused undue strain for only a small set of fingers. So, a while back I learned how to properly use touch typing, which slowed my typing speed a bit but increased my accuracy while decreasing strain.

One thing that I learned while learning touch typing is the fact that the qwerty-layout of most English-language keyboards was designed to slow down typists to avoid typewriters jamming up because of people typing too swiftly. Computers, clearly, don’t have this concern anymore, though most English-language keyboards do still use the qwerty-layout. So, some people have worked on changing the layout to increase typing speed once more, resulting in layouts such as Dvorak and Colemak, which focus on putting more common letters right on the home-row (the central row where your fingers rest with touch-typing).

The past week or so, I’ve been learning about Colemak via learncolemak.com and practicing with this new layout over on keybr.com. I was quite happy to discover that switching keyboard layouts is tremendously easy in Linux, requiring just a simple command:

setxkbmap -model pc104 -layout us -variant colemak

As you can imagine, switching back just involves the same command but without the listed variant. The only thing remaining is that the actual letters on the keys would then be incorrect. However, since touch typing involves learning all the keys through muscle-memory, there’s no point in looking at the keys in any case. In the long run, though, since I do have a Keychron K2 mechanical keyboard, I could just switch the keycaps over to a Coleman layout if I would be interested.

Still, it’ll take quite some daily practice before I’m comfortable enough to switch over like that. For the time being, I may end up using the following command to easily switch between three separate layouts:

setxkbmap -model pc104 -layout us,us,us -variant ,intl,colemak -option grp:alt_shift_toggle

Using this snippet in my i3wm configuration file, I can use alt+shift to on-the-fly change between standard qwerty-layout, an international “dead keys” layout that allows me to use punctuation to insert diacretics for typing in Dutch, and the Colemak keyboard layout.

I’m looking forward to seeing how this new layout will be treating me. In the meantime, I’ll keep on enjoying how easy things are to set up in Linux (never thought I’d be saying that, but here I am!).

Learning About Debugging

I was reading a Reddit post on how Guido van Rossum stated he wanted to make Python work faster when I came across a comment chain where somebody joked that you can make any program faster by using fewer print() statements. A reply quipped “How else am I supposed to debug my programs?”. I’m assuming it was a joke for them, but this is certainly a weakness on my part. Since I’m self-taught, I find myself using standards that make sense to me, but would be bad practice for a formally taught individual. In this case, that’s using the print() statement to help me debug by running the program and seeing what I end up with, tweaking that, running it again, and so on.

The Reddit thread includes a comment chain where somebody jokes about using fewer print() statements.

Fortunately, though, somebody in the replies provided a serious answer to the quip: debuggers and logging. Because I use Linux, I’ve gotten quite familiar with log files – any time something doesn’t work, you have to check a log file to see where the problem lies. Without log files, you’re utterly confused; with log files, you’re a quick Internet search away from your answer. However, I never applied logging in my own programs. Firstly, they’re often so small that they’re easy to review without a log file; secondly, I just didn’t know how. It’s still bad practice, though.

Fortunately, one person linked to a really solid explanation of how debuggers work from last year’s PyCon:

A video by Nina Zakharenko explaining debugging in Python

It was so enlightening to watch – she described exactly how I muck around debugging my programs, using print() statements to analyze my output, adjusting it, and so on. Clearly, my bad habit was a common amateur move that happened often enough for her to be motivated to tell people about this debugger. Not only does it look like a good habit to build, the way she explains it makes it just so easy that I don’t see why I never bothered to use it in the first place.

Setbacks and new hurdles

I’d left the pomodoro project in the freezer for quite a while. Last week, I figured I’d pick up on it again, and make some time for that hobby. It’s been challenging to pick it up again, however. KiCad, apparently, has had an update, and currently does not work on my PC. However, the error message that I get does not seem to be a common enough experience to warrant many questions about it online. It might be that it’s too recent for it to become an issue (it was a few days ago); it might be that the problem is unique to my setup; or, alternatively, it might be that the problem is really so trivial that most people have just applied some solution that I do not know.

Regardless, I figured to just work directly on the programming of the new uC, as that is a thing I can experiment with on the go without needing KiCad. Everything works fine for a few iterations as I test the LEDs by blinking them all in order, and I test various waiting functions, and so on. As I was testing out interrupt states, I started getting an odd message – for some reason, Avrdude was setting new fuses. On the next flash, it mentions that fuses have changed, and asks whether I want to reset them. Naturally, I do. However, the fuse setting issue seems to repeat, and then I can’t seem to connect at all anymore.

So, I figure to test some alternatives. Can I upload something else to the chip? Can I use the USBTiny to upload something to my first prototype? Is it the connections on the breadboard, is it the USBTiny PCB itself? What’s going on? All tests fail. So, a new challenge presents itself. I need to figure out where the problem is, followed by what the problem is, and then whether or not this is something that I can fix.

I use Arch, BTW

This week was a holiday for me, fortunately. Technically, my work doesn’t offer standard holidays, except for the usual summer break in university education. However, after the merger with another local university, our new colleagues brought with them a lovely cultural shift: the assumption that the education-free weeks are holidays. So many of them take this as just a right, that our employer has basically given up and accepted this is a holiday week for educational staff. Part of that, I suspect, is the fact that 1 in 5 employees in education suffers from some sort of burn-out, according to the Dutch national statistics centre, CBS (at the top of that first graph, “Onderwijs” means “Education”). I imagine that this certain lax attitude towards the enforcement of certain policies is just considered risk management – the potential damage of formally enforcing this as not being a holiday would likely outweigh the financial benefits of having teaching staff available during this week.

So, I finally had some time and energy on my hands to do some things I’d been looking forward to for a while. One of which was to transfer my desktop PC over to Arch from Manjaro Linux. I’d originally kept my main PC on Manjaro based on the idea that I wanted a PC that just worked without much fuss, and assumed that personally maintaining Arch would cause more issues than having a more curated OS. Ironically, over time, my Manjaro-based desktop ended up having issues more often than my Arch-based laptop. Every so often, an update to Manjaro would prevent me from logging in to the X Server (an update to NVidia drivers would end up conflicting with lightdm-slick-greeter, or reset a configuration file), and I’d be left fixing that. Alternatively, I’d set things up exactly the way I wanted to on my laptop, and when I’d go back to using my desktop, I realized that it had a different setting or program that I did not prefer at all. So, with some more time on my hands, it was time for a change.

As I’d written before, I’ve installed Arch Linux a number of times over the past years, so the installation itself wasn’t that much of a problem. I was, of course, irrationally worried that I’d irreparably mess something up but that’s just usual nerves; the reality is that very little in Linux is irreparable. In fact, the process was surprisingly smooth and easy. Particularly given how Linux separates out files in its system hierarchy, it was incredibly easy to change the OS and still keep my old files and even installed games. So, now I finally fully run Arch Linux as a main driver, and I’m happily configuring and customizing when I want and as I see fit.

Hit My Tax with Assembly

Wow, Assembly is hard! I’ve been following that tutorial I mentioned a while back, and I’ve gotten to the point where I can’t follow along anymore. In lesson 6 about timer interrupts there’s quite a jump to new programming concepts and program structure, without much more guidance. So far, I’ve been able to follow along quite well, and though I wouldn’t be able to easily program something in Assembly just yet, I at least felt comfortable enough to read and understand an Assembly program. At this point, though, it’s becoming quite tricky to follow.

Part of the difficulty, I think, is that it’s clear that the website is originally written in German, and while I’m able to follow the text (fortunately, knowing some German helps me follow the sentence structure), the English translation can be a little rough. It’s terse and clipped prose, packed with information. The tutorial is also written more as a demonstration than a lesson plan. Educationally speaking, I would have put in exercises inbetween or some small challenges at the end of lessons with proposed answers at the back. At least something to provide a little more structure to the whole.

The other part, though, is the massive jump from lesson 5 to 6. The earlier lessons all involved a single pin goal (blinking or dimming it) with only a few registers. In lesson 6, though, suddenly we’re setting multiple registers, running two processes in parallel, on top of learning timer interrupts and the way it completely changes the program structure. Simply said, I’m lost!

I already knew that I wouldn’t end up actually programming much in Assembly – seeing how easy Arduino programming is predisposes me to trying C – but I figured to at least learn the basics so I can understand better what’s happening in C. At this point, the cost of investing time into learning this doesn’t really outweigh the benefit of understanding the workings. I figure that, at least for now, I understand more than I did before, and must rather work on the next project.

My First Makefile

Following that AVR Assembly tutorial I’ve been mentioning recently, I’m finding myself compiling and flashing programs to the ATtiny13 quite frequently. So, after doing that twice, I was done with repeating those steps over and over, when I could just automate that part. My first instinct was to use a simple Shell script to put the commands in, but Linux wouldn’t be Linux if somebody hadn’t already run into and solved that very same issue. So, today I learned about Makefiles.

The way I understand it, make is a program designed help in compiling – it checks whether any source files have changed and, if so, compiles them again. In programs of the size I’m writing, that’s really a negligible advantage, but I can see with larger programs that more than a second to compile having to wait over and over can get tiring quickly. The more interesting part for me is that I can use it to immediately say “flash this to the ATtiny13!” and the makefile will take care of the configuration of the commands to compile and send it. Right now, I have the following file, which I’m sure will expand as I learn about makefiles and how they work:

MICROCONTROLLER = attiny13
PROGRAMMER = usbtiny
INCPATH = /home/nextcloud/lib/avra
SRCFILE = 5_fast_pwm.asm

$(SRCFILE).hex: $(SRCFILE)
        avra -l $(SRCFILE).lst -I $(INCPATH) $(SRCFILE) -o $(SRCFILE).hex

flash: $(SRCFILE).hex
        avrdude -v -p $(MICROCONTROLLER) -c $(PROGRAMMER) -U flash:w:$(SRCFILE).hex:i

clean:
        rm -f $(SRCFILE).hex $(SRCFILE).obj $(SRCFILE).eep.hex $(SRCFILE).asm.lst

Right now, the filenames it creates are a little ugly (“5_fast_pwm.asm.hex”, for instance), so I’m sure I’ll learn how to deal with that better in time. One of my favorite features is the clean command, though. This way of doing it ends up with me having a lot of cluttering files in there, and having one command in the same set to clean it is great.

AVRDUDE Permissions for USB Programmer on Arch Linux

Thanks to a very helpful blog post, I managed to resolve the permissions issue for my USBtiny ISP programmer. As I mentioned in my previous post, I had a permissions error when using it with AVRDUDE, and I wasn’t sure where or how to resolve the issue. As it turns out, there’s a standard config folder where I could just edit a simple file that resolves everything.

Firstly, I needed to see what the vendor and product ID’s of the USB device where. That’s easy to get in Linux with the following command:

[paulo@deku ~]$ lsusb
. . .
Bus 002 Device 004: ID 1781:0c9f Multiple Vendors USBtiny
. . . 

That ID lists the vendor:product IDs. Once you have those, browse on over to /etc/udev/rules.d to make a file called 99-usbtiny.rules, and enter the following:

# Set Group for USBtiny
SUBSYSTEM=="usb", ATTRS{idVendor}=="1781", ATTRS{idProduct}=="0c9f", GROUP="uucp"

As long as you make sure that your user is a member of the group uucp, then you can now use the programmer without a hitch. As the original blog post lists, I have to note that uucp is a group for Arch Linux, and that this will no doubt be different for other distros. Either way, I’m happy to have another issue resolves, and another lesson learned.

Assemly Programming on Linux

As I mentioned a few posts back, I’ve been interested in following a tutorial on programming microcontrollers in Assembly. There’s been some interesting challenges to overcome, but this morning I’ve finally done it and can actually start the tutorial proper. Considering how tricky it was for an average schmuck like me to do this, I figured it’d be worth a small write-up for other people wanting to try this as well.

Firstly, I noticed that the originating pool of information on Assembly programming online now has a surprisingly small number of sources. Mostly, if you’ll look into the uses of AVRDUDE to upload programming to microcontrollers, it will be about uploading C code. The same goes for the main Arch Wiki article on compiling for AVR. That makes perfect sense, of course, because it’s easier to do more with C than with Assembly for the average person; and, from what I’ve read, the speed advantages of Assembly over C for a lot of these purposes are negligable. Apparently, AVR microchips are already optimized quite a bit for C, and the slight advantage that you may get by programming in Assembly pales in comparison to just getting a faster microchip to begin with. So, given that programming these in Assembly mostly seems to be a niche hobby (the programming equivalent of choosing off-the-grid survival rather than camping, perhaps?), there seem to be less than a dozen people online that are a solid source for programming in Assembly, and less when you want answers directly for Linux.

The issue there is that one of the main methods of doing Assembly coding and uploading seems to be Microchip Studio (formerly Atmel Studio, and often referred to as such online, still). The big issue being that this is online available for Windows. So, for Linux, you have to get a little creative. The actual programming, of course, isn’t that tough: use whatever editor or IDE you prefer; my favorite is still Vim, largely because my needs are small and this seems quite fine. The first problem is compiling your assembly, however.

As wonderful as the Arch Wiki usually is, the only relevant post I could find about Assembly is their listing of programming languages. All that tells me is that as is the compiler provided by the packages binutils, fasm, nasm, and yasm. Those, though, are general Assembly compilers. What it doesn’t mention is that there’s also a compiler available specified for the Atmel AVR microchips: avra. Furthermore, the person who wrote the tutorial I linked to at the start also coded his own assembler: gavrasm. Once you’ve written your Assembly file, compiling is ridiculously easy. For example, using gavrasm, you just type:

gavrasm -seb example_file.asm

Or for avra, it’s:

avra example_file.asm -o example_file.hex

Both will give you a hex file that you can upload to your microcontroller. A crucial difference in my case with the ATtiny13 is that gavrasm had its own definitions file for AVR microcontrollers, so didn’t need me to include those. Avra, however, doesn’t, and needed me to supply it. Fortunately, somebody has kindly provided a whole list of them over on GitHub, so I just downloaded the relevant one for me, which turned out to be “tn13def.inc”.

After compiling the .hex file, I needed to figure out how to upload it to the microcontroller via ISP. The tutorial linked above explained a good setup for an ISP header set, and the right connections to the microchip in the first lesson, but it used Microchip Studio for the uploading. So, I had to figure out the Linux alternative. AVRDUDE is that solution, but it took a little more fiddling to get working. Firstly, I needed to figure out what exactly my usb programmer was called (as there are multiple types). Using the command “lsusb” gave me the following output:

Bus 002 Device 013: ID 1781:0c9f Multiple Vendors USBtiny

So, I guessed that “usbtiny” would be the correct name. Using the information provided by Mitchel Davis in episode 5 of his YouTube playlist on Bare-Metal MCU, I ended up with the following command for AVRDUDE:

avrdude -v -p attiny13 -c usbtiny -U flash:w:2_led_on.hex:i

That calls AVRDUDE in verbose (-v) mode for more feedback, telling it we’re programming an attiny13 (-p attiny13) using a usbtiny programmer (-c usbtiny), and that we’re uploading and writing to flash memory, the hex file (-U flash:w:2_led_on.hex) and doing that immediately (:i). That looked great, except it didn’t work. Luckily, the verbose command let me know that it was a permissions issue for the USB programmer. So, sudo-ing the command made it work. Ideally, I’d like to find the correct permissions setting for my user so I wouldn’t have to use sudo for this, but I haven’t been able to find the right one yet. Neither adding uucp nor tty has helped, nor has installing usbasp-udev and avrisp-udev.

So, there’s more to learn still. Possibilities to improve the workflow, for instance, lie in using makefiles to set up some standard settings. And, of course, trying to fix those permissions errors. Either way, with the process described here, I managed to write, compile, and upload the relevant program from the tutorial linked above. So, more learning lies ahead!

Arch Linux and Regular People

Often, when I read about Arch Linux online, there is an odd sense of gatekeeping in comment threads and articles. Arch Linux is seen as a hardcore OS to use. In fact, a common post on /r/archlinux is a proud post describing how somebody spent several hours, trying multiple times to install Arch until they finally managed to make it work. Hardcore fans disparage users of Manjaro as though using Manjaro is a display of weakness. It’s gotten to the point where “btw I use Arch” has become an Internet meme. And yet, I would really recommend Arch.

It’s easy to think that Arch Linux is for programmers; many of the posters online are, or at the least work in IT or have studied Computer Science, or something similar. I am no such thing. In fact, Linux use has taught me some basic programming skills. A common point of praise for Arch is the ability to tweak the system completely to your liking. For some, that means squeezing the absolute most out of their machines: I once read a post where people almost competed to have the quickest boot-up times, arguing over miliseconds as though it was a speedskating event. For me, it’s more about knowing what’s on my laptop. I enjoy the experience of wanting to do something, then just immediately installing a program to do it, and then to go and do that thing. What’s on my laptop is what I put on there, because I wanted it to be.

Doing things this way has taught me about computers, however. Having to learn to program taught me a little about how computers work. Having to understand the Linux filesystems has taught me about the implicit choices in filesystem hierarchies. Having to learn about user and group permissions have made me consider sharing options in Windows as well. Basically, using Linux has made me think about things that I never bothered to think about, even though it’s good to have done so.

I find I reap the benefits of this at work. Now, when we have to switch to a new system, such as when we had to make full use of Office365 when the COVID-19 lockdowns hit last March, I found it easy to adjust to new systems, because I was used to analysing how they interacted. When we needed to set up new interactions between these (such as, let’s say, Microsoft Planner and Microsoft Lists), I could see some analogues to Linux systems, and I could recognize a type of WYSIWYG-interface that tried to abstract code in the background. Despite not working in an IT environment, understanding just a little more of how these systems work has helped me bridge gaps there that I couldn’t have before.

And, on top of that, using Linux is just fun. When I’m back to using Windows 10 on my day-to-day work laptop, it surprises me how clunky Windows is as a system, and how I never noticed that before I really started optimizing my workflow on Linux using i3WM. Let alone that I never bothered to think about workflow before doing that. So, even for us non-programmer, non-IT, non-computery people, I really recommend using Linux.