What I learned when investigating temperature and humidity monitoring for my apartment

18 June 2021

It's extremely hot and humid in my apartment. For no particular reason, I'd like to monitor how hot and humid it is. In theory, I could buy an air conditioner and see how much of a difference that makes, but in practice, I'll just feel that. But before it occurred to me that there isn't really much utility in temperature monitoring the apartment that I spend nearly all of my time in, and can monitor myself, I went digging into how I might do so.

I haven't done much physical computing since I put the Buttery Biscuit Bass on the shelf when I finished at university and moved to Sweden. One or two little Spotify Web API hacks - but connecting a Spark Core to wifi with fancy corporate auth wasn't easy, and I mostly gave up. So I'm a little bit rusty - back in the day, I learned how to use a Raspberry Pi and the Arduino platform. I had a few nice shields, most of which I had at least hacked a little bit with. I had heard of some communication protocols like SPI and I2C, but hadn't used them. And I had designed a few circuits (I made a synthesizer in a lemsip box once). But a lot has changed since then, companies have pivoted, I've learned how to do computers gooder, and I have less tolerance for stringing wires all over the place.

This article isn't a tutorial, it's more "personal notes about stuff I've discovered so that I remember them and hopefully they help you too", I don't claim to have a deep understanding about any of this. Your mileage may vary, I may be wrong about stuff - but hopefully you can use this as a jumping off point.

Platforms

Arduino

Arduino is still going strong. They have a few new boards now, including an uno with wifi. I remember having to order wifi shields for a project, only to discover too late that they didn't support access point mode, and ending up having to prototype with a Raspberry Pi instead. Arduino-compatible chips are used in custom mechanical keyboards quite often, even though they run the QMK program, rather than Arduino's software, so I have had a little exposure to them when building my keebio iris.

Raspberry Pi

Raspberry Pi have released a whole load of new products, including an $80 fully loaded pi with usb 3, wifi, two hdmi ports, and a bunch of other fancy things. They've also got a line of smaller devices, including a Raspberry Pi zero. I was lucky enough to attend a christmas themed hackathon in the UK in maybe 2017 or so where all the attendees were given a Raspberry Pi zero, although I regret to say that I haven't used mine yet. They also have a Raspberry Pi Pico, which uses a custom processor, and a Raspberry Pi contained entirely in a keyboard.

Spark Core

One other chip I had picked up at some point was a Spark Core. I discovered that unfortunately they don't exist anymore - or at least, not in the same way. The Spark Core was a wifi enabled development board that could phone home to Spark's servers, and be programmed via a web IDE. If I understand correctly, Spark also was a sink for telemetry and data. Some time since I got my Spark Core, Spark renamed to Particle, started offering new chips (I think the 'photon' is the analog to the core), and pivoted to target enterprise use cases, rather than the developer focused marketing I had previously seen. I tried to get it working, but I couldn't discover the ssid in setup mode, so RIP to that lil chip - and I don't think I'll buy into the new iteration.

Feather

Feather is a newer board that I don't remember being extremely aware of at the time. They're an ecosystem made by Adafruit, of boards on nice breadboard compatible prototyping kits. Feathers are available with all kinds of different actual microprocessors and onboard goodies like wifi, bluetooth, connectors, etc. Because of the range of microprocessors, they're available for all manner of programming platforms - arduino, c/c++, rust, circuitpython/micropython. I definitely intend to try some of these out some time.

Espruino

I remember having heard of the Espruino back when I was more into hardware, but I think I assumed it was an ardunio clone of some kind. I also wasn't as into Javascript then as I was now, so I might not have appreciated an array of devices that contain lots of fun stuff like screens and sensors that are programmable in Javascript! I definitely want to try these out too.

Writing embedded code

Arduino

Back in the day, I wrote Arduino code. It's a great ecosystem full of libraries and driver support for lots of peripherals. It supports I2C directly, and has a really nice developer tooling setup. So nice that I never really needed to learn much about how the programming of the ROM actually works, what instruction set the code is compiled down to, how the microprocessor works directly, or any of that. I'd like to learn a bit more, but not at the expense of getting stuck in the weeds and never getting anything done.

C/C++

I vaguely understand Arduino code to be an SDK for C/C++ (not even sure which one), that makes it easy to write code using high level languages. Writing embedded code using C/C++ opens up access to writing code that directly runs on lots more boards, but also requires a much heavier understanding of developer tooling than Arduino requires. C/C++ is reputed to be a difficult language, and while I have some experience in it, I definitely remember a high footgun ratio, and inscrutible error messages. However, lots of development boards provide libraries in C/C++, and the tooling evidently exists, so it could be a good way to go. I just don't really know the developer experience.

Rust

Rust is another way to write code that compiles down into the bytes that go into the ROM on the chip and run directly on the bare metal, with no Operating System required. Rust is a great language, with lots of safety features that make it hard to write code with tricky bugs, like C/C++. There's also loads of great learning material for embedded development. One minor drawback is that I haven't been able to find libraries for each microprocessor that I've checked, and certainly fewer libraries for onboard utilities like wifi chips, bluetooth, etc (although I should have a thorough read of the books before I give up. My hesitancy comes mostly from not wanting to get bogged down in learning about developer tooling, rather than spending time writing useful code.

MicroPython/CircuitPython

I actually have used MicroPython before, since it ran on the emfcamp tilde mkiii, the badge that was handed out at the electromagnetic field camp in 2016. I enjoyed that, and it felt like just writing standard python, mostly. I quite like python as a language, and I accomplished what I intended to, rather quickly if I recall correctly. Unfortunately, the embedded Pythons require quite a lot of memory, which isn't necessarily ideal for a lightweight sensor beacon. I'm not ruling it out, but I'm hoping to run into an option that's this easy to use, without the memory overhead. Which is Arduino, I guess, for the simplicity level I'm going for here.

Sensor protocols

In the past, I had mainly used simpler analog components like light activated resistors, and buttons that activated different resistance amounts, and used analog arduino pins to read in data. I never quite got to the stage where I was using devices like neopixels, displays, or more complex sensors. I've since heard of I2C and SPI, and have become aware that there exist tons of sensors with drivers that are easy to use from most of the platforms listed above. I've also learned that connecting these sensors together and between boards isn't always standardised. Some boards have pins dedicated to the important uses in I2C and SPI, but don't have connectors for a standard socket. Of course, breakout boards can help with this, and are available for most of the boards listed above. There are some standard connectors though. Adafruit has a great resource on I2C connector standards, but here's a quick summary.

Grove

A company called Grove makes lots of cool looking I2C sensors, and have a connector that seems to have a significant ecosystem around it. Lots of sensors are available, and there seem to exist breakout boards for this standard. It looks a little bulky, but I haven't seen it in real life, so am not qualified to know this.

Adafruit Stemma/Stemma QT

A lot of Adafruit's products are made with Stemma/Stemma QT connectors. These are 4 pin connectors that carry I2C, and are cross compatible. Stemma QT is a smaller version of the original Stemma connectors, but still carries I2C, so it's useful for smaller profile devices. Like the grove, they use 3.3-5v, so they're cross compatible, but have a different connector, so you'll have to get Grove-Stemma/Stemma QT wires.

SparkFun Quiic

Lots of SparkFun's I2C devices use the Quiic connectors, which also carry I2C. They operate only at 3.3 volts, but they use the same connectors as Stemma - so if you make sure your Stemma device is using 3.3 volts, they're compatible.

I haven't chosen an ecosystem to go with here yet, but since they're all reasonably compatible, I don't expect too much lock-in.

Communicating to other devices

I think I've only ever communicated to/between development boards using either serial over usb, or wifi (except of course that I've plugged an arduino into an ethernet cable), so I'm rather unfamiliar with protocols here. Here's what I learned:

Bluetooth

Bluetooth Low Energy provides a really cool way for devices to communicate with each other. There are two modes, GAP for devices that aren't connected, but can broadcast their advertising ID for connections or information in GAP mode, and GATT, for devices that are connected to each other to do two-way communication. It's pretty flexible and low power, so I'd

WiFi

WiFi is wifi, it's good. It connects the device to a WiFi network, it can connect to other devices over TCP and UDP, which means http requests, presumably websockets, irc if you like, connecting to things like mqtt servers etc. The downside (as I understand it at least) is that many WiFi chips can be rather power hungry, and if you have too many devices on your network without configuring your DHCP server you may run out of local IP addresses. WiFi seems to have a shorter range than Bluetooth in my apartment, but of course, it's wifi, you know about WiFi.

Zigbee

Zigbee is another wireless communication protocol that isn't WiFi or Bluetooth. Devices can talk to each other over it, but it doesn't buy you a connection to the internet. It has a good range, but it can interfere with WiFi and in order to get data from a Zigbee device, you have to have another device listening and talking to something else.

Zwave

Zwave is another wireless communication that isn't WiFi or Bluetooth or Zigbee. It's older than Zigbee and doesn't interfere with WiFi, but it's slower, and it's another protocol that isn't already built into my phone.

LoRa

LoRa seems like a really cool protocol because in perfect conditions it has a 10km range. However, I don't need my apartment's temperatures broadcast all over the city, and I don't feel like doing encryption, so I'll avoid this one.

Thread

Thread seems like a newer wireless communication protocol that has IPv6 built in, and lets devices mesh together, rather than requiring a hub like an access point. Of course, in order for a device to bridge to the internet, at least one of the devices has to be connected to both Thread and an ordinary network.

What can I do about storing data and triggering actions?

My project is about taking measurements and using those measurements in a program later. Here are the options I've considered for storing the measurements.

Grafana

Either self hosted, or by the Grafana company as a service, I could use Grafana to store the info. Strictly speaking in the case of self hosting, I would use a time series database like Prometheus, Grafite, or Heroic (not really that one) to store the data for viewing in Grafana. I've seen some really cool uses of Grafana - at Electromagnetic Field Camp 2016 the bar used it to keep track of stock, and to let people know when the next keg of their favourite beer was in from storage. I've also had to use Grafana to set up custom alerts at work - it turns out (at least in my experience) that Grafana is nice to use as a viewer, and not as nice to use as an editor.

If I wanted to take actions based on the results, I think theoretically I could have bots listening to alerts. But that definitely seems squarely in the 'overengineering' territory, and not something I want to get into.

Custom code writing to a database

Naturally, I could choose to write custom code - a script that either listens to bluetooth using the radio on my own computer that I leave on all the time, or a cloud function that my device can call over WiFi and responds. If this really is the quickest option to accomplish what I need, I would go for it. I would need a database, though, to store measurements, and to manually write code to manage the database size (deleting old records), and any frontend would need to either be able to call my backend in the right format, or be entirely custom. Seems like a lot of work, not necessarily ruling it out.

io.adafruit.com

Adafruit is an awesome company that builds hardware. The feather development board and many of the sensors I would need for this project are their design. They know that a lot of their customers buy their hardware to monitor information, and might benefit from focusing on the hardware, rather than having to write a load of software for their project too. So they built io.adafruit.com, a service where internet connected devices can write their data to. They handle the UI, security, they support data uploading and triggering things based on cloud state. It seems absolutely worth trying out - in fact, I intend to give them a go with the hardware I buy regardless of whether I end up doing something else in the end.

They have a generous free tier which allows pushing 30 data points per minute, 30 days of storage, triggers every 15 minutes, and a 5 feed limit, with a higher limit available on a subscription. They even provide a lot of clear information about data privacy, which is a breath of fresh air on the internet.

MQTT

MQTT is a really cool system designed specifically for IoT telemetry. It's a pubsub protocol, with a broker - one centralized system accepts connections from agents (whether IoT devices, or servers performing actions based on data, etc) that can each both publish and subscribe messages on topics. There's a lightweight protocol, so it shouldn't take up too much power, and it should be fairly easy to write code for. Slight bother that you have to run an MQTT server, and decide whether that should be local or in the cloud. If it's in the cloud, there are security concerns. If it's local you have to keep it up and running (on my computer I switch between booting into Ubuntu and into Windows fairly often).

It's a really cool protocol and I want to try it out, but I think maybe not for this project - at least v1. On the plus side, all the hardware I'd need for the build should be compatible with MQTT, so if I want to try it out later, I won't need to buy a whole new set of stuff.


So that's what I've learned! Writing this post has coalesced my own knowledge of these assorted hardware things, so a) thanks for indulging me, b) I hope you've found it useful too. If you have comments, are annoyed that I got something wrong, or that I missed a technology you're interested in and would like to see included in the comparisons, let me know either on twitter or on mastodon, and let's talk about it. Thanks for reading!!!