Noted: Let’s make an app: part 2

Another week has passed, and my baby is taking shape.

It has been a week of dizzying highs, crushing lows, and a general feeling of drowning in a sea of choice.

The end result is that I have a prototype/mvp installable app, across Windows, Mac OS, IOS and Android, with login via email address or google account.

Which I’m overall pretty ecstatic about!

Behold, my (probably highly bug ridden) product:

The eagle eyed among you may notice that I am using my notes app to write a todo list for things I need to do to improve the notes app.

INCEPTION STYLE

Where did we start?

At the beginning of the week, I had a hosted API, connected to a database, and an installable IOS app, which could be installed to multiple devices, and could sync notes between them.

It did its syncing by repeatedly calling the notes API via a GET request, polling for changes, which was gross. It also had a save button to update the notes via a PUT request.

The protocol was a GET/PUT on a single notes endpoint.

It had no authentication, so anyone using the app got the same notes, and could edit them.

So what’s the plan

I wanted to continue with the ‘get shit working fast end to end‘ approach, in order to properly test that what I was trying to do was possible with the tech choices I have made. To meet my MVP requirements, I still needed:

  • IOS client
  • Android client
  • Windows client
  • Mac OS client
  • API which allows syncing between devices (ideally by pushing changes)
  • Authentication/login so you only get to edit your own notes
  • Figure out how to distribute the installable bits… (app stores and the like)

Let’s fix this broken notes protocol

As you can see from the list above, a fairly key requirement was to sort out a way of syncing changes to all clients, ideally without resorting to the basic, heavyweight polling solution I had in place.

A major theme of this week was one of bewilderment at the sheer dazzling array of technology choices available, each with their own subtle pros and cons.

This began with the choice of technology for syncing notes with the server.

During the previous week I played around with Server Sent Events, which would have suited my requirements pretty well, but didn’t play nicely with the IOS client, and didn’t seem to be being used a lot (meaning I couldn’t find many good tutorials/resources on how to use them with mobile clients).

I also tried web sockets, which, again, would have been pretty nice to work with in a web app, but had not great integration with IOS.

I then lucked out (I think), and gave socket.io a go.

https://socket.io/

In their own words:

Socket.IO is a library that enables real-time, bidirectional and event-based communication between the browser and the server. It consists of:

  • *a Node.js server *
  • a Javascript client library for the browser (which can be also run from Node.js)

The client will try to establish a WebSocket connection if possible, and will fall back on HTTP long polling if not.

So it is a really nicely written wrapper around web sockets, with a fallback.

It has great documentation, and they actively maintain clients for Swift, Java and Javascript (among others), which means I can use it easily in all of the places I need to (more on this later).

After integrating this with the IOS application, my process now looked like:

1) Establish a socket connection with the notes server

2) On updates from the server, update the local client ‘notes’ variable

3) On the client saving notes, push them up to the server via the socket connection

4) On the server receiving an update, push the changes out to all connected clients

Desktop clients

I was pretty sure I could smash together an Android client if needed, and that would use Java/Kotlin, so I knew it would work with socket.io.

I was less sure about what to do about desktop clients though.

I have worked at companies using Electron, and I have a decent amount of experience with web technologies, so first of all I hammered together a crappy Electron application, just to have another client running. It was pretty simple to get going.

My previous gut feel about Electron is that it can be slow, resource hungry, and generally it feels a bit hacky to develop. I wanted to explore other options.

My preference given unlimited time would (I think) be to write desktop applications from scratch for each platform.

Given my simple UI (just a text input screen), and the fact that I develop on a Mac, I think I could have made a Mac OS application without too much hassle.

Windows, on the other hand was far less obvious how to get started, and would likely involved virtual machines/dual booting and other annoyances.

That is not to say it wouldn’t be possible, it absolutely would, but the developer experience would probably be painful. I did briefly considered actually finding a used windows laptop and using that for the Windows client…

However, I want to move quickly, and minimise frustration, so that seemed like a no.

I then read a bunch of opinionated blog posts, which said alternately some form of:

  • ‘Electron is trash, write native apps instead’
  • ‘Electron is trash, use this other framework instead’
  • ‘Electron is trash, but its the best option we have, and who has time to to annoying costly alternative anyway’
  • ‘Electron is great’

I wasted half a day trying to get JavaFX to work and gave up.

Then I revisited my Electron app, cleaned it up a bit, integrated it with socket.io (was super easy), implemented their suggestions for making the app a bit more secure, and, after some experimentation with other libraries, used Electron Builder to produce installers for Mac and Windows.

It was about as seamless as any development I have ever done, and so after dicking around trying other solutions, I did what many companies/individuals seem to have done, and said ‘Fuck it I’ll just use Electron‘, a choice that I’m pretty happy with on balance.

If I get curious in the future, I can always try my hand at moving to native apps, or some other framework, but for now it’s just so convenient.

Cool, two clients up and running, very satisfying. No horrific things have appeared yet, and it seems like socket.io is a semi sensible choice.

I still had one connection for everyone, meaning there was just one set of notes, and it was editable by everyone who installed the app.

There was no getting around it, it was time for authentication to rear its ugly head…

Authentication/Identity/My brain is melting

TL;DR After thoroughly confusing myself about Auth0 and OpenId Connect, I have ended up using Auth0, and their universal login, to authenticate users on both mobile and desktop clients.

The net result of this is that clients can easily get a JWT access token, which the server can validate to ensure they are who they say they are, and then grant them access to a socket connection with only their own notes.

This is achieved via socket.io’s rooms functionality, where each user is assigned to a room with their userId (socket.join(userId)), as well as this handy library for validating JWTs for use with socket.io https://www.npmjs.com/package/socketio-jwt

const io = require("socket.io")(server);

io.sockets
  .on(
    "connection",
    socketioJwt.authorize({
      secret: jwks.expressJwtSecret({
        cache: true,
        rateLimit: true,
        jwksRequestsPerMinute: 5,
        jwksUri: process.env.JWKS_URI,
      }),
      timeout: 15000, // 15 seconds to send the authentication message
    })
  )
  .on("authenticated", (socket) => {
    const userId = socket.decoded_token.sub;

    socket.join(userId);

    Notes.findOne({ username: userId }).then((notes) => {
      if (notes) {
        io.to(userId).emit("notesUpdated", notes);
      }
    });

    socket.on("updateNotes", (payload) => {
      debug(`updating ${userId} notes`);
      Notes.find({ username: userId }).then((notes) => {
        if (notes && notes.length) {
          Notes.updateOne(
            {
              username: userId,
            },
            {
              username: userId,
              content: payload.content,
            }
          ).then(() =>
            io.to(userId).emit("notesUpdated", {
              username: userId,
              content: payload.content,
            })
          );
        } else {
          Notes.create({
            username: userId,
            content: payload.content,
          }).then(() => {
            io.to(userId).emit("notesUpdated", {
              username: userId,
              content: payload.content,
            });
          });
        }
      });
      socket.on("disconnect", () => {
        debug("user disconnected");
      });
    });
  });

From the Electron/any JavaScript client’s perspective, this looks like this:

  connectToNotesStream: () => {
    socket = io(apiIdentifier);

    const token = getAccessToken();
    socket.on("connect", () => {
      socket.emit("authenticate", { token });

      socket.on("authenticated", () => {
        socket.on("notesUpdated", (data, ack) => {
          _updateNotes(data.content);
        });
      });
    });
  }

Mobile clients

Similar to my decision anxiety around desktop technologies, I flirted heavily with the idea of using React Native, Ionic, or Flutter etc. before eventually deciding that my UI needs were so minimal that I may as well just develop native apps.

This project is largely a learning endeavour, and I like the idea of getting a shallow understanding of the IOS and Android development ecosystems, rather than Facebook’s wrapper around them.

I can easily develop IOS and Android applications from my Mac, socket.io has well supported Swift and Java clients, and Auth0 integrates (kind of) well with Java and Swift.

How I picked things

Reading this through, it seems like I made decisions quite easily, but I’ll be honest this week has been overwhelming.

There are so many different ways to build mobile and desktop apps these days.

In order to make decisions, I had to come up with some criteria for how to pick one technology over an other. The criteria ended up being pretty simple:

  • Does it work well with socket.IO and Auth0?
  • Will it give me knowledge of an underlying protocol/technology that could be useful?

As an example, React native fulfilled number 1 (good Auth0 support), but not number two (I don’t want to learn React Native, I’d rather understand the native platforms a bit)

Electron also only fulfilled number 1, but there wasn’t a clear alternative given the time constraints I have (at some point I’ll have to get a job again!)

I’d still have preferred to have a go at making a native windows application, and I still might…

Although Electron also makes it super easy to make linux apps which is also kind of appealing.

I’m basically making this tool for myself, and I regularly work on Windows, Mac and Linux, as well as switch between IOS and Android phones…

Conclusion

Another great week.

Next I want to design a logo, come up with a more unified UI/UX flow for the different platforms, to make them look more similar.

Also try and actually get some stuff in various app store(s).

Wish me luck…

Leave a Reply

Your email address will not be published. Required fields are marked *