Sol 11

Been stuck in a martian sandstorm, but it finally cleared up.
I’m lucky it didn’t go global this time.

DALL.E 3 generated image

Here’s what I’ve learnt in my short hiatus:

I should really learn system design, it can save lives

Planning out systems in advance might seem cumbersome, or slow down the time to deployment, but it will save you time later on. I definitely learnt this the hard way. Taking a “let’s do it on the fly” approach will cost you precious time later, planning and prep will save time later. A simple AI app may have the following components:

• Models
• Data / Database
• API endpoints
• Backend Server / Serverless deployment
• Frontend

Having a rough idea of these components and how they will shape up in the development process, along with envisioning what the final workflow looks like, will definitely help in easier debugging and development in the long run.

Unit testing & system design should be a separate course in undergrad

Unit tests are pretty great. As systems scale, things start breaking, a sort of “entropy in software systems”. Testing each component individually ensures that each building block of this larger system works as intended, and more importantly shrieks at the developer whenever it doesn’t. At the end of the day, software will break, but how quickly we fix it is entirely in our hands. This testing mentality also suits well when using system design, wherein you can design your system while keeping in mind the possible foot-guns you could step on, and staying clear of them with some well written, and extensive unit tests.

When working with API’s, bigger is definitely NOT better

When working with API endpoints, it seems tempting to chuck all your requirements into a single endpoint and call it a day. I’ve made this mistake too, at a rather unfortune time as well. Even though it seems like a slightly long-winded approach, creating separate microservices for separate models / functions is quite a good approach to build functionality for an app / service. Rather than having one large service, splitting this up into multiple micro-services allows for convenience in unit testing, as well as molding existing functionality to suit new use-cases. Also, pooled inputs from requests to a main API endpoint can be distributed conveniently for use by individual micro-services.

In short, design your system in advance, use abstractions whenever necessary and focus on modularity of components for flexibility & ease of maintenance.