How not to reinvent the wheel — the hybrid system approach
lessons learned in a startup before it got acquired
Back in my early days as a dev, I worked for a transport startup that broke up a monopoly by doing weird things. University had given me an impression of a structured and orderly approach towards running a business. Perhaps that’s how things work for other places, but not at the place I ended up.
I was hardly prepared for the way my first proper boss worked.
He hated reinventing the wheel, and any proposal always had to provide alternative existing solutions that could be modded to fit the project’s purpose. In short, the entire system was completely bootstrapped together in such an ad-hoc way that was completely unique to the business itself.
Here are four different kinds of hybrid creations we made for the various projects we booted up on a monthly cycle.
WordPress + Angular
When a third party company proposed to build the content system for the low low price of $45,000 plus on-going maintenance fee, a part of my brain instantly flashed red.
I told my boss that what he’s after is a blog. Content system was just a fancy name for it. Besides, he’s already got an established WordPress installation. The problem was not because WordPress itself sucked, but how it was being used.
A content system can only remain organized if the people using it do it properly.
With a backlog of over 500 posts as pages with no tags or categories, of course WordPress is going to feel like it’s failing the business.
After a bit of a revamp with a new theme and re-categorization of content, it allowed the business to utilize the widget areas for the booking form. Within two weeks of sorting out WordPress and porting parts of the Angular form over as a plugin, we ended up with a smoother content to booking system conversion process, leveraging the SEO work that was bringing in the traffic.
Lesson of the day: if it sounds expensive, it probably is.
Angular micro-frontends in Java
When I started, there was already an established booking system. However, the original system was clunky and the backend guy wasn’t exactly the best when it came to front-end implementation.
There were also limitations in where we could put the frontend and for a particular project I worked on, it had to be within the Java application itself. While it wasn’t ideal, especially when we were trying to move everything out and into their own separate spaces, I had to come up with a workaround and ended up with Angular micro-frontends.
While micro-frontends and Angular isn’t quite synonymous, it is possible. There’s a misconception that for an Angular app to work, you had to create an entire Angular app from scratch, set it up and then only work inside that monolith. However, Angular does allow for modularization through Angular elements — meaning that you can build your Angular component separately and then drop it into the project. The result is a neatly contained piece of code that remains isolated from everything else while drawing from a centralized data management service to keep states in-sync with other parts of the Java app.
Why didn’t we build in Java directly? In part, it was because we wanted to create a clear separation between the Java part and the frontend technologies. Things were starting to get muddled over time and clean separation was only achievable through a transitional approach.
Lesson of the day: when building from scratch isn’t possible, micro-frontends can help speed up the transition process
Angular + microservices backend
There was a time where the Java backend got so bad and we still had to ship out features. Java itself isn’t bad, but the way previous devs stitched together code led to a spaghetti and meatballs kind of situation.
It’s the perks of working with legacy code and past extensions haven’t been systematically modularized.
The time limitation and reduced number of developers on the team meant that we had to get creative with what we’ve got. Rather than trying to wrestle with the Java backend, we ended up composing a series of microservices backend and attaching it to our existing Angular frontend.
There were a lot of adapters to the Java API — but the new structure also made way new backend code that wasn’t tightly coupled with the existing system. States were saved to a database and coordinated through a series of chained AWS lambdas. We ended up using Serverless for local development and then pipelined it so that all the developers on the team could work on a particular part of the backend without encroaching on another dev’s code. It also got rid of the pesky
git conflict issues that we were having.
Lesson of the day: modular extension can be a blessing when you’re bogged down by legacy code
Microservice backend with SQL and DynamoDB
One of the major issues of building an entire backend with microservices is how do you deal with states?
The transient nature of microservices means that data that needs to be passed through the composition chain needs to be managed by an external source. You also need the data structures to be flexible enough without the need to create new schemas to accommodate for every little bit of data you want to hold.
So how did we balance the need for data to be informative but also flexible at the same time? We ended up going with an AWS lambda + DynamoDB approach. Being part of the same ecosystem meant easier and faster accessibility speeds than trying to work over the network with an external database.
Not only that, the tableless structures allowed for data to be any shape required. This allowed for quicker experimentation on data models and transformations as needed without impacting the rest of the application.
The way DynamoDB deals with billing is similar to Firebase — charge based on usage. The only other difference is the provisioning of resources, meaning that your bill can be hard-capped if needed.
When things needed to be stored permanently, we’d move the data over the SQL database, where the Java backend would have access to it and do what it needed to do.
Lesson of the day: hybrid data solutions can work, especially if you have clear purposes for the database types. It doesn’t have to be an exclusive one or the other kind of approach when it comes to SQL and tableless databases.
While it’s nice to have everything in one place and code that’s composed in a highly modular manner, it is not always 100% guaranteed that this will happen in real life.
Code can be messy, grown in strange environments that led to decisions that made sense at the time. A hybrid approach can help balance the need for a developer’s sustained sanity against delivery requirements. It also allows for the ability to modernize your systems without freaking out the boss.
Telling your boss that the entire system is rubbish will never fly. Why? Because to most non-developers, they see code as a permanent structure — like a physical object that doesn’t change or erode. There’s a monetary value attached to it and most businesses need to make a return on that investment before they think of discarding what they’ve currently got.
The Internet itself is now at a mature state, along with the development techniques and ideologies. Things that are built today have a higher chance of surviving the next decade than what was built during the infancy of hybrid system ideas.
When it comes to the business perspective, the point is to keep moving forward, and sometimes, as developers, we just have to figure out that part without getting rid of everything.