Rewriting the website from Bol.com

At the start of this year, I was presented with a challenge I could not resist. Rewriting an old code base of a high-traffic e-commerce website. The company I work for could be compared with Amazon. However, within the Netherlands, we're still outperforming them. Together with ~1000 other engineers, I work on maintaining and innovating this e-commerce platform (bol.com).

As the year started, I had the goal of growing into a Full-Stack position in which I could balance tech-heavy challenges with my creative side on the frontend. To give some context, the webshop runs on an old, in-house created template engine in Java. Funnily enough, this framework is quite performant and baked into the culture of the company. However, after running over 20 years on this old technique - and seeing the new frameworks advance - the time has come to move on.

Because of this massive transition from a legacy system to a completely new technology stack, I've got a load of interesting stuff to share with you all. This blog post will spill the beans on the thought process behind our journey. From the challenges we faced, to the exciting 'wow' moments, and our first steps into this exciting new architecture. So, grab a good cup of coffee, get comfy, and let's dive right in together!

The decision of a new stack ↫

Choosing the right technology for a platform at scale is a massive decision. It doesn't just impact the developers who have to work with the technology daily, but also defines the company's future growth and scalability. So a while ago, the foundation teams within our ‘Shopping’ product were given the trust to select a framework that would suit the company, and be promising with regards to the future. It can be hard to select the correct one that suits your project, however in my opinion it boils down to a couple of questions:

• What is the community behind a framework?
• What knowledge do you currently have within the company and what is the learning curve to onboard/ teach others?
• What technologies are known to new talents/ new hires on the market?
• What are the technical implications of the chosen framework (performance, rendering strategy, UX, etc)

As we are working with high-load pages (thousands of requests a second), we had to think of a performant option that was highly scalable across the technical landscape and our engineers. The current webshop was a mono-repository which is deployed every morning (automatically). This seems nice, but makes it really hard to rollback, move and deploy fast. Therefore, the autonomy part by every individual team is also important. To explain, within our Product-led structure, every team is responsible for their own page. As an example, a team works on the check-out, whereas the other team works on search pages. This was also something the foundation team had to take into account when selecting a suitable framework.

The decision was made to go for the following stack:

NextJS, React, Typescript, GraphQL and Tailwind 🚀

You might think, why these techniques? Well, let me explain:

• NextJS:
  Primarily, NextJS provides an optimal balance of performance and flexibility. It supports both server-side rendering and static site generation, which are crucial for an e-commerce website like ours. Next to this, it’s widely adopted by the frontend community - which makes it easy to onboard new hires, and get help from the community.
  
• React:
  Within the company, we mostly used two frameworks, Angular and React. Our decision to go for React had to do with our vision in the future of React. We prefer to use React within bigger code bases to be able efficiently split and re-use components across the full webshop. Aside from the adoption, we’ve seen a huge community around the framework which gives us trust into the future of React.
  
• Typescript:
  Well, pretty explanatory I’d say. Do people still use Javascript these days? 🤪 No, all jokes aside. We chose for Typescript because it provides a more robust and safe coding environment. With static typing and other powerful features which will significantly improve code quality and maintainability over the long term.
  
• GraphQL:
  We opted for GraphQL as it allows us to efficiently load data, reduce the amount of data transferred, and simplify the code on the client side. This change made the aggregation layer completely redundant (big service), that we used to have in the old architecture.
  
• Tailwind:
  Finally, Tailwind was selected for its utility-first CSS approach. This makes building responsive designs incredibly efficient, and it provides a consistent styling approach across the entire project. Next to this, it has the biggest community behind it for styling languages.

Rewriting a single page

Alright, the stack is set in stone, so we’re ready to go. Right?!

I still remember the kick-off we had together with the foundation team. On this project, I work as a Frontend engineer in the team that’s responsible for all search pages within the shop. Since the (horizontal) teams have to be onboarded on the new architecture, we always do a kick-off with the foundation. This enables us to onboard quickly, and understand how the full architecture works. Next to that, it ensures that we’re using the same standards across all horizontal teams.

So, the week started and within the first day I’ve been surprised multiple times. Of course, I read into the new architecture - as a good engineer always does, right? However, you must imagine coming from an old piece of legacy code. It’s like you’ve been flying in the first airplane from the Wright Brothers and being transferred over to use a new F35 aircraft. Lots of new features, new practices and especially lots of ‘wow’ moments! Firstly, we created our own micro-frontend. The new architecture has been set-up in a way that we can release independently from each other. Something that’s super normal in today's engineering standards, but hard to imagine from a mono-repository.

Anyway, this MFE (micro-frontend) was up and running within seconds. The app, of course, also supports hot-reloading, something the old webshop did not have. Since it was written in Java, you had to compile the code on every change (happy coding!). Next to this, the foundation team added loads of boiler-plating to provide us with a smooth onboarding. Things like, jest, storybook, design systems, tailwind and GraphQL clients - fresh from the start. This enabled us to focus within hours on creating actual code. Within the first week, we created an initial Top10 page. This is a page which shows - as the title spoils - the Top10 products of a certain category (duh). This demo app worked end-to-end with the graph implementation in which the data was fetched across the landscape. This - again - might feel like it’s super easy, however within a huge company with over 1.5k microservices it’s quite complex to shape the architecture in such a way.

So the weeks continued, and we kept working on the page. In here, we worked towards a first MVP to be able to test - as soon as possible - in production. This was important to us, but also to our stakeholders to verify if the new architecture acts up to the promises. The biggest changes we encountered were the efficiency and performance of the development process. Because we moved from a Rest-based architecture to a GraphQL layer, we were able to be way more efficient with our requests towards the server. Next to this, it made loads of frontend logic redundant. We pushed back most of the knowledge as we could. Ideally, the frontend should only apply frontend logic based on the response. We don’t want to deal with heavy computations like pricing logic or delivery options.

Next, the application of a micro-frontend structure allowed us to work independently and deploy changes quickly. It was really funny to see the mental mindshift of the team and the stakeholders.

‘Well, this will be live tomorrow right? ’, nope, we’ll deploy straight away!

During the first month, we had to do some small maintenance/ tweaks in the ‘old’ webshop - as life goes on - even if you’re building something new. It was extremely funny to hear from the webshop engineer that he had to search for a couple of hours where the code was that was addressing this issue. Whereas in React he could find it instantly. This also showed us that we’re moving towards the correct direction. Making things more easy, more performant, easier to expand and easier to code in!

So, alright, lots of promoting and nice words on the new architecture - looks like I’m a fan. What is the downside of the new architecture? The only thing I can think about is a blocker we faced. Whenever the new architecture launched, the foundation team chose StitchesJS as CSS framework. At the start, I was a bit hesitant about CSS in JS solutions, but I start to love it! However, the only issue with Stitches is that it does not support the React Server Components. Literally the moment after we launched the new architecture, NextJS announced the App Router - which made a massive change in using SSR on component level. If you’d like to learn more about RSC (React Server Components) I’d highly recommend this blog post by Josh.

So long story short, StitchesJS, currently doesn't support React Server Components due to the way it handles styles. React Server Components (RSC) aim to provide a way to render components on the server, and then send the rendered HTML and minimal JavaScript to the client. However, StitchesJS creates dynamic styles at runtime, which is a client-side operation. Therefore, it can't generate and inject styles during server-side rendering, which is a key aspect of RSC.

This was the only bummer I could think of when starting. Because at the moment of starting on the page router for our SPA (single page application), we already knew that we had to migrate soon to the app router. Fast forward a couple of months, we’ve decided to move forward with TailwindCSS, and therefore rewrite the styling from StitchesJS to Tailwind. This will enable us to make optimal use of the new React Paradigm.

How to go to production?

So alright, we’ve created a micro-frontend which enables us to deploy quickly. However, we want to push this to production independently without breaking/ affecting other pages. So how do we do this? Well, before the traffic reaches the internals of Bol, we firstly route you through a gateway/ proxy. This layer will check what page you’re requesting, and based on that - route to the correct virtual service. This enables us to do different roll-out strategies.

During the creation of our MVP we used the following:

• Full rollout (100%) on STG.
• Full rollout (100%) to internal colleagues. This means, everyone who’s logged into the VPN will be routed directly to the ‘new’ page (micro-frontend). This has the benefit of testing with production data - without exposing to the customers yet.
• Phased rollout to customers. Through an A/B experiment we can route certain groups of users towards either the ‘old’ or ‘new’ page. This helps us to see differences and monitor the performance against the stable system that has been running for 20 years. The distribution is mostly 50/50, but depending on the ‘risk’ we can scale up or down.

After two months of working on this page, we went fully live on production. This might look long, however these pages have loads of hidden logic (legacy) associated with them. Next to that, all side conditions should also be up to standard. Topics as SEO, performance, alerting, monitoring, logging, dashboard, night support, web vitals etc.

Overall learnings

In looking back at our journey, we've gleaned some crucial insights. Transitioning from a legacy system to a new tech stack is not a walk in the park, but it's an exciting challenge that brings with it immense learning and growth opportunities. We experienced first-hand the power of modern technologies such as NextJS, React, and GraphQL, and the efficiency they brought to our processes. We learned the importance of being able to deploy changes quickly and independently, and how this can shift the mindset of the team and stakeholders. We also discovered that obstacles are part of the journey. Our encounter with the StitchesJS and React Server Components compatibility issue was a bump in the road, but it pushed us to find another solution and keep moving forward. Finally, we saw the value of a phased roll-out strategy, which allowed us to test our new system thoroughly before a full launch.

All in all, this journey has been an amazing learning opportunity in which we’ve been given a lot of trust, autonomy and fun. Together with my teammate, I’ve had loads of meetings/ huddles in which we paired up on tasks. Not because we couldn’t do them on our own, but because we did not want to miss out on the fun. The project really made me fall in love again with the thing I truthfully love about programming. Working in a fun & challenging environment, together, where we make direct impact for our customers/ users.