Kensington Diamonds (KD) operates as a luxury diamond boutique located in Omotesandō
avenue, Shibuya, Tokyo. Matt and his team specialize in selling unique shaped handcrafted
diamonds from engagement, fashion rings to wedding rings.
Matthew Hooper, one of the owners, reached out to us due to his team having difficulties
integrating and connecting their Magento based online store to the platform of his biggest
diamond supplier. KD business relies on direct meetings with potential clients but the online
store was the connection point. Automation was the key here!
Issues
Matthew’s strategy was to expand the business by offering a wide pallet of differently sized
diamond shapes to his clients directly through KD’s online store. The custom script developed
by his former employee was built to fully automate KD inventory with the inventory and stock of
all KD diamond suppliers.
After months of effort the script still wasn’t working like expected. Fragments were missing, and
the products database was sync partially. There was a gap somewhere. Matt was frustrated!!
The whole project was handed over to us! We introduced our workflow as a development
agency quick to Matt. Staging repository was quickly deployed and the whole environment was
taken off from the production server. We started with the work.
Lots of Customization
After we dug into KDs code we located 4 custom scripts straight away. These scripts should
have been called in a specific order for the sync (between 2 entirely different systems) to work.
Cron was set to trigger each script individually on a time basis. From the log files we noticed that
execution time of current sync was average 2 hours?! That was a HUGE problem! 4 custom
scripts defined in a Cron have been overlapping with each other because execution time was
different every time. That explains the issue with partial synchronization that Matt was
complaining about the very first moment.
Discovery
The algorithm was a complete disaster! For example, the script that had to auto disable stock
within Kensingtons database, based on a supplier's stock (DisableProducts.php) was making
approximately `4000-5000` calls to KD database!! One call for each product in the database?! We
saw so many badly written lines of code during our existence as developers (that started more
than 10 years ago), but this was too much!! This was a simple example of a code that breaks
the whole system - a performance downgrader in all aspects!
Slow page loads
Beside the lack of performance most of the pages were over sized in average 20-25MB per
page. Huge, compared to an optimized page that shouldn’t go over 1,5 to 2,5 Mb in size! But
that ended up not being the only problem.
Most of JS code was not deferred on category and product pages which in turn decreased load
time even more. That was very noticeable on slower mobile networks (3G). Lot of consumers
today still use slow mobile networks to browse the Internet.
After another deep dive into the code we noticed a bunch of redundant code deployed as a part
of implementation that was started but not accomplished by another individual developer. Sadly
to say this developer didn’t follow magento coding standards at all. He managed to add another
burden to performance with his concept of altering Magento code on core level.
Google score
Google insights score was awful low! Beside a decent number of unoptimized images on
category and product pages we noticed quick few more issues affecting google speed score.
Sadly to say this wasn’t on Matt’s priority list at the moment! His business was closing a decent
number of sales through several PPC campaigns. Google and search traffic wasn’t something
that the owners wanted to invest into right now. As Matt explained to us, that will be something
we will look into later - the long term strategy. Okay! So let's tie up the horse where the boss
wants the horse tied.
Our Solution
01. Sync and API calls - Neven our lead developer had something to say about the
current time execution of the custom script and synchronization! He drew up a new
algorithm on our whiteboard. The API should be called only once after it gets triggered
by the cron job. The idea was simple: instead of having 4 custom script Neven created
one file to call up the logic in the correct order. This approach of comparing KD’s product
database with API results from the memory reduced ten of thousands unneeded API
requests to just one. This way execution time has been reduced from average 2 hours to
under 10 minutes. That was an amazing performance upgrade! There was no need of
splitting Magento backend to a separate instance after the code was optimized. This in
turn reduced automatically server costs. One update, two wins!
02. More efficient Hosting - KD was hosted on a full dedicated server at OVH, the data
center was located in London. MY GOODNESS!! Who would host a business for local
traffic on the opposite side of the world? We called up Matt and told him that we need to
move KD from UK data center to a data center in Japan, closer to customers. We
migrated from OVH London to Linode (cloud based hosting) Tokyo. Matt’s happiness
after migration was remarkable. Page load improved decently. Matt was able to feel the
improved page load time even better since he was located in Tokyo.
03. Image optimization - We created a small NFS (separate storage - network file system)
instance for media files only. All product images were optimized with a custom module,
the image optimization technology was based on ‘jpegoptim’ tool. Cron (automation tool)
was set to trigger optimization once a day. Every new product image that was added to
the product database after that was optimized on the fly. This reduced page size from
20-15Mb to 2-3Mb in size.
04. Magento Upgrade - KD online store was running on an older 1.9.3 Magento version, the
upgrade to 1.9.4.3 was necessary. This is usually a straightforward process but this time
we had to localize all core updates so the functionality doesn’t get affected. It took us a
week to localize all core code that was altered by someone else. This time we set up a
continuous delivery method based on AWS codePipelines, a repository of optimized
code was deployed to bitbucket. No updates on code level won't be allowed from now
one.
The Results
It took us 3 weeks to sort out core code issues, to improve performance and to solve all
problems related to API integration. More than 300k products are getting sync perfectly in no
time now.
KD was a successful project that was completed quickly. Matt felt very pleased about the
progress. Finally he got managed to merge his idea into the business. In turn we got more tasks
that we had to fix. Migration to Magento2 was scheduled as the most important project in the
first quarter of the incoming 2022 year.
Technologies
being utilized