The Produce Market Guide (PMG) is a resource to produce commodity information, trends, and data analysis. PMG is a product of Farm Journal which shares information on fresh fruits and vegetables and is the industry's leading source for news, information, and analysis.

The advanced search feature on the website was slow which ruined the overall user experience with an increasing bounce rate. The previous version would take 17-25 seconds for a search to complete including the page load time and another 15-20 seconds for the results to populate.

Moreover, the product team wanted to monitor the user behavior on the search and provide them with the best-personalized experience. The lack of any technical support and missing integration with a search would cause the admin to only guess the user behavior pattern.

The organization was familiar with Drupal since the website was already on it sans the features they were looking for.

Since there were not many changes required in the architecture, the joint client-vendor team chose to progressively decouple the search functionality. The new search of the website was built with React & elastic whereas the overall architecture remained the same.

Goal

• To build search applications with facets and track user activities for a better and personalized user experience.
• Improve the user experience of the website by providing quicker/faster results

Requirements

• Develop the architecture to index and map the data on the elastic server from Drupal to build an efficient modern search kit system
• Develop a backend service for the mapping and index the data on CRUD (Create, Read, Update, Delete) operations on the website
• Develop a Logstash pipeline to log queries (helps to track user behavior) on the elastic server and set up Kibana for visualization of data
• Write various test cases for the backend services and React app
• Improve the UI of the application as per the requirement
• Keep the launch time within the brackets of one month

A new design was planned to integrate the faceted search. It required three simultaneous implementations.

Implementing the ElasticSearch Connector module

As per the requirement of the client a mapping was created on the ElasticSearch server and the data was indexed on the same. To meet the requirement of the indexing patterns used by Kibana and search kit, many custom preprocesses were written for the Search API.

Create a faceted search application with React

Once the Elastic backend architecture was developed, one of the biggest challenges in building the faceted search was adding the Searchkit and React in the same search application.

For that, the front-end, the logic and project structure for the search was designed and developed in the sandbox with modern tools like Babel and Webpack and 3rd-party libraries like Search-kit.

Integrating Kibana and Logstash

Both Logstash and Kibana are based on Elasticsearch. While Logstash is an open source tool for collecting and storing logs for future use, Kibana is a web interface which is a free open-source analytics and visualization platform to give a better understanding of the data.

Outcome

The new PMG website is a lot user-friendly with search application and loads the results in lesser time than before. Since not much was added to the already existing website, the new version isn’t difficult to maintain.

Because of Logstash and Kibana, the admin can now track the user behavior on search application and personalize the user experience.

The project was completed within a month following the principles of SCRUM and AGILE with teams fluidly arranging themselves according to the needs.

Earlier the product would take 17-25 seconds for the search to load (including the page load time) and 15-20 seconds more after the page load for the results to populate.

We helped PMG decrease their load time by 1200%.

The product now loads in less than 2 seconds and it takes less than 500 milliseconds for the results to populate.