Hetarth's blog
Google Summer of Code 2023 - Julia

Google Summer of Code 2023 - Julia

Hetarth Shah's photo
Hetarth Shah
·

7 min read

Table of contents

Project Title: Building an Intelligent Search System for Documenter.jl

Student: Hetarth Shah

Mentors: Morten Piibeleht

Organisation: Julia

About Me:

I am Hetarth Shah from India and have been working with Julia throughout the summer. I've been actively engaged in the enhancement of the search system within the Documenter package. It's been an enriching experience, aligning my passion for programming with the opportunity to optimize and elevate user interactions within the Julia ecosystem.

About Julia:

The Julia organization is a driving force behind the development of the Julia programming language. As a pioneering open-source project, Julia combines the best of both worlds by offering remarkable performance with a user-friendly syntax. With a growing community of dedicated contributors and users, the Julia organization is committed to pushing the boundaries of technical computing, enabling innovative solutions across diverse fields. My GSoC project involved collaborating with this dynamic organization to advance the capabilities of an intelligent search system, contributing to Julia's ongoing mission of empowering developers and researchers worldwide.

Project Overview:

The goal of this project is to improve the user experience of search functionality provided by Documenter.jl; by building an intelligent search system that will enable users to easily and quickly find the information they need.

Project Goals and their Status:

  1. Task 1: Make a CI/CD for faster search load times

    • Status: Not Needed

    • Problem: The duration for the initial load seems to be a bottleneck in improving search functionality as a whole. Because the site's whole index needs to be loaded and it also takes time to build the entire index. This seems to be a problem for devices with a limited amount of resources and sometimes for all devices in general too. We may say that it serves as a preliminary "warm-up" before the search operation.

    • Proposed Solution: To address this we can transfer the "warm-up" to the CI/CD phase. As a result, only the index file would need to be loaded on the client side, thereby lowering the initial load.

    • Current Implementation: Fortunately, shifting to MinisearchJs for client-side search engine improved the initial load times and hence this feature wasn't needed anymore.

  2. Task 2: Changing the Search Engine

    • Status: Achieved

    • Problem: At that time, the search was implemented using LunrJs. But since LunrJs has not been maintained for such a long time we decided to move to another search engine which is better maintained.

    • Proposed Solution: After much searching and comparing, we decided to shift to MinisearchJs as it was properly maintained and was made with the motive of working in a less resource-intensive way which was an added benefit for us.

    • Current Implementation: Currently, the search engine has been shifted to MinisearchJs.

  3. Task 3: Updating the Search UI

    • Status: Achieved

    • Problem: At that time, the search result was merely a list of page links with not much context. Hence, the entire Search UI needed to improve.

    • Proposed Solution: Taking inspiration from established Search UIs from TailwindCSS, Algolia's Modal UI and many more. A solution was proposed to remove the search page and have a modal which would be accessible from everywhere on the website.

    • Current Implementation: Currently, the UI for search has been changed to that of a modal heavily inspired by Algolia's modal design and some custom features like filters integrated specifically for Julia's ecosystem.

Weekly Progress Overview:

  1. Week 0-2 (Community Bonding):

    • During community bonding, I engaged in Google events and discussed the project with my mentor. We aligned on a new UI vision and explored transitioning from Lunr.js to MiniSearch.js for improved search efficiency and developer experience. These weeks laid the foundation for exciting advancements, elevating expectations for an efficient revamped search UI.

    • ℹ️Related Blogs: Cracking GSoC for Julia, Two Weeks of Coding

  2. Week 3-4:

    • These weeks were a whirlwind of research and design brainstorming, driving the mission to enhance user-friendliness in revamping Julia's search. Inspired by established searches from TailwindCSS, Laravel, etc. I explored various search implementations and recognized the need for an improved search in Documenter. Working with my mentor, we initiated GitHub issues for UI and search engine changes. Transitioning to coding, I developed a script to pre-build the search index, aiming for faster, more reliable user experiences.

    • ℹ️Related Blogs: Steps towards Crafting An Enhanced Search Experience

  3. Week 5-6 (Midterm Evaluation):

    • During weeks five and six of my GSoC project, I transitioned from Lunr.js to Minisearch.js to optimize load times and enhance search efficiency. This shift involved key search components like tokenization, trimming, stopwords, field boosting, and fuzzy searching. Minisearch.js significantly improved cold start times and reduced index size. I also crafted user-friendly search UI designs and built a static version with multiple variations. An unexpected turn involved improving the current search listing before JuliaCon 2023. As mid-term evaluations approached, I felt both nervous and prepared, eventually passing successfully.

    • ℹ️Related Blogs: Transitioning from Lunr.js to Minisearch.js

  4. Week 7-10:

    • Weeks 7 to 10 provided a breather after mid-term evaluations, letting me recharge before delving into tasks. The main focus was bringing the new user-friendly search UI to life, involving engaging with the Julia community for feedback. This collaborative process of coding, testing, and refining resulted in a polished UI that marked growth and camaraderie.

    • ℹ️Related Blogs: Relax, Code, Repeat

  5. Week 11-12 (Final Evaluation):

    • As my GSoC journey with Julia drew to a close, the final two weeks were a whirlwind of collaboration, coding, and growth. Reflecting on these months, I iterated on community feedback, refining the search UI.

    • ℹ️Related Blogs: Where It All Starts and Ends

Future Work:

There are many useful feedback given by the community. Some of them are features like storing search history, support for pluggable architecture in the search system and support for both the search page as well the modal UI. Hence, improvements can be made to the existing implementation to make it more feature-rich.

  • Project/Organisation Related Links

  • Related Issues and PRs

    • Issue for Changing Search Engine: Issue #2141 (Closed)

    • Issue for Upgrading the Search UI: Issue #2147 (Closed)

    • Issue for Showing more context in search results: Issue #1437 (Closed)

    • Issue for using another search engine: Issue #1832 (Closed)

    • PR for Search Engine Change: PR #2172 (Merged)

    • PR for Semi-Upgrade of Search UI: PR #2172 (Merged)

    • PR for Search Modal UI: PR #2202 (Merged)

  • Unrelated Issues and PRs

    • Issue for Fixing Extra Navbar Space: Issue #2215 (Closed)

    • Issue for Collapse/Expand Docstring: Issue #2103 (Open)

    • PR for Fixing Extra Navbar Space: PR #2216 (Merged)

    • PR for Collapse/Expand Docstring: PR #2217 (Pending)

Post GSoC:

GSoC is just the beginning. After spending a summer on this project, I believe that this project can greatly benefit from improving the generation of the search index.

Communication Mediums:

Communication with the mentors mainly happened through bi-weekly video meetings where discussion and feedback regarding the search UI and further plans occurred. Julia's slack was also used as a means of communication.

Acknowledgements:

I am immensely thankful to my mentor, Morten, for their invaluable guidance and unwavering support during my GSoC project. The Julia community's collaborative spirit has been a constant source of inspiration, and I extend my appreciation to them. I also want to express my gratitude to the GSoC team for providing me with this incredible opportunity to contribute and learn.

Conclusion:

In conclusion, the GSoC period has been a thrilling ride of learning and growth. From tackling challenges to crafting solutions, every step has added to my skills. I'm excited about the positive impact my contributions will have on the Julia community. This experience has deepened my commitment to open-source collaboration, and I'm eager to continue contributing and learning from it. The GSoC journey has sparked a passion for exploration and teamwork, and I'm enthusiastic about my future open-source endeavours.

If you’d like to know more about my project and its implementation, please feel free to reach out!

Github: Hetarth02
LinkedIn: Hetarth Shah
Website: Portfolio

💖Thank you for joining me on this journey, and look forward to more updates as I continue contributing to the Open Source community.

Open SourceJuliagsocDevJourney