Solr Cloud for Sitecore – Step 1 – Install and Configure ZooKeeper

Dear Sitecore Family,

Last week, I was successfully able to configure Solr Cloud environment and I struggled a bit in the beginning when I got started with it.

Hence I decided to blog about it when I was able to successfully do it.

Step 1 to Solr Cloud environment is setting up Zoo Keeper!

So what is Zoo Keeper and why is it required?

Great Question!

ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. All of these kinds of services are used in some form or another by distributed applications. Each time they are implemented there is a lot of work that goes into fixing the bugs and race conditions that are inevitable. Because of the difficulty of implementing these kinds of services, applications initially usually skimp on them ,which make them brittle in the presence of change and difficult to manage. Even when done correctly, different implementations of these services lead to management complexity when the applications are deployed.

Source: https://zookeeper.apache.org/

One more important responsibility of Zookeeper is to keep an eye on the state of all Solr nodes in the cluster. If any node goes down and a search request comes in for that node, Zookeeper routes it to an alternative replica node.

How do we setup ZooKeeper?

Well, here are the steps:

Unzip the zookeeper zip to a location – say – E:\ZooKeeper
Create a folder Data and create a file myid without an extension inside it
- This is where we set the node number
Add the node number of the zookeeper – like 1, 2 or 3.
Create a folder log.
- This is where we will have the log file to check whether the ZooKeeper Service is running successfully or not.
Go to conf folder, rename the zoo_sample.cfg to zoo.cfg
Update the following things:
- PortNumber to 2141
- Update dataDir to E:\ZooKeeper\data
- Add dataLogDir= E:\ZooKeeper\log
- Add all the 3 ZooKeeper server Hostnames with the portnumber ranges :2888:3888
Next, Execute the zkserver.cmd in bin – it should run successfully with the given configuration.
Now we want to run it as a Windows service hence, use NSSM to do so.

So I have taken care of all these things in a PowerShell Script here:

It will require the following things to be done:

Download the gz of Zookeeper from the archives https://archive.apache.org/dist/zookeeper/ unzip it and put the zip in the same folder as the $downloadFolder mentioned in the script. I am planning to tweak it to take the gz and unzip it but the reason I have kept this version is because unzipping gz will require 7-zip and when we setup things on client environments, sometimes we cant install things on client servers. The best way is to use a zip which can be extracted by Windows as default.

https://gist.github.com/varunvns/d9143e03d44503ae2806c827e3ab5012

References:

During the course of learning about Solr Cloud, I stumbled upon some articles or documentation which I felt were awesome and great in clearing my concept regarding Solr Cloud and ZooKeeper:

https://lucene.apache.org/solr/guide/6_6/solrcloud.html
https://github.com/ivanbuzyka/Sitecore.SolrCloudConfiguration/wiki/3.-Install-Zookeeper-as-a-service
https://sitecore-community.github.io/docs/search/solr/Install-and-configure-SolrCloud/
https://doc.sitecore.com/developers/90/platform-administration-and-architecture/en/walkthrough–setting-up-solrcloud.html

Credits:

In no particular order, these guys deserve the credit to handle all my questions and answer me patiently, and some also troubleshooting things patiently with me.

Happy Sitecoring!