The Learning Search Multiplier/Popularity boost is based on click tracking for up to the previous 30 days (this is the system default, but it can be changed in the client system parameters). Each click on a product is saving the absolute position of the item in search results (if a product is on the second page, and it's position in the results was 2 and there are 12 items per page, the absolute will be 14) and the number of results from the current search. For Popularity boost, only items with the number of results >5 and absolute >3 are taken into account.
We recommend that you keep the value between 0 and 1 to ensure that the search result's relevancy still has a strong impact on the order the items are displayed in. This helps safeguard that an item does not appear on top because it is popular, even though it is not necessarily the most relevant search term. In essence, the score should always have more weight on the order than on the popularity for search requests. For example, let's say a user searches for keyword "watch". A watch accessory appears in the result, but so does a replacement watch band. Perhaps users have been clicking on the watch accessory more in the last 30 days, so now the accessory might start displaying at the top of the results if the Learning Search Multiplier has been set to favor popularity. This is a less relevant result for the keyword "watch" and most likely is a lower priced item than an actual watch, thus these results might be deemed undesirable.
It is also valuable to set the value greater than zero so that popularity is able to play a role in the display order. When you are searching by department, relevancy doesn't play an important role as you are using navigation as opposed to keywords.For example, if you search a category, all items that will return will have the same category and will match the same score. However, in this case if the popularity metric is turned on, the items that are more popular will display closer to the top regardless of the popularity setting being at 1 or 3. We recommend that you turn this feature on in the staging account and ensure that the dev site is connected to the staging site for testing.