- Technical Workflow
- Modifying the Sequence of Search Results
- Category Adjustments
- Template Adjustments
- Possible Problems and their Solutions
Please note that this is the documentation for IntegerNet_Solr Pro for Magento 1. The documentation for IntegerNet_Solr Pro for Magento 2 will soon be published.
- Install Solr and create at least one working core
- Copy the files from the
solr_confdir of the module repository to the
confdir of your Solr core
- Reload the Solr core (or all of Solr)
- (If activated: deactivate the Magento compiler)
- Copy the files and directories from the
srcdirectory of the module repository into your Magento installation. If you are using modman and/or composer you can find a
modmanfile and a
composer.jsonin the root directory.
- Clear the Magento cache
- (Recompile and reactivate the Magento compiler – it is not recommended to use the compiler mode of Magento, independent of the IntegerNet_Solr module)
- Go to the Magento backend, go to
System -> Configuration -> Solr.
- Enter the Solr access data and configure the module (see below)
- Click Save Config. The connection to the Solr server will automatically be tested. You’ll get a success or error message about that.
- If you are using the Magento Enterprise Edition, you have to switch off the integrated Solr engine by switching
System -> Configuration -> Catalog -> Catalog Search -> Search Engineto
- Reindex the integernet_solr index. We recommend doing this via shell. Go to the
shelldir and call
php -f indexer.php -- --reindex integernet_solr
- Try typing a few letters in the search box on the frontend. A box with product and keyword suggestions should appear.
For each product and store view combination, a Solr document is created on the Solr server. This happens through the Magento indexing mechanism which allows to react on every product change. You can either have a full reindex process which processes all products efficiently (in batch of 1000 products each, configurable) or a partial reindex.
A partial reindex will happen if any product is created, modified or deleted and will recreate the corresponding documents in the Solr server for the affected products only, so the Solr index is always up to date.
The data which is stored on Solr contains the following information:
- Product ID
- Store ID
- Category IDs
- Contents of all product attributes which are marked as “searchable” in Magento
- Generated HTML for autosuggest window, containing the defined data and layout (i.e. name, price, image, …)
- If configured: Generated HTML for results page, once for grid mode and once for list mode
- IDs of all options of filterable attributes for the layered navigation
If you are using the full reindex regularly, we recommend using the swap functionality. You can configure the module to use a different Solr core for indexing and swap cores afterwards (
System -> Configuration -> Solr -> Indexing -> Swap Cores after Full Reindex).
When using the autosuggest functionality, there will be an AJAX call whenever a customer has typed the first few letters into the search field on the frontend. The AJAX response will be the HTML of the Autosuggest window, including product data, keyword suggestions, matching categories and/or attributes. The target of the AJAX call will differ depending on the configuration setting
System -> Configuration -> Solr -> Autosuggest Box -> Method to retrieve autosuggest information
This is the basic method which uses Magento methods only, as does the MySQL default or the Solr functionality of the Magento Enterprise Edition. It’s the slowest but the most flexible. It’s intended as a fallback if other methods are not working due to whatever reason.
This will call a separate PHP file
autosuggest-mage.php in the Magento root dir directly. It skips the routing process of Magento and thus will deliver the contents faster. Still, all Magento functionality should be working.
We haven’t found any disadvantages of this method yet, except its speed (see below).
This will call a different PHP file
autosuggest.php in the Magento root dir directly. It doesn’t use most of the Magento functionality and is a lot faster in most environments. As it doesn’t do any database calls, all the data which is needed for the autosuggest window will have to come either from Solr directly or from a text file. The module automatically generates text files which contain the information used for the default autosuggest window:
- The Solr configuration
- Some additional configuration values
- All category data (names, IDs and URLs)
- All attribute data which is configured to be used in autosuggest (option names, IDs and URLs)
- Some additional information like the Store Base URL or the filename of the template file (see below)
- A copy of the
template/integernet/solr/result/autosuggest.phtmlfile which is used in your theme. It has all the translation text already translated.
The information is stored in
var/integernet_solr/store_x/config.txt (as a serialized array) and
var/integernet_solr/store_x/autosuggest.phtml. These files will be automatically recreated in any of the following events:
- AJAX call on the frontend occurs while the file
- The configuration of the Solr module is changed.
- All Cache is cleared.
- Button “Rebuild Solr Autosuggest Cache” on the Magento admin cache management page is used.
So if you want to force recreating that information, trigger any of the above events.
Note that you won’t have all Magento functionality available if you are using this method. Please try to stick to those methods used in
app/design/frontend/base/default/template/integernet/solr/autosuggest.phtml. For example, you cannot include static blocks or other external information without further modification.
You will find the configuration in the admin area of Magento at System -> Configuration -> Solr:
The configuration option are listed and described here:
In the upper area, success messages, error messages, warnings and information messages are display. For example, there is an automated check if the module is activated, if access data to the Solr server is filled in and if the connection is working correctly.
If this switch is set to “No”, the search module cannot be used on the frontend. Instead, the default search of Magento will be used. You can set the options for single websites and store views separately.
The module needs a correct license key in order to work correctly. You will get it from us after purchase and payment of the module. Please contact email@example.com if you are having problems with your license key.
You can test the module for two weeks without any license key. Only after this period, the license key will be necessary for the module to work.
A license key is valid for one live instance and an arbitrary number of corresponding development, test and staging instances.
Attention: there will be no internet connection to a license server. As soon as a valid license key is entered, the module will work on its own without any external dependencies (except the Solr server of course).
If this switch is activated, all requests to the Solr server will be saved in a log file. This affects the autosuggest function and the search results. You can find the logs in the directory
/var/log with the file names
The log files are used for bug tracing and for optimization of search results only. As the files can get pretty large with a frequently used search function, we usually recommend switching off logging on live environments.
You can enter the connection and access data to your Solr server here. If the data is correct, you will see according success messages on the upper part of the configuration page – error messages otherwise.
In case you don’t know the access data, you can get them from your administrator or hosting company which has installed the Solr server.
If you have access to the admin area of the Solr server, you can find the access data as follows:
- Select your core in the Core Selector on the lower left part of the page:
- Select “Query” below the Core Selector
- Click “Execute Query”
- In the upper part on the right you will now find the URL which was used for your sample request:
The URL can be divided into the single parts as follows:
The single parts can then be entered into the configuration:
Please take care that the field Core doesn’t contain any slashes, while the field Path must contain at least one slash at the front and one at the end.
If you don’t get an error message after entering your credentials for the Solr server, you should stick to the default method cURL. Otherwise you can try switching to file_get_contents. The availability of both methods depends on server settings of the Magento server.
Please enter username and password here if those are needed for the access from Magento to the Solr server.
To make sure that the connection to your Solr server is not lost unnoticed, the module is able to automatically perform a connection check.
When the value “Yes” is selected, an automatic connection check will be performed every 5 minutes.
If you would like to be notified about each failed connection check, enter the value 1.
Notifications are sent to the email addresses provided in this field. For multiple recipients, divide addresses by comma.
You are able to set up your own email template for connection check notifications. The template is stored in
System -> Transactional Emails in the Magento backend.
If you have modified the email template, please make sure that the one selected in the configuration matches the one you wish to use.
Here you can change the sender of your connection check notifications.
The number entered here is the number of products which is processed by the indexer (see above) at the same time. That’s also how many product data sets are transferred to the Solr server in a single request. The performance of the indexing process depends heavily on this setting. You can reduce this value if you are getting errors during indexing.
You should only deactivate this setting if you recreate the index completely (i.e. during each night) but can’t use a Swap core. If this setting is active the Solr index will be emptied completely at the start of every reindexing process before rebuilding it.
If you rebuild the Solr index regularly (i.e. nightly) we recommend to use the functionality to swap cores. You need a second core for that. In this case, you should activate this setting and enter the name of the second core into the field Name of Core to swap active Core with below.
If this setting is deactivated, only exact search matches will be registered. No spelling error correction will be performed. On the other hand, searches are conducted faster if this setting is deactivated.
Here you can enter how sensitive the fuzzy search should be. The value must be between 0 and 1, i.e. 0.75. The lower the value the more matches you will get, as spelling mistakes will be corrected more generously. You should test different values to get the optimal value for your shop. We recommend settings between 0.6 and 0.9.
Direct search results are automatically complemented by fuzzy search results if fuzzy search is activated.
You can limit this functionality by entering a number of sufficient direct search results. If at least this many direct search results are found, no fuzzy search is performed.
If you enter no value or 0, fuzzy search will always be performed.
Like above, but individually adjustable for the autosuggest box. It can be interesting to deactivate this function for the autosuggest only due to performance reasons.
Like above, but individually adjustable for the autosuggest functionality.
Just like for search requests you can limit the activity of fuzzy search for autosuggest, too.
If the sufficient number of direct search results for autosuggest is reached, a fuzzy search for autosuggest terms will not be performed.
In case the entered value is 0 or empty, fuzzy search will always be performed.
If this setting is activated, the HTML code which displays a single product in the search results will already be generated during indexing. Of course, this will take a bit longer, but on the other hand, the output will be faster in the search results. This is because this part won’t have to be generated on the fly for every product.
Thus, we recommend to activate this setting. There is an exception if the product data in the search results should be displayed customer dependent or customer group dependent, i.e. if the prices differ for the different customer groups.
Please deactivate this setting in that case.
You can choose between AND and OR. The search operator is used if there is more than one search word in the request, i.e. “red shirt”. When using AND, only search results will be displayed which match both (or all) search words.
When using OR, results which match only one of the search words will be displayed.
In most cases, AND is the better setting.
Filters can be displayed either in the left column next to the list of products or above the products. The latter is recommended if you have a rather narrow template.
If there are many filter options for a filter, for the sake of clarity you can limit the number of displayed filter values. Entering the value “0” means that all filter values will be displayed.
Usually filter options are sorted by number of results. In some cases it makes sense to sort them alphabetically instead. To activate alphabetical sorting, set the value to “Yes”.
Configure with which priority category names are handled in the Solr index. For example, if the search term “black shirts” should primarily return those products as search results which are contained in a category named “shirts”, you might want to enter a higher value than 1.
The default value is 1. If you enter a higher value, category names have a higher priority in the Solr index.
This setting is used by the price filter. You can set the steps which are used for the single intervals. I.e. 10 leads to the intervals 0.00-10.00, 10.00-20.00, 20.00-30.00 and so on.
This setting is used for the price filter as well. This value defines the topmost interval. If set to 200, this would be from 200.00. All products which cost more than 200.00 will be combined in this interval.
If the entered search term is an exact match with an important attribute of a product, you can here activate a direct redirect to the matching product page. As a result, the way to the product is shortened, because you skip the step of showing the search results page.
It is recommended to only use this redirect for attributes which have unique values for each product.
Just like a redirect to a product page, you can also activate redirects for search terms which exactly match a category’s attribute. Please make sure to only use this feature for attributes which allow for unambiguous matching with a category page.
If you don’t want to have a linear arrangement of intervals and you are using Solr 4.10 or above, you can set the desired interval borders for the price filter individually here. In the example 10,20,50,100,200,300,400,500 this would be the intervals 0.00-10.00, 10.00-20.00, 20.00-50.00 and so on until 400.00-500.00 and from 500.00.
If you activate this setting, Solr will be used to displayed products on category pages. Especially for stores with a huge amount of products or filterable attributes for layered navigation, this will speed up the load time of category pages.
Independent of the filters’ position on search result pages, you can choose where to display filters on category pages: either in the left column next to the products or above the products. This is a default value which can be overwritten by the category’s configuration.
When you activate this setting, categories that match the search term will be displayed in the autosuggest box. To finetune suggested categories, you can exclude single categories from being indexed.
When activated, matching CMS pages are displayed in the autosuggest box. It works similar to indexing categories. To finetune suggested CMS pages, you can exclude single CMS pages from being indexed.
If you deactivate this setting, no autosuggest window will be displayed.
This settings is described in the chapter Technical Workflow in detail.
Depending on your products, the given search word(s) will be expanded to meaningful variants. For example: if the text “re” is entered, the following suggestions will appear: regular…, resistant…, refined…, red….
The number of products which will be displayed in the autosuggest window.
The number of categories which will be displayed in the autosuggest window. If “Use Solr to index category pages” is activated, too, the displayed categories are those whose name and description match the search term. If not, only those categories are displayed which contain products matching the search term.
The number of CMS pages which will be displayed in the autosuggest window. This feature only works, if “Use Solr to index cms pages” is set to “Yes”.
If this setting is active, not only the category names will be displayed, but their parent categories as a path as well.
For example, this will be “Electronics > Cameras > Accessories” instead of “Accessories”.
The link which is behind the displayed categories. It can be:
– Search results page while the category filter is set to show only products of the selected category
– Category page
You can enter an arbitrary number of attributes here which will be displayed in the autosuggest window, including the options which are contained in most of the corresponding products. For every row you can select the attribute and the number of displayed options. Additionally you can define the sorting of the attributes – the attribute with the lowest value in the “Sorting” field will be shown first.
Only attributes with the property “Use In Search Results Layered Navigation” will be selectable.
Here you are able to select which of the pages processed by IntegerNet_Solr shall be hidden from bots and search engines. As a results, these pages’ meta element robots has the value “NOINDEX,NOFOLLOW”. Please note that this configuration may have a great impact on your store’s ranking in search engine results.
Even with the default settings of this module, the search results will be put in a sequence which depends on the frequency of the occurrences of the search words in the product attributes. This already leads to good results – much better than with the default search of Magento.
Still, there are some possibilities to adjust the sequence of search results:
If search words occur in the name or the SKU of a product, it should be valued higher than an occurrence in the product description. By default some attributes are already valued higher than others.
The prioritization follows the value “Solr Priority” which you can set for each product attribute. This new property can be seen in the attribute grid (*Catalog -> Attributes -> Manage Attributes*):
In this case, the grid is sorted by the value “Solr Priority”. You can set this value in the attribute properties:
The calculated priority of the search word(s) for the product will be multiplied with this value if the search word occurs in the attribute. Thus, 1.0 is the default – no multiplication happens here. You can increase or decrease the value of individual attributes. We recommend values between 0.5 and 10 (maximum).
Please note that you have to rebuild the Solr index after adjusting the Solr priority of one or more attributes.
From time to time, products should be emphasized, either because they are top sellers or because they should be sold out.
This module makes it possible to increase or decrease the priority of individual products.
Therefore, the new product attribute “Solr priority” is used. You can see it in the tab “Solr” on the product view page in the Magento backend.
With that, you have the possibility to position a product, as far as it matches the search word(s), further up or down, relative to its default position. We recommend using values between 0.5 and 10 (maximum). The mechanism is the same as with the boosting of attributes. Though, a reindexing isn’t necessary after you have adjusted this value for one or more product(s), as far as automatic index updates are activated.
If you would prefer to exclude certain CMS Pages from search results, the settings to do so can be found in the corresponding CMS page in the tab named “Solr”.
Set “Exclude this Page from Solr Index” to “Yes” to exclude the page from search results.
Use the field “Solr Priority” to weight this page more heavily in the search results. The higher the entered number, the higher the boost factor for this CMS page.
If need be you can exclude categories from Solr search results. The necessary settings can be found in your Magento backend in the corresponding category in the tab named “Solr”. When the value is set to “Yes”, this category will no longer be displayed in the autosuggest window.
Next to excluding a category, you can also opt to exlude all child categories. The excluded categories will no longer be shown in the search suggestions. However, all products connected to the excluded categories will still be shown as product suggestions and as search results.
Even if you don’t use IntegerNet_Solr to load product lists on category pages, you can use the extension’s feature to remove unnecessary filters from a category page. For example, you can thus prevent the filter “Gender” from being shown on a category page for male clothing.
For each category, you can change the position of filters, overwriting the default value from the IntegerNet_Solr configuration. Filters can be displayed either in the left column next to the product list or above the product list.
If you are using a non-standard template, probably some adjustments need to be made. The template of the autosuggest box and the results page is defined in
app/design/frontend/base/default/template/integernet/solr/ (PHTML files) and
skin/frontend/base/default/integernet/solr/ for the CSS file which is included into every page. Copy the files to your own theme directory (same directory and file name) and adjust them there.
Most probably you already have a template for the search results page. Usually it can be found at
template/catalog/product/list.phtml in your theme directory. To generate the according content for the PHTML files of the IntegerNet_Solr module, you have to split the content of your file into three parts.
- The parts inside
- The remainder goes to
template/integernet/solr/result.phtml. The previously cut out part must be replaced with the following code: ->getChild(‘item’)
grid depending on the part you are replacing.
You should switch off the configuration option
Search Results -> Use HTML from Solr Index while modifying the template files. If you have this option activated, you have to do a full reindex after activating / changing a list or grid template file.
You can copy and modify the
template/integernet/solr/autosuggest/item.phtml files to modify the appearance of the autosuggest window. Attention: as the generated HTML for each product is stored in the Solr index, you’ll have to reindex after you made changes to the
Pay attention: as the autosuggest functionality isn’t delivered by Magento but by a raw PHP version in order to improve performance, you cannot use all Magento functions in your
Try to stick to the functions which are used in
app/design/frontend/base/default/template/integernet/solr/result/autosuggest.phtml. As the HTML is generated by Magento instead, you can use all Magento functions in your
If you aren’t using product, category, attribute or keyword suggestions on your autosuggest page, please switch them off in configuration as well because this will improve the performance.
In order to further customize the module, we integrated several events which can be observed by an another module. The following events are included in IntegerNet_Solr:
For further information about these events, their parameters and usage, as well as an example module, please see our blog post.
- Rewrite conflicts with modules which affect the layered navigation
You can’t avoid this. But you can resolve the conflicts. You can see how we resolved such a conflict with one of those modules in the file
- Saving products in the backend takes a long time
This may happen if you have many store views. We recommend switching the indexing mode of the
integernet_solrindex to “Manually” and do a full reindex at night via cronjob if possible.
- Product information on the results page should be different for different customer groups, but it’s the same for all
Search Results -> Use HTML from Solr Indexin this case so the product HTML will be regenerated at every call.
Please note that this will affect the performance of the search result page.
- Product information on the autosuggest window should be different for different customer groups, but it’s the same for all
As the product HTML will always be stored in the Solr index, this is impossible. Try to modify the HTML in
template/integernet/solr/autosuggest/item.phtmlso it doesn’t contain customer specific information anymore (e.g. prices).
Something is odd here?
We love a good documentation so we do our best to keep this installation guide, user guide and developer manual of IntegerNet_Solr complete and up-to-date. If you think we missed something or if the information could be improved, let us know and we will fix it.