Table of Contents
Overview
The Hawksearch service enables online retailers and publishers the ability to drive a rich, compelling user experience. This experience drives visitors to the products and information that they are seeking.
Hawksearch provides the ability to power the product listing pages for categories and brand pages on the site in addition to driving the search page on the site. This document will cover integration between an existing site and Hawksearch to drive: Search Results and Landing Pages. The focus will be the Proxy/Hybrid UI integration approach.
The Hawksearch Recommendation service requires tracking events from a user on your website. This document also describes the steps to install client side tracking on your website to send relevant data to the Recommendation engine.
Domains
Hawksearch has three environments available: Development, Staging/Test and a load-balanced Production. When performing integration, each engine can be accessed by using these domains:
Development Environment
When including the scripts below, please use the appropriate domains for the development environment.
Hawk URL: http://dev.hawksearch.net/
Tracking URL: http://tracking-dev.hawksearch.net/
Recommendations URL: http://recs-dev.hawksearch.net/Staging/Test Environment
When including the scripts below, please use the appropriate domains for the test environment.
Engine Reference URL: http://test.hawksearch.net/
Tracking URL: http://tracking-test.hawksearch.net/
Recommendations URL: http://recs-test.hawksearch.net/Production Environment
When including the scripts below, please use the following domain for the production/live environment.
Engine Reference URL: http://yourenginename.hawksearch.com/ Provided at time of launch.
Tracking URL: http://tracking-na.hawksearch.com/
Recommendations URL: http://recs-na.hawksearch.com/
Search Results
Search Results display after a visitor performs a keyword search on a site. Relevant products, facets, paging controls, and other objects are returned to guide a visitor to a product or article. Once the Hawksearch engine is available, Search Results can be received through an API call.
API Request for Search Results
http://dev.hawksearch.net/sites/demo/?keyword=jackets&hawkoutput=custom&hawkitemlist=json
Query Parameters
keyword: This value for this parameter is a search term used by the visitor.
hawkoutput: The value is set to “custom” for receiving data in the hybrid format (some components as HTML and
results as json)
hawkitemlist: The value for this is set to json indicating that the results set will be returned as json.
NOTE: Other components of the response can be returned back as json as well, instead of HTML. If you’re interested in this, please inquire with your project team.
Landing Pages
Landing Pages showcase unique product listings on a site. This feature within Hawksearch is typically used to drive the following pages:
Category Pages
Brand Pages
Custom product listing pages
Content Landing Pages
The same Hawksearch engine that drives Search Results also drives these Landing Pages. The big difference is that a merchandiser will need to create Landing Pages in the Hawksearch Workbench.
API Request for Landing Pages
http://dev.hawksearch.net/sites/demo/?lpurl=/brands/columbia/&hawkoutput=custom&hawkitemlist=json
Query Parameters
lpurl: This value is the relative path of the visitors URL, which corresponds to a Landing Page URL as specified in the Hawksearch Dashboard.
hawkoutput: The value is set to “custom” for receiving data in the hybrid format (some components as HTML and results as json)
hawkitemlist: The value for this is set to json indicating that the results set will be returned as json.
NOTE: Other components of the response can be returned back as json as well, instead of HTML. If you’re interested in this, please inquire with your project team.
Creating A Landing Page
For the landing pages to work correctly, a Landing Page needs to be created in Hawksearch. There are two methods to create a landing page. There is no limit on the number of Landing Pages created. Create Landing Pages by brand, category, sale, featured items, and any other custom data that is available in the Hawksearch engine. Landing Pages allow you to provide a consistent user experience, similar to Search Results.
1. Creation through the Hawksearch Workbench
The Hawksearch Workbench provides the ability to create Landing Pages. After logging in, click Workbench tab on the top navigation. On the left navigation, click the link to Landing Pages management section.
https://dev.hawksearch.net/settings/pages/
2. Creation through the REST API
Hawksearch provides a REST API available for creating landing pages:
https://dev.hawksearch.net/api/v9
The API allows CRUD operations for Landing Pages. For additional details for the Landing Page API, please reference the Hawksearch API document.
NOTE: If you are not familiar or have not seen the Hawksearch API document please contact your Hawksearch Project Team to get access to the document.
When to Request Landing Pages
Every site is different and how it requests a landing page needs to be determined.
For example, a visitor browses to a site’s page showcasing the brand Columbia (e.g. http://demo.hawksearch.com/brands/columbia/). The website needs to know if that particular URL should be powered by Hawksearch. If a Hawksearch Landing Page exists, the site will make an API request behind the scenes.
To handle Landing Pages, these steps are performed.
Determine when to query the Hawksearch engine for a Landing Page
Store a flag on specific pages in the existing CMS that forces a query to the Hawksearch Engine
Use the API to get a list of existing Hawksearch Landing Pages and cache it. If the URL exists in the cached list, send a query to the Hawksearch Engine.
Query Hawksearch for the specific Landing Page
Use the API format mentioned above
Hawksearch Example API Request Formats
Below are the sample requests of how you should be requesting data from Hawksearch and the various output modes you can use to request data in a specific format.
REQUEST FOR SEARCH
Results as JSON, Top Pager and all other objects as HTML
http://dev.hawksearch.net/sites/demo/?keyword=jackets&hawkoutput=custom&hawkitemlist=json%20
Results as JSON, Top Pager as JSON and all other objects as HTML
REQUEST FOR LANDING PAGE
Results as JSON, Top Pager and all other objects as HTML
http://dev.hawksearch.net/sites/demo/?lpurl=/accessory%20&hawkoutput=custom&hawkitemlist=json%20
Results as JSON, Top Pager as JSON and all other objects as HTML
Hawksearch API Response
When the Hawksearch Engine is queried, the following objects are returned as part of the API Response.
Success: This indicates if the API Request was successful.
Data: This object will contain the components related to the API Response. The data object will contain the following objects:
Title: This is the title of the page.
TopText: This object will contain the HTML text above the search results (did you mean, expand results).
BreadCrumb: This object contains the HTML to display the breadcrumb with selected facets and/or keywords.
TopPager: This object contains the HTML for the pager above the search results.
BottomPager: This object contains the HTML for the pager below the search results.
Results: This object contains the results in JSON, HTML, or XML format depending on how the data is being requested.
JSON Example
"Results": [
{
"Score": 0.005307952,
"ItemName": "Logo Hat - The North Face",
"ImageURL": "/assets/1/14/DimThumbnail/{DE7A166C-24F5-453B-B970-A30BD746C21D}.jpg",
"ImageAlt": "10187",
"Id": "Item_74303",
"CustomURL": "/products/logo-hat-the-north-face/",
"TrackingId": "1213044",
"Brand": "The North Face",
"Price": "18",
"SalePrice": "18",
"IsOnSale": false,
"BestFragment": "... - the <b>north<\/b> <b>face<\/b> the logo hat has the <b>north<\/b> <b>face<\/b> logo and '1968' embroidered on this washed cotton twill hat.<br> <br> men accessories hats & headwear specials sat dec 15th kids women accessories hats & headwear women hats & headwear caps men hats & headwear caps the <b>north<\/b> ...",
"MinPrice": "18",
"MaxPrice": "18",
"MinSalePrice": "0",
"MaxSalePrice": "0"
},
{
"Score": 7.017364E-4,
"ItemName": "Baby Llama Beanie",
"ImageURL": "/assets/1/14/DimThumbnail/The-North-Face-Baby-Llama-Beanie-jake-blue.jpg",
"ImageAlt": "Llama",
"Id": "Item_118551",
"CustomURL": "/products/baby-llama-beanie-/",
"TrackingId": "1213044",
"Brand": "The North Face",
"Price": "25",
"SalePrice": "11.97",
"IsOnSale": true,
"BestFragment": "... design, the <b>north<\/b> <b>face<\/b> baby llama beanie is about the cutest thing you can pull on your baby's head. made from lightweight acrylic fibers, this <b>north<\/b> <b>face<\/b> hat has a fleece ear band for extra warmth and a soft feel against the skin.</div><div>&nbsp;</div></div></div> ...",
"MinPrice": "25",
"MaxPrice": "25",
"MinSalePrice": "11.97",
"MaxSalePrice": "11.97"
}
]
Merchandising: This object contains the HTML to display merchandising and banner rules set up in the Dashboard. Selections: This object contains HTML for the selected Facets used for filtering.
Facets: This object contains HTML for the available Facets used for filtering.
FeaturedItems: If the Featured Items Banner has been set up for this query, this object contains HTML for this merchandising feature.
Location: This object contains the value of a redirect URL if set up through the Workbench. If exists, send the visitor to the URL and ignore the rest of the API response.
DidYouMean: This object contains the value for the spell check override specified in the Workbench for the appropriate keyword. This value will only contain the recommended keyword.
TrackingId: This object contains the Tracking ID that is passed around for links on the Search Results page. Please refer to the Click Tracking section for more details.
MetaDescription: This object contains the SEO Field, Meta Description, for the requested Landing Page. MetaKeywords: This object contains the SEO Field, Meta Keywords, for the requested Landing Page. MetaRobots: This object contains the SEO Field, Meta Robots.
HeaderTitle: This object contains the SEO Field, Header Title.
RelCanonical: This object contains the Canonical links.
For Additional Details regarding the objects and the hawkoutput format, please refer to the API - front end integration .
Integration Code & Behavior
Once the Landing Page is set up, create a server-side request to Hawksearch using the sample request formats shown above.
For additional details regarding the code structure, please reference the VB.NET code sample attached with this documentation. The code will contain details regarding the structure of the pages and the modules.
Steps to Integrate and Code For the Proxy:
Create The Hawksearch Helper Class
This class is responsible for sending the request to the Hawksearch engine with appropriate parameters. It also contains the definition and format for the API Response objects.Create The Search Page / Module
This step calls the Hawksearch Helper Class and then parses the generated JSON response into the appropriate DIV elements on the page. The DIV elements have the specific IDs as listed below and in the code sample. The IDs need to be consistent as they are utilized for AJAX requests. If the results are parsed and rendered by other means, please include the reference to the appropriate module on this page and make sure it is included inside the appropriate DIV. For example if you are overriding and generating the results, please make sure it is put inside the hawkitemlist div.Create The Proxy Page
This is an exact copy of the Search Results page. The only difference is that the search results page writes out the response as HTML that is rendered on the page and the proxy page generates the HTML and then converts it to JSON with the HTML object containing the full HTML. Since the proxy page is triggered from the JavaScript the proxy page needs to write the response as JSON that the JavaScript callback can process.Add Parse and Generate HTML Logic
For the results and pager components, add server side code to render the results.Embed the Hawksearch JavaScript Block on the page
Hawksearch Helper Class
The sample class below acts as a helper for Hawksearch. The code blocks show how to create a request with either a keyword or landing page URL as a parameter. Depending on how the request is generated, you will get the appropriate components.
public class HawkSearchHelper
{
public static HawkSearchResult LoadHawkSearchDoc()
{
//Keep document in the context to avoid duplicate queries
HawkSearchResult res = null;
string url = string.Empty;
url = "http://test.hawksearch.net/sites/demo/?";
//If this is a search request
if (SearchRequest) {
url += "keyword=" + keyword;
}
//If this is a landing page reequest
if (CategoryPageRequest) {
url += "lpurl=" + categorypageurl;
}
// Retrieve document from Hawksearch. Please pass in the true client // IP and user agent
HttpWebRequest r = WebRequest.Create(url);
if (Request != null) {
r.UserAgent = Request.UserAgent;
r.Headers.Add("HTTP_TRUE_CLIENT_IP",Request.UserHostAddress);
}
try {
System.Net.HttpWebResponse resp = r.GetResponse();
using (System.IO.StreamReader sr = new System.IO.StreamReader(resp.GetResponseStream())) {
string html = sr.ReadToEnd();
res = JsonConvert.DeserializeObject<HawkSearchResult>(html);
}
HttpContext.Current.Items("hawksearchdoc") = res;
} catch (Exception ex) {
}
return res;
}
}
Search Page/Listing Page
The Search Page/listing page handles the initial server load. It uses the Helper Class above and processes the response returned from the engine. Once the response has been processed, the HTML is rendered on to the page server side.
Proxy Page
The Proxy Page acts as an intermediate page between Hawksearch and the site for any load following the initial server load (which is handled by the search page/listing page). The Proxy Page uses the Helper Class above and processes the response returned from the engine. The behavior is exactly similar to the search page, the only difference being once the final HTML is generated the HTML is converted to JSON response with the HTML property. This will help the callback function process the response client side and refresh the appropriate elements on the page.
Private void LoadInitialResults() {
HawkSearchResult doc = HawkSearchHelper.LoadHawkSearch(DB, GlobalSiteId, SitePage.SearchFilter, Request);
if ((doc == null)) {
return;
}
// Handle redirect
if (!(doc.Location == null)) {
Response.Redirect(doc.Location);
}
// Handle items and other html components such as pager below in places where
}
protected override void Render(System.Web.UI.HtmlTextWriter writer)
{
string RenderedHTML = string.Empty;
using (System.IO.StringWriter sw = new System.IO.StringWriter()) {
using (HtmlTextWriter tw = new HtmlTextWriter(sw)) {
base.Render(tw);
RenderedHTML = sw.ToString();
}
}
string jsoncallback = HttpContext.Current.Request("callback");
Response.ContentType = "application/json";
Response.Write(jsoncallback + "(" + JsonConvert.SerializeObject(new {
Success = true,
html = RenderedHTML,
location = ""
}) + ")");
}
True Client IP and User Agent
With the Hybrid Proxy approach, a server side call is made to the Hawksearch engine and because of this all header data comes from the server. Therefore, in order for Hawksearch to be user specific, please pass the user IP and the user-agent of the user when making the request to Hawksearch. This will prevent from skewing reporting data with internal searches and bot searches.
C# Code Sample
string url = string.Empty;
url = "http://test.hawksearch.net/sites/demo/?";
//If this is a search request
if (SearchRequest) {
url += "keyword=" + keyword;
}
//If this is a landing page request
if (CategoryPageRequest) {
url += "lpurl=" + categorypageurl;
}
// Retrieve data from Hawksearch. Please pass in the true client // IP and user agent
HttpWebRequest r = WebRequest.Create(url);
if (Request != null) {
r.UserAgent = Request.UserAgent;
r.Headers.Add("HTTP_TRUE_CLIENT_IP",Request.UserHostAddress);
}
JavaScript Block
The following JavaScript needs to be installed on every page of your site. The block should look as follows. Please either adjust or add as needed. The enginename is the engine ID associated with your Hawksearch account. The ClientGuid is the Tracking Key associated with your account. To lookup the Engine Name and Tracking Key for your account please look in the Hawksearch workbench under Admin >> Account Info.
<!-- Hawksearch Header Includes -->
<script type="text/javascript">
//<![CDATA[
(function (HawkSearch, undefined) {
HawkSearch.BaseUrl = '%%ProxyPageURL%%';
HawkSearch.HawkUrl = 'http://dev.hawksearch.net/sites/%%enginename%%';
HawkSearch.TrackingUrl = 'http://tracking-dev.hawksearch.net';
HawkSearch.RecommenderUrl = 'http://recs-dev.hawksearch.net';
HawkSearch.ClientGuid = '%%tracking_key%%';
if ("https:" == document.location.protocol) {
HawkSearch.BaseUrl = HawkSearch.BaseUrl.replace("http://", "https://");
HawkSearch.HawkUrl = HawkSearch.HawkUrl.replace("http://", "https://");
HawkSearch.TrackingUrl = HawkSearch.TrackingUrl.replace("http://", "https://");
HawkSearch.RecommenderUrl = HawkSearch.RecommenderUrl.replace("http://", "https://");
}
}(window.HawkSearch = window.HawkSearch || {}));
var hawkJSScriptDoc = document.createElement("script");
hawkJSScriptDoc.async = true;
hawkJSScriptDoc.src = HawkSearch.HawkUrl + '/includes/hawksearch.min.js?v1.0';
var hawkJSTag = document.getElementsByTagName('script')[0];
hawkJSTag.parentNode.insertBefore(hawkJSScriptDoc, hawkJSTag);
var hawkCSSScriptDoc = document.createElement("link");
hawkCSSScriptDoc.setAttribute("rel", "stylesheet");
hawkCSSScriptDoc.setAttribute("type", "text/css");
hawkCSSScriptDoc.setAttribute("href", HawkSearch.HawkUrl + '/includes/hawksearch.css');
document.head.appendChild(hawkCSSScriptDoc);
//]]>
</script>
HTML Elements
In order for this approach to work you will need to make sure your markup has the following div elements in place. This way when the Hawksearch JavaScript processes the response from the JSON object it knows the div elements to refresh. Please place the divs in the corresponding locations on your page.
NOTE: Please make sure the ids of the divs match exactly as per the specification shown below
<div id="hawkbannerlefttop"><!-- Area for Banner above filters on Top --> </div> <div id="hawkfacets"><!-- Area for Filters--></div> <div id="hawkbannerleftbottom"><!-- Area for Banner below filters --></div> <div id="hawktitle"><!-- Area for page title --></div> <div id="hawkbreadcrumb"><!-- Area for breadcrumb --></div> <div id="hawkbannertop"><!-- Area for Banner Top --></div> <div id="hawktoptext"><!-- Area for message above search results --></div> <div id="hawktoppager"><!-- Area for Top Pager --></div> <div id="hawkitemlist"><!-- Area for Result Item/Content list --></div> <div id="hawkbannerbottom"><!-- Area for Banner below search results--></div> <div id="hawkbannerbottom2"><!-- Area for Banner below search results--></div> <div id="hawkbottompager"><!-- Area for Bottom Pager --></div>
Understanding and Troubleshooting the Proxy Page
Once the JavaScript Block is on the site, to test if the proxy was installed correctly, please use the firebug plug-in in Firefox. This plug-in is also great for troubleshooting issues with AJAX Behavior in Hawksearch. Developer Tools on Chrome works just as well.
Check that:
There are no errors on the console in firebug.
View the source of the page to make sure the script is registered correctly and the base URL points correctly to the proxy page on the site.
If everything is installed correctly, selecting a facet should produce a request using AJAX to your Proxy Page. View the request in the “Net” tab in the firebug window. If the request does not fire, check the console tab for any JavaScript errors.
Auto-Complete for Search Results
Within the Hawksearch Workbench, settings can be modified to control what shows in the Auto-Complete section.
To enable it on your page, there are two steps:
Add the Auto-Complete JavaScript block to the end of the <BODY> tag.
textboxname is the ID of the search text box. Look to your existing HTML code to add the matching ID.
textboxnamefooter is the ID of the search text box if you have one on the footer. Look to your existing HTML code to add the matching ID.
<!-- Hawksearch - Auto-Suggest -->
<script type="text/javascript">
//<![CDATA[
HawkSearch.initAutoSuggest = function () {
HawkSearch.suggestInit('%%textboxname%%', {
lookupUrlPrefix: HawkSearch.HawkUrl + '/?fn=ajax&f=GetSuggestions',
hiddenDivName: '',
isAutoWidth: true
});
HawkSearch.suggestInit('%%textboxnamefooter%%', {
lookupUrlPrefix: HawkSearch.HawkUrl + '/?fn=ajax&f=GetSuggestions',
hiddenDivName: '',
isAbove: true
});
};
//]]>
</script>
SEO Best Practices for On-Site Search Results Page
To improve SEO on the site, ensure the search page has been added to the site’s robots.txt and disallowed. Always have a no index / no follow on the search pages. For improving SEO functionality for popular search terms, contact us about the SEO Booster functionality we offer.
NOTE: By default all major bots requests for SEARCH PAGES are blocked from being served by the Hawksearch servers. The idea is to use landing pages for any custom category/brand/custom landing page that needs to be indexed by search engines. However if you have custom requirement where you still need the search page to be served for web crawlers please include the parameter below with the corresponding request to Hawksearch.
REQUEST FOR SEARCH
http://dev.hawksearch.net/sites/demo/?keyword=jackets&hawkoutput=json&hawkrobots=allow
REQUEST FOR LANDING PAGE
http://dev.hawksearch.net/sites/demo/?lpurl=/accessory&hawkoutput=json&hawkrobots=allow
Http/Keep-Alive
In the event that you see unexpected timeout errors from the site when testing, please confirm your http web request settings for the request you make to Hawksearch. In some cases the clients reset the connection instead of leaving it in Time_Wait/Close_Wait causing IIS to log it as an interruption. In this scenario please force the request to be Http 1.0 with no keep alive and see if timeouts no longer appear.
Tracking Code
NOTE: In order to populate data in the Hawksearch Reports, and to leverage Personalized Search, it is CRITICAL to implement Tracking Integration.
Click here for instructions on adding Hawksearch Event Tracking Code .