Proxy/Hybrid Integration

This information is for older versions of Hawksearch and not the latest v4.0. This is not compatible with the Search Api or Indexing Api. Please contact Hawksearch for more information

Table of Contents

1 Overview
2 Domains
3 Search Results
4 Landing Pages
5 Hawksearch Example API Request Formats
6 Hawksearch API Response
7 Integration Code & Behavior
8 JavaScript Block
9 HTML Elements
10 Understanding and Troubleshooting the Proxy Page
11 Autocomplete for Search Results
12 SEO Best Practices for On-Site Search Results Page
13 Http/Keep-Alive
14 Tracking Code

Overview

The Hawksearch service enables online retailers and publishers the ability to drive a rich, compelling user experience. This experience drives visitors to the products and information that they are seeking.

Hawksearch provides the ability to power the product listing pages for categories and brand pages on the site in addition to driving the search page on the site. This document will cover integration between an existing site and Hawksearch to drive: Search Results and Landing Pages. The focus will be the Proxy/Hybrid UI integration approach.

The Hawksearch Recommendation service requires tracking events from a user on your website. This document also describes the steps to install client side tracking on your website to send relevant data to the Recommendation engine.

Domains

Hawksearch has three environments available: Development, Staging/Test and a load-balanced Production. When performing integration, each engine can be accessed by using these domains:

Development Environment
When including the scripts below, please use the appropriate domains for the development environment.
Hawk URL: http://dev.hawksearch.net/
Tracking URL: http://tracking-dev.hawksearch.net/
Recommendations URL: http://recs-dev.hawksearch.net/
Staging/Test Environment
When including the scripts below, please use the appropriate domains for the test environment.
Engine Reference URL: http://test.hawksearch.net/
Tracking URL: http://tracking-test.hawksearch.net/
Recommendations URL: http://recs-test.hawksearch.net/
Production Environment
When including the scripts below, please use the following domain for the production/live environment.
Engine Reference URL: http://yourenginename.hawksearch.com/ Provided at time of launch.
Tracking URL: http://tracking-na.hawksearch.com/
Recommendations URL: http://recs-na.hawksearch.com/

Search Results

Search Results display after a visitor performs a keyword search on a site. Relevant products, facets, paging controls, and other objects are returned to guide a visitor to a product or article. Once the Hawksearch engine is available, Search Results can be received through an API call.

API Request for Search Results

http://dev.hawksearch.net/sites/demo/?keyword=jackets&hawkoutput=custom&hawkitemlist=json

Query Parameters

keyword: This value for this parameter is a search term used by the visitor.

hawkoutput: The value is set to “custom” for receiving data in the hybrid format (some components as HTML and

results as json)

hawkitemlist: The value for this is set to json indicating that the results set will be returned as json.

NOTE: Other components of the response can be returned back as json as well, instead of HTML. If you’re interested in this, please inquire with your project team.

Landing Pages

Landing Pages showcase unique product listings on a site. This feature within Hawksearch is typically used to drive the following pages:

Category Pages
Brand Pages
Custom product listing pages
Content Landing Pages

The same Hawksearch engine that drives Search Results also drives these Landing Pages. The big difference is that a merchandiser will need to create Landing Pages in the Hawksearch Workbench.

API Request for Landing Pages

http://dev.hawksearch.net/sites/demo/?lpurl=/brands/columbia/&hawkoutput=custom&hawkitemlist=json

Query Parameters

lpurl: This value is the relative path of the visitors URL, which corresponds to a Landing Page URL as specified in the Hawksearch Dashboard.

hawkoutput: The value is set to “custom” for receiving data in the hybrid format (some components as HTML and results as json)

hawkitemlist: The value for this is set to json indicating that the results set will be returned as json.

NOTE: Other components of the response can be returned back as json as well, instead of HTML. If you’re interested in this, please inquire with your project team.

Creating A Landing Page

For the landing pages to work correctly, a Landing Page needs to be created in Hawksearch. There are two methods to create a landing page. There is no limit on the number of Landing Pages created. Create Landing Pages by brand, category, sale, featured items, and any other custom data that is available in the Hawksearch engine. Landing Pages allow you to provide a consistent user experience, similar to Search Results.

1. Creation through the Hawksearch Workbench
The Hawksearch Workbench provides the ability to create Landing Pages. After logging in, click Workbench tab on the top navigation. On the left navigation, click the link to Landing Pages management section.

https://dev.hawksearch.net/settings/pages/

2. Creation through the REST API

Hawksearch provides a REST API available for creating landing pages:

https://dev.hawksearch.net/api/v9

The API allows CRUD operations for Landing Pages. For additional details for the Landing Page API, please reference the Hawksearch API document.

When to Request Landing Pages

Every site is different and how it requests a landing page needs to be determined.

For example, a visitor browses to a site’s page showcasing the brand Columbia (e.g. http://demo.hawksearch.com/brands/columbia/). The website needs to know if that particular URL should be powered by Hawksearch. If a Hawksearch Landing Page exists, the site will make an API request behind the scenes.

To handle Landing Pages, these steps are performed.

Determine when to query the Hawksearch engine for a Landing Page
- Store a flag on specific pages in the existing CMS that forces a query to the Hawksearch Engine
- Use the API to get a list of existing Hawksearch Landing Pages and cache it. If the URL exists in the cached list, send a query to the Hawksearch Engine.
Query Hawksearch for the specific Landing Page
- Use the API format mentioned above

Hawksearch Example API Request Formats

Below are the sample requests of how you should be requesting data from Hawksearch and the various output modes you can use to request data in a specific format.

REQUEST FOR SEARCH

Results as JSON, Top Pager and all other objects as HTML

http://dev.hawksearch.net/sites/demo/?keyword=jackets&hawkoutput=custom&hawkitemlist=json%20

Results as JSON, Top Pager as JSON and all other objects as HTML

http://dev.hawksearch.net/sites/demo/?keyword=jackets&hawkoutput=custom&hawkitemlist=json&hawktoppager=json%20

REQUEST FOR LANDING PAGE

Results as JSON, Top Pager and all other objects as HTML

http://dev.hawksearch.net/sites/demo/?lpurl=/accessory%20&hawkoutput=custom&hawkitemlist=json%20

Results as JSON, Top Pager as JSON and all other objects as HTML

http://dev.hawksearch.net/sites/demo/?lpurl=/accessory%20&hawkoutput=custom&hawkitemlist=json&hawktoppager=json%20

Hawksearch API Response

When the Hawksearch Engine is queried, the following objects are returned as part of the API Response.

Success: This indicates if the API Request was successful.

Data: This object will contain the components related to the API Response. The data object will contain the following objects:

Title: This is the title of the page.
TopText: This object will contain the HTML text above the search results (did you mean, expand results).
BreadCrumb: This object contains the HTML to display the breadcrumb with selected facets and/or keywords.
TopPager: This object contains the HTML for the pager above the search results.
BottomPager: This object contains the HTML for the pager below the search results.

Results: This object contains the results in JSON, HTML, or XML format depending on how the data is being requested.

JSON Example

"Results": [
      {
         "Score": 0.005307952,
         "ItemName": "Logo Hat - The North Face",
         "ImageURL": "/assets/1/14/DimThumbnail/{DE7A166C-24F5-453B-B970-A30BD746C21D}.jpg",
         "ImageAlt": "10187",
         "Id": "Item_74303",
         "CustomURL": "/products/logo-hat-the-north-face/",
         "TrackingId": "1213044",
         "Brand": "The North Face",
         "Price": "18",
         "SalePrice": "18",
         "IsOnSale": false,
         "BestFragment": "... - the <b>north<\/b> <b>face<\/b> the logo hat has the <b>north<\/b> <b>face<\/b> logo and &#39;1968&#39; embroidered on this washed cotton twill hat.&#60;br&#62; &#60;br&#62;   men accessories hats & headwear specials sat dec 15th kids women accessories hats & headwear women hats & headwear caps men hats & headwear caps the <b>north<\/b> ...",
         "MinPrice": "18",
         "MaxPrice": "18",
         "MinSalePrice": "0",
         "MaxSalePrice": "0"
      },
      {
         "Score": 7.017364E-4,
         "ItemName": "Baby Llama Beanie",
         "ImageURL": "/assets/1/14/DimThumbnail/The-North-Face-Baby-Llama-Beanie-jake-blue.jpg",
         "ImageAlt": "Llama",
         "Id": "Item_118551",
         "CustomURL": "/products/baby-llama-beanie-/",
         "TrackingId": "1213044",
         "Brand": "The North Face",
         "Price": "25",
         "SalePrice": "11.97",
         "IsOnSale": true,
         "BestFragment": "... design, the <b>north<\/b> <b>face<\/b> baby llama beanie is about the cutest thing you can pull on your baby&#39;s head. made from lightweight acrylic fibers, this <b>north<\/b> <b>face<\/b> hat has a fleece ear band for extra warmth and a soft feel against the skin.&#60;&#47;div&#62;&#60;div&#62;&#38;nbsp&#59;&#60;&#47;div&#62;&#60;&#47;div&#62;&#60;&#47;div&#62; ...",
         "MinPrice": "25",
         "MaxPrice": "25",
         "MinSalePrice": "11.97",
         "MaxSalePrice": "11.97"
      }
   ]

Merchandising: This object contains the HTML to display merchandising and banner rules set up in the Dashboard. Selections: This object contains HTML for the selected Facets used for filtering.

Facets: This object contains HTML for the available Facets used for filtering.

FeaturedItems: If the Featured Items Banner has been set up for this query, this object contains HTML for this merchandising feature.

Location: This object contains the value of a redirect URL if set up through the Workbench. If exists, send the visitor to the URL and ignore the rest of the API response.

DidYouMean: This object contains the value for the spell check override specified in the Workbench for the appropriate keyword. This value will only contain the recommended keyword.

TrackingId: This object contains the Tracking ID that is passed around for links on the Search Results page. Please refer to the Click Tracking section for more details.

MetaDescription: This object contains the SEO Field, Meta Description, for the requested Landing Page. MetaKeywords: This object contains the SEO Field, Meta Keywords, for the requested Landing Page. MetaRobots: This object contains the SEO Field, Meta Robots.

HeaderTitle: This object contains the SEO Field, Header Title.

RelCanonical: This object contains the Canonical links.

Integration Code & Behavior

Once the Landing Page is set up, create a server-side request to Hawksearch using the sample request formats shown above.

Steps to Integrate and Code For the Proxy:

Create The Hawksearch Helper Class
This class is responsible for sending the request to the Hawksearch engine with appropriate parameters. It also contains the definition and format for the API Response objects.
Create The Search Page / Module
This step calls the Hawksearch Helper Class and then parses the generated JSON response into the appropriate DIV elements on the page. The DIV elements have the specific IDs as listed below and in the code sample. The IDs need to be consistent as they are utilized for AJAX requests. If the results are parsed and rendered by other means, please include the reference to the appropriate module on this page and make sure it is included inside the appropriate DIV. For example if you are overriding and generating the results, please make sure it is put inside the hawkitemlist div.
Create The Proxy Page
This is an exact copy of the Search Results page. The only difference is that the search results page writes out the response as HTML that is rendered on the page and the proxy page generates the HTML and then converts it to JSON with the HTML object containing the full HTML. Since the proxy page is triggered from the JavaScript the proxy page needs to write the response as JSON that the JavaScript callback can process.
Add Parse and Generate HTML Logic
For the results and pager components, add server side code to render the results.
Embed the Hawksearch JavaScript Block on the page

Hawksearch Helper Class

The sample class below acts as a helper for Hawksearch. The code blocks show how to create a request with either a keyword or landing page URL as a parameter. Depending on how the request is generated, you will get the appropriate components.

public class HawkSearchHelper
{

    public static HawkSearchResult LoadHawkSearchDoc()
    {
        //Keep document in the context to avoid duplicate queries

        HawkSearchResult res = null;
        string url = string.Empty;
        url = "http://test.hawksearch.net/sites/demo/?";

        //If this is a search request
        if (SearchRequest) {
            url += "keyword=" + keyword;
        }

        //If this is a landing page reequest
        if (CategoryPageRequest) {
            url += "lpurl=" + categorypageurl;
        }


// Retrieve document from Hawksearch. Please pass in the true client // IP and user agent
        HttpWebRequest r = WebRequest.Create(url);
        if (Request != null) {
            r.UserAgent = Request.UserAgent;
            r.Headers.Add("HTTP_TRUE_CLIENT_IP",Request.UserHostAddress);
        }


        try {
            System.Net.HttpWebResponse resp = r.GetResponse();
            using (System.IO.StreamReader sr = new System.IO.StreamReader(resp.GetResponseStream())) {
                string html = sr.ReadToEnd();
                res = JsonConvert.DeserializeObject<HawkSearchResult>(html);
            }

            HttpContext.Current.Items("hawksearchdoc") = res;
        } catch (Exception ex) {
        }


        return res;
    }
}

Search Page/Listing Page

The Search Page/listing page handles the initial server load. It uses the Helper Class above and processes the response returned from the engine. Once the response has been processed, the HTML is rendered on to the page server side.

Proxy Page

The Proxy Page acts as an intermediate page between Hawksearch and the site for any load following the initial server load (which is handled by the search page/listing page). The Proxy Page uses the Helper Class above and processes the response returned from the engine. The behavior is exactly similar to the search page, the only difference being once the final HTML is generated the HTML is converted to JSON response with the HTML property. This will help the callback function process the response client side and refresh the appropriate elements on the page.

Private void LoadInitialResults() {
        HawkSearchResult doc = HawkSearchHelper.LoadHawkSearch(DB, GlobalSiteId, SitePage.SearchFilter, Request);
        if ((doc == null)) {
            return;
        }
        // Handle redirect
        if (!(doc.Location == null)) {
            Response.Redirect(doc.Location);
        }
        // Handle items and other html components such as pager below in places where 
        
    }
protected override void Render(System.Web.UI.HtmlTextWriter writer)
{
	string RenderedHTML = string.Empty;

	using (System.IO.StringWriter sw = new System.IO.StringWriter()) {
		using (HtmlTextWriter tw = new HtmlTextWriter(sw)) {
			base.Render(tw);
			RenderedHTML = sw.ToString();
		}
	}

	string jsoncallback = HttpContext.Current.Request("callback");
	Response.ContentType = "application/json";
	Response.Write(jsoncallback + "(" + JsonConvert.SerializeObject(new {
		Success = true,
		html = RenderedHTML,
		location = ""
	}) + ")");
}

True Client IP and User Agent

With the Hybrid Proxy approach, a server side call is made to the Hawksearch engine and because of this all header data comes from the server. Therefore, in order for Hawksearch to be user specific, please pass the user IP and the user-agent of the user when making the request to Hawksearch. This will prevent from skewing reporting data with internal searches and bot searches.

C# Code Sample

JavaScript Block

The following JavaScript needs to be installed on every page of your site. The block should look as follows. Please either adjust or add as needed. The enginename is the engine ID associated with your Hawksearch account. The ClientGuid is the Tracking Key associated with your account. To lookup the Engine Name and Tracking Key for your account please look in the Hawksearch workbench under Admin >> Account Info.

HTML Elements

In order for this approach to work you will need to make sure your markup has the following div elements in place. This way when the Hawksearch JavaScript processes the response from the JSON object it knows the div elements to refresh. Please place the divs in the corresponding locations on your page.

Understanding and Troubleshooting the Proxy Page

Once the JavaScript Block is on the site, to test if the proxy was installed correctly, please use the firebug plug-in in Firefox. This plug-in is also great for troubleshooting issues with AJAX Behavior in Hawksearch. Developer Tools on Chrome works just as well.

Check that:

There are no errors on the console in firebug.
View the source of the page to make sure the script is registered correctly and the base URL points correctly to the proxy page on the site.

If everything is installed correctly, selecting a facet should produce a request using AJAX to your Proxy Page. View the request in the “Net” tab in the firebug window. If the request does not fire, check the console tab for any JavaScript errors.

Autocomplete for Search Results

Within the Hawksearch Workbench, settings can be modified to control what shows in the Autocomplete section.

To enable it on your page, there are two steps:

Add the Autocomplete JavaScript block to the end of the <BODY> tag.

textboxname is the ID of the search text box. Look to your existing HTML code to add the matching ID.

textboxnamefooter is the ID of the search text box if you have one on the footer. Look to your existing HTML code to add the matching ID.

SEO Best Practices for On-Site Search Results Page

To improve SEO on the site, ensure the search page has been added to the site’s robots.txt and disallowed. Always have a no index / no follow on the search pages. For improving SEO functionality for popular search terms, contact us about the SEO Booster functionality we offer.

REQUEST FOR SEARCH

http://dev.hawksearch.net/sites/demo/?keyword=jackets&hawkoutput=json&hawkrobots=allow

REQUEST FOR LANDING PAGE

http://dev.hawksearch.net/sites/demo/?lpurl=/accessory&hawkoutput=json&hawkrobots=allow

Http/Keep-Alive

In the event that you see unexpected timeout errors from the site when testing, please confirm your http web request settings for the request you make to Hawksearch. In some cases the clients reset the connection instead of leaving it in Time_Wait/Close_Wait causing IIS to log it as an interruption. In this scenario please force the request to be Http 1.0 with no keep alive and see if timeouts no longer appear.

Tracking Code

Click here for instructions on adding Hawksearch Event Tracking Code .