****README.txt - Google Rank Extractor**** ################################################################ Version : Perl v1.6 Beta Released : November 24, 2009 Copyright : (c)2009 Craig Chant (1DMF) http://dance-music.org Coded In : X/HTML - CSS - AJAX - JAVASCRIPT - PERL - FLASH Licence : Released under GNU/GPL version 3, as shown below. ################################################################ Acknowledgements -> Google Rank Extractor : Original concept and PHP coding by Randy Cullom - http://www.randycullom.com -> gre_db_setup.sql : mySQL setup file (c)2009 Randy Cullom -> GRE_Template Module : Copyright (c)2000-2002 Sam Tregar (sam@tregar.com) -> FusionCharts : vFree.1.2 (c)2007 - http://www.fusioncharts.com ################################################################ --Requirements-- 1. A web site. 2. A server that supports both PERL and SQL databases. SSI is optional. --Files Included In This Distribution-- README.txt - This file with instructions and documentation. rank_extractor.pl - Perl file that collects the google data. gre_config.pl - User configuration settings. gre_admin.pl - Perl file that provides the admin interface with query tool. GRE_SQL.pm - Perl SQL Module utilising the DBI / WIN32:ODBC Modules for database connectivity. GRE_Template.pm - Perl HTML Template module. gre_start.tp - HTML template for admin user interface. gre_search.tp - HTML template for displaying query results. gre_css.tp - CSS template for admin tool and results display. gre_ajax.js - Javascript AJAX file to extract and save google data (SSI option available). gre_db_setup.sql - The SQL instructions to set up your database. index.html - Admin user interface. FusionCharts.js - Javascript file which powers the FusionCharts flash graphs engine. FCF_Area2D.swf - FusionCharts flash file. FCF_Bar2D.swf - FusionCharts flash file. FCF_Column2D.swf - FusionCharts flash file.. FCF_Column3D.swf - FusionCharts flash file. FCF_Doughnut2D.swf - FusionCharts flash file. FCF_Line.swf - FusionCharts flash file. FCF_Pie2D.swf - FusionCharts flash file. FCF_Pie3D.swf - FusionCharts flash file. FCF_MSBar2D.swf - FusionCharts flash file. FCF_MSColumn2D.swf - FusionCharts flash file. FCF_MSColumn3D.swf - FusionCharts flash file. FCF_StackedBar2D.swf - FusionCharts flash file. FCF_StackedColumn2D.swf - FusionCharts flash file. FCF_StackedColumn3D.swf - FusionCharts flash file. --Quick Installation Instructions-- 1. Download and unzip the files contained in GRE_Perl.zip file. 2. Upload the following files to your cgi-bin folder - rank_extractor.pl - gre_config.pl - gre_admin.pl - GRE_Template.pm - GRE_SQL.pm - gre_start.tp - gre_search.tp - gre_css.tp NOTE: Remember to CHMOD all perl (.pl / .pm) files to 755 if using *nix hosting. You can usually do this via your FTP program by changing the 'File Attributes' settings for each file to allow execution. Always upload using ASCII transfer mode. 3. Chose your method of capturing the google data a) Javascript(AJAX) - Requires visitor to have Javascript enabled browser - Upload gre_ajax.js file to the root of your website. - Insert a script reference in each of the files of your site, or in all of the pages you wish to track. - A typical HTML Javascript include would look like:- - The best place is just before the end tag or in your 'footer' include file if using includes. b) SSI (Server Side Includes) - Requires server to have SSI installed (99% of server do by default) - Insert one of the following lines in each of the files of your site, or in all of the pages you wish to track. OR - Depending on the OS platform and SSI configuration of the server will determine which SSI include you will need to use. NOTE: If you choose to use the SSI method within another include file be sure to check the version of SSI installed on the server is capable of handling nested includes as some servers are not. If you wish to use the SSI method but your website is not using either of the default .shtml or .shtm file extentions, you should be able to set your server to parse other file extentions such as .htm & .html through the SSI interpreter via your hosting Control Panel or contacting your hosting support. Only use one method of trigering the rank_extractor.pl code. If you use the Javascript(AJAX) & SSI methods together you will most likely create duplicate entries in the database for each visit. 4. Create a SQL database via your hosting Control Panel to store the data. You can name this whatever you would like. - If you are using mySQL you can run the SQL commands included in the file named gre_db_setup.sql to set up your database. 5. Upload the index.html file to a subdirectory of your website. Something like gre_admin. This will be your Google Rank Extractor Admin area. 6. Upload the FusionCharts subfolder and contents to the admin subdirectory of your website. This powers the FusionCharts graph engine. Ensure you upload flash files using Binary mode. 7. Edit the gre_config.pl file with your settings for database connection, table name, rank position required flag and admin password. Ensure when altering colours you use Hexadecimal. - Here is a breakdown of the gre_config.pl configuration user settings $GRE_SQL::DRIVER = 'mysql'; - Set for mySQL default, change to ODBC for MS SQL or another DB driver $GRE_SQL::HOST = 'localhost'; - Only change this if the default 'localhost' is not correct $GRE_SQL::DSN = 'yourDSN'; - Set this to your Data Set Name (DSN) created for the database instance $GRE_SQL::USER_ID = 'yourUSERID'; - Set this to your databse user account $GRE_SQL::PASSWORD = 'yourPASSWORD'; - Set this to your database user account password our $dbtable = 'grank'; - Set this to the table name you create for collecting the data (default = grank) our $pos_req = 0; - Flag when set to 0 will mean every visit from google with be stored (default) Flag when set to 1 will only collect visit data if Rank Position data is present our $password = ''; - No password is set by default, we strongly recommend you use a password to secure your data. our $path_to_cgi = ''; - Set physical path to website cgi-bin (remember to use forward slashes and end with a slash). Depending on server setup this can be left blank (usually only needed for windows OS). e.g. our $path_to_cgi = 'c:/websites/myweb/cgi-bin/' our $logo = ''; - Set image file for company logo. Ensure you upload the image to the GRE admin directory e.g. our $logo = 'yourLOGO.gif' our $logo_width = '200'; - Set width of logo our $logo_height = '80'; - Set height of logo our $page_bg = 'FFFFFF'; - Set main background colour our $page_font = 'Arial'; - Set main font our $page_font_size = '12'; - Set main font size (sizes are in pt measurements) our $th_border = '669966'; - Tabular results header border colour our $th_bg = '006600'; - Tabular results header background colour our $th_font_colour = 'FFFFFF'; - Tabular results header font colour our $td_results_border = '669966'; - Tabular results border colour our $td_results_bg = 'CCFFCC'; - Tabular results background colour our $td_results_font_colour = '000000'; - Tabular results font colour our $td_main_border = '669966'; - Main admin border colour our $td_main_bg = 'CCFFCC'; - Main admim background colour our $td_main_font_colour = '000000'; - Main admim font colour our $hl_link_colour = '0000FF'; - Hyperlink colour our $hl_hover_colour = '0000FF'; - Hyperlink hover colour our $hl_visited_colour = 'FF00FF'; - Hyperlink visited colour our $hl_active_colour = 'FF00FF'; - Hyperlink active colour our $important = 'FF0000'; - Important notice colour our $message = 'FF0000'; - Main messages colour our $canvasBgColour = 'D6FFD6'; - Graph canvas background colour our $canvasBorderColour = '7B3F00'; - Graph canvas border colour our $bgColour = 'CCFFCC'; - Graph background colour our $divLineColour = 'A82925'; - Graph dividing line colour our $baseFontColour = '000000'; - Graph base font colour (tool tip and numbers) our $outCnvBaseFontColour = '000000'; - Graph canvas font colour (labels and legends) our $baseFontSize = '12'; - Graph base font size (sizes are in pt measurements) our $outCnvBaseFont= 'Arial'; - Graph base font our $colours - This holds various pre-set colours used for plotting graphs. You can alter them or add to them but you must use Hexadecimal colours and separate them with a pipe (|). Remember if you change your admin template colours and alter the background colour. Ensure you change any colours in the $colours variable or other graph colour variables which may clash. Otherwise you may not see some graph data or legends. ############################################################################################# # Ensure you understand the settings below before altering them from their default settings # ############################################################################################# our $age_delay = 0; - Set this to the number of days (based on visit date), that you want to keep records in your database. If set to zero (default), no records will be deleted or archived. Ensure you do not make a mistake, if you set this incorrectly you could wipe all the data in your table! If you want to keep 12 months worth of data, set to 366, e.g. our $age_delay = 366; All records older than 366 days will then be deleted. It is worth noting that the trigger for deleting / archiving records is when the admin page is loaded. If any records are deleted / archived, a message will appear in the admin screen notifying you of your current $age_delay setting and how many records were deleted. Records are only archived if you correctly set the $arc_aged setting. Ensure perl has write permissions to the CGI-BIN folder before setting an age delay. our $arc_aged = 1; - This sets whether records which have aged should be archived to a CSV file before they are deleted. 1 = True (default) - All aged records will be archived to a CSV file and stored in your CGI-BIN folder. 0 = False - Aged records will not be archived before being deleted. Use with caution! Archived records are stored in files named with the 'cut off date' and deleted / archived records count. e.g. 2009-11-23(591).csv - All records up to but NOT including the 23rd November 2009 will be stored. CSV files will need to be managed manually and downloaded via FTP or other method. --Use of the Google Rank Extractor-- GRE uses the Google Referral String to store visitor data and Google Rank Position (when present) GRE Admin user interface requires Javascript enabled browser to function & Flash player plugin for FusionCharts. --GRE Admin Functions-- 01. Maximum Rank Filter 02. Ignore Zero Rank Filter. 03. Phrase Filter 04. Page Filter 05. Google Version Filter 06. Ignore Localisation Option. 07. Date Range Filter 08. Sort By Option. 09. Display As Option. 10. Report Type option. 11. Cross reference 'By' option. 12. Graph Type option. 13. Zoom option. --Leaving all options / filters in their default or blank state-- Leaving everything blank will return the maximum results. In essence it will return every single hit your site's pages received from a qualified Google Referrer. --Maximum Rank-- The Maximum Rank filter allows you to place a limit on the results being returned, based upon the ranking position of your site for the search phrase each user entered to find your site. For instance, if you only wanted to see the hits you received from your site's Top 5 ranked keyword phrases you would put the number 5 in this field before submitting your query. This would exclude all referrals that came through with a ranking notation of position 6 or higher. --Ignore Zero Rank-- If you have set your config file to collect all visits from Google regardless of whether a Ranking Position was suplied. You can still exclude them from the results by selecting this option. --Phrase-- The Phrase filter allows you to narrow down the returned results based upon the keyword phrase used by the visitor. As a default this field conducts a MySQL "LIKE" type of search on the database, not an Exact type of search. In English this means if you plugged in a phrase of "red lederhosen" the tool would of course return any results where the user searched specifically for the phrase "red lederhosen". However because the default is a LIKE type of search the results would also include phrases like "pretty red lederhosen", "german red lederhosen" and "red lederhosen germany". --Page-- The Page filter is quite a useful piece of data, in that it tells you quickly and easily which pages of your site people land on when they conduct certain searches. Not only does this show you which are your most important landing pages, but if you use the information wisely it will allow you to tailor the message you're communicating on each of your landing pages so that it better matches what visitors are looking for when they hit those pages. This one can have some important implications not only on your SEO efforts, but also on your conversion efforts. Because knowing exactly what people were searching for when they landed on one of your pages can give you a major hint at what you might need to say to them in order to improve your conversions. You may use CTRL to select multiple pages for direct comparisons. --Google Version-- This tells you which version of Google the visitor used to find your site, as well as where your site ranks for search phrases in each version of Google. By version of Google I mean the global and regional or geo-location versions. eg Google.com, Google.co.uk (United Kingdom), Google.com.au (Australia), Google.ca (Canada), Google.co.in (India) and so on. The results of this one surprised me a bit with the sites I used to test the original GRE code. I was frankly amazed at how well my .com web sites that are housed on US based servers rank all over the world. For their most competitive phrases in many, many cases my sites seem rank better on several of the regional search engine versions than they do on the .com version. Which makes no sense at all given the search engines general push towards personal search results. Additionally, the tool will tell you if someone searched one of the localized versions of Google with a Global type of search, or if they had selected "Show sites from XYZ Country". At the time of this writing the Google Local versions are defaulting to Global types of searches. It takes an additonal action by the searcher to change to a Country-Only type of search Note: There is nothing in the back end that lists all of the various Google versions. Instead the tool looks at the actual referral string to see where the hit came from, and this information is used to populate the Google Version dropdown box. So if you see a regional engine listed in the dropdown box, your site has received at least one hit from that version of Google. You may use CTRL to select multiple versions for direct comparisons. --Ignore Localisation-- As visits are stored showing whether the visitor had selected a local search or not, you can still choose to have this excluded from the results by selecting this option. This is very usefull to collate data when displaying results as graphs, but can also save you the trouble of having to select both '(Local)' and '(Global)' if making multiple selections. Tabular: If you had visits from both 'co.uk (Global)' and 'co.uk (Local)', you would only need to select one them and your search would still include both in the results. Graphs: If you had 10 visits from 'co.uk (Global)' and 5 visits from 'co.uk (Local)', the graph will only show 'co.uk' with a total of 15 visits. --Date Range-- This one does exactly what you would expect it to do. It allows you to filter the results being displayed by the date of the original hit. So if you wanted to only look at results from a certain one week period you can. A default date is initially set for the last 7 days, you can either alter the date range as desired, use the 'clear' option to remove date selection or use the 'reset' to reset the date range back to the last 7 days. --Sort By-- Sort By is NOT a filter and does not exclude any results. Instead it simply changes the order in which results are displayed. So it allows you to group the results in different ways, to hopefully make it a bit easier to use. The currently available sort options are as follows. a. By Date in Ascending Order (the Default action) b. By Date in Descending Order c. By the search phrase entered by the user, Alphabetically in Ascending Order d. By the search phrase entered by the user, Alphabetically in Descending Order e. By the Ranking Position in Ascending Order f. By the Ranking Position in Descending Order g. By the Google Version used by the user h. By the Landing Page name, Alphabetically in Ascending Order i. By the Landing Page name, Alphabetically in Descending Order --Display As-- This option allows you to switch between a tabular result, a flash graph or a CSV export file. --Report Type-- There are several reports available for you to choose from. All reports will work in conjunction with all other search functions. 1. Google Version - Shows count of each Google version your visitors came from. 2. Landing Page - Shows count of each landing page your Google visitors landed on. 3. Search Phrase - Shows count of each search phrase used in Google to find your site. 4. Ranking Position - Shows count of each SERPs rank position ('No Rank' shown if not available). 5. Visit Date - Shows count for each day you received Google visitors. -- By -- You can choose to select an additional cross reference 'by' option to produce stacked and multi-series graphs. You may select any type of report and cross reference by any option except by the same report type. ie. If you select a report type of 'Google Version' you cannot then choose to cross reference this by 'Google Version'. However you can choose to cross reference by any of the other options or set to 'Not Required' to produce single series graphs. Using 'Report Type' in combination with the cross reference 'By' option gives you a total of 25 different graphs! --Graph Types-- There are eight graph types to choose from which work with single series graphs. 01. Column Chart 3D 02. Column Chart 2D 03. Pie Charts 3D 04. Pie Chart 2D 05. Bar Chart 2D 06. Area Chart 2D 07. Line Chart 08. Doughnut Chart There are six graph types to choose from which work with multi-series graphs. 01. Stacked Column 3D 02. Stacked Column 2D 03. Stacked Bar 2D 04. Multi-Series Column 3D 05. Multi-Series Column 2D 06. Multi-Series Bar 2D --Zoom-- You can magnify the graph size up to 300% --Combining Filters-- Though it may not be evident at first, you can combine the various filters and sorting functions available in the tool to focus in on more specific data. --Hyperlinks-- Tabular reports show hyperlinks for Search Phrase & Landing Page. 1. Phrase Check - This enables you to run a search with the Search Phrase & Google version used by your visitor. 2. Page View - This enables you to view your own Landing Pages. --Copyright, License To Use and Other Legal Stuff-- File: README.txt Application: Google Rank Extractor. Application: README.txt file for GRE. Copyright (C) 2009 Craig Chant This program is free software: You may use, redistribute and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation. It is released under version 3 of the License. This program is distrbuted in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License at http://www.gnu.org/licenses/ for more details. The only requirements for redistribution are that any such distributions, with or without improvements, carry the original copyright notice above, and that any derivitive versions also carry a GNU GPL License. ################################################################ --Version History-- Version : Perl v1.0 Beta August 22, 2009 - Original Release Version : Perl v1.1 Beta August 23, 2009 - Added TimeStamp to help with potential caching issues. - Added SQL injection protection for input attempts using single quotes. Version : Perl v1.2 Beta August 24, 2009 - Added Win32::ODBC to GRE_SQL Module to ensure Windows OS & MS SQL compatability. - Added path_to_cgi parameter needed for Windows OS & template path. - Added (Local) or (Global) search identifier to version data. - Fixed Admin area AJAX error when password not used. Version : Perl v1.3 Beta October 26, 2009 - Updated 'date range' selector tool. - Added hyperlinks -> landing page & Google search phrase checker (Table View Only). - Added FusionCharts (vFree.1.2) flash graph engine. - Added Report -> Google Version - Added Report -> Landing Page - Added Report -> Search Phrase - Added Report -> Ranking Position - Added Report -> Visit Date Version : Perl v1.4 Beta October 30, 2009 - Updated 'date range' selector tool. - Updated 'sort by' tool. - Added 'ignore zero rank' option. - Added 'ignore localisation' option. - Added multiple select functionality for 'landing page' and 'google version'. - Added cross reference 'by' function. - Added FusionCharts (vFree.1.2) stacked and multi-series graphs. - Added graph 'colours' to config file. - Added Report -> Google Version (cross reference option). - Added Report -> Landing Page (cross reference option). - Added Report -> Search Phrase (cross reference option). - Added Report -> Ranking Position (cross reference option). - Added Report -> Visit Date (cross reference option). Version : Perl v1.5 Beta October 31, 2009 - Added company logo options. - Added template colour scheme options. - Added graph colour scheme options. - Added screen resolution 'auto size' function. Version : Perl v1.6 Beta November 24, 2009 - Fixed 'Ignore Zero Rank' display error. - Added CSV export function. - Added Aged records database purge option. - Added Aged records CSV archiving option.