To Get Started:
Start by reading the Installing Plugins section in our comprehensive WordPress and WordPress Multisite Manual if you are new to WordPress.
Note: If you have an older version of the plugin installed in /mu-plugins/ please delete it.
When activated on regular WordPress installs – the SmartCrawl menu item is added to Settings.
When network-activated on WordPress Multisite installs – the SmartCrawl menu item is added to Settings in the Network Admin dashboard.
Enabling Access on Sub-sites
In a Multisite install, you can allow your site admins to adjust the settings for SmartCrawl on their sites by doing the following:
1. With the plugin network-activated, make your selection of which tabs you wish to allow your admins to change. You’ll see a setting for that at the bottom of each tab:
- Please note, that for any settings tab that you don’t wish to give subsite admins access to, you’ll want to make sure that everything inside that tab is configured the way you want it to be, as it won’t be manageable inside the Network Dashboard when you enable per-blog mode.
2. Once you’re done with configuring which tabs you wish to show up, open your wp-config.php and above this line:
/* That's all, stop editing! Happy blogging. */
Add in this line of code:
define( 'WDS_SITEWIDE', false );
Once this is done, the SmartCrawl settings pages will disappear from the Network Dashboard, and will now display inside each subsite Dashboard under:
Settings -> SmartCrawl
Any settings in tabs you mark as inaccessible should be applied to every site on the network, but everything your site admins can control should be site-specific.
If you ever need to take network control over the settings for SmartCrawl in the future, simply set the define you just added to “true”. The settings will then disappear from individual sites, and will once again be accessible in your network settings.
Configuring the General Settings
Let’s begin by configuring the global settings under the first tab – Step 1: Settings.
2. Select the minimum user role who can edit.
1. The Active Components setting allows to enable or disable selected SEO components. In most situations, you would want to leave all four of the active components checked. Unchecking any of these boxes will disable the settings for the corresponding component.
2. Show metaboxes to users enables you to select the minimum user role you would like to have access to the SEO metabox settings. These are the SEO settings that appear in the edit/create post pages. This can be especially useful on a multisite network to keep things simple for users by letting the Super Admin manage the settings.
Save your settings before moving to the next tab – Step 2: Sitemaps.
Configuring the Sitemap Settings
There’s a lot of things to cover here, so let’s take it in 3 parts.
XML Sitemap & Exclude
2. Select the post types & taxonomies to exclude from the sitemap.
1. The XML Sitemap setting is entirely optional, and you may want to simply leave it at the default. When network activated, you likely won’t see anything at all in the “Path to the XML Sitemap” field. The plugin assumes that the path is something like this:
/home/user/public_html/wp-content/uploads/sitemap.xml
If for any reason you need your sitemap in a different location on your server, enter the path to the file location here.
On single site installs (and on the main site if you have toggled the per-blog mode with the define as above), you will see the full path displayed in that field. It will look like this by default:
/home/user/public_html/wp-content/uploads/sitemap.xml
In per-blog mode on Multisite, the default will look like this on sub-sites:
/home/user/public_html/wp-content/uploads/sites/2/sitemap.xml
The “URL to the XML Sitemap” will display the web address of your sitemap in case you want view or link to it. The default is:
http://yoursite.com/sitemap.xml
2. The Exclude settings enable you to exclude selected post types and/or taxonomies from your sitemap. As an example, this can be especially useful if you have private, members-only content that you don’t want indexed in any way.
Options
2. Select to apply the style-sheet or not.
3. Optionally enable a dashboard widget.
4. Disable automatic updates if desired.
1. If you enable Include image items with the sitemap, it’s important to know that plugin memory consumption will considerably increase. How much depends on how much of your content is images, as well as your server configuration & capabilities.
2. The Include stylesheet setting will simply make the sitemap easier to read by human eyes. If you have no intention of viewing it or linking to it, you can leave this disabled.
3. Enabling Show dashboard widget will display a widget with SEO stats in your site’s dashboard. If you are in network mode, the widget will display in the dashboard of every site in your network, and the data will be specific to each site.
4. Disable automatic sitemap updates is useful if you want to update them manually only, either by visiting the SEO Settings page, or by clicking the “Update sitemap now” link in the widget.
Search Engines
2. Choose where to add verification code.
3. Toggle search engine notification on/off.
1. You can enter your Google & Bing site verification codes in the first fields of the Search Engine options. For more on how and why to do that, click below:
2. This setting allows you to Add verification code to all of the pages on your site, or just the home page.
3. The next setting of this tab enables you to Automatically notify search engines when your sitemap updates. No explanation necessary.
Again, save your settings before moving on to Step 3: Title & Meta.
Configuring the Title & Meta Settings
The Site Title & Meta settings is one area where SmartCrawl really gets to shine. You can set global SEO meta using any of over 2 dozen macros for every post type on your site.
Search engines read the title and description for each element of your site. So it’s important to take the time to configure these settings.
You can override the global meta you set here for individual posts, and fine-tune them at will. But we’ll get to that a bit later; let’s first take a closer look at the global settings. 🙂
Home
2. Set your homepage meta description.
3. Optionally enter any homepage keywords.
4. Set blog meta robots.
1. The Home Title setting is where you can enter exactly what you want search engines to prominently display on results pages. This is what identifies your site in the results.
2. The Home Meta Description is likely the most important setting here, as it is the most likely to be indexed by search engines.
- The Title and Description fields for every post type on your site can be configured using any of over 2 dozen macros. Macros are powerful little function tags that pull in information related to your post types & taxonomies.
- You can view the full list of macros available for you to use by clicking the “Help” tab in the upper-right corner of your screen while on this page.
- You can also manually enter text that you want to appear in search results. For example, if your site was called Bill’s Emporium, and if you were to enter this into the Home Title field:
%%sitename%% - A sample siteThe title you’d see in search results would look like this:Bill’s Emporium – A sample site
3. Here, you can optionally enter any Home page keywords you want search engines to pay special attention to.
4. The Main Blog archive Meta Robots settings affect your site’s blog archive pages. You can set your archive to No Index to help prevent duplicate content appearing in search results (which can negatively impact your rankings).
You can also tell search engines to index the main archive page, but leave the subsequent pages alone by ticking Leave the first page alone, but apply to subsequent pages. That can be especially useful if your front page is your blog archive.
For more about No Index and other meta robots, see the Configuring SEO for Individual Posts section further down this page.
Posts & Pages
2. Configure the post type meta description.
1. The Post Title/Page Title setting is where you configure how your individual post/page titles should appear to search engines and on search results pages.
2. The Post Meta/Page Meta Description will adapt and display information specific to each post/page. Here is where you can really play around with the available macros to customize exactly how your site content should appear in search results.
- If you have a blog with multiple authors, and would like search results for each one of them, try including the %%name%% macro which is replaced with the post/page author’s “nicename”.
- Or if your site is a photoblog, you may want to use the %%caption%% macro, which is actually replaced with the caption of the post attachment!
Media, Categories, Tags, etc.
Every post type & taxonomy that is used on your site will be available for SEO customization in the same manner: media, categories, tags, archives… even “Search” and “404” pages.
Configuring Moz Settings
Moz is the the industry leader in SEO reports, and we make it easy to integrate with their API. Note that configuring this is entirely optional.
All you need to do to take advantage of all their reporting tools is enter your API Access ID and Secret Key in the corresponding fields.
To get your Access ID & Secret key, sign up for a free account at https://moz.com/community/join.
Then visit http://moz.com/products/api/keys to generate your API credentials.
Once you’ve entered your credentials and saved the settings, it’ll only take a few minutes for you to begin to see metrics specific to your site.
Click any linked text in the metrics to view more information and data on Moz.com
- Your site metrics also appear in the dashboard of your site, just under the Sitemaps Widget (if you had enabled that earlier).
- Plus, in a multisite install, metrics specific to each site in your network appear in the dashboard of each site. How cool is that?
Here’s an added bonus: there is also a Moz metrics metabox in the post editor of every post, page & custom post type on your site. That’s right, you can get detailed metrics for individual posts too. Check it out!
You’ll find a wealth of information about good SEO practices, and details about your site metrics, by visiting the “Learn” section at Moz: http://moz.com/learn/seo
Configuring Automatic Links
This section enables you to specify certain key words on your site that should always link to content on your site, or even a different site altogether.
For example, maybe anytime you write the words ‘WordPress news’ you want to automatically create a link to the WordPress news blog, wpmu.org. Without this plugin, you would have to manually create these links each time you write the text in your pages and posts – which can be no fun at all.
There are lot of things to cover here too, so let’s take this in 3 parts again:
In the beginning…
2. Select site content to link to.
3. Set the minimum post title length for linking.
1. The first setting allows you to select exactly which post types the plugin should automatically Insert links in. Every post type active on your site will be available for selection here.
2. The second setting tells the plugin what post types & taxonomies it should look for and Link to. For example, if you have a “Really Cool Stuff” category on your site, the plugin could automatically link to that category archive any time it finds “really cool stuff” on your site.
3. The Minimum post title length allows you to set the minimum number of characters a post title must have in order for the plugin to automatically create links to it.
But wait, there’s more…
2. Check to allow linking to empty taxonomies.
3. Check to disallow linking in heading tags.
4. Enter the posts and/or pages you wish to exclude.
5. Enter any keywords you want the plugin to ignore.
6. Set the maximum number of links per post.
1. Similarly, the Minimum taxonomy title length sets the minimum length of taxonomy titles to autolink to. For example, if you do not want the plugin to automatically create links to your “Really Cool Stuff” category, set this to at least 18 (Really Cool Stuff = 17 characters).
2. Sometimes you may have Empty taxonomies on your site, but want to link to them anyway. Checking this allows the plugin to do that automatically.
3. Check the Exclude Headings setting if you do not want links automatically inserted in heading tags. No worries, h1, h2, h3 and h4 tags are exempted by default, so this setting only affects h5 and h6 headings.
4. If you want the plugin to Ignore posts and pages, enter the IDs, slugs or titles for the post and/or pages you wish to exclude and separate them by commas.
5. The Ignore keywords setting enables you to enter any keywords that the plugin should not use for links. This can be very handy if you notice certain words or phrases are linking to places you don’t want them to. Or if there are certain areas of your site you don’t necessarily want to be linked to (the “Uncategorized” category for example).
6. The Maximum autolinks number limit is exactly that: the maximum number of autolinks that will be added to each individual post.
To the end…
2. Enter any custom keywords the plugin should link.
3. Select additional link processing options.
1. The Maximum single autolink occurrence sets the number of times any specific link will appear in individual posts.
2. The Custom Keywords setting enables you to really tweak things. If there are any keywords, or key phrases, that you want to automatically link to specific URLs, enter them here in the following format:
keyword, keyword, keyword phrase, http://sitename.com/page
3. The Other Settings section actually has several additional settings you can select:
- Process only single posts and pages will ensure that autolinking does not occur in places like archives or search results pages.
- Process RSS feeds will ensure that links are automatically included in your RSS feeds.
- Case sensitive matching will ensure that links are automatically created only if uppercase and lowercase spelling is an exact match.
- Prevent duplicate links ensures that only the first occurrence of any matched text in any post will be linked. Note that this overrides the Maximum single autolink occurrence setting above.
- Open links in new tab/window will, um, open links in new tab/window .
Configuring SEO for Individual Posts
We mentioned earlier that you can tweak the SEO settings for individual posts too. Let’s take a look at that now, shall we?
Open the post editor for any post, page or custom post type on your site, then scroll down until you see the SmartCrawl tab.
You will see familiar settings there to optimize that specific post with title and other meta information. As can be seen in the screenshot above, this can be especially handy when the post is composed entirely of shortcodes (they aren’t parsed by search engines, and don’t help your rankings at all).
You also have a few more advanced settings that can be tweaked for individual posts.
2. Enter a canonical URL.
3. Set a 301 redirect.
4. Adjust the sitemap priority.
1. The Meta Robots settings tell the search engines how, or if, this particular site content should be indexed. Engines will, by default, Index all content for search results, and Follow all links for page-rank and such. You can modify that behavior here.
If you do not want your content indexed in the Open Directory Project or the Yahoo! Directory, you can block them by checking the appropriate checkboxes.
Check No Archive to prevent a cached copy of your content from appearing in search results. This is handy for frequently updated content where you really want the most recently crawled version of your page to appear.
Check No Snippet to prevent a description from appearing below the page in the search results. This also prevents caching of the page.
For more on what these settings actually do when search engine bots crawl your site, see this Google article: Using the robots meta tag
2. You can also specify the Canonical URL for individual posts & pages. A canonical page is the version of a set of pages with highly similar content that you want the engines to prioritize and index.
3. The 301 Redirect setting tells search engines that the content has permanently moved to a new location. If set, it will redirect visitors to the URL you enter here if they try to access this post/page.
For more information about Canonical URLs and 301 Redirects, see this Google article and this one at SearchEngineWatch.com.
4. The Sitemap Priority setting enables you to adjust the priority that your post or page should be given in your sitemap, thus giving it higher (or lower) priority in search engine results. For example, if you have set a canonical URL for one page in a set of pages with related or similar content, you can raise the priority of that one, and lower it for the others.
While the sitemap priority does not affect your site’s search engine ranking, here a couple of things to keep in mind:
- Pages with high priority values are likely to get indexed faster and crawled more often.
- They help search engines to decide which URL to show if multiple pages from the site rank for a search query.
Now you have everything you need to truly optimize your site’s content in one easy bundle. If you hit a snag with any aspect of configuring this plugin, head on over to the SmartCrawl community forums where support staff and members just like you are standing by to help.
SmartCrawl Features
Make it easier for search engines to find your content.
- Simple guided settings
- Offer SEO options by user role
- Direct interaction with Google
- Direct interaction with Bing
- Send sitemap updates automatic to search engines
- Title and meta data optimization
- Automatic sitewide linking
- Complete Moz integration
- Multisite and BuddyPress compatibility
- Process posts and pages individually
- Process RSS feeds
- Conduct case sensitive matching
- Prevent duplicate links
- Open links in new tab/window
- Exclude posts or pages from sitemap
- Exclude custom post types
- Exclude categories
- Exclude tags
- Exclude custom taxonomies
- Include or exclude images
- Include or exclude stylesheets
- Include or remove the sitemap dashboard widget
- Disabling automatic sitemap updates
- Custom home title
- Home meta description
- Keywords
- Main blog archive meta robots options
- Post title defaults
- Post meta description defaults
- Media title defaults
- Media meta description defaults
- Custom post titles (per custom post type)
- Custom post meta descriptions (per custom post type)
- Post categories, tags and custom taxonomy title and meta defaults
- Post categories, tags and custom taxonomy robots tags
- Author and date archives
- 404 page title and description defaults