Development

Configuring a robots meta tag

This article walks through a sample setup that will allow for a "robots" meta tag to be included in the source of one or more pages. Using these options, it's possible to instruct search engines as to whether or not they should index particular pages and/or crawl links on those pages.

Template setup

The very first step here will be to include a region in your Template where the resulting <meta> tag will be placed in the source code of your page(s).

  • Edit the Template that your Pages are using
  • Within the <head> tags somewhere, add the following code and save:
<system-region name="ROBOTS"/>

Metadata Set setup

We'll add a field to our Metadata Set which will allow for users that edit Pages to select whether or not a Page should be indexed by search engines as well as whether or not links should be analyzed by search engines.

  • Edit the Metadata Set that your Pages are using
  • Switch to the Custom Fields tab
  • Click to add a new Checkbox field
  • Fill out the following fields with the values seen below:
    • Name: search-engine-options
    • Field Label: Search engine behavior
    • Help Text: Specify how this page should be treated by search engines in terms of appearing in search results and link crawling
  • Under the Configuration field, switch to the XML tab
    • Copy/paste the following XML into that field and click Submit
<checkbox>
 <item label="Prevent indexing of this page">noindex</item>
 <item label="Prevent crawling of links in this page">nofollow</item>
</checkbox>

Format setup

With our metadata option set up, we now need to write a Velocity Format which will check to see which (if any) of the options have been selected for a Page and then output the corresponding <meta> tag based on those selections.

  • Create a new Velocity Format
  • For the contents of the Format, enter the following code:
#set ($searchEngineOptions = $currentPage.metadata.getDynamicField("search-engine-options"))
#set ($robotsContent = $_DisplayTool.list($_ListTool.toList($searchEngineOptions.values), ","))
#if (!$_PropertyTool.isEmpty($robotsContent))
<meta name="robots" content="${robotsContent}"/>
#end

Final steps

Now that all of the pieces are in place, the final step of the process is to attach our Velocity Format to the "ROBOTS" region that we created in our Template. This assignment can be done at the Template level, the Configuration, or the individual Page level (just depending on which Pages you want to inherit that setup).

The most common method is to attach the Format to the region at the Template level (so that all Pages using that Template have the region available as a placeholder for that <meta> tag).

End result

When users go to edit a Page that uses both the Template and Metadata Set as described above, the options will appear as indicated in the screen shot below:

Image showing the check box options for the robots tag

If no options are selected, no <meta> tag will be added to the Page's source code. If one or both of the options are selected, the tag will be included with the corresponding attribute values. For example, if both options are selected, the output will be:

<meta content="nofollow,noindex" name="robots"/>
Notes:
  • The sample above is just one of many different ways that you could approach this type of setup. Feel free to modify any of the steps here as needed for your specific use case.
  • The term "indexing" in this article is not to be confused with an asset's "Include when indexing" option (which is specific to the CMS and has no effect/is unrelated to search engine behavior).
Back to Top ↑