I recently worked with a client at the beginning of their SEO strategy. This case study outlines our analysis and recommendations to improve their search visibility using programmatic SEO techniques. I wrote up my general recommendations for anyone interested in Programmatic SEO here.

Initial Assessment

The client had a significant amount of unique content, but it was not being indexed properly due to a number of technical issues. Some of the problems we identified:

1. Most of the content lived on a subdomain, not the primary domain – not a huge issue, if everything else is good
2. The primary domain had pages with redirects to the subdomain, but they were 302 instead of 301 redirects
3. Neither the subdomain nor the primary domain had a robots.txt or sitemap.xml
4. Page content was largely being loaded through React Javascript calls, instead of being pre-rendered on the page, leading to various page speed issues
5. No directory index pages
6. No inter-linking across pages

I showed them the Zapier app directory, which is an excellent example of how to generate many thousands of pages with unique content, as well as directory index pages for browsing the content. One interesting question they had was “are more pages better?”. The answer is: yes, more pages are better, but only if each page has mostly unique content. All the pages can follow the same template structure, but if you diff across pages, there should be plenty of significant content differences.

Recommendations

Based on our findings, we figured out a set of prioritized recommendations, starting with the easiest wins:

1. Implement robots.txt and sitemap.xml: This is the first step to guide search engines on how to crawl and index the site. Both the top-level domain (TLD) and subdomains need these files. A simple robots.txt that allows all user agents is fine to start with, as long as there’s something. And you can also link to the sitemap.xml in the robots.txt.
2. Switch to 301 Redirects: We advised the client to use 301 redirects instead of 302 to ensure that link equity is passed correctly, which is crucial for SEO.
3. Optimize On-Page Elements: Titles, meta descriptions, and H1 tags should be optimized to highlight the unique keywords on each page. Each page should have a single H1 tag that closely matches the page title.
4. Use SEO Tools: Setup Google Search Console, Bing Webmaster Tools, and an SEO tool like ahrefs to see how the content is being indexed and how well it is performing. Most people don’t realize that Bing is the backend search index for most alternative search engines like duckduckgo.com, so it’s important to factor them in to improve your reach.
5. Generate Directory Index Pages: Create index pages from top-level category items to enhance navigation and SEO.
6. Inter-Linking Across Content: Create links between related pages using categories, or any other content relationships.
7. Static HTML Generation: Instead of loading content through React in real-time, generating static HTML pages can greatly improve load times and indexing quality.
8. Load Balancer Implementation: By putting a load balancer in front of the main domain, they can then point a specific path to the subdomain, thus eliminating the need for a visibly separate subdomain and 301 redirects.
9. Mobile Optimization: Ensuring the site is optimized for mobile rendering is crucial for improving search ranking. All search engines now prioritize mobile pages and mobile rendering.

Conclusion

Programmatic SEO can provide substantial improvements in search visibility and user experience when implemented correctly. Generating thousands of pages programmatically is a great way to scale your content and visibility. But it’s crucial to get the technical implementation correct, otherwise the search engines will not properly index the content, and your pages just won’t show up in search results.

If you’d like some specialized help on your programmatic SEO strategy, feel free to reach out to me at Streamhacker Technologies.