PostNuke Community

• ZarToXidE
  
  

  • Rank: Helper
  • Registered: 20.01.05
  • last visit: 27.06.09
  • Posts: 452

  Posted: 30.10.2006, 14:38

  Since I'm only running the News module, I do not need the googlesitemap module (yet), and added my rss feed as sitemap.
  .
  Then I nocited some hits on my site from search engines after 1 day I posted an item. So, as noosy as I am, i wanted to know where google sent the people that came to my site. So I clicked the links from the referer module, and saw that my backend was indexed aswell.
  .
  Now can I read those feeds fine, but it isn't really what the idea was. Was discussing this item on another board, and got the advice to modify the backend a bit.
  .
  If this:
  

  Code

  header("Content-Type: text/xml; charset=ISO-8859-1");

  
  Would be changed to:
  

  Code

  if(!empty($_SERVER['HTTP_USER_AGENT']) && strpos($_SERVER['HTTP_USER_AGENT'], "Mozilla") !== false) {
      header("Content-Type: text/xml; charset=iso-8859-1");
  } else {
      header("Content-Type: application/rss+xml; charset=iso-8859-1");
  }

  
  Then search engines won't index these files. Since I modified the backend to give an RDF and an ATOM feed items got listed 4 to 5 times in search engines. (3 feeds, 1 frontpage, and the article itself)
  .
  I got told that by using this, search engines wouldn't index the feeds. Just made the modification myself, so I can't really conform it's working yet. But will keep an eye on it.
  .
  Just wanted to share this with ya'll
  
  --
  Doe Maar Freak

  • 
  •  
• larsneo
  
  

  • Rank: Software Foundation
  • Registered: 31.12.69
  • last visit: 27.06.09
  • Posts: 4458

  Posted: 30.10.2006, 14:42

  what about simply using a robots.txt rule?
  
  --
  regards from germany
  ..::[Zikula Application Framework]::.. ..::[SEO-Blog]::.. ..::[CMS Sicherheit]::..

  • 
  •  
• ZarToXidE
  
  

  • Rank: Helper
  • Registered: 20.01.05
  • last visit: 27.06.09
  • Posts: 452

  Posted: 30.10.2006, 14:54

  larsneo

  what about simply using a robots.txt rule?

  
  LOL.. Doh... You're right.. Haven't thought of that one..
  
  Now I'm wondering.. Because this is changed, will google still accept it as a sitemap, by using robots.txt it won't be accessible after it updates the robots.txt. [s]Did a resubmit, I'll get back on that..[/s]
  
  Edit: Google still accepts the RSS feed as a sitemap. Hope the feeds don't show up in the results in a week or so.
  
  
  
  edited by: ZarToXidE, Oct 30, 2006 - 04:05 PM
  
  --
  Doe Maar Freak

  • 
  •  
• slam
  
  

  • Rank: Helper
  • Registered: 04.05.04
  • last visit: 28.06.09
  • Posts: 339

  Posted: 30.10.2006, 16:53

  Feel free to use this one and adapt it to your needs.
  Greetings,
  Chris
  
  --
  Cozi - development is life.
  
  sidux - an operating system must operate.

  • 
  •  
•  
•  

Users online

This list is based on the users active over the last 60 minutes.