Processing and deleting phpsessid and sessionID (session identifiers on PHP- and ASP-applications)


During the process of crawl your website may form session IDs. Our crawler processes and deletes session identifiers. To the Sitemap file all links will be input "clear", without identifiers, passed in URL - phpsessid (for PHP) or objects sessionID (for ASP). This helps to avoid pasting into Sitemap duplicate links, when bot receives the same page with different URLs.

Example of session identifier in PHP:
    http://website.tld/page.html?PHPSESSID=123456session6789
Example of session identifier in ASP:
    http://website.tld/(S(123456session6789))/page.html
Finally, the URL will be transformed back to a basic form:
    http://website.tld/page.html