The Google Sitemaps is a new program that allows webmasters to create a sitemap of their site. Why is it worth the extra trouble? Well, partially because the website masters don’t have a choice, but also because it leads to the two things that can improve search engine ranking: it’s easier for Google to index all the pages (thus avoiding the problem of accidentally ‘skipping’ key information) and it’s easier to upload new and relevant content.

Essentially, you use the XML program. That’s easy enough for webmasters. After that, three options await—ranging from the simple and straightforward, to what we’d like to call “Challenging”.

Let’s start with challenging. Of course that’s a relative term; if you’re so comfortable wit your server and so smart you can install scripts in your sleep, then this won’t be much of a problem. You use Google’s Python Generator. You’ll need to install this, and that requires a little more technical know-how than reading “IT Management for Dummies.” How does it work? The search engine gives you a link to a generator. Download this link, and set it up on your server. While you go off and get yourself a well-deserved cup of coffee, the computer does it thing and you come back and find that your sitemap XML file has been automatically fed to Google. Once that’s done, open a Google Account so you can follow your sitemap’s progress and read any diagnostic data. Or, alternatively, you can sign in using your accounts for GMail and Google Alerts.

Not too keen on Python Generator? Not a problem. Use is a PHP Code Generator. This program spiders your site, then takes charge of developing the XML sitemap file. Cool—it does everything for you! Except you’ll have to have a basic understanding of PHP files and scripts, or you’ll be lost. Not something for the average Joe (or Jane) who just wants to make a personal website, but relatively simple for professional webmasters or a particularly smart weekend tekkie.

If you think Python Generator and PHP are way too complex, or just too complicated, don’t panic. You still have plenty of options—the numerous free online generators that are all over the web, ready to rescue you from the hassles of XML sitemaps. Google itself keeps a directory of these third party suppliers (for lack of a better word), so just visit its website or Google them yourself.

One such option that is provided by XML-Sitemaps requires minimal supervision. Click it, and let it spider your site and index as many as 500 pages into a handy sitemap XML file. (Although, if you really do have 500 pages, it may be a good idea to get a webmaster—that’s a lot of content to handle on your own. Then you can make him worry about the XML sitemaps for you.)

Once the sitemap is generated, these programs will also let you download files and make minor changes. Another advantage: you can establish the priority of the pages, indicate the kind of frequency you want and adjust it accordingly, and other nifty features that give you more control over your website. If you are doing daily updates, however, you should use PHP or Python Generator.

Upload the file to your website, tell Google Sitemaps that you’ve “done the deed”, and let the hits come.

The Robots.txt protocol, also called the “robots exclusion standard” is designed to lock out web spiders from accessing part of a website. It is a security or privacy measure, the equivalent of hanging a “Keep Out” sign on your door.

This protocol is used by web site administrators when there are sections or files that they would rather not be accessed by the rest of the world. This could include employee lists, or files that they are circulating internally. For example, the White House website uses robots.txt to block any inquiries on speeches by the Vice President, a photo essay of the First Lady, and profiles of the 911 victims.

How does the protocol work? It lists the files that shouldn’t be scanned, and places it in the top-level directory of the website. The robots.txt protocol was created by consensus in June 1994 by members of the robots mailing list (robots-request@nexor.co.uk). There is no official standards body or RFC for the protocol, so it’s difficult to legislate or mandate that the protocol be followed. In fact, the file is treated as strictly advisory, and does not have absolute guarantee that those contents won’t be read.

In effect, robot.txt requires cooperation by the web spider and even the reader, since anything that is uploaded into the internet becomes publicly available. You aren’t locking them out of those pages, you are just making it harder for them to get in. But it takes very little for them to ignore these instructions. Computer hackers can also easily penetrate the files and retrieve information. So the rule of thumb is—if it’s that sensitive, it shouldn’t be on your website to begin with.

Care, however, should be taken to ensure that the Robots.txt protocol doesn’t block the website robots from other areas of the website. This will dramatically affect your search engine ranking, as the crawlers rely on the robots to count the keywords, review metatags, titles and crossheads, and even register the hyperlinks.

One misplaced hyphen or dash can have catastrophic effects. For example, the robots.txt patterns are matched by simple substring comparisons, so care should be taken to make sure that patterns matching directories have the final ‘/’ character appended: otherwise all files with names starting with that substring will match, rather than just those in the directory intended.

To avoid these problems, consider submitting your site to a search engine spider simulator, also called search engine robot simulator. These simulators—which can be bought or downloaded from the internet— use the same processes and strategies of different search engines and give you a “dry run” of how they will read your site. They will tell you which pages are skipped, which links are ignored, and which errors are encountered. Since the simulators will also reenact how the bots will follow your hyperlinks, you’ll see if your robot.txt protocol is interfering with the search engine’s ability to read through all the necessary pages.

It’s also important to review your robot.txt files, which will enable you to spot any problems and correct them before you submit them to real search engines.

Anyone who is serious about building wealth on the Internet MUST master the search engines. Therefore, learning how the search engines actually work and how they present information to the customer initiating a search, is key to anyone optimizing their website for search engine indexing.

The most common methods the search engines use to index a site is by using content scanning robots called “Web Crawlers” or “Spiders”.

Search Engines use spiders to index websites. When you submit your website pages to a search engine by completing their required submission page, the search engine spider will index your entire site. A “spider” is an automated program that is run by the search engine system. Spider visits a web site, reads the content on the actual site, reads the site’s Meta tags, and also follows the site’s outbound links. The spider then returns all that information back to a central depository, where the data is indexed. It will visit each link you have on your website and index those sites as well. Some spiders will only index a certain number of pages on your site, so don’t create a site with 500 pages!

The spider will periodically return to the sites to check for any information that has changed. The frequency with which this happens is determined by the moderators of the search engine, although there is a metatag within many websites’ header that contains instructions for the crawlers to return in a specified number of days.

A spider creates a file almost like a book; it contains the table of contents, the actual content and the links and references for all the websites it finds during its search, and it may index up to a million pages a day!

Example: Excite, Lycos, AltaVista and Google.

When you ask a search engine to locate information, it is actually searching through the index which it has created and not actually searching the Web. Different search engines produce different rankings because not every search engine uses the same algorithm to search through the indices.

One of the things that a search engine algorithm scans for is the frequency and location of keywords on a web page, but it can also detect artificial keyword stuffing or spamdexing. Then the algorithms analyze the way that pages link to other pages in the Web. By checking how pages link to each other, an engine can both determine what a page is about, if the keywords of the linked pages are similar to the keywords on the original page.

These algorithms are largely a mystery; a deep secret known only to the search engine staff. Huge amounts of money are paid to SEO (search engine optimization) experts (as well as some who claim to be experts) to optimize pages for the highest possible search engine rankings. Top rankings on engines like Google mean market domination for the well optimized site, and that top rank is highly contested, as you can guess!

Although Apple had nothing to do with the podcasting technology, the iPod and podcasting quickly became linked. With the advent of the iPod Video, podcasting entered a whole new level. Now, more than the usual audio broadcasts, aspiring film makers can also share their work with the world.

So how do you go about making your own video podcast? To make it easier on you, use a digital video camera. If you have an analog video camera, it is still possible to convert the footage to digital format but it adds another step to the process. So assuming you have a digital video camera, what do you do?

First, take great footage. Of course, making a film is not simply shooting. Create a concept and make it good. When you have finalized your ideas, it is time to shoot. When taking your shots, bear in mind your target media – the iPod. The small screen will not be able to display too many details so it is a good idea to get close when shooting. Do not use the wide-screen mode but use the standard mode instead. Optimize bandwidth and battery power by shooting several short segments instead of one full-length film. Edit! Small screen viewing will not show text clearly unless they are large enough. Also bear in mind that fast motion and high contrast might not show very clearly on the small screen.

Now that you have your video, it’s time to compress it. This needs to be done so that the video will be optimized for web and iPod viewing. There are different ways you can do this. If you have a MAC, it’s no big deal. You can compress and export your video using iMovie HD6 simply by clicking the following: FILE&gtExport&gtiPod&gtShare. It’s as easy as that – the software automatically converts the file and shares it on iTunes. For PC users, you can also use QuickTime Pro, version 7.0.3 or higher. Again, simply click Export > Using Quick Time Conversion. The resulting *.m4v file is the one you will upload to your web server. Other options are Sorenson Squeeze and Spasm Software’s Podner.

Describe or tag your movie. This is very important as all the information you supply will be the basis of searching later on. This is how your movie can be found and singled out, out of all the video podcasts out there. If you’re using the latest iMovie version, all you need to do is drag the file onto iTunes and click the  Tab. Then enter the necessary information. You can also change the filename. For PC users, create an XML document. You can find a sample XML file at http://playlistmag.com/features/200…dcast/index.php. Just change the details to suit your movie. When this is done, all you need to do it upload your file to your web server.

Now what? Distribute! Let people know that your video podcast exists. You can do this in a variety of ways. To reach as many people as possible, submit your podcast to iTunes. You can also provide a URL for your podcast by including it in your blog. First create an RSS feed (www.feedburner.com is a good one) so that you can attach your video to your blog. Then create an entry in your blog and then place a link to your video. Of course, if you really want people to notice your work, you have to promote it. Go to websites such as www.podcastalley.com. Now wait for the world to discover the budding film maker in you.

Technorati Tags: technology
Technorati Tags: software
Technorati Tags: apple
Technorati Tags: ipod

Google calls Sitemaps “an experiment in web crawling”. In general, you place a sitemap on to your web server. This helps the crawlers identify the pages you have and add it to the search index. And when you update your site map (after changing a page or adding new content), it also marks the pages you’ve changed and makes a note of what order to review it.

It’s a lot more systematic and efficient than random crawling, where there are no guarantees that the web crawlers will pick up on the most important pages. So even if it requires an additional step for web developers, it’s greater assurance that all the hard work you put into your content actually shows up in the searches.

Google Sitemaps was developed partly to resolve the problem encountered by big websites, wherein web crawlers would skip over some pages and fail to index some of the content. Considering the effect this can have on your search engine optimization efforts, that glitch can have a big impact on your ranking. And for websites that regularly change content (like product sites), you’d want your new stock to actually register—especially when you’re promoting a hot, trendy item.

So in short, to get your site into Google, you’ve got to make it easier for Google to find you—via their map. It’s not as complicated as it sounds. You just need to use a software tool to create a sitemap in Google’s chosen XML format, upload that to your site, and then send a URL notification to Google. And since you’re doing that anyway, make an HTML sitemap for the other search engines (Yahoo, MSN, etc.) and then include a link to it in an unobtrusive corner of your website. Many of those software tools can do that for you.

Is it expensive? Not really. For smaller sites (less than 500 pages) you can find several free Google Sitemap Generators. Look for those that allow you to restrict the directories that are being searched, and give instructions as to which files to index and which to avoid. This can be programmed through the filters and settings. For your convenience, you’ll also want one that lets you include PDF, .doc, .xls, and .zip documents.

Larger sites may require a trained webmaster, who can install a program on the site to help you index it and generate a sitemap. Of course, you can do it on your own (and again, you’ll find many XML sitemap tutorials and software on the Internet), but with 800 pages to handle you probably have other web administration worries to think about. It’s just easier to pass it on to a professional. Or at least, invest in a very efficient XML software that will take care of most of the details and has several tools and functions at its disposal.

Once you’ve chosen a program (or hired someone to set it up for you), you’ll have to create a Google account. They don’t charge for the service, and you’ll want to get features like tracking the status of your Sitemaps and reviewing the diagnostic information for your sites.

Now, you’re ready to conquer Google.

Technorati Tags: software