Robots.txt buat wordpress

user_name · Dec 26, 2009

keren imponya gan..

buka kitab SEO lagi dah.. biar lebih mudheng..

tara reng kayu youwch...

abeng · Dec 26, 2009

Tolong di list juga dong spam bot crawler nya.. Biar bisa antisipasi..

engkoh · Jan 27, 2010

Wah, simpen dulu dh. . Thx master n srandal yg udh refer ke sini.

2mk · Jan 27, 2010

Ini punyaky :

# This rule means it applies to all user-agents
User-agent: *

# Disallow all directories and files within
Disallow: /cgi-bin/
Disallow: /stats/
#Disallow: /tag/
Disallow: /wp-admin/
Disallow: /wp-includes/

# The Googlebot is the main search bot for google
User-agent: Googlebot

# Disallow all files ending with these extensions
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /*.gz$
Disallow: /*.wmv$
Disallow: /*.tar$
Disallow: /*.tgz$
Disallow: /*.cgi$
Disallow: /*.xhtml$

# Disallow Google from parsing indididual post feeds and trackbacks..
Disallow: */feed/
Disallow: */trackback/
Allow: /feed/

# Disallow all files with ? in url
Disallow: /*?*
Disallow: /*?

# The Googlebot-Image is the image bot for google
User-agent: Googlebot-Image

# Allow Everything
Allow: /*

# This is the ad bot for google
User-agent: Mediapartners-Google*

# Allow Everything
Allow: /*

Click to expand...

xilver2dragon · Jan 27, 2010

yang paling bagus diantara 3 contoh yang ada yang mana yah??? pengen nyobain juga

indrajid · Jan 27, 2010

scarface said: ↑

nu pasti namah kitu weh pi
Click to expand...

menurutku jgn disallow semuah.. klo trafik kecil biarin ajah spam bot masuk biar jd trafik.. CMIIW

cute_v3 · Sep 6, 2010

berat juga nih topik..klo di wp, kita masangnya di hostingnya gitu tah?
nubitol bingung

musyrifin · Sep 8, 2010

gan mo nanya neh, struktur web ogut kaya gini:

www.domain.com/wordpress

nah kalo buat robot.txt narohnya dimana gan?mohon bantuannya...........

setyonugroho · Sep 8, 2010

musyrifin said: ↑

gan mo nanya neh, struktur web ogut kaya gini:

www.domain.com/wordpress

nah kalo buat robot.txt narohnya dimana gan?mohon bantuannya...........
Click to expand...

di folder wordpress-nya gan..

alginus · Sep 8, 2010

Wah pada pamer robot ne para master.jd bingung pake yg mana..

dwisucipto · Oct 10, 2010

mantap brow...langsung dicoba

avionick · Jan 23, 2011

banyak erornya aku cek di
http://tool.motoricerca.info/robots-checker.phtml

aleydoank · Jan 23, 2011

wahh.. trit lama bangkit dari kubur.. hahahaha

nenkai · Jan 23, 2011

aleydoank said: ↑

wahh.. trit lama bangkit dari kubur.. hahahaha
Click to expand...

tp sangat membantu buat para newbi nic gan,, btw,, kasih tau cara na yg bgs donk,,,, trus dri k-3 robot yg d atas mna yg bgs mnrut agan?? tnk gan sblumnya

mubaroki · Feb 17, 2011

yang ampuh yang mana nih

khabibf · Feb 17, 2011

scarface said: ↑

nih versi saya..

HTML:

# This rule means it applies to all user-agents
User-agent:  *
 
# Disallow all directories and files within
Disallow: /cgi-bin/
Disallow: /stats/
#Disallow: /tag/
Disallow: /wp-admin/
Disallow: /wp-includes/
  
# The Googlebot is the main search bot for google
User-agent: Googlebot
 
# Disallow all files ending with these extensions
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /*.gz$
Disallow: /*.wmv$
Disallow: /*.tar$
Disallow: /*.tgz$
Disallow: /*.cgi$
Disallow: /*.xhtml$
 
# Disallow Google from parsing indididual post feeds and trackbacks..
Disallow: */feed/
Disallow: */trackback/
Allow: /feed/
 
# Disallow all files with ? in url
Disallow: /*?*
Disallow: /*?
 
# The Googlebot-Image is the image bot for google
User-agent: Googlebot-Image
 
# Allow Everything
Allow: /*
 
# This is the ad bot for google
User-agent: Mediapartners-Google*
 
# Allow Everything
Allow: /*

# This is the ad bot for google
User-agent: Mediapartners-Google*
 
# Allow Everything
Allow: /*# BEGIN XML-SITEMAP-PLUGIN
Sitemap: http://www.domainmu.com/sitemap.xml.gz
# END XML-SITEMAP-PLUGIN

Click to expand...

Set dahh.. kompleks banget... kayanya boleh nih di copas...

Eff Killms · Feb 18, 2011

khabibf said: ↑

Set dahh.. kompleks banget... kayanya boleh nih di copas...

Click to expand...

dari ketiga robot.txt td yg mna nih yg paling optimal ??
ane coba dlu versi atas ane ... hmpir mirip sama dngan yg saya pke saat ini

---------- Post added at 08:11 AM ---------- Previous post was at 08:09 AM ----------

Klu punya ane kayak gini ....

User-agent: Googlebot
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /*?*
Disallow: /*?

User-agent: *
Disallow: /cgi-bin
Disallow: /wp-content/cache/
Disallow: /wp-content/tmp/
Disallow: /wp-content/themes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/
Disallow: /wp-config.php
Disallow: /wp-login.php
Disallow: /wp-register.php
Disallow: /wp-includes/
Disallow: /wp-admin/
Disallow: /author/

# Google Image
User-agent: Googlebot-Image
Disallow: /

# Google AdSense
User-agent: Mediapartners-Google*
Disallow: /

# Adsbot-Google
User-agent: Adsbot-Google
Disallow: /

#Googlebot Mobile
User-agent: Googlebot-Mobile
Disallow: /

# Disable Internet Archiver Wayback Machine
User-agent: ia_archiver
Disallow: /

# Disable digg mirror
User-agent: duggmirror
Disallow: /

Sitemap: http://domainkamu/sitemap.xml
Sitemap: http://domainkamu/sitemap.xml.gz

bintang_kecil · Feb 18, 2011

Robot.txt itu sebaiknya di sesuaikan dg kondisi blog masing2 <<-- sok tahu mode on

inuzuka · Mar 14, 2011

waduh pengen masang tapi taut.. untungnya apa ya pake robot.txt, klo gak cocok ma web kita efeknya apa ya?

ifeytea · Mar 15, 2011

pelajari dulu ah....

Log in or Sign up

Robots.txt buat wordpress

user_name Super Hero

abeng Super Junior

engkoh Super Hero

2mk Super Hero

xilver2dragon Ads.id Pro

indrajid Super Hero

cute_v3 Ads.id Fan

musyrifin Ads.id Fan

setyonugroho Super Hero

alginus Hero

dwisucipto Newbie

avionick Ads.id Pro

aleydoank Super Hero

nenkai Super Hero

mubaroki Super Hero

khabibf Hero

Eff Killms Ads.id Starter

bintang_kecil Super Hero

inuzuka Ads.id Starter

ifeytea Ads.id Fan

Share This Page