macropin / django-googlebot

Django middleware to allow googlebot to index logged-in accessible content.

Django middleware to allow googlebot to index logged-in accessible content

Django Middleware to allow Googlebot access to paywalled, or login only content. This Django middleware automatically logs in Googlebot as the googlebot user.

Install

Add to your system / virtualenv:

pip install git+git://github.com/macropin/django-googlebot#egg=django-googlebot

Add to Django settings.py:

MIDDLEWARE_CLASSES = (
    ...
    'googlebot.middleware.GooglebotMiddleware',
)

Finally, create a googlebot user. This account will be used when Googlebot is automatically logged in. If you don't create an account, then one will be created automatically.

NB If you wish to disable the Google Cache feature, create a robots.txt with Noarchive . Eg:

User-agent: *
Disallow:
Noarchive: /restricted-content/