(With changeset 1440 there exist a new way, to setup context["robots"])
The context variable "robots" are set to "index,follow" or "NONE,NOARCHIVE". It's for the html head line, who locks like this:
PyLucid set this value dynamically. Every non anonymous response should set the value to "NONE,NOARCHIVE" and all views for anonymous users to "index,follow".
Here a chronological way for setup context["robots"]:
context_processors.static set context["robots"] to "index.follow" and index._get_context() set request.anonymous_view = True
plugin_manager._run() change request.anonymous_view if one plugin is not viewable for anonymous.
any plugin can set request.anonymous_view = False, too (e.g. the login view)
index._render_cms_page() set context["robots"] to "NONE,NOARCHIVE" if request.anonymous_view == False