jwm.robotstxt.googlebot.RobotsParseHandler¶

class jwm.robotstxt.googlebot.RobotsParseHandler¶

Bases: pybind11_object

Handler for directives found in robots.txt. These callbacks are called by ParseRobotsTxt() in the sequence they have been found in the file.

__init__(self: jwm.robotstxt.googlebot.RobotsParseHandler) → None¶

HandleAllow(self: jwm.robotstxt.googlebot.RobotsParseHandler, line_num: int, value: str) → None¶

HandleDisallow(self: jwm.robotstxt.googlebot.RobotsParseHandler, line_num: int, value: str) → None¶

HandleRobotsEnd(self: jwm.robotstxt.googlebot.RobotsParseHandler) → None¶

HandleRobotsStart(self: jwm.robotstxt.googlebot.RobotsParseHandler) → None¶

HandleSitemap(self: jwm.robotstxt.googlebot.RobotsParseHandler, line_num: int, value: str) → None¶

HandleUnknownAction(self: jwm.robotstxt.googlebot.RobotsParseHandler, line_num: int, action: str, value: str) → None¶: Any other unrecognized name/value pairs.

HandleUserAgent(self: jwm.robotstxt.googlebot.RobotsParseHandler, line_num: int, value: str) → None¶

ReportLineMetadata(self: jwm.robotstxt.googlebot.RobotsParseHandler, line_num: int, metadata: jwm.robotstxt.googlebot.RobotsParseHandler.LineMetaData) → None¶