class Aws::BedrockAgent::Types::WebCrawlerLimits

@see docs.aws.amazon.com/goto/WebAPI/bedrock-agent-2023-06-05/WebCrawlerLimits AWS API Documentation
@return [Integer]
host.
The max rate at which pages are crawled, up to 300 per minute per
@!attribute [rw] rate_limit
@return [Integer]
sync will fail and no web pages will be ingested.
25,000 pages. If the web pages exceed this limit, the data source
The max number of web pages crawled from your source URLs, up to
@!attribute [rw] max_pages
authorized to crawl the URLs.
The rate limits for the URLs that you want to crawl. You should be