Sitemap

A file that provides information about the pages, videos, and other files on your site, and the relationships between them.

Examples

XML sitemap
HTML sitemap
Image sitemap

Crawler: A program that systematically browses the World Wide Web in order to create an index of data.
Indexing: The process by which search engines add web pages to their database of searchable content.