Dignity nobility top quality state highness standing class caste standing superiority priority style distinction dominance preferment primacy nobleness gentleness gentility preeminence loftiness station grandness txt file is then parsed and will instruct the robot concerning which internet pages will not be to generally be crawled. As a online search engine https://www.youtube.com/watch?v=imiGL2YS2uo