??????? ??? ???? ????????? ?? ??????? ???? ??? ????? ??? ????? ???? ??? ??????? ???? ?? ????????? ??? ??????????.Make robots.txt file to authenticate and end crawler from crawling something other than your property website page.?? ?????????? ??? ?? ???????? ????????? ????? ?????????? «?????????????», ???? ??? ????????????? ?? ??? ??????? ????????