How to get into robots.txt?

califax
10 years ago

0

So when I go to robots.txt of one site, it tells me that I dont have permission to see that. How to get there? Thanks for every advice, I tried to find that out but if it was here Im really sory

tl0tr
10 years ago

0

It should be in the root of the website

dloser
10 years ago

0

If you don’t have access to it, it either doesn’t exist or something’s probably not configured right. Most likely there is nothing you can do about it (except perhaps letting the owner know).

califax
10 years ago

0

[quote=dloser]Most likely there is nothing you can do about it[/quote]
So there is no way to get into it? I dont know but….

Reply has been removed

HyunMi
10 years ago

0

can you maybe just specify the site so we can check it out

dloser
10 years ago

0

As tl0tr mentioned, there is only one place where it should be: the root.

You’ll never hear me say that there is no way to get it, because it’s not impossible that there is, for example, some vulnerability that can be exploited. It’s just not that likely and definitely not worth the effort in general. I have done no research at all, but I don’t expect you’d find anything interesting in most cases.

Cyan Wind [freewind1012]
10 years ago | edited 10 years ago

0

The file is supposed to be read by bots/crawlers so, well, you can understand the website structure a bit better if you can find one. However, as @dloser said above, nothing is interesting enough.

![Image](https://www.hackthis.co.uk/user/userbar.png?user=freewind1012%20//%20Navigator)

Reply has been removed