Should I allow /attachments/ in robots.txt?

[VisEntities]

Hey everyone, quick question about SEO and attachments.

I'm trying to understand what exactly falls under /attachments/ in robots.txt.
Right now, my file has this line:

Disallow: /attachments/

I'm not sure what the site is putting under that path, is it just images? Videos? Uploaded files from users? Something else?

Also wondering if it’s a good idea to allow it so search engines can index useful stuff like that, or if it’s better to keep it blocked to avoid thin content or crawl bloat.

Would appreciate any clarification or advice from anyone who’s looked into this!

 

[Max Taxable]

Hey everyone, quick question about SEO and attachments.

I'm trying to understand what exactly falls under /attachments/ in robots.txt.
Right now, my file has this line:

Disallow: /attachments/

I'm not sure what the site is putting under that path, is it just images? Videos? Uploaded files from users? Something else?

Also wondering if it’s a good idea to allow it so search engines can index useful stuff like that, or if it’s better to keep it blocked to avoid thin content or crawl bloat.

Would appreciate any clarification or advice from anyone who’s looked into this!

Nothing is blocked by robots.txt.

Noble crawlers will honor it. The attachments folder will have all images, video and media that is uploaded as attachments in posts by users. It's going to be a huge file potentially. If you want noble crawlers to index it that's fine.