Hello all.
When my site was migrated from vB to XF...some custom member badge images came along for the ride. When I scan the site for errors...these badge image file URLs are showing up quite a bit in the scans. I would like to block them from the Google crawler. A couple other misc URLs thrown in as well I would like to block.
Here are the URLs:
I did some research on how to block things in the robots.txt. Kind of new to this...some of it was confusing. Want to get it right...so I don't end up blocking lots of possibly important stuff.
What lines would I need to add to block each of the 5 URLs above in the sites robots.txt? Just want to block these URLs. If each one requires a separate line in the robots.txt...that would be ok.
Thanks for the help![Smile :) :)](https://cdn.jsdelivr.net/joypixels/assets/8.0/png/unicode/64/1f642.png)
When my site was migrated from vB to XF...some custom member badge images came along for the ride. When I scan the site for errors...these badge image file URLs are showing up quite a bit in the scans. I would like to block them from the Google crawler. A couple other misc URLs thrown in as well I would like to block.
Here are the URLs:
I did some research on how to block things in the robots.txt. Kind of new to this...some of it was confusing. Want to get it right...so I don't end up blocking lots of possibly important stuff.
What lines would I need to add to block each of the 5 URLs above in the sites robots.txt? Just want to block these URLs. If each one requires a separate line in the robots.txt...that would be ok.
Thanks for the help
![Smile :) :)](https://cdn.jsdelivr.net/joypixels/assets/8.0/png/unicode/64/1f642.png)