![[DigitalPoint] App for Cloudflare®](/community/data/resource_icons/8/8750.jpg?1692633961){kind=icon}
[DigitalPoint] App for Cloudflare® 1.9.1.1
- Thread starter digitalpoint
- Start date
digitalpoint
Well-known member
Define, “it”.
ekool
Well-known member
Ahh yes, my bad the Country Flag. Some users don't like the flag that's showing or just don't want people to know.
digitalpoint
Well-known member
Primarily the whole intent is that it’s viewed only by mods/admins. It’s not intended as a way for users to “show off” where they are located. Honestly, I’d remove the ability for normal users to see it.
Basically it’s intended for the users that can see the underlying IP addresses of users (usually mods/admins).
ekool
Well-known member
We have a few sites with international users and it's super handy to show it. Just my 2c. though.
digitalpoint
Well-known member
Ya, it certainly can be done if you want, but there’s not a mechanism to hide it from certain users.
Fullmental
Active member
Just a dumb question about this - finally got around to trying this out in our test environment. We went with a separate bucket so we can easily purge the data and delete the bucket if it doesn't work out. Are we supposed to see this new bucket pop up in the R2 page of the addon, or will it be invisible in the ACP? Because I did not see it pop up when I added the config line.
digitalpoint
Well-known member
If you mean the stats, no… you wouldn’t see stats for things you added manually via config.php.
No idea, never seen it myself and I’m using R2.
MentaL
Well-known member
is it exclusively that image?
securedme
Well-known member
Not only that one. Actually this happens after I resumed the development of a halted forum after returning from a 3 month holiday.
Also, when I clicked on some existing images in messages, the lightbox showed "image not found". Then I right-clicked the message image and have it opened in a new tab, the following appeared:
I'm using the webp feature for uploaded images by the way
Having updated all add-ons didn't help.
So I disabled all add-ons. Inserting images and lightbox images worked. Guess they were stored locally.
After enabling only this add-on, the problem appeared.
digitalpoint
Well-known member
Looks like you are trying to add an image by URL and that image is served with presigned URLs. Presigned URLs are only valid for the user they are given to (so the URL isn’t going to be valid for a user on a different IP address… like the server trying to fetch it).
Presigned URLs are also very short-lived. The error you got there says the URL expired.
Disable presigned URLs and it will probably work better for you.
Presigned URLs are also very short-lived. The error you got there says the URL expired.
Disable presigned URLs and it will probably work better for you.
securedme
Well-known member
Disabling the Presigned urls option solved all problems. Thanks a lot!
Fullmental
Active member
If anyone does try to use this for their image proxy cache, note that you need to sync the contents of your
image_cache folder on your local server to r2:your_bucket/image_cache, unlike the data and internal_data/attachments links in the addon itself that expect to be in the root of those buckets as per the addon FAQ (or, y'know, without any directory prefix before the numbered folder prefixes, because "directories" don't technically exist in R2). I'm not sure why the inconsistency is there, but that threw me for a loop when I first tested it and old proxied images didn't work when I copied them over, but newly proxied ones did. It was only until I inspected the bucket in an S3 browser that I saw that XF had created a new "image_cache" directory inside the bucket.So, for example this was the rclone command I used:
rclone copy /path/to/image_cache/ cloudflare:bucket/image_cache --progress --transfers=32 --checkers=32I'd also recommend using at least 32 checkers and transfers, maybe even 64 to speed up this folder as a lot of images are very small and rclone by default limits itself to just 5 of each). This cut our migration of ~200k images/16GB total file size from ~7 hours in the default config during testing to maybe 1.5 hours in production when we added the extra transfers and checkers. We did not run into any rate limiting errors with 64 checkers ourselves, but limited it to 32 in production to avoid excessive I/O slowing the rest of the site down. YMMV, we're on SSD storage but I'm not sure if it's nvme or if it is what generation/speed it's at.
Last edited:
digitalpoint
Well-known member
It’s also a terrible idea… just throwing that out there. The reason the addon doesn’t do it by default isn’t because it’s a bad idea, not because someone just didn’t think of it.
Imagine putting your browser cache off your computer and thousands of miles away on a remote server. That’s effectively what you are doing there. The cache will end up being slower than not having a cache. But if that’s something you are going for, for whatever reason.. have at it.
Imagine putting your browser cache off your computer and thousands of miles away on a remote server. That’s effectively what you are doing there. The cache will end up being slower than not having a cache. But if that’s something you are going for, for whatever reason.. have at it.
Fullmental
Active member
We have no choice. Our VPS is limited to 80GB of storage - our image proxy needs to retain images indefinitely, and we amassed 15GB in just one day. We expect to be around 200GB total or ~2 million images. In order to get that amount of storage from our VPS provider, we would need to nearly quadruple our server costs to do so, or rely on our VPS provider's block storage which is nearly as expensive due to an unfavorable pricing structure. In our testing it's no slower than fetching the attachment images directly either.It’s also a terrible idea… just throwing that out there. The reason the addon doesn’t do it by default isn’t because it’s a bad idea, not because someone just didn’t think of it.
Imagine putting your browser cache off your computer and thousands of miles away on a remote server. That’s effectively what you are doing there. The cache will end up being slower than not having a cache. But if that’s something you are going for, for whatever reason.. have at it.
Last edited:
digitalpoint
Well-known member
Why does the image proxy need to retain forever? By definition right there, it’s no longer a “cache”.
Fullmental
Active member
Because we've lost over a million historical images linked to our site over the 15 years we've been around, including some that were so popular in our early days we still regularly get complains and requests to "fix" them. We're tired of relying on random external image hosts that our members use for various reasons, that then change policies and delete the images over time, causing our member's comics and screenshot runs to be lost forever, often without the original author's knowledge. And don't get me started on the number of people who still think Discord is a good place to host images with the current 24 hour limit on their URLs, that's a whole other restoration project we need to get moving on before they start purging images for good there too...And frankly I personally am just sick of seeing this all over the forum when I browse old threads:
Now, we can't bring back the vast majority of broken links that already exist, but we can make absolutely sure that we don't allow the number of broken images to grow any longer. We've tried alternative methods to force users to use the attachment system but got major pushback because the interface is so clunky to use when people are trying to insert 100+ images into their posts and need to manage them. And the media gallery is not well-liked either, so we have hundreds of members that are trying to get around that because it's more efficient to use something like imgbb or whatever they are used to.
Our only remaining option that we're aware of is the proxy image cache. So yeah, we're not treating it as a cache. We're treating it as the only way to preserve as much of our site as possible and safeguard against link rot. And that is an intended use case for the cache, the options menu specifically calls out how to use the cache to protect against link rot by setting a refresh time when images are retained indefinitely:
Fun note there, if you use the cache refresh time instead of also disabling it, when your image hosts replace the original images with a stock "this image can't be found" piece instead, it doesn't protect against that and we still lose the image. So that and the proxy lifetime are both set to zero now. We learned to set them the hard way. But I digress...
If we're not supposed to use it this way, why would they tell us to do it? Our members have been asking us for recommendations on an image host that will truly retain their work indefinitely. We don't know of one, because any time we've suggested one in the past it seems the terms always change, and then we're ultimately responsible (due to making the recommendation) when (not if) something breaks. If we're going to own that anyway, we'd better handle it and host it ourselves.
If you do have a better option, please tell us, because this is all we know how to do in a way that will work within our budget.
EDIT: Oh, we've also tried Andy's convert image and convert image all addons to automatically convert linked images to attachments but there were too many bugs and edge cases where it actually destroyed some of our posts and I had to restore them from a SQL backup manually, we can't rely on it. I'm grateful for his addon collection but it seems his code in these in particular just isn't well-suited for our needs, so after several months trying to make it work (all the while link rot still occurring...) we were forced to remove that as well. Nothing we try does the job we need it to do. And, honestly, what's the difference between us loading it through an "attachments" bucket in R2 vs a "proxy" bucket in R2? Same resources, same CDN, same fetch times, same storage and transaction pricing...Am I missing something?
Last edited:
