[bd] Attachment Store [Deleted]

[Mouth]

So how on earth can we scan attachments for viruses when these are uploaded to S3 bucket or DO Spaces?
Has anyone figured this out?

There is several products/integrations available for AV scanning S3 buckets. Using SNS topics for new file notifications I believe. Don't know if similar is available for DO spaces, or same products work with both.

 

[Alpha1]

It should work as usual, not cleared why your members complained that way?

With delayed processing it takes 5-15 minutes before the images appear. That may be related to the use of a http proxy. Or just the size / activity of the board.

Some scanner supports URL scanning, so maybe look into that?

It seems to me that this means the url scanner will download all 31GB of attachments for every scan and that an index of the urls is needed.
This seems to defeat the purpose of hosting on a DO space to save bandwidth costs.

There is several products/integrations available for AV scanning S3 buckets. Using SNS topics for new file notifications I believe. Don't know if similar is available for DO spaces, or same products work with both.

Yes, I found one but no mention of DO.
https://cloudonaut.io/antivirus-for-s3-buckets/
It would be great if it would be available for DO spaces as well.

BTW: I did file a suggestion with DO to get a built in scanner:
https://digitalocean.uservoice.com/...estions/34830775-antivirus-scanner-for-spaces

 

[Solari]

I noticed a constant error in our browser consoles, how can we fix this?

"[Error] Failed to load resource: the server responded with a status of 404 (Not Found) (attachment_editor.min.js.map, line 0)"

 

[xfrocks]

It seems to me that this means the url scanner will download all 31GB of attachments for every scan and that an index of the urls is needed.
This seems to defeat the purpose of hosting on a DO space to save bandwidth costs.

Ideally, you should only submit the new URL for scanning. No point to scan the old files over and over again.

I noticed a constant error in our browser consoles, how can we fix this?

"[Error] Failed to load resource: the server responded with a status of 404 (Not Found) (attachment_editor.min.js.map, line 0)"

That error pops up only when you open the console because the browser tries to download the map file which does not exists. We have modified our build to avoid this error, the fix will be included in the next release.

 

[Solari]

That error pops up only when you open the console because the browser tries to download the map file which does not exists. We have modified our build to avoid this error, the fix will be included in the next release.

Thanks for the heads up, will look out for it.

Ray

 

[maxicep]

I have problem about change the amazon s3 bucket details.
I've installed this add-on for a long time and working fine, I'm using the amazon s3 storage.

But i'm experiencing the problem about moving/changing the s3 bucket details.

Because, I switched to another amazon s3 account so my bucket name, api and secret keys are changed. So, i must change the attachment data now. But move attachment data tool doesn't work. I type old bucket details in data moving tool but script can't move so i'm sure that this feature has bug.

I want to help from @xfrocks but he didn't help me yet.

Anyone did change the amazon s3 bucket info on their attachment datas before ?
Thanks

 

[maxicep]

@xfrocks Helped me and fixed the problem so all attachments have been moved to another s3 bucket without any problem. He helped me perfect even connect with team viewer so he is very very care about his products.

Thank you!

 

[mcatze]

Preparing for XF2.1..

Open to /src/addons/Xfrocks/AttachmentStore/XF/Entity/AttachmentData.php and search for:

public function getThumbnailUrl()

Replace it with:

public function getThumbnailUrl($canonical = false)

 

[rdn]

Preparing for XF2.1..

Open to /src/addons/Xfrocks/AttachmentStore/XF/Entity/AttachmentData.php and search for:

public function getThumbnailUrl()

Replace it with:

public function getThumbnailUrl($canonical = false)

Thanks, solved my 1st issue with 2.1 & this addon.

 

[rdn]

On XF 1.5, What is the right way of un-installing this addon?
Without losing any data with this settings:



Thanks!

 

[Vlady]

Preparing for XF2.1..

It doesn't work with digitalocean spaces in xf2.1 for me.
@xfrocks, could you help?

 

[AtomicZombie]

Hello,

I have XF2.1 and Media Gallery, and want to do the following...

• Run the forum on my VPS where it is now.
• Deliver images from Media Gallery from shared hosting (different domain).
• Be able to back up all images using FTP of single directory.

The forum I am migrating from (VB Gallery) does this now.

I looked through this thread, but most are discussing attachments. I do not allow attachments.
Does this addon do what I am asking?

Thanks,
Brad

 

[fingon]

Does anyone knows that the xf2 version of this is working with XFMG 2 now?

thanks

 

[xfrocks]

Does anyone knows that the xf2 version of this is working with XFMG 2 now?

thanks

It works but you need to configure the data adapter if you want to XFMG thumbnail to be uploaded too, see latest update https://xfrocks.com/resources/bd-attachment-store-for-xenforo-2-0.35/update?update=184, basically just add something like these into config.php:

$config['fsAdapters']['data'] = function () {
    return \Xfrocks\AttachmentStore\Fs::getAdapterDefault();
};
$config['externalDataUrl'] = function ($externalPath, $canonical) {
    return 'https://s3.amazonaws.com/bucket/' . $externalPath;
};

 

[Dakis]

Unfortunately when I upgraded from 1.5 to 2.1, some of my attachments have disappeared. I don't know how to even start troubleshooting this. I am using the external /data folder as the point for the attachments, and I didn't change this in 2.1.

Look at two examples, first one works, second they are gone:

https://www.avsite.gr/forum/threads/Το-σαλόνι-που-ατμίζει-dloger.160622/post-2453948


https://www.avsite.gr/forum/threads/Το-σαλόνι-που-ατμίζει-dloger.160622/post-2455869

 

[rdn]

What could be the reason for this errors?



Under XenForo 2.1.3 and executing "Move attachment data".
Attachment ID seems to exist fine.

 

[xfrocks]

What could be the reason for this errors?

View attachment 209507

Under XenForo 2.1.3 and executing "Move attachment data".
Attachment ID seems to exist fine.

How many attachments do you have and how many of those error logs are being produced? It's normal if some files are missing, e.g. if you have 200k attachments and lost 100 of them, it's expected.

 

[rdn]

How many attachments do you have

527,237 items right now.

how many of those error logs are being produced?

2 pages of error log I think.

It's normal if some files are missing

Based on those Data # ID, I can still load the attachment fine.

 

[xfrocks]

527,237 items right now.


2 pages of error log I think.


Based on those Data # ID, I can still load the attachment fine.

That's about 40 errors for 500k files, 0.8% -- not too bad.

Based on those Data # ID, I can still load the attachment fine.

Oh, the attachment ID is different from data ID. If you want to find the attachment ID for data ID 123, run this MySQL query:

SELECT attachment_id FROM xf_attachment WHERE data_id = 123;

 

[rdn]

Oh, the attachment ID is different from data ID. If you want to find the attachment ID for data ID 123, run this MySQL query:

I'll test this now from the Data ID screenshot above.

Yeah, seems to be missing:

This attachment cannot be shown at this time. Please try back later.

How to clean all those orphaned attachments?