XF 2.1 internal_data attachment limitation

[ChrisTERiS]

Hello,

I'm coding a script to move data from a custom PHP script to xenForo 2.1.6. So far so good, everything works fine, except something which appears to be a limitation.

In details. I'm moving photos to internal_data/attachments/0/ and a thumbnail to data/attachments/0. The problem is that when I reach 9999 files, nothing else save to those directories. I'm not getting any error (have display_errors on), the rest thread data are saving to database, the procedure continues without any break.

So my question is, if there is a limitation on the amount of the files that I can store there. Maybe I need to count the files and every 9999 files to create a new directory like internal_data/attachments/1/ etc? If yes then do I need to update anything to table xf_attachment_data? I can see a field file_path there. Maybe I need to add 1 or 2 etc corresponding to the new folder in internal_data/attachments/ ?

Thank you
Chris

 

[Paul B]

XF automatically limits files per directory to 1000.

You should replicate the same behaviour in your script.

 

[TickTackk]

internal_data/attachments/0/

^ That and this:

data/attachments/0

Group or the /0/ will change every 1000 attachments in the database. Take a look at method starting from src/XF/Entity/AttachmentData.php:60.

 

[ChrisTERiS]

@Brogan @TickTackk

Thank you both !! Is ok, I can follow this style creating a new folder every 1000 files, and if I understood well, xenForo automatically recognizes the path, or do I need to set the folder name in file_path field?

 

[TickTackk]

XF will not automatically detect files in any of those directories and add them to the database.

 

[ChrisTERiS]

XF will not automatically detect files in any of those directories and add them to the database.

Got it, thank you very much !!

 

[ChrisTERiS]

@TickTackk
If you don't mind, could you please tell me something. I did the transfer, each directory /0/, /1/ etc created ok in both data/attachments/ and internal_data/attachments/, each folder has max 1000 photos.

The only problem is that I've is that while all thumbnail to whichever directory they exist, appear fine, the big photo with lightbox has an issue. Photos which exist in /0/ folder appear fine. All other photos that exist in /1/, /2/ etc have broken image link.

Any idea what can causes this issue? Maybe the number infront of filename must be reset each time that I'm changing directory. eg the first file in /1/ directory must be 1-filename or 1001-filename? I'm using the value of data_id, so the the filenames are 1001-filename, 1002-filename etc.

Thank you
Chris

 

[TickTackk]

AFAIK the format is something like {data id}-{flash hash}.data.

 

[ChrisTERiS]

Thank your for your quick response. Really appreciate your help.
Unfortunately, this is what I've, and it works fine for photos in /0/ directory, but not in /1/, /2/
That {flash hash} is typo? Because I'm working with file_hash.data

 

[TickTackk]

Both are same; I just didn't use _. What response do you get when you open the full attachment?

 

[ChrisTERiS]

The photo below is in /0/ folder, has file_hash: 5c65f152ad074c8c5e8fd30545229b8b file_path: 0 and works fine in both thumbnail and lightbox

<div class="attachment-icon attachment-icon--img">
    <a href="/index.php?attachments/056a3121c256aae8f36f56492164f297-jpg.9/" target="_blank" class="js-lbImage">
        <img src="/hfs/data/attachments/0/9-5c65f152ad074c8c5e8fd30545229b8b.jpg" alt="056a3121c256aae8f36f56492164f297.jpg" />
    </a>
</div>

unlike photo in /1/ folder, with file_hash: ffb89fdb592a7d9d8fa44e0e471c4e57 file_path: 1 which works fine as thumbnail, but in lightbox appears as borken link

<div class="attachment-icon attachment-icon--img">
    <a href="/index.php?attachments/cbed358ba4456365a8a902259e0f3e5b-jpg.1353/" target="_blank" class="js-lbImage">
        <img src="/data/attachments/1/1353-ffb89fdb592a7d9d8fa44e0e471c4e57.jpg" alt="cbed358ba4456365a8a902259e0f3e5b.jpg" />
    </a>
</div>

 

[fahad ashraf]

Second one doesn't have hfs prefix before /data

 

[ChrisTERiS]

Second one doesn't have hfs prefix before /data

Εagle's sight But it has it. I just removed the folder name as it's not my own site and I forgot to remove it from that path.

 

[ChrisTERiS]

Here is the actual link

Classifieds

Till around page 20 photos are in /1/ folder, so only thumb works. Lightbox shows broken link.
From page 20 and then, photos are in /0/ folder and everything works fine.

Keep in mind that all those created with the same php code, so I count as impossible some to be wrong and some others not.

 

[ChrisTERiS]

Table xf_attachment has 2 fields that I left out on importing. temp_hash and unaccociated. Actually I don't know what values to store there.

 

[TickTackk]

<div class="attachment-icon attachment-icon--img">
    <a href="/index.php?attachments/cbed358ba4456365a8a902259e0f3e5b-jpg.1353/" target="_blank" class="js-lbImage">
        <img src="/data/attachments/1/1353-ffb89fdb592a7d9d8fa44e0e471c4e57.jpg" alt="cbed358ba4456365a8a902259e0f3e5b.jpg" />
    </a>
</div>

Why is it trying to look into the data directory? You should really look into how the attachment system works.

 

[ChrisTERiS]

Why is it trying to look into the data directory? You should really look into how the attachment system works.

Honestly, have no idea Actually, I know nothing about xenForo. Just trying to import data from a vBulletin classifieds mod. I'm just saving data to database. Have no any deal with templates and links.

Edited: I'm doing it just for education purposes. Actually my work is just to assign relationship between old database structure and new database structure. eg This_old_field value must goes to This_new_field value. But I like to play around

 

[ChrisTERiS]

If it didn't worked at all, I had to say "Sorry, don't know the system". But that issue, where everything works fine for photos in /0/ folder, but semi works in other folders, struggles me

 

[ChrisTERiS]

Just in case that someone reads this thread in the future, the file_path field in xf_attachment_data table, does not needs any value. I was adding there the folder number.