GoodForNothing Image Optimizer [Paid] [Deleted]

[Nuno]

The way Convert Image does things is not compatible with GFNIo.

Hi,

I have thousand of images converted with MetaMirror and Convert Image.
Some where resized with Resize image from Andy others with the default Xenforo/XFMG upload system

Are you saying that images that where rehosted/imported with MetaMirror/Convert Image can't be optimized?

Thanks

 

[Mr. Goodie2Shoes]

I am using other addons which could affect attachments, although I ran your previous GFN Kraken addon recently without any problem.

Then the next version (1.0.3) will hopefully work fine for you.

Are you saying that images that where rehosted/imported with MetaMirror/Convert Image can't be optimized?

What I meant by that is: This add-on ports in XenForo's Attachment Model to fetch the file and process them/put them in queue. The way Convert Image does things, the add-on will not be able to port into that. But running the image rebuild should compress those fetched images.

 

[Mr. Goodie2Shoes]

Mr. Goodie2Shoes updated GoodForNothing Image Optimizer with a new update entry:

New Image Handlers & Code Refactoring

This version adds support for two new image handlers:

• Imagify.IO - https://imagify.io/ as requested by @drastic
  
• Ewww.IO - https://ewww.io/ as requested by @Mouth
  Special shout-out to Shane for the developer license

The code has also be refactored to fix issues regarding attachments getting corrupted. Now every line of execution is handled carefully and a temp image is created which will be...

Read the rest of this update entry...

 

[Sunka]

The next version I am working will skip the images that have already been optimized.

Is that covered in this 1.3 version?

 

[Mr. Goodie2Shoes]

Is that covered in this 1.3 version?

Yes, yes... forgot to add it to the resource update

 

[Mouth]

v1.0.3
Could post attachments being miscalculated/handled incorrectly?

Forum Statistics
Messages:2,067,225
Number of Attachments:59,679
Attachment Disk Usage (GB):13.86

... vs ...

Statistics
Images Optimized:14,101
Disk Space Usage:1.9 GB
Disk Space Saved:649.6 MB
Percentile:24.80%

Enabled Image Handlers
GIF:Gifsicle
JPEG:JpegOptim
JPEG2000:JpegOptim
PNG:OptiPng

I have rebuild all image content types. XF is saying 60k attachments for 14Gb, whereas GNF IO is saying 14k images for 2Gb.


With at least 95% of my site attachments being images, I would have though GFN IO stats (images optimised) would have been much higher.

 

[Mr. Goodie2Shoes]

I have rebuild all image content types. XF is saying 60k attachments for 14Gb, whereas GNF IO is saying 14k images for 2Gb.

What does this SQL return?

SELECT COUNT(*) from xf_attachment WHERE content_type IN ('user', 'conversation_message', 'post', 'resource_update', 'xengallery_media');

With at least 95% of my site attachments being images, I would have though GFN IO stats (images optimised) would have been much higher.

This yields the best results if most of them are uncompressed. And what is the quality set for jpegoptim?

 

[Mouth]

What does this SQL return?

SELECT COUNT(*) from xf_attachment WHERE content_type IN ('user', 'conversation_message', 'post', 'resource_update', 'xengallery_media');

mysql> SELECT COUNT(*) from xf_attachment WHERE content_type IN ('user', 'conversation_message', 'post', 'resource_update', 'xengallery_media');
+----------+
| COUNT(*) |
+----------+
|    61035 |
+----------+
1 row in set (0.03 sec)

This yields the best results if most of them are uncompressed. And what is the quality set for jpegoptim?

90

Gifsicle = Level 2
OptiPNG = Level 1

 

[Mr. Goodie2Shoes]

61035

Wait... this one's even higher than what your forum stats shows:

59,679

What is the handler error log count?

 

[Mouth]

Wait... this one's even higher than what your forum stats shows:

Going up, /admin.php now at 62,389.

What is the handler error log count?

67,757

 

[Mr. Goodie2Shoes]

67,757

Yup, that's the culprit. What are some of the error messages?

 

[Mouth]

Yup, that's the culprit. What are some of the error messages?

Lots of ...

Message: File size increased after optimization. The original data is retained.
user(55353), Today at 14:53

Message: File size increased after optimization. The original data is retained.
image_proxy(2612), A moment ago

Error Info
Message: proc_open(): fork failed - Cannot allocate memory
post(32650), Today at 15:07
Extra Data
array(5) {
  ["error_date"] => int(1477793256)
  ["error_type"] => string(14) "ErrorException"
  ["file"] => string(40) "library/GFNIo/ImageHandler/JpegOptim.php"
  ["line"] => int(81)
  ["trace_string"] => string(1176) "#0 [internal function]: XenForo_Application::handlePhpError(2, 'proc_open(): fo...', '', 81, Array)
#1 library/GFNIo/ImageHandler/JpegOptim.php(81): proc_open(''/usr/bin/jpego...', Array, NULL, NULL, NULL, Array)
#2 library/GFNIo/ImageHandler/Abstract.php(56): GFNIo_ImageHandler_JpegOptim->_optimize(Object(GFNIo_Image))
#3 library/GFNIo/Application.php(74): GFNIo_ImageHandler_Abstract->optimize(Object(GFNIo_Image))
#4 library/GFNIo/StorageHandler/Attachment.php(24): GFNIo_Application::optimize(Object(GFNIo_Image))
#5 library/GFNIo/Deferred/ProcessQueue.php(54): GFNIo_StorageHandler_Attachment->processFromQueue(Array)
#6 library/XenForo/Model/Deferred.php(295): GFNIo_Deferred_ProcessQueue->execute(Array, Array, 8, '')
#7 library/XenForo/Model/Deferred.php(429): XenForo_Model_Deferred->runDeferred(Array, 8, '', false)
#8 library/XenForo/Model/Deferred.php(374): XenForo_Model_Deferred->_runInternal(Array, 8, '', false)
#9 deferred.php(23): XenForo_Model_Deferred->run(false)
#10 {main}"
}

The latter looks to be the issue.
php (php-fpm 7.0) running/active memory_limit = 386M

 

[Mr. Goodie2Shoes]

Message: File size increased after optimization. The original data is retained.

Meaning the images were already optimized. So this error can be safely ignored.

Message: proc_open(): fork failed - Cannot allocate memory

386M should've been more than enough. Does this error persist if that single image is requeued?

 

[Mouth]

Does this error persist if that single image is requeued?

How is this achieved?

 

[Mr. Goodie2Shoes]

How is this achieved?

Select the checkbox that's on the right hand side of the error log and then hit the "Requeue Selected" button at the bottom of the list.

 

[Mr. Goodie2Shoes]

Mr. Goodie2Shoes updated GoodForNothing Image Optimizer with a new update entry:

Fixed a serious issue with Attachment Store

The image was getting removed from the system if "Maintain File Name" was enabled for External and FTP upload. The issue was from the fact that the file name (if that option is enlabed) was being handled by [bd] Attachment Store differently which I didn't notice and apologies for that.

Thank you @Axel B for bringing this up to me on a short notice and being cooperative during the debugging process.

Read the rest of this update entry...

 

[Mouth]

386M should've been more than enough. Does this error persist if that single image is requeued?

No, but perhaps it's not processing it?

Message: File size increased after optimization. The original data is retained.
post(33848), A moment ago

 

[Robru]

After the last update 1.0.4, processes the items no longer to 100, but one by one.





How to stop:


This script can't stop the rebuild proces:

DELETE FROM xf_deferred WHERE unique_key = 'gfnioq';

 

[Sunka]

If I understand correct, images are optimised, but still there is queue filled with this (30 hours and growing), and error list to.
Is there could be process of requeueing If I press Rebuild Attached Images and Resume Image Optimization?
If that is problem, how could I stop process?

 

[Mr. Goodie2Shoes]

@Sunka You'll have to clear the queue list

TRUNCATE TABLE gfn_io_image_queue

and delete the deferred process

DELETE FROM xf_deferred WHERE unique_key = 'gfnioq';

The reason why the stats got messed up may be because you've cleared the process log and reran the "Rebuild Attached Images" as the system checks inside the table of process log to see if the image was processed or not. If not then it would be added to the queue. Since clearing the log basically truncates the table the system thinks the image as a new one.