Link Checker by AddonsLab [Paid] 1.17.0

[Alpha1]

The link checker only checks the status code. Which will be distorted because the target servers will block the check. So we need humans to go check all those links so that we can really confirm what links are dead.

It would be nice if there would be a function for members to check links and mark it as working or dead. Let them select either a green checkmark or a red X. If its a confirmed dead link then there could be a function to replace it with a suggested link.
And have an admin setting on how to handle dead links. For example:

1. delete
2. replace with default text
3. replace with suggested link

Personally I would just want to automatically remove the confirmed dead links and be done with it. Other admins may have different preferences.

 

[Alpha1]

@AddonsLab your demo admin panel doesnt work. ' The access to this page is denied for demo admin accounts. '

 

[AddonsLab]

Hello, @Alfa1

I see. Yes, the product could never 100% correctly check if the links are dead or no due to the involvement of remote servers.

One thing we plan to implement to let the admins off-load the check to the moderators is the implementation of moderation queue support. Moderators of a forum will see the dead links deleted in their forum, and decide what should be the replacement.

So, as I understand your suggestion is to give permission-based control to all users to mark a link as broken. This can definitely be a powerful tool for checking the dead links, so we would like to implement it. We would like, however, to implement it differently than described. We could show icons for users to mark a link as dead/live. This would increase the "rank" as a dead link. In Batch Update page in admin, you could just have this rank as another criterion, so admins could see only links, which are marked as dead more than 5 times for example, and be able to manipulate the links in batch, setting text and link to whatever they want or delete (the default replacement is already an option in admin panel).

Please let us know your thoughts regarding this.

Admin panel access is fixed now. Please see product options here - https://addonslab.com/admin.php?options/list/allm_main

Thank you!

 

[Alpha1]

Are you interested to add support for major / popular addons? Like:

• XFRM
• XFMG
• Showcase
• Vaultwiki

 

[AddonsLab]

Are you interested to add support for major / popular addons? Like:

• XFRM
• XFMG
• Showcase
• Vaultwiki

We are interested in it, but it is not on our priority list for now. We are planning a large rewrite that will allow us handling other BB Codes as well (media, possibly img), and implement moderation queue and other high-priority tasks. Once the system is stable and all important features are implemented, we will consider expending it to other content types as well.

Thank you!

 

[Nirjonadda]

@AddonsLab Why this URL are not valid? What going on the new implement?

https://translate.google.com/

 

[Nirjonadda]

@AddonsLab What going on? No response here?

 

[AddonsLab]

@AddonsLab What going on? No response here?

Sorry, it seems we missed your reply here. The link mentioned works perfectly for us, both locally and on our demo board - https://addonslab.com/threads/testing-link-checker.134/

Please let us know how you see the problem, is it in Admin Panel or front-end and what is the status exactly. As the issue is most likely specific to your setup or server, please update your ticket in our Support System with the information requested and we will assist you there.

Thank you!

 

[AddonsLab]

Hello, everyone!

This is an update regarding the development of this add-on. We are now working on it and building the upcoming version, that will have support for multiple BB Code tags, particularly URL, IMG, and MEDIA. An important change in the way the product is developed is updating the coding technique to be fully modular, which allows us to put the foundation needed for further XenForo 2.x version development without duplication of the entire system. This does not mean direct compatibility of the product for two versions, but rather a large re-use of its business logic in both platforms, which decreases the number of bugs and allows to apply the generic fixes to both platforms at once.

With this version, we also introduce automatized tests of the code using framework PHPUnit. We target full test coverage and deep testing of the product, which should decrease the number of bugs drastically and prevent regressions when we implement a new feature.

We changed the way the links are stored in the database, making it more optimized, which will be obvious especially if you have many links to the same URL in different posts. This will also solve some issues reported when the same URL is considered as valid in one post and invalid in another.

This version will also contain the most suggested features - moderation queue support, filtering Batch Update results by thread ID and post date, with other front-end enhancements like Admin alerts on a broken link, a user page listing all their broken links coming in the next version.

Please note, that with this release the price for the add-on will be set to 34.99 USD considering its increased feature set. This will not affect current customers, with 25 USD yearly renewal. This will not affect current customers in any way. This will also be the price for XenForo 2.x version, again applying to all new customers, but the existing customers will get a free upgrade to the XenForo 2.x version whenever they decide to upgrade.

We will update here once the release is ready.

Thank you!

 

[Nirjonadda]

Great improvements @AddonsLab Please make separte dashboard for IMG and Media BB code. Showing link are separated.

Example:

1. Mange Link URL
2. Mange IMG BB Code
3. Mange Media BB Code

 

[AddonsLab]

Great improvements @AddonsLab Please make separte dashboard for IMG and Media BB code. Showing link are separated.

Example:

1. Mange Link URL
2. Mange IMG BB Code
3. Mange Media BB Code

This will be a filter available on all current pages, to choose if you want a list of all items, or only one type of items (URL, IMG, MEDIA), just like now you have a filter for the link status.

 

[dutchbb]

Testing on big board now (9mil posts), indexing of posts is going good (30min for 1mil). Will keep posted here how dead link check go and removal/replace of links.

Server load isn't even moving (php 7.1 + php-fpm). Disabled everything at frond-end.

 

[dutchbb]

Took 5 hours to complete 10mil posts (had more then 9?) at standard setting. So pretty fast., found 700k links.

Now testing for dead links, have it set to 10 per run else i got an page not found error or lock wait. But this is as fast as 100 per run. It took 20 seconds per 100 links and per 10 links it is 2 seconds. So lower setting does not mean slower.

Will take a while to test them all, 38 hours in total if i am correct so not to long for 700k links to test

 

[AddonsLab]

Took 5 hours to complete 10mil posts (had more then 9?) at standard setting. So pretty fast., found 700k links.

Now testing for dead links, have it set to 10 per run else i got an page not found error or lock wait. But this is as fast as 100 per run. It took 20 seconds per 100 links and per 10 links it is 2 seconds. So lower setting does not mean slower.

Will take a while to test them all, 38 hours in total if i am correct so not to long for 700k links to test

Thank you for the detailed update.

Exactly, the lower setting does not mean lower speed in general, as most of the time is spent on actual remote requests checking for link validity, and not on page reloads or MySQL queries.

The product checks one URL only one time, so I expect it will finish earlier than calculated, as most likely not all 700k links are unique.

Just let us know here or via a Support Ticket if you need our assistance with anything.

Thank you!

 

[dutchbb]

Would it also be possible to do this with a cron job?

150k checked at the moment btw. Also did some more redirects from an older domain we had, easyer to check with this add-on.

 

[Nirjonadda]

Would it also be possible to do this with a cron job?

Also We needed this functionality, The major featuring of this add-on. Please do this functionality.

 

[AddonsLab]

Would it also be possible to do this with a cron job?

150k checked at the moment btw. Also did some more redirects from an older domain we had, easyer to check with this add-on.

The job is registered as a deferred task in XenForo, and XenForo just allows you to run it from admin panel as well. If you close the window in Admin Panel, the task will still run in the background every time a user visits your board. I think you will see "there are pending tasks" message in Admin Panel whenever you visit admin panel. We will consider enhancing this functionality with showing you on top of admin pages more precise information on how many links are already checked and how many are left. We are almost done with a major upgrade of the product. This part will be also tested and implemented as mentioned. You can expect the release in 3-4 days.

Thank you!

 

[dutchbb]

The job is registered as a deferred task in XenForo, and XenForo just allows you to run it from admin panel as well. If you close the window in Admin Panel, the task will still run in the background every time a user visits your board. I think you will see "there are pending tasks" message in Admin Panel whenever you visit admin panel. We will consider enhancing this functionality with showing you on top of admin pages more precise information on how many links are already checked and how many are left. We are almost done with a major upgrade of the product. This part will be also tested and implemented as mentioned. You can expect the release in 3-4 days.

Thank you!

So i can close my browser when rebuild data deffered task (dead link check)? Normally if you close the window it will stop and will state when opening admin panel again that there are manual rebuild processes stopped and click to continue.

An option to (re)check a specific domain would be great. I added some redirects from a very old domain we had, but like 500 are already checked (404) and doing this manual is a lot of work.

 

[AddonsLab]

Also with rebuild data deffered task? Normally if you close the window it will stop and will state when opening admin panel again that there are manual rebuild processes stopped and click to continue.

I had it run last night but was logged out and process stopped.

And one more thing, an option to (re)check a specific domain would be great. I added some redirects from a very old domain we had, but like 500 are already checked (404) and doing this manual is a lot of work.

Rebuild data uses deferred tasks as well, but I am not completely sure they run automatically but will make sure this works for link checker in the upcoming version. Domain filter will be implemented as well.

Thank you!

 

[dutchbb]

When a url does not have http or https in front it gives this error: 999 Local or invalid URL (URL has unsupported protocol or tries to load local data)

But most of them are working correct, like www.xenforo.com