Post Content Find / Replace 1.0.0

[EQnoble]

Well if you have 8 million posts it stands to reason that you could easily have 20-100k img tags within post content.

Given that potential circumstance and knowing you used Jake's edit it's highly probable that this indeed did match the 5000 result limit of posts containing [img] (as set by limit in the edit you made) and then when it went to run the regex across those 5000 popsts there is an equally great chance that out of the first 5000 matched quickfind posts that there were no imageproxy.php images.

To test this theory you have a few options.

You could either:

increase the limit in the edit from 5000 to 10000​

or

you could change the value of offset to '5000' (leaving limit at 5000) so that it skips over the posts you already know don't have matches and tries the next 5000 quickfind matches​

(in this post I gave instructions for how to add options to the admin interface rather than editing)

or

you could try to change you quickfind value to 'imageproxy.php' and ensure only posts with that string are included limiting out the potential thousands of posts you have with image tags that are not using the imageproxy.php link.​

 

[The Dark Wizard]

Well if you have 8 million posts it stands to reason that you could easily have 20-100k img tags within post content.

Given that potential circumstance and knowing you used Jake's edit it's highly probable that this indeed did match the 5000 result limit of posts containing [img] (as set by limit in the edit you made) and then when it went to run the regex across those 5000 popsts there is an equally great chance that out of the first 5000 matched quickfind posts that there were no imageproxy.php images.

To test this theory you have a few options.

You could either:

increase the limit in the edit from 5000 to 10000​

or

you could change the value of offset to '5000' (leaving limit at 5000) so that it skips over the posts you already know don't have matches and tries the next 5000 quickfind matches​

(in this post I gave instructions for how to add options to the admin interface rather than editing)

or

you could try to change you quickfind value to 'imageproxy.php' and ensure only posts with that string are included limiting out the potential thousands of posts you have with image tags that are not using the imageproxy.php link.​

I've raised the numbers even more then that, nothing. I am using imageproxy.php in the quickfind.

This is what the database spits back:

In-regards to how many posts I have with the content I want fixed. Is there no way to do this directly bypassing this tool? Just seems like I passed a certain level, since I 'm in the millions.

I'm not familiar with the queries I'd need.

 

[EQnoble]

I would need more information specific to your needs before I just arbitrarily and irresponsibly give you answers and with such a large DB to work with I would rather not poke around on your site with regex by relay.

If you have a clone of your site setup with all private content removed I can take a look for you and see if I can find a problem.

 

[The Dark Wizard]

I would need more information specific to your needs before I just arbitrarily and irresponsibly give you answers and with such a large DB to work with I would rather not poke around on your site with regex by relay.

If you have a clone of your site setup with all private content removed I can take a look for you and see if I can find a problem.

I have been doing everything on a clone .

This is all in preparation to migrating to xF. What exactly do you need me to send to you? I'll send a PM.

 

[EQnoble]

Smart man.

The short list for me to debug a potential problem with this addon goes as such:

• Login credentials to the ACP so I can get hands on the addon.
• Links to 1 or 2 posts with the example you are trying to find with the subject string pointed out clearly
• What those 1 or 2 example subject strings should be converted to once complete

As a side note if you implement the changes in that small link in the post of mine a few back I can manipulate the values of the edits without needing FTP access to edit the addon file on your server.

If I need anything else beyond that we can communicate in PM.

 

[The Dark Wizard]

Smart man.

The short list for me to debug a potential problem with this addon goes as such:

• Login credentials to the ACP so I can get hands on the addon.
• Links to 1 or 2 posts with the example you are trying to find with the subject string pointed out clearly
• What those 1 or 2 example subject strings should be converted to once complete

As a side note if you implement the changes in that small link in the post of mine a few back I can manipulate the values of the edits without needing FTP access to edit the addon file on your server.

If I need anything else beyond that we can communicate in PM.

PM sent !

 

[Movie Prop Sites]

Our forum have been migrated a couple of times, once prior to my buying it. That migrations left quotes like this:

<div class='quotetop'>QUOTE(Sarge Christi @ Aug 28 2007, 07:36 PM) <{POST_SNAPBACK}></div><div class='quotemain'>I thought about that, but, the brutes might cross paths, which would be bad for cloning, right?</div>

I am not familiar enough with regular expressions to be able to create a find/replace string that would address quotes like this. Would it be possible for someone to help?

 

[Movie Prop Sites]

Well... I finally found my own answer:

Quick Find: <div class='quotetop'
Regular expression: #\<div class='quotetop'>QUOTE\(([^)]+)(?: @ )(?:[^>]+)>(?:[^>]+)>(?:[^>]+)>([^<]+)</div>#siu
Replacement String: [QUOTE="\1"]\2[/QUOTE]

 

[Brent W]

We are doing an IPB import and have quite a bit [SIZE=XX] tags that aren't parsed. What would I need to use here to remove these from posts.

 

[Breixo]

Hi, is there any list of examples? I tried looking for my solution (or something near) at this thread by it's quite complicated.
I'm trying to redirect old urls to the actual ones, but I'm able only to change the code inside the [url not the whole link.
My attempt:

https://www.movilesdualsim.com/foro/threads/
#https:\/\/www\.movilesdualsim\.com\/foro\/threads\/([0-9]+)-([^./]+")#siU
https://www.movilesdualsim.com/tema/\1/"

From this:
https://www.movilesdualsim.com/foro/threads/18157-Como-instalar-Gapps-en-un-Bedove-x12-rooteado

To this:
https://www.movilesdualsim.com/tema/como-instalar-gapps-en-un-bedove-x12-rooteado.18157/

Or at least this:
https://www.movilesdualsim.com/tema/18157/

Thanks in advance!

 

[EQnoble]

Hi, is there any list of examples? I tried looking for my solution (or something near) at this thread by it's quite complicated.

I actually remember doing this for someone already though I admit I am not sure if it was here publicly or privately and because I probably have close to 400 posts in this thread and because I have about 850 separate convos in my inbox with individual conversations having up to 650 back and forth messages...I am not really sure where to start looking. I assume you have a similar problem when looking through this thread.

The irony is here is that to find what you need concerning the regex within post content, you need a regex style fuzzy logic search feature.

I'm trying to redirect old urls to the actual ones, but I'm able only to change the code inside the [url not the whole link.
My attempt:

https://www.movilesdualsim.com/foro/threads/
#https:\/\/www\.movilesdualsim\.com\/foro\/threads\/([0-9]+)-([^./]+")#siU
https://www.movilesdualsim.com/tema/\1/"

From this:
https://www.movilesdualsim.com/foro/threads/18157-Como-instalar-Gapps-en-un-Bedove-x12-rooteado

To this:
https://www.movilesdualsim.com/tema/como-instalar-gapps-en-un-bedove-x12-rooteado.18157/

Or at least this:
https://www.movilesdualsim.com/tema/18157/

Thanks in advance!

Always put example content from your site in [CODE][/CODE] tags to preserve it as is, if the fix for this particular issue wasn't so simple it could present big problems.

Also, when getting the post content at your site, click "edit" on the post and switch to the "bbcode editor view" which will display all tag info rather than the rendered output which is what anyone offering help will need to best help you.

Preferably I like to have the entire tag with link as an example because there is always the possibility that I will see something that needs to be accounted for and that you would not be looking for .

That said, I can't afford the time for many long winded explanations which I do so love giving but I see that you are making your own attempts and I haven't assisted you in about 4 years so I will throw something out there that may work for you or that might establish a base for you or anyone else to modify and test.

Always remember to test WITHOUT SAVING and REVIEW the results before finalizing changes to your DB and ALWAYS take a backup before you begin.

Good luck.

! UNTESTED

Quick Find:

movilesdualsim

Regular expression:

#(https://www\.movilesdualsim\.com/)foro/threads/([0-9]+)-([a-z0-9-%]+)#si

Replacement String:

\1tema/\3.\2

 

[Breixo]

Great, thanks a lot @EQnoble
Sure, it's quite difficult to search for a solution here, the only way is page by page
That's why a list of examples such as at conditionals would be so helpful.

I tried your solution, there's only one problem related to romanized titles, when regex found an accent, it doesn't continue with the rest of the title:

The replacement url works, however not as smart as having the complete title or none.
The coded example is:

[URL='https://www.movilesdualsim.com/foro/threads/243-Bater%C3%ADa-para-clones-del-iphone-sciphone-i68']Batería para clones del iphone - sciphone i68+[/URL]

My guess is that

([a-z0-9-]+)

should be changed in order to admit also those characters.
However, after google I can't find the right code to get the whole rango.

The alternative, if none can be found, is to change the replacement string to just:

\1tema/\2

Or to keep with your solution despite the urls with accents (those are not the majority and they also work...).

I'll wait for your reply in case you know an alternative before changing it.

Thanks for your help!

 

[EQnoble]

Great, thanks a lot @EQnoble
I tried your solution, there's only one problem related to romanized titles, when regex found an accent, it doesn't continue with the rest of the title:

I actually purposely tried to not give you the whole answer, didn't want to take away a chance for you to learn stuff if that is what you were trying to do. ;-)

My guess is that

([a-z0-9-]+)

should be changed in order to admit also those characters.

Your guess is indeed correct, the match breaks as soon as it hits a % (per your example).

The alternative, if none can be found, is to change the replacement string to just:

There are always alternatives but as I pointed out when I posted above I was being cautious to not give you an entire answer because thinking about the problems myself is how I learned and I would not want to do you the injustice of removing a valid opportunity for learning regex which carries over to just about every programming language that is not esoteric.

I have updated the expression above for you with a fix in the area you pointed out (take note of the changes and predict what they will effect before testing it), remember to keep an eye on your result list without saving, you may still come up with more match cases not satisfied.

 

[Breixo]

Thanks @EQnoble
I tried with that example and work for most exceptions. However, some new elements appear like famous Spanish ñ:

[URL]https://www.movilesdualsim.com/foro/threads/28085-Problema-Envio-a-España-(Despatched-to-overseas-(Country-code-UA[/URL]

I tried playing around with

. = any char
\. = the actual dot character
.? = .{0,1} = match any char zero or one times
.* = .{0,} = match any char zero or more times
.+ = .{1,} = match any char one or more times

but was unsuccessful

Like:

([0-9]+)-([.*]+)#si
([a-z0-9-.]+)
([a-z0-9-.*]+)
([a-z0-9-ñ]+) (but stops there)

Any extra idea?

 

[EQnoble]

Thanks @EQnoble
I tried with that example and work for most exceptions. However, some new elements appear like famous Spanish ñ:

[URL]https://www.movilesdualsim.com/foro/threads/28085-Problema-Envio-a-España-(Despatched-to-overseas-(Country-code-UA[/URL]

When you say appear I assume you mean these have been in your posts this way and you are discovering these problems when running the expression? These seems like a problem cause by import or something which could have improperly encoded the string.

(ñ) being display as (ñ) appears to be a symptom of a larger problem that would not be fixed with this addon. If the ñ is displaying as ñ it points to being encoded as something other than UTF-8.

An explanation: http://xenforte.com/regex/breixo.php

I have methods to work around this but they are tricky and I will not post them because they take too much time on the back and forth to explain how to do and what to look for.

However if you would like me to check a few expressions on your site myself I can determine very quickly without wasting much of my time if we can solve this issue using this addon without needing to know the exact root of the encoding issue and without compounding it.

 

[Breixo]

Thanks a lot again for your help @EQnoble
Sure, probably I made it wrong when importing those urls.
Fortunately, that only happens in old urls, not a problem with the new ones.
My aim with these replacements are mostly to avoid 404 errors, as they are not heavy-used threads/links, it is not such a problem to make it just
\1tema/\2 with those improperly encoded.
I really appreciate your help but don't want to take more of your valuable time.
I'll look further during the weekend but probably will go with your previous solution.
Thanks!!

 

[EQnoble]

Thanks a lot again for your help @EQnoble
Sure, probably I made it wrong when importing those urls.
Fortunately, that only happens in old urls, not a problem with the new ones.
My aim with these replacements are mostly to avoid 404 errors, as they are not heavy-used threads/links, it is not such a problem to make it just
\1tema/\2 with those improperly encoded.
I really appreciate your help but don't want to take more of your valuable time.
I'll look further during the weekend but probably will go with your previous solution.
Thanks!!

No don't use the old one, keep trying if you have the weekend. It is totally possible to keep the full link.

I will give you a hint , you need to run two completely different expressions to not break anything.

 

[Alpha1]

I would like to replace:

 [IMGL=white]https://my-forum.com/forum/attachment.php?attachmentid=53915&stc=1&d=1485187416[/IMGL]

with:

[FLOAT_LEFT]https://my-forum.com/attachments/53915/[/FLOAT_LEFT]

Does anyone have an idea how?

 

[eberkund]

@Alfa1

Quick find:

[IMGL=white]https://my-forum.com/forum/attachment.php

Regex:

\[IMGL=white\]https:\/\/my-forum\.com\/forum\/attachment\.php\?attachmentid=(\d*)

Replacement:

[FLOAT_LEFT]https://my-forum.com/attachments/$1/[/FLOAT_LEFT]

 

[SeaDog]

I have a number of large multi-line tables in several threads after an import, how would I go about removing or converting those? I've used Brogan's Table BBCode plugin here: https://xenforo.com/community/resources/cta-table-bb-code.2847/ with only a little success due to the tables being on multiple lines. An example of a row is below:

[TR="class: before"]
[TD="class: cell-date"] 08/04[/TD]
[TD="class: cell-team-a"] MHR
54
[/TD]
[TD="class: cell-logo-a"] *IMAGE*[/TD]
[TD="class: cell-before-score"][/TD]
[TD="class: cell-score"] 18h30[/TD]
[TD="class: cell-after-score"][/TD]
[TD="class: cell-logo-b"] *IMAGE*[/TD]
[TD="class: cell-team-b"] FCG
14
[/TD]
[/TR]

Is there anything I can do to convert these multi-line tables?