What is the recommended process on cleaning user accounts?

Helmuts

Helmuts

New member
Good morning,

What is the recommended process on cleaning user accounts? Last April I acquired 2 old forums (est. in 2001 and 2004) and need to clean the users so that we can start sending monthly DMs (newsletters with industry updates) to our users, and maintaining low email bounce-rate.

Received an interesting suggestion:
eva2000 said:
you don't need to delete those users, just disable email sending to those users after you determine and clean your users' email addresses.
Click to expand...

1 thing I am not prepared to do - removing users who haven't looged in the forum within the last 1 year (or, even 5 years). I keep seeing old members returning once they learn that a forum is in good hands now. .. my main issue is a forum with 530k users > test email blast showed around 50% bounce-rate (and we stopped it, of course).

Thank you for sharing your expertease. H
 
.. manually removed around 70k users by identifying domains that are used for creating bulk user accounts + removing users that have a following pattern in their email address: (dot)character(dot)

for a new webmasters, more details on the last one:
example of the last one: hjk.i.sa.fgg@gmail.com (I am betting that no normal user would use this pattern .i. in their email address)
section: Users > Batch update users
scr:
Screenshot 2024-05-02 at 08.51.45.webp

On the big forum: down from 536k to 466k users
 
Helmuts said:
my main issue is a forum with 530k users > test email blast showed around 50% bounce-rate (and we stopped it, of course).
Click to expand...
Yikes 50% bounce rate!

Paid commercial email list cleaners and manual Xenforo MySQL query updates is what I did at https://xenforo.com/community/threa...cleanup-scrubber-services.129553/post-1153470 with SQL queries https://xenforo.com/community/threa...cleanup-scrubber-services.129553/post-1153497

FYI, paid cleaners only charge for good emails usually, so at 50% bounce rate, you'd only be charged for 50% of your emails!

I've only used https://www.emaillistverify.com but as mentioned in other thread by @Mouth seems https://www.millionverifier.com/#prices is cheaper

As mentioned at https://xenforo.com/community/threa...-moment-with-speed-issues.221295/post-1680335, you could do self hosted email cleans like @DragonByte Tech Mail addon I think. I tried my hand at coding a Python version with free email domain, disposable email database checks and added Xenforo support to display MySQL query for bad emails only to set their status to email_bounce passing flags -xf and -xfdb xenforo and -xfprefix xf_
Code: 
python validate_emails.py -f user@domain1.com -l emaillist.txt -xf -xfdb xenforo -xfprefix xf_
[
    {
      "email": "user+to@domain1.com",
      "status": "ok",
      "free_email": "no"
    }
    {
      "email": "xyz@domain1.com",
      "status": "unknown_email",
      "free_email": "no",
      "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'xyz@domain1.com'; xenforo\""
    }
    {
      "email": "user@tempr.email",
      "status": "disposable",
      "free_email": "unknown"
    }
    {
      "email": "user@domain2.com",
      "status": "ok",
      "free_email": "no"
    }
    {
      "email": "user@gmail.com",
      "status": "ok",
      "free_email": "yes"
    }
    {
      "email": "user@yahoo.com",
      "status": "ok",
      "free_email": "yes"
    }
    {
      "email": "user1@outlook.com",
      "status": "ok",
      "free_email": "yes"
    }
    {
      "email": "user2@hotmail.com",
      "status": "ok",
      "free_email": "yes"
    }
]
Using jq tool to only list MySQL queries need for bad emails only email_bounce updates. The listed MySQL query can be run within SSH session via command line.

Code: 
python validate_emails.py -f user@domain1.com -l emaillist.txt -xf -xfdb xenforo -xfprefix xf_ | jq -r '.[] | select(.xf_sql) | .xf_sql'

mysql -e "UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'xyz@domain1.com'; xenforo"
or you'd run query
SQL: 
UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'xyz@domain1.com';

But there is a risk of doing it yourself is potentially damaging your sending email domain's reputation on the server you run the script from. So a paid email list clean might be better - especially if it's a once off anyway.

Then, you can set a global forum notice for email_bounce users, notifying them to update their email addresses
 
Last edited:
Self hosted email validator requires port 25 to be open on your web server / your firewall, so bear that in mind :)

I had the same problem when first switching to SES, I had to beg them not to kick me out due to the number of bounces, that’s what inspired me to add the feature to DB Mail.
I also told XF to treat any soft bounce as a hard bounce and limit hard bounces to 1. Took a few more emails but it’s under control now :)
 
DragonByte Tech said:
Self hosted email validator requires port 25 to be open on your web server / your firewall, so bear that in mind :)
Click to expand...
Indeed. I added to my Python script 3rd party smtp support so can send from there though still potentially risky for whatever sending domain is used
DragonByte Tech said:
I had the same problem when first switching to SES, I had to beg them not to kick me out due to the number of bounces, that’s what inspired me to add the feature to DB Mail.
Click to expand...
So SES allowed you to stay? If they did, guess there's hope for folks :)

I just ban hotmail and yahoo emails and for years have less problems :D
 
eva2000 said:
I just ban hotmail
Click to expand...
I still have my old Hotmail and use it for a test user on my site (you know, the guy I can ban to test changes to warning rules and that sort of thing) so I have to have a bit of tolerance for it. And, to be honest, I find Gmail as problematic as those two, maybe moreso.
 
eva2000 said:
So SES allowed you to stay? If they did, guess there's hope for folks :)
Click to expand...
It actually happened twice, the second time was due to unauthorised access to SMTP sending. I forget where exactly the bug was.

I had to go into explicit detail on what was wrong and how I had taken steps to fix it so it wouldn’t happen again 😅
 
How weird.
I have a rule that all email accounts must be valid and verified by the free isp.
If someone refuses they cannot post on my forum.
This might be tedious but it stops the rot of bounced emails.
It does the job.
 
You COULD utilise the Activity Summary email functionality to slowly and progressively weed out bouncing email addresses.
Start with very narrow parameters for a small user delivery, then slowly expand out the timeframes.
 
Mendalla said:
I still have my old Hotmail and use it for a test user on my site (you know, the guy I can ban to test changes to warning rules and that sort of thing) so I have to have a bit of tolerance for it. And, to be honest, I find Gmail as problematic as those two, maybe moreso.
Click to expand...
Bans only apply to new regos so won't impact an existing account with Hotmail usually. So your old Hotmail user account can still be used for a test user.

DragonByte Tech said:
It actually happened twice, the second time was due to unauthorised access to SMTP sending. I forget where exactly the bug was.

I had to go into explicit detail on what was wrong and how I had taken steps to fix it so it wouldn’t happen again 😅
Click to expand...
lol pushing the limits. I'd imagine this was a long time ago? Probably Amazon SES's tolerance isn't that great any more?

Mouth said:
You COULD utilise the Activity Summary email functionality to slowly and progressively weed out bouncing email addresses.
Start with very narrow parameters for a small user delivery, then slowly expand out the timeframes.
Click to expand...
Yes, I've done that before. You can also query the XF database last visit and break down user counts into time frame buckets so you have an idea of how many users you'd touch with each parameter/timeframe.

@Helmuts question led me down a rabbit hole coding a script for myself/clients to do this cleaning in-house if necessary including support 3rd party SMTP providers and rotating multiple SMTP providers for SMTP checks and optional syntax/dns checks that byself SMTP checks https://gist.github.com/centminmod/769500a1b92c6791995408883e4aaef2 :D Though for most folks using a paid email cleaner service is probably a better idea :)
 
eva2000 said:
@Helmuts question led me down a rabbit hole coding a script for myself/clients to do this cleaning in-house if necessary including support 3rd party SMTP providers and rotating multiple SMTP providers for SMTP checks
Click to expand...
Might be a dumb question, but:
How do you verify email addresses via "multiple" or "3rd party" SMTP providers?

SMTP checks (eg. "MAIL FROM ... RCPT TO", check response & abort) only produce meaningful results if a direct SMTP session to the recipient domain MX is established; a SMTP provider / smarthost would most likely always accept mail for relay (if syntactically correct and passes relay checks)
 
So to use a verify program, Iupload my list from the ACP and they will give me a list of bad emails or how does this work?
 
Kirby said:
SMTP checks (eg. "MAIL FROM ... RCPT TO", check response & abort) only produce meaningful results if a direct SMTP session to the recipient domain MX is established; a SMTP provider / smarthost would most likely always accept mail for relay (if syntactically correct and passes relay checks)
Click to expand...
Haven't got that far. But from testing so far with AWS SES SMTP at least, ah yes I see what you mean hmmm. Will try other SMTP providers and see if they do the same

For direct local server tests of my script is fine though

example for Xenforo support to display MySQL query for bad emails only to set their status to email_bounce passing flags -xf and -xfdb xenforo and -xfprefix xf_ with disposable_email status field

Code: 
python validate_emails.py -f user@domain1.com -l emaillist.txt -tm all -xf -xfdb xenforo -xfprefix xf_
[
    {
        "email": "user@mailsac.com",
        "status": "ok",
        "status_code": 250,
        "free_email": "yes",
        "disposable_email": "yes"
    },
    {
        "email": "xyz@centmil1.com",
        "status": "invalid_format",
        "status_code": null,
        "free_email": "unknown",
        "disposable_email": "no"
    },
    {
        "email": "user+to@domain1.com",
        "status": "ok",
        "status_code": 250,
        "free_email": "no",
        "disposable_email": "no"
    },
    {
        "email": "xyz@domain1.com",
        "status": "unknown_email",
        "status_code": 550,
        "free_email": "no",
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'xyz@domain1.com'; xenforo\""
    },
    {
        "email": "abc@domain1.com",
        "status": "unknown_email",
        "status_code": 550,
        "free_email": "no",
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'abc@domain1.com'; xenforo\""
    },
    {
        "email": "123@domain1.com",
        "status": "unknown_email",
        "status_code": 550,
        "free_email": "no",
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = '123@domain1.com'; xenforo\""
    },
    {
        "email": "pop@domain1.com",
        "status": "unknown_email",
        "status_code": 550,
        "free_email": "no",
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'pop@domain1.com'; xenforo\""
    },
    {
        "email": "pip@domain1.com",
        "status": "unknown_email",
        "status_code": 550,
        "free_email": "no",
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'pip@domain1.com'; xenforo\""
    },
    {
        "email": "user@tempr.email",
        "status": "ok",
        "status_code": 250,
        "free_email": "no",
        "disposable_email": "yes"
    },
    {
        "email": "info@domain2.com",
        "status": "ok",
        "status_code": 250,
        "free_email": "no",
        "disposable_email": "no"
    },
    {
        "email": "user@gmail.com",
        "status": "ok",
        "status_code": 250,
        "free_email": "yes",
        "disposable_email": "no"
    },
    {
        "email": "op999@gmail.com",
        "status": "unknown_email",
        "status_code": 550,
        "free_email": "yes",
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'op999@gmail.com'; xenforo\""
    },
    {
        "email": "user@yahoo.com",
        "status": "ok",
        "status_code": 250,
        "free_email": "yes",
        "disposable_email": "no"
    },
    {
        "email": "user1@outlook.com",
        "status": "ok",
        "status_code": 250,
        "free_email": "yes",
        "disposable_email": "no"
    },
    {
        "email": "user2@hotmail.com",
        "status": "ok",
        "status_code": 250,
        "free_email": "yes",
        "disposable_email": "no"
    }
]

a comparison using a commercial paid service EmailListVerify for the same emaillist.txt tested above. You can sign up using my affiliate link for EmailListVerify and free accounts get 100 free email verifications for starters.

Code: 
disposable,"user@mailsac.com"
dead_server,"xyz@centmil1.com"
ok,"user+to@domain1.com"
disposable,"user@tempr.email"
ok,"info@domain2.com"
email_disabled,"xyz@domain1.com"
email_disabled,"abc@domain1.com"
email_disabled,"123@domain1.com"
email_disabled,"pop@domain1.com"
email_disabled,"pip@domain1.com"
ok,"user@gmail.com"
email_disabled,"op999@gmail.com"
ok,"user@yahoo.com"
ok,"user1@outlook.com"
ok,"user2@hotmail.com"


RandallC said:
Amazon allows 10% bounce, I however got suspended at 14% and had to go under review.
Click to expand...
So you fixed the issue and ok'd for review?
RandallC said:
So to use a verify program, Iupload my list from the ACP and they will give me a list of bad emails or how does this work?
Click to expand...
Yeah basically it - how I previous did it outlined at https://xenforo.com/community/threa...cleanup-scrubber-services.129553/post-1153470
 
Last edited:
eva2000 said:
Haven't got that far. But from testing so far with AWS SES SMTP at least, ah yes I see what you mean hmmm. Will try other SMTP providers and see if they do the same


So you fixed the issue and ok'd for review?

Yeah basically it - how I previous did it outlined at https://xenforo.com/community/threa...cleanup-scrubber-services.129553/post-1153470
Click to expand...
Ya, They will suspend you sending ability put your account under review, Like Dragonbyte said you have to explain what happened, why it happend, what you have done to correct this going forward and if it happens in the next 40 days they will suspend your account.
 
eva2000 said:
example for Xenforo support to display MySQL query for bad emails only to set their status to email_bounce passing flags -xf and -xfdb xenforo and -xfprefix xf_ with disposable_email status field
Click to expand...
Any consideration to making this functionality, non-remote MX checking (dns, disposable, etc.) , read and write/update direct to the XF db?
 
RandallC said:
Ya, They will suspend you sending ability put your account under review, Like Dragonbyte said you have to explain what happened, why it happend, what you have done to correct this going forward and if it happens in the next 40 days they will suspend your account.
Click to expand...
Good to know the procedure. Fortunately, never had to experience it yet :)
Mouth said:
Any consideration to making this functionality, non-remote MX checking (dns, disposable, etc.) , read and write/update direct to the XF db?
Click to expand...

Guess that is an option too - maybe add -xfrun argument to directly run the commands on the XF database :)

For now it's manually done. But I did add external EmailListVerify API support so you can sign up generate API key and use my script that way too to get email results + custom XF MySQL queries. Or do it locally without paid services https://gist.github.com/centminmod/769500a1b92c6791995408883e4aaef2#api-support

The status field value comes from EmailListVerify API check while free_email and disposable_email field values from from script's own database check. The status_code is null as it's not applicable in -api mode

Code: 
python validate_emails.py -f user@domain1.com -l emaillist.txt -tm all -api emaillistverify -apikey $elvkey -xf -xfdb xenforo -xfprefix xf_
[
    {
        "email": "user@mailsac.com",
        "status": "unknown",
        "status_code": null,
        "free_email": "yes",
        "disposable_email": "yes",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'user@mailsac.com'; xenforo\""
    },
    {
        "email": "xyz@centmil1.com",
        "status": "unknown",
        "status_code": null,
        "free_email": "no",
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'xyz@centmil1.com'; xenforo\""
    },
    {
        "email": "user+to@domain1.com",
        "status": "valid",
        "status_code": null,
        "free_email": "no",
        "disposable_email": "no"
    },
    {
        "email": "xyz@domain1.com",
        "status": "email_disabled",
        "status_code": null,
        "free_email": "no",
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'xyz@domain1.com'; xenforo\""
    },
    {
        "email": "abc@domain1.com",
        "status": "email_disabled",
        "status_code": null,
        "free_email": "no",
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'abc@domain1.com'; xenforo\""
    },
    {
        "email": "123@domain1.com",
        "status": "email_disabled",
        "status_code": null,
        "free_email": "no",
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = '123@domain1.com'; xenforo\""
    },
    {
        "email": "pop@domain1.com",
        "status": "email_disabled",
        "status_code": null,
        "free_email": "no",
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'pop@domain1.com'; xenforo\""
    },
    {
        "email": "pip@domain1.com",
        "status": "email_disabled",
        "status_code": null,
        "free_email": "no",
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'pip@domain1.com'; xenforo\""
    },
    {
        "email": "user@tempr.email",
        "status": "unknown",
        "status_code": null,
        "free_email": "no",
        "disposable_email": "yes",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'user@tempr.email'; xenforo\""
    },
    {
        "email": "info@domain2.com",
        "status": "valid",
        "status_code": null,
        "free_email": "no",
        "disposable_email": "no"
    },
    {
        "email": "user@gmail.com",
        "status": "valid",
        "status_code": null,
        "free_email": "yes",
        "disposable_email": "no"
    },
    {
        "email": "op999@gmail.com",
        "status": "email_disabled",
        "status_code": null,
        "free_email": "yes",
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'op999@gmail.com'; xenforo\""
    },
    {
        "email": "user@yahoo.com",
        "status": "valid",
        "status_code": null,
        "free_email": "yes",
        "disposable_email": "no"
    },
    {
        "email": "user1@outlook.com",
        "status": "valid",
        "status_code": null,
        "free_email": "yes",
        "disposable_email": "no"
    },
    {
        "email": "user2@hotmail.com",
        "status": "valid",
        "status_code": null,
        "free_email": "yes",
        "disposable_email": "no"
    }
]

Using jq tool to just filter for MySQL queries.

Code: 
python validate_emails.py -f user@domain1.com -l emaillist.txt -tm all -api emaillistverify -apikey $elvkey -xf -xfdb xenforo -xfprefix xf_ | jq -r '.[] | select(.xf_sql) | .xf_sql'

mysql -e "UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'user@mailsac.com'; xenforo"
mysql -e "UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'xyz@centmil1.com'; xenforo"
mysql -e "UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'xyz@domain1.com'; xenforo"
mysql -e "UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'abc@domain1.com'; xenforo"
mysql -e "UPDATE xf_user SET user_state = 'email_bounce' WHERE email = '123@domain1.com'; xenforo"
mysql -e "UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'pop@domain1.com'; xenforo"
mysql -e "UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'pip@domain1.com'; xenforo"
mysql -e "UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'user@tempr.email'; xenforo"
mysql -e "UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'op999@gmail.com'; xenforo"

EmailListVerify API dashboard :D

1714888979870.webp

edit: seems the disposable emails i test give unknown status via EmailListVerify API versus local gives ok status, so probably need a code update :D

Code: 
python validate_emails.py -f user@domain1.com -l emaillist.txt -tm all -xf -xfdb xenforo -xfprefix xf_ | jq '.[] | select(.disposable_email == "yes")'
{
  "email": "user@mailsac.com",
  "status": "ok",
  "status_code": 250,
  "free_email": "yes",
  "disposable_email": "yes"
}
{
  "email": "user@tempr.email",
  "status": "ok",
  "status_code": 250,
  "free_email": "no",
  "disposable_email": "yes"
}

Looks like Emaillistverify might be correct = unknown so they differentiate disposable emails = unknown I think regardless if the email passes SMTP check. Guess that makes sense, so I should also mark disposable emails so they show xf_sql query

1714891392246.webp

Updated

Code: 
python validate_emails.py -f user@domain1.com -l emaillist.txt -tm all -xf -xfdb xenforo -xfprefix xf_
[
    {
        "email": "user@mailsac.com",
        "status": "ok",
        "status_code": 250,
        "free_email": "yes",
        "disposable_email": "yes",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'user@mailsac.com'; xenforo\""
    },
    {
        "email": "xyz@centmil1.com",
        "status": "invalid_format",
        "status_code": null,
        "free_email": "unknown",
        "disposable_email": "no"
    },
    {
        "email": "user+to@domain1.com",
        "status": "ok",
        "status_code": 250,
        "free_email": "no",
        "disposable_email": "no"
    },
    {
        "email": "xyz@domain1.com",
        "status": "unknown_email",
        "status_code": 550,
        "free_email": "no",
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'xyz@domain1.com'; xenforo\""
    },
    {
        "email": "abc@domain1.com",
        "status": "unknown_email",
        "status_code": 550,
        "free_email": "no",
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'abc@domain1.com'; xenforo\""
    },
    {
        "email": "123@domain1.com",
        "status": "unknown_email",
        "status_code": 550,
        "free_email": "no",
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = '123@domain1.com'; xenforo\""
    },
    {
        "email": "pop@domain1.com",
        "status": "unknown_email",
        "status_code": 550,
        "free_email": "no",
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'pop@domain1.com'; xenforo\""
    },
    {
        "email": "pip@domain1.com",
        "status": "unknown_email",
        "status_code": 550,
        "free_email": "no",
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'pip@domain1.com'; xenforo\""
    },
    {
        "email": "user@tempr.email",
        "status": "ok",
        "status_code": 250,
        "free_email": "no",
        "disposable_email": "yes",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'user@tempr.email'; xenforo\""
    },
    {
        "email": "info@domain2.com",
        "status": "ok",
        "status_code": 250,
        "free_email": "no",
        "disposable_email": "no"
    },
    {
        "email": "user@gmail.com",
        "status": "ok",
        "status_code": 250,
        "free_email": "yes",
        "disposable_email": "no"
    },
    {
        "email": "op999@gmail.com",
        "status": "unknown_email",
        "status_code": 550,
        "free_email": "yes",
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'op999@gmail.com'; xenforo\""
    },
    {
        "email": "user@yahoo.com",
        "status": "ok",
        "status_code": 250,
        "free_email": "yes",
        "disposable_email": "no"
    },
    {
        "email": "user1@outlook.com",
        "status": "ok",
        "status_code": 250,
        "free_email": "yes",
        "disposable_email": "no"
    },
    {
        "email": "user2@hotmail.com",
        "status": "ok",
        "status_code": 250,
        "free_email": "yes",
        "disposable_email": "no"
    }
]
 
Last edited:
eva2000 said:
I did add external EmailListVerify API support so you can sign up generate API key and use my script that way too to get email results + custom XF MySQL queries. Or do it locally without paid services https://gist.github.com/centminmod/769500a1b92c6791995408883e4aaef2#api-support
Click to expand...

added MillionVerifier API support too https://gist.github.com/centminmod/769500a1b92c6791995408883e4aaef2#millionverifier :cool:

Code: 
python validate_emails.py -f user@domain1.com -l emaillist.txt -tm all -api millionverifier -apikey_mv $mvkey -xf -xfdb xenforo -xfprefix xf_
[
    {
        "email": "user@mailsac.com",
        "status": "disposable",
        "status_code": null,
        "free_email": false,
        "disposable_email": "yes",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'user@mailsac.com'; xenforo\""
    },
    {
        "email": "xyz@centmil1.com",
        "status": "invalid",
        "status_code": null,
        "free_email": false,
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'xyz@centmil1.com'; xenforo\""
    },
    {
        "email": "user+to@domain1.com",
        "status": "ok",
        "status_code": null,
        "free_email": false,
        "disposable_email": "no"
    },
    {
        "email": "xyz@domain1.com",
        "status": "invalid",
        "status_code": null,
        "free_email": false,
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'xyz@domain1.com'; xenforo\""
    },
    {
        "email": "abc@domain1.com",
        "status": "invalid",
        "status_code": null,
        "free_email": false,
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'abc@domain1.com'; xenforo\""
    },
    {
        "email": "123@domain1.com",
        "status": "invalid",
        "status_code": null,
        "free_email": false,
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = '123@domain1.com'; xenforo\""
    },
    {
        "email": "pop@domain1.com",
        "status": "invalid",
        "status_code": null,
        "free_email": false,
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'pop@domain1.com'; xenforo\""
    },
    {
        "email": "pip@domain1.com",
        "status": "invalid",
        "status_code": null,
        "free_email": false,
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'pip@domain1.com'; xenforo\""
    },
    {
        "email": "user@tempr.email",
        "status": "disposable",
        "status_code": null,
        "free_email": false,
        "disposable_email": "yes",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'user@tempr.email'; xenforo\""
    },
    {
        "email": "info@domain2.com",
        "status": "ok",
        "status_code": null,
        "free_email": false,
        "disposable_email": "no"
    },
    {
        "email": "user@gmail.com",
        "status": "ok",
        "status_code": null,
        "free_email": true,
        "disposable_email": "no"
    },
    {
        "email": "op999@gmail.com",
        "status": "invalid",
        "status_code": null,
        "free_email": true,
        "disposable_email": "no",
        "xf_sql": "mysql -e \"UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'op999@gmail.com'; xenforo\""
    },
    {
        "email": "user@yahoo.com",
        "status": "ok",
        "status_code": null,
        "free_email": true,
        "disposable_email": "no"
    },
    {
        "email": "user1@outlook.com",
        "status": "ok",
        "status_code": null,
        "free_email": true,
        "disposable_email": "no"
    },
    {
        "email": "user2@hotmail.com",
        "status": "ok",
        "status_code": null,
        "free_email": true,
        "disposable_email": "no"
    }
]
Code: 
python validate_emails.py -f user@domain1.com -l emaillist.txt -tm all -api millionverifier -apikey_mv $mvkey -xf -xfdb xenforo -xfprefix xf_ | jq -r '.[] | select(.xf_sql) | .xf_sql'

mysql -e "UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'user@mailsac.com'; xenforo"
mysql -e "UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'xyz@centmil1.com'; xenforo"
mysql -e "UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'xyz@domain1.com'; xenforo"
mysql -e "UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'abc@domain1.com'; xenforo"
mysql -e "UPDATE xf_user SET user_state = 'email_bounce' WHERE email = '123@domain1.com'; xenforo"
mysql -e "UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'pop@domain1.com'; xenforo"
mysql -e "UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'pip@domain1.com'; xenforo"
mysql -e "UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'user@tempr.email'; xenforo"
mysql -e "UPDATE xf_user SET user_state = 'email_bounce' WHERE email = 'op999@gmail.com'; xenforo"
 
I took a stick of dynamite to the rabbit hole that is email verification LOL

My email verification script now supports 5 commercial email verification services' APIs for - EmailListVerify, MillionVerifier, MyEmailVerifier, CaptainVerify, Proofy.io . Comparison tables for pricing and results here and full details and demo examples at https://github.com/centminmod/validate-emails :D

For @Helmuts to compare pricing for >500k

Provider200k250k300k500k1m2.5m5m10m
EmailListVerify (demo, results)-$349 (0.001396)-$449 (0.000898)$599 (0.000599)$1190 (0.000476)$1990 (0.000398)$3290 (0.000329)
MillionVerifier (demo, results)---$259 (0.000518)$389 (0.000389)-$1439 (0.000288)$2529 (0.000253)
MyEmailVerifier (demo, results)-$349 (0.001396)-$549 (0.001098)$749 (0.000749)$1249 (0.0005)$1849 (0.00037)-
CaptainVerify (demo, results)-$250 (0.001)-$500 (0.001)$650 (0.00065)-$2000 (0.0004)-
Proofy.io (demo, results)$229 (0.001145)-$289 (0.000963)$429 (0.000858)$699 (0.000699)$1399 (0.00056)--
 
Last edited:
Excerpt from larger comparison table showing how each commercial service handles a disposable email that works, an invalid domain, a valid working email and a user account that doesn't exist versus local script lookup result.

Table comparing the JSON field values for each email address across the 5 different Email cleaning service APIs and also compared to local script non-API queries results.

Tested on the same sample emaillist.txt of email addresses. These are their respective returned values for status JSON field which retrieved from the respective API services. While status_code (not used with external APIs), free_email and disposable_email JSON fields are from local script code/databases where applicable.

EmailAPIstatusstatus_codefree_emaildisposable_email
user@mailsac.comEmailListVerifyunknownnullyesyes
user@mailsac.comMillionVerifierdisposablenullfalseyes
user@mailsac.comCaptainVerifyriskynullnoyes
user@mailsac.comProofy.ioundeliverablenullnoyes
user@mailsac.comMyEmailVerifierinvalidnullyesyes
user@mailsac.comLocal Scriptok250yesyes
xyz@centmil1.comEmailListVerifyunknownnullnono
xyz@centmil1.comMillionVerifierinvalidnullfalseno
xyz@centmil1.comCaptainVerifyinvalidnullnono
xyz@centmil1.comProofy.ioundeliverablenullnono
xyz@centmil1.comMyEmailVerifierinvalidnullnono
xyz@centmil1.comLocal Scriptinvalid_formatnullunknownno
user+to@domain1.comEmailListVerifyvalidnullnono
user+to@domain1.comMillionVerifieroknullfalseno
user+to@domain1.comCaptainVerifyvalidnullnono
user+to@domain1.comProofy.iodeliverablenullnono
user+to@domain1.comMyEmailVerifiervalidnullnono
user+to@domain1.comLocal Scriptok250nono
xyz@domain1.comEmailListVerifyemail_disablednullnono
xyz@domain1.comMillionVerifierinvalidnullfalseno
xyz@domain1.comCaptainVerifyinvalidnullnono
xyz@domain1.comProofy.ioundeliverablenullnono
xyz@domain1.comMyEmailVerifierinvalidnullnono
xyz@domain1.comLocal Scriptunknown_email550nono
 
You must log in or register to reply here.

Similar threads

Helmuts
What is your self-hosted SMTP solution? We are testing Postal at the moment, with speed issues
2
Replies
23
Views
802
eva2000
eva2000
DomainArchitect.com
  • Question Question
XF 1.1 What is the recommended way to install PostRelease Script
Replies
7
Views
1K
JBS
JBS
Back
Top Bottom