XF 2.2 Auto subscription failing - Information: Invalid custom field. Unrelated payment, no action to take.

[bushcraft Tone]

I would appreciate some help.

I'm having some problems with my subscriptions, there's quite a few of them failing with the message Information: Invalid custom field. Unrelated payment, no action to take. Although it's not all of them, some of them seem to go through ok.
The callback address is correct

After a number of failures in delivering the notifications from PayPal they disable the notification system which then means that no one can have their recurring or new subscriptions validated.

This is a typical log:
array(29) {
["payer_id"] => string(13) "YR6RCQNJ4RUBY"
["ipn_track_id"] => string(13) "3e30ae2cf1401"
["charset"] => string(12) "windows-1252"
["payment_gross"] => string(0) ""
["verify_sign"] => string(56) "ADr0NHzeCFe.CSxO7pGJJRhPURJ6ArMARMNskx.NrU78S3pyZbu-5.F-"
["item_name"] => string(47) "Account upgrade: Full member - Gold (xxxxxxxx)"
["txn_type"] => string(14) "subscr_payment"
["receiver_id"] => string(13) "DC6ESBY32JQYN"
["payment_fee"] => string(0) ""
["mc_currency"] => string(3) "GBP"
["transaction_subject"] => string(47) "Account upgrade: Full member - Gold (xxxxxxxxx)"
["custom"] => string(32) "i-jX5SR825FPHDGBHculoBpk1AW4gSM8"
["protection_eligibility"] => string(8) "Eligible"
["payer_status"] => string(8) "verified"
["first_name"] => string(7) "Stephen"
["subscr_id"] => string(14) "I-C0PJRYS9RDSH"
["mc_gross"] => string(5) "15.00"
["payment_date"] => string(25) "07:03:09 Apr 25, 2021 PDT"
["payment_status"] => string(9) "Completed"
["business"] => string(20) "shop@bushcraftuk.com"
["last_name"] => string(5) "xxxxx"
["txn_id"] => string(17) "3YD131352G037174T"
["mc_fee"] => string(4) "0.74"
["resend"] => string(4) "true"
["payment_type"] => string(7) "instant"
["notify_version"] => string(3) "3.9"
["payer_email"] => string(20) "xxxxxxxxxx"
["receiver_email"] => string(20) "shop@bushcraftuk.com"
["residence_country"] => string(2) "GB"

Another:
array(38) {
["payer_id"] => string(13) "QDR3PUUPHBBZ4"
["address_country_code"] => string(2) "GB"
["ipn_track_id"] => string(13) "89066c24fe26f"
["address_zip"] => string(8) "xxxxx"
["charset"] => string(12) "windows-1252"
["payment_gross"] => string(0) ""
["address_status"] => string(9) "confirmed"
["address_street"] => string(21) "xxxxxxxx"
["verify_sign"] => string(56) "AX8ieAkKpYYejWHnkQpJQhiSXfgJAVxTINmwARf.Etnl0.5bPTeTiNQH"
["item_name"] => string(31) "Full member - Gold Subscription"
["txn_type"] => string(14) "subscr_payment"
["receiver_id"] => string(13) "DC6ESBY32JQYN"
["payment_fee"] => string(0) ""
["mc_currency"] => string(3) "GBP"
["transaction_subject"] => string(31) "Full member - Gold Subscription"
["custom"] => string(8) "Stingray"
["protection_eligibility"] => string(8) "Eligible"
["address_country"] => string(0) ""
["payer_status"] => string(8) "verified"
["first_name"] => string(4) "xxxx"
["address_name"] => string(10) "xxxxxx"
["subscr_id"] => string(14) "I-K755CNBCHYVK"
["mc_gross"] => string(5) "15.00"
["payment_date"] => string(25) "04:54:10 Apr 23, 2021 PDT"
["payment_status"] => string(9) "Completed"
["business"] => string(20) "shop@bushcraftuk.com"
["item_number"] => string(32) "d480e98a5adf78059715c73761f62719"
["last_name"] => string(5) "xxxxxx"
["address_state"] => string(4) "xxxxxx"
["txn_id"] => string(17) "03E32194RS213673D"
["mc_fee"] => string(4) "0.74"
["resend"] => string(4) "true"
["payment_type"] => string(7) "instant"
["notify_version"] => string(3) "3.9"
["payer_email"] => string(26) "xxxxxxx"
["receiver_email"] => string(20) "shop@bushcraftuk.com"
["address_city"] => string(19) "xxxxxxxxxx"
["residence_country"] => string(2) "GB"

This is the corresponding IPN form PayPal

Instant Payment Notification (IPN) details​

Message ID03D151821X974135C
Date/time created 23/04/2021 12:54 BST
Original/Resent Original
Latest delivery attempt date/time 27/04/2021 10:28 BST
Notification URL http://www.bushcraftuk.com/forum/payment_gateway.php?method=paypal
HTTP response codeWhat's this? 500
Delivery status Failed
No. of retries 16
Transaction ID 03E32194RS213673D
IPN type Transaction made

mc_gross=15.00&protection_eligibility=Eligible&address_status=confirmed&payer_id=QDR3PUUPHBBZ4&address_street=40 xxxxxxxx Avenue&payment_date=04:54:10 Apr 23, 2021 PDT&payment_status=Completed&charset=windows-1252&address_zip=xxxxx xxxx&first_name=xxx&mc_fee=0.74&address_country_code=GB&address_name=xxxxx&notify_version=3.9&subscr_id=I-K755CNBCHYVK&custom=xxxxx&payer_status=verified&business=shop@bushcraftuk.com&address_country=&address_city=xxxxxx,xxxxxx&verify_sign=AT2OxmoAsrt3V2phmge.GWuB7ltWAXFm7fsvcP9lrNBaLAeLZlB1rtUP&payer_email=xxxxxx@googlemail.com&txn_id=03E32194RS213673D&payment_type=instant&last_name=xxxxx&address_state=xxx&receiver_email=shop@bushcraftuk.com&payment_fee=&receiver_id=DC6ESBY32JQYN&txn_type=subscr_payment&item_name=Full member - Gold Subscription&mc_currency=GBP&item_number=d480e98a5adf78059715c73761f62719&residence_country=GB&transaction_subject=Full member - Gold Subscription&payment_gross=&ipn_track_id=89066c24fe26f

Any thoughts?
I've been resending them from PayPal side after resetting the IPN to active and some have gone through, others not. Issues first started a while ago and I figure best to get this sorted before it cascades into something more complicated and harder to sort out.

Any help greatly appreciated.
Thanks
Tony

 

[Mike]

The "invalid custom field" message only applies when we think the callback URL is a "legacy" (XF 1.x) URL. This only happens if the callback URL doesn't have an _xfProvider value.

What you've shown in this part of your message is probably relevant:

Notification URL http://www.bushcraftuk.com/forum/payment_gateway.php?method=paypal

That's not and has never been an XF payment callback URL. I presume that is from an add-on that you may have used in the past (or are still using?). The callback seems to be returning a 500 error as well, so I don't think that's the "invalid custom field" message.

Looking at the IPN example data you included, two of the three aren't ones that XF itself would've generated, so I have to presume they relate to payments from an add-on. The first one does look like a request generated by XF itself, but the question then is why does it not seem to have the expected callback URL. We explicitly send this through with the request so the value in the PayPal account generally doesn't matter.

Do any of the PayPal IPN calls indicate they've gone to payment_callback.php?_xfProvider=paypal? That would be the only expected URL for XF 2.x payments. If someone subscribed in XF 1.x, then the URL would be payment_callback.php only.

 

[bushcraft Tone]

Thanks Mike, I appreciate you helping with this.

I've a good number of XF1.x subscribers as well as 2.x

The notification URL setup in PayPal is - https://bushcraftuk.com/community/payment_callback.php

I've always run my forums with very few add-on's, UI.X style, TH user profile notes, tapatalk with nothing set up on it, and XF enhanced search. Nothing that should have an impact on payments.

OK, I've searched through the last 14 days and I've got about a dozen that failed with the callback
]http://www.bushcraftuk.com/forum/payment_gateway.php?method=paypal

and the rest sent on from PayPal with the callbacks:

https://bushcraftuk.com/community/payment_callback.php?_xfProvider=paypal

&

https://bushcraftuk.com/community/payment_callback.php

So, is there a way that I can change the callback URL on those members that have the faulty one to one that works?(http://www.bushcraftuk.com/forum/payment_gateway.php?method=paypal / https://bushcraftuk.com/community/payment_callback.php?_xfProvider=paypal)

and if I could would that resolve the issue? I've no idea how these got messed up, it's affected a fair amount of subscribers...

 

[Mike]

Unfortunately, you can't change the IPN (callback) URL on an existing subscription -- this is a PayPal limitation and not something we can do anything about.

Certainly in the case of the payment_gateway.php examples, there isn't really anything we can do with this. Doing a search here, this might relate to vBulletin subscriptions (so from prior to your migration). This isn't something we've ever tried to handle and it's not just a matter of redirecting the request because the format of the input data and how it looks up things like subscriptions would be entirely different. At best, you could create an empty payment_gateway.php file and handle the processing of payments manually (extending their subscriptions, etc). Though on the assumption that you've been running XF for quite a long time and these subscriptions pre-date it, they are quite likely expired within XF already -- although I can't say what might've happened to their user groups, as the XF group approach likely differed from the software you used previously. If this is the case, you may be best cancelling the subscription within PayPal and having them potentially resubscribe.

In terms of payment_callback.php without _xfProvider, this is a normal callback for a subscription that was created in XF 1.x, so there isn't anything inherently wrong with it. The fact that you do seem to have some callbacks with _xfProvider mentioned would imply that XF 2.x is passing the IPN URL into the payments as expected (because otherwise you wouldn't have any callbacks to that). You may need to look at the specific PayPal logs to see whether these payments are failing to process as you expect and what the IPN results show for ones that do fail to process (we specifically need to know what it's sending for "custom", as that format differs significantly between XF 1 and 2 requests).