![[DigitalPoint] Better Google Analytics](/community/data/resource_icons/8/8752.jpg?1653152083){kind=icon}
[DigitalPoint] Better Google Analytics 1.1.2.2
- Thread starter digitalpoint
- Start date
digitalpoint
Well-known member
Nothing code-wise, no. If you have a URL, I could look at it though… traffic/data looks normal on my sites.
frm
Well-known member
I’ve had recent reports from forum owners that have indicated a noticeable uptick in bot activity while using the [DigitalPoint] Better Google Analytics add-on for XenForo, with a subsequent decline in such activity after uninstalling it. While the add-on's code has been audited and confirmed to send no information to DigitalPoint or any external entity—only making API calls to Google—this still raised some important questions for me, that also may be useful for the community, about how Better Analytics (the Google Application, not the XenForo add on) interacts with a user’s Google Analytics account and what its underlying processes entail.
Upon reviewing the Privacy Policy and Terms of Service, it appears that they were originally written with forum usage in mind. However, given their broad and loosely defined language, it is unclear how they apply to Better Analytics itself, particularly in relation to Google Analytics data, which could potentially be classified as "Content" under these terms.
To ensure transparency and clarify any privacy concerns, it would be helpful to address the following questions:
I am making this request on the public forum, as opposed to in private, so that forum administrators, who may not be aware of what a Google Application can do once granted permission, have the opportunity to learn what could be done with their data once their Google Analytics account is linked. I have nothing personal against you or DigitalPoint—I simply want to ensure that my privacy, as well as the privacy of others, is handled with the utmost care.
I appreciate your taking the time to address these concerns and look forward to your response.
Upon reviewing the Privacy Policy and Terms of Service, it appears that they were originally written with forum usage in mind. However, given their broad and loosely defined language, it is unclear how they apply to Better Analytics itself, particularly in relation to Google Analytics data, which could potentially be classified as "Content" under these terms.
To ensure transparency and clarify any privacy concerns, it would be helpful to address the following questions:
- Google Analytics Account Linking – What specific data is accessed, or can be accessed, or modified when a user links their Google Analytics account to Better Analytics?
- Google’s Data Usage – Once connected, how does Google Analytics handle the data that Better Analytics interacts with, and vice versa—how does Better Analytics process, modify, or utilize data received from Google Analytics?
- Third-Party Data Transmission – Does the Service harvest, store, or transfer any Google Analytics data to another application, external service, or third party, either directly or through another application?
- Data Retention Policy – If any data is collected, does Better Analytics adhere to the stated policy of deleting such data after four days?
- Data Processing & Security – If data is stored, transmitted, or processed externally, what security measures are in place to protect it, and how is it handled?
- Access & Sharing – Other than yourself or the DigitalPoint entity, does anyone else have access to the data, or is it shared with any external parties?
- Privacy Policy & Terms of Service Updates – Given the nature of these concerns, do you plan to update the Privacy Policy and Terms of Service for Better Analytics to provide greater transparency regarding the handling of Google Analytics data once an account is connected?
- Independent Audits – Would you allow an independent third party to audit Better Analytics (the Google Application, not the add on) at any given notice to verify compliance with privacy and security standards?
I am making this request on the public forum, as opposed to in private, so that forum administrators, who may not be aware of what a Google Application can do once granted permission, have the opportunity to learn what could be done with their data once their Google Analytics account is linked. I have nothing personal against you or DigitalPoint—I simply want to ensure that my privacy, as well as the privacy of others, is handled with the utmost care.
I appreciate your taking the time to address these concerns and look forward to your response.
digitalpoint
Well-known member
My servers can technically see the single use code when a Google Analytics account is linked because it has to bounce off my server and redirect back to the user's server. The user's server is what redeems that code for an API token though, and the API token is never stored or seen outside of their server (never passes through my servers). If my server was to maliciously redeem that code for an API token, someone would immediately know because they couldn't redeem the code for a token themselves (it's a single redemption). Long story short is I can't see anything about anyone's Analytics because the API token never leaves their server. There's some comments/info on it in the source code already:
PHP:
/*
* Yes, these ($credentials) are supposed to be here. No, you can't do anything malicious with it.
* client_id and secret are fine to be distributed when it's an "installed application" (such as an add-on).
*
* See: https://developers.google.com/identity/protocols/oauth2#installed
* "The process results in a client ID and, in some cases, a client secret, which you embed in the source code of your application. (In this context, the client secret is obviously not treated as a secret.)"
*
* As part of the OAuth2 process, Google issues a code that can be redeemed once for a token (which is what is used to authenticate to user's Analytics account via API).
* The api.digitalpoint.com URL is used to redirect the code back to the user's site, where the code is redeemed for a token.
* Google's OAuth2 system requires a static auth_url that is pre-defined (you can't tell Google to redirect back to user's site directly since each user's site is different), so this is how it's done.
* There is no security risk because the code Google passes to api.digitalpoint.com is invalid after it's redeemed for a token, and can only be redeemed once.
* api.digitalpoint.com can't redeem the code and then pass it along to the user's site because then the user themselves wouldn't be able to get a token.
*/It doesn't process, modify or utilize the data in any way beyond the obvious... The API can help a user setup a web property, it can fetch reporting data on demand for reports in the admin area (under Admin -> Logs -> Analytics). API calls are never made beyond those cases, and the data from the reporting API calls is never saved or transmitted. The API calls for creating a web property does result in the Measurement ID being saved and also transmitted to Google (it's how Google knows what property to assign events to).
No, API tokens and data retrieved from use of those tokens never leaves the user's server.
See above. The only data that's ever collected is the Measurement ID for the web property (and by "collected", I mean it's saved to the user's server, not mine or anyone else's), as it's needed to use Google Analytics.
Data is never stored, transmitted or processed externally. Data also isn't stored internally on a user's server with the exception of the Measurement ID (see above).
No one has access to user's Analytics accounts except I guess maybe Google since it's their product. That includes me. I couldn't make any API call or get any data from any site using this addon even if they wanted me to. I don't have (and never had) any user's API tokens, nor do API tokens ever transit through my servers at any point.
No plans, no. The API tokens and data never leave the user's server. Again, I can't (and could never) see anything about anyone's site. A privacy policy about something I never had access to or could get access to seems a little silly. Kind of along the lines of me adding a privacy policy about what I'll do with your bank account or credit card info (that I also never had, nor can get).
Sure, but not really sure how that would happen, as it's not an "application", it's a project under Google's API Console. Google itself had to audit requested scope usage in order to use their APIs for the project. Giving a third party access to the project in the Google API Console really doesn't do much other than them being able to see, "yep, it's a Google API project..." but we already know that.
Again, the only thing you can really see there are the scopes that Google has already audited/approved for use on the project. Which isn't any big secret (it's the same scopes someone approves when they do the initial OAuth2 connection).
The "app" is this addon. So auditing the app means auditing the code you can already see. There is no external part of the app beyond the part that does the initial OAuth2 redirection so a user can redeem the OAuth2 code for an API token on their server). This is that code that redirects the code back to the user's site in it's entirety (yes, I know it's still on XenForo 1.. hah):
PHP:
public function actionBetterAnalytics()
{
$apiAction = $this->_input->filterSingle('api_action', XenForo_Input::STRING);
if ($apiAction == 'link')
{
$state = $this->_input->filterSingle('state', XenForo_Input::STRING);
if ($code = $this->_input->filterSingle('code', XenForo_Input::STRING))
{
return $this->responseRedirect(XenForo_ControllerResponse_Redirect::SUCCESS, $state . '&code=' . urlencode($code));
}
else
{
return $this->responseError('Invalid request.');
}
}
}No data is ever collected, stored or used, nor do I have access to user's API tokens (the initial OAuth2 code is a single-use code that a user's server uses to get an API token). And that API token never leaves their server (beyond the obvious use of the user's server using it for making calls directly to Google Analytics APIs).
You could offer me a billion dollars to give you any info about your own site, and I could not. Because I don't have, nor did I ever have access to user's API tokens. You don't need to rely on my reputation for anything (nor should you). It's simply not something I ever had or can get.
I'm not worried about it and don't take it personally. Again, the whole system was designed/built in a way that you don't have to trust me because I never had, nor can I get access to anyone's data. API calls are made directly to Google Analytics API endpoints with your own API tokens. The requests don't pass through my servers or anything like that.
Last edited:
frm
Well-known member
My main concern was whether the Project owner could access any pass-through data, like a user's Analytics data, once an account is linked to Better Analytics.
You've made it clear that there is no way to request or extract data from Better Analytics—whether through a separate application or any other method.
I appreciate the clarification.
frm
Well-known member
About the same here, but I forgot my original username and password. While DigitalPoint has dealt with legal matters in the past, they aren’t relevant to this discussion, so I won’t revisit old issues. That said, those events have nothing to do with my skepticism about Better Analytics. My concerns stem purely from observing a spike in bot activity when it was installed and a drop when it was removed, which made me curious about what was happening behind the scenes, whether it was an issue with the add on or with the Project.
digitalpoint
Well-known member
Right... like I said, you could literally offer me a billion dollars and I couldn't make an API request for your site because I simply don't have (nor ever had) access to your API tokens. If you were offering me a billion dollars and you needed me to make an API call with your token, I could make a malicious addon you could then install into XenForo that could send me the API token. But you'd have to install it... and along the same lines, I could make an addon to create me as an admin on your site and also wipe out your entire database/all your users. But again... you'd have to go out of your way to install that. Without installing something new, I couldn't make an API call to your Analytics account even if my life depended on it.
Well, the good news is you don't have to trust me directly or indirectly. Like if my servers were hacked and someone got everything... they STILL couldn't get anyone's data because API tokens never existed on my servers (nor could they ever have... it's the whole reason the OAuth2 protocol uses single-use OAuth2 codes that get redeemed for API tokens). As I mentioned above, even if my server were maliciously intercepting the OAuth2 code, if I were to redeem it for an API token, it would be immediately apparent to the end-user because THEY couldn't redeem it themselves (they would never get an API token... which all happens on their server).
digitalpoint
Well-known member
A spike in bot activity definitely isn't coming from this addon... at least not directly. Maybe if a bot or process was monitoring your site just to see if it was using Google Analytics (the event tracking requests aren't hidden to a browser), they could do something. But seems like a lot of work and not particularly practical to spend time monitoring and waiting for a site to use Google Analytics before it sends bots.
Maybe if you mention what bots I could tell you why they would visit? But again... nothing from this addon. I can't even tell you the domains/sites using this addon, so... I don't think Google's API Console will even tell me the number of total sites have granted OAuth2 credentials (not that I know about anyway), they definitely don't tell me "who".
digitalpoint
Well-known member
FWIW, this is the extent of the info I can get from the Google Api Console... I can see how many requests are being made to various API endpoints... but I can't tell you what site made those requests or what the underlying data was. Google let's API projects see this sort of info for debugging purposes... in case API calls are throwing errors or being slow. Like I can see that in the last 30 days users collectively made 18,237 API calls for realtime data (what you see on the index of the admin screen) and made 652 reporting API calls (what would be used when someone looks at reports in the admin area). But beyond that... nada. I can't tell you who or what data the API calls contained.
Again, this is just the tools Google gives developers to debug... like if there's a massive spike in API errors, maybe the application (the addon in this case) is doing something wrong.
Again, this is just the tools Google gives developers to debug... like if there's a massive spike in API errors, maybe the application (the addon in this case) is doing something wrong.
I dunno, he seems pretty sus to me. Lets throw him out the airlock.
frm
Well-known member
Since this is third-party reporting, I'll need to follow up to identify the specific bots, IP ranges, and other anomalies they've mentioned. If they're willing to share more details, I'll provide an update to determine whether this is actually related to the add-on or just a coincidence.
For now, it's 0230 in Japan, so I'm heading to bed. But, I appreciate you taking the time to address my concerns with such openness and transparency that you have.
MentaL
Well-known member
Tbh I was thankful when he sent me the nudes I lost. Unsure how he had them backed up but who am I to complain? Jk.
Please ignore. My bad.
I disabled the template modification insert GA JS by mistake. Turned it back on and it works again.
Last edited:
MentaL
Well-known member
one more, assuming its this. Cannot replicate this.
#1 src/vendor/guzzlehttp/guzzle/src/Client.php(328): GuzzleHttp\Client->applyOptions(Object(GuzzleHttp\Psr7\Request), Array)
#2 src/vendor/guzzlehttp/guzzle/src/Client.php(169): GuzzleHttp\Client->transfer(Object(GuzzleHttp\Psr7\Request), Array)
#3 src/vendor/guzzlehttp/guzzle/src/ClientTrait.php(207): GuzzleHttp\Client->requestAsync('POST', Object(GuzzleHttp\Psr7\Uri), Array)
#4 src/addons/DigitalPoint/Analytics/GoogleApi/Measurement.php(47): GuzzleHttp\Client->postAsync('https://www.goo...', Array)
#5 src/addons/DigitalPoint/Analytics/GoogleApi/Measurement.php(76): DigitalPoint\Analytics\GoogleApi\Measurement->makeApiCall('https://www.goo...', Array)
#6 src/addons/DigitalPoint/Analytics/Repository/Measurement.php(11): DigitalPoint\Analytics\GoogleApi\Measurement->collect('d559cac7-6a33-4...', Array, 0, Array, true)
#7 src/addons/DigitalPoint/Analytics/Pub/Controller/Beacon.php(35): DigitalPoint\Analytics\Repository\Measurement->collect('d559cac7-6a33-4...', Array, [invalid])
#8 src/XF/Mvc/Dispatcher.php(362): DigitalPoint\Analytics\Pub\Controller\Beacon->actionImage(Object(XF\Mvc\ParameterBag))
#9 src/XF/Mvc/Dispatcher.php(264): XF\Mvc\Dispatcher->dispatchClass('DigitalPoint\\An...', 'Image', Object(XF\Mvc\RouteMatch), Object(DigitalPoint\Analytics\Pub\Controller\Beacon), NULL)
#10 src/XF/Mvc/Dispatcher.php(121): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(DigitalPoint\Analytics\Pub\Controller\Beacon), NULL)
#11 src/XF/Mvc/Dispatcher.php(63): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#12 src/XF/App.php(2826): XF\Mvc\Dispatcher->run()
#13 src/XF.php(806): XF\App->run()
#14 index.php(23): XF::runApp('XF\\Pub\\App')
#15 {main}
["url"] => string(163) "/b/image.gif?p=msaXihPQ2uaplMWUAIjic58p71rpFTCmsAire2P17WYVE90nHXTiVGo%2BxL82M7OUmeBzU4XALQvBKsEHQkItxUY8wFBbvgXfhi%2FpYd6mSM9zPJq4fH%2FcvVUxShfZpfF6ug%2FbbIDTIo2X"
["referrer"] => bool(false)
["_GET"] => array(1) {
["p"] => string(140) "xxx"
}
["_POST"] => array(0) {
}
}
- GuzzleHttp\Exception\InvalidArgumentException: json_encode error: Malformed UTF-8 characters, possibly incorrectly encoded
- src/vendor/guzzlehttp/guzzle/src/Utils.php:298
- Generated by: Unknown account
- Mar 6, 2025 at 7:15 AM
Stack trace
#0 src/vendor/guzzlehttp/guzzle/src/Client.php(377): GuzzleHttp\Utils::jsonEncode(Array)#1 src/vendor/guzzlehttp/guzzle/src/Client.php(328): GuzzleHttp\Client->applyOptions(Object(GuzzleHttp\Psr7\Request), Array)
#2 src/vendor/guzzlehttp/guzzle/src/Client.php(169): GuzzleHttp\Client->transfer(Object(GuzzleHttp\Psr7\Request), Array)
#3 src/vendor/guzzlehttp/guzzle/src/ClientTrait.php(207): GuzzleHttp\Client->requestAsync('POST', Object(GuzzleHttp\Psr7\Uri), Array)
#4 src/addons/DigitalPoint/Analytics/GoogleApi/Measurement.php(47): GuzzleHttp\Client->postAsync('https://www.goo...', Array)
#5 src/addons/DigitalPoint/Analytics/GoogleApi/Measurement.php(76): DigitalPoint\Analytics\GoogleApi\Measurement->makeApiCall('https://www.goo...', Array)
#6 src/addons/DigitalPoint/Analytics/Repository/Measurement.php(11): DigitalPoint\Analytics\GoogleApi\Measurement->collect('d559cac7-6a33-4...', Array, 0, Array, true)
#7 src/addons/DigitalPoint/Analytics/Pub/Controller/Beacon.php(35): DigitalPoint\Analytics\Repository\Measurement->collect('d559cac7-6a33-4...', Array, [invalid])
#8 src/XF/Mvc/Dispatcher.php(362): DigitalPoint\Analytics\Pub\Controller\Beacon->actionImage(Object(XF\Mvc\ParameterBag))
#9 src/XF/Mvc/Dispatcher.php(264): XF\Mvc\Dispatcher->dispatchClass('DigitalPoint\\An...', 'Image', Object(XF\Mvc\RouteMatch), Object(DigitalPoint\Analytics\Pub\Controller\Beacon), NULL)
#10 src/XF/Mvc/Dispatcher.php(121): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(DigitalPoint\Analytics\Pub\Controller\Beacon), NULL)
#11 src/XF/Mvc/Dispatcher.php(63): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#12 src/XF/App.php(2826): XF\Mvc\Dispatcher->run()
#13 src/XF.php(806): XF\App->run()
#14 index.php(23): XF::runApp('XF\\Pub\\App')
#15 {main}
Request state
array(4) {["url"] => string(163) "/b/image.gif?p=msaXihPQ2uaplMWUAIjic58p71rpFTCmsAire2P17WYVE90nHXTiVGo%2BxL82M7OUmeBzU4XALQvBKsEHQkItxUY8wFBbvgXfhi%2FpYd6mSM9zPJq4fH%2FcvVUxShfZpfF6ug%2FbbIDTIo2X"
["referrer"] => bool(false)
["_GET"] => array(1) {
["p"] => string(140) "xxx"
}
["_POST"] => array(0) {
}
}
digitalpoint
Well-known member
Most likely, that's going to be someone reading an email with an invalid (non-UTF8) subject somehow. JSON encode doesn't work for non-UTF8 characters. I ended up making a change to strip out non-UTF8 before it's sent to the API just in case there's some in there for whatever reason (for next version).
SatGuyScott
Active member
Just a heads up, since applying the update no one has reported the issue.
Thanks!
ichpen
Well-known member
Yeah, fixed it right up. Thanks @digitalpoint
dunowat
Member
Out of curiosity, I've recently upgraded to 2.3.6 and the most recent better analytics. Since then I've seen a rise in Event > Exceptions > Not Set. What's the best way to get this set so I can see the issue that's happening. I didn't change any of the defaults and I have over 1 million events in a day.
